I asked this same question on Reddit and I got zero engagement, so perhaps Lemmy has people that care more about their hardware.
I recently decided to use some of the tools provided by Mr Salter (netburn) and I have to ask the community if you want to see multi-client stress tests (4K streaming, VoIP, web browsing) used on a wireless router or if the single-client iperf tests are good enough. Bear in mind that pretty much all publications that still test their devices (most don’t) rely on the single-client test method.
Removed by mod
Oh, I agree wholeheartedly that collecting the data is not that much fun, especially since yes, I will have to do it. But I think users may benefit to see if the non-enterprise wireless routers can accomplish a certain task. For example, can that expensive Netgear router actually handle four client devices streaming 4K at the same time? What if we add browsing in the mix? The point of this thread was to get an idea if it’s actually worth running these tests (which take quite a bit) and if people are interested in seeing this type of data on the web.
Removed by mod
Can your ISP? If so, yes. Because ~25 Mbps * 4 is not a lot of data. And the NAT for four clients mapped to the same firewall/router is pretty trivial. And no, adding “browsing” is not going to be an issue.
On paper, it is not a lot of data, but then adding more clients requesting 25Mbps continuously and then adding some spontaneous, but intensive web browsing can lead to latency spikes. And the user no longer gets a good streaming/browsing experience. I’ve even seen it on an expensive (by consumer-based networking standard) router such as the GT-AX6000.
So what matters is the amount of packets per second that can be processed which these speed tests already cover (albeit, somewhat obfuscated because most people don’t understand the network layers.
I am just trying to better understand this stuff, so I have to ask if seeing how long it takes for a client device to accomplish a certain task wouldn’t be better than just glancing over the average Mbps in a graph? That’s what most publications are showing.
Removed by mod
You’re talking about real-world scenarios, but I am just trying to get something simulated that resembles general real-life conditions. So, no CDN, the modem and even the ISP don’t matter in this particular scenario. You have written a phenomenal response, so I am sorry that I ask to take some more of your time and check out this article: smallnetbuilder.com/wireless/wireless-reviews/2x2-ac-access-point-roundup-part-2/ This is pretty much what I am trying to accomplish and it does seem that the APs can be stressed by fewer client devices than expected. or maybe again, there’s something that I am missing.
Removed by mod
I understand perfectly where you’re coming from and I would love to find some way to objectively test wireless networking hardware which can be easily replicated in pretty much any other site. The Octoscope tools (now assimilated by Spirent) may be the closest since we get to put the wireless AP/router in a box and then simulate the conditions we want. But, for those that don’t have fat wallets, I guess these open-source tools are good enough, even if the results are heavily subjective. I know that people don’t like good enough, but at this point in time, even the fine edge between realistic and unrealistic is better than nothing.
And funny thing, WiFi 6 is not really that much better than WiFi 5, unless some very specific conditions are met - I’ve seen it in testing. So yes, I know what hype and advertising can do…
Gamer Nexus are my favorite when it comes to PC hardware as well and I would love to see them give it a try at testing wireless networking hardware. Who knows, maybe they’ll create a standard for testing these non-enterprise wireless routers.
Well, I have a MiniPC running a VyOS router and the only time I’ve seen it even break a sweat is when I have to run openvpn or wireguaurd with lots of throughput which is probably because of the encryption involved.
I’m curious as to how the wireless access points part of the network work. I have no problem saturating my bandwidth on wired connections but on wireless, I do get choking when say 3-4 devices try to stream 4k.
I totally be interested in this sort of testing methodology being published. Maybe in a wiki?
Getting comparable numbers for buffer bloat and queuing would be great for commercial routers. Of course you would want to compare against Enterprise solution so that people know where on the spectrum they’re landing.
Full disclosure I roll my own GLI net open WRT router and I enforce different queues for qos seperation… i.e. downloading and streaming shouldn’t interfere with VoIP calls and gaming
Multi client test seems better honestly. I end up running 3-4 iperfs from different clients to a wired server to see how the bandwidth chokes. I wonder how it will be if one of the clients are running the iperf server as well.
Real life workloads like 4k and VoIP with multiple clients seem much more realistic and representative.
Seriously Lemmy is the best. Few minutes in and people are already answering questions. The concept behind the multi-client tests are to SSH into the server and then simulate whatever type of traffic one wants. I have already done it on a couple of wireless routers and it worked great. I hope at least. Iperf can’t really accomplish this, as far as I know, it’s only one instance on a single client at a time. Netburn seems to be the best tool so far, while keeping things free and open-source.
Seconding your opinion about lemmy :) Do you think you could write up that you did? I would be interested in reading. Found this article on ars as well:
https://arstechnica.com/gadgets/2020/01/how-ars-tests-wi-fi-gear-and-you-can-too/
I’ll be checking out netburn.
I am not really a fan of self promotion and I would really like Lemmy to be free of it, as much as possible. If you want to check what I did so far, just write “router review multi-client stress test” in any browser. You really won’t find anybody else trying out these tools at the moment, even Arstechnica seems to have moved away from wireless router reviews.
Sure :)
I recently ran into an issue with my home network where I suspected that the current wifi router (3-point mesh) couldn’t handle all the clients simultaneously. Not in a manner of throughput, but just with keeping all the devices online in the first place.
I have at minimum 30 clients online all the time, up to a max of 40 or something, depending on who’s home or what is active. (Went a bit overboard with home automation and stuff.)
I was getting random disconnects or stalling wifi on some of the devices. The coverage was fine, so I figured it was just the number of wifi signals that was overwhelming the AP.
Point of the story: I was disappointed that absolutely no review/benchmark ever pays attention to this kind of upper limit that seems to exist in practice. It’s all range and speed, but never about the maximum number of active clients.
I’ve replaced the mesh network and everything is fine now. But I had to trial and error that shit…
It isn’t really about the maximum number of client devices, it’s what they do, what type of standard are they a part of, how far away, the interference (!). This is why it’s pretty much impossible to put a number and say: hey, this TP-Link router will handle 30 client devices, while this Asus router goes up to 100… In a sense, a multi-client stress test kind of addresses this issue, but it kind of doesn’t. It’s because it’s extremely dependent on the conditions that the tester has in their lab/office/home.
One thing to check on a review could be the attenuation as a better factor than the distance (this way, you can reproduce the result in your house even if just with a single-client test).
Not really, to be honest. I’d rather see how compatible a particular router is against popular open-source firmware, how frequently the updates are delivered, etc.
For instance, the Asuswrt Merlin is a pretty good firmware for ASUS routers, but the updates (stable) are irregular - the last stable update currently was two months ago, which to me is unacceptable considering there have been critical vulnerabilities in ASUS routers. Given how malware and botnets are increasingly targeting routers these days, it’s imperative that updates get delivered at least once a month - with an out-of-band policy for critical vulnerabilities.
When I run iPerf tests I almost always use multiple clients because of exactly what another comment said, it’s more realistic and a better representation.
So you basically spawn multiple instances of iperf3 and then connect all clients to a single server (using the same port?) What do you think about checking the latency experienced at the client level when various tests are running at the same time?
Yeah, I spawn multiple instance of iperf3. Checking the latency would be a very useful metric, it should give a good result of what the connection will be like under load.
One client downloading a torrent, second client viewing 1080p60 video from YT and clients 3&4 transferring a few TB of data through lan.
I would love to see this via WiFi.
Sounds good to me
