Shaping performance

I will take the praise happily until you actually see the implementation :wink: (full disclosure any bright idea that might be found in there was heavily inspired by @richb-hanover's betterspeedtest.sh, I just hacked pieces together and twiddled until they seemed to work). It was Rich that first pointed me at the fact that shell scripts do not need to look like spaghetti, so thanks for that.

Same here, from my background I simply can not see the rationale for sports with such massive head injury risk like football (I even worry about "headers" in soccer, at least before full brain maturation), but I digress. I hope you had a nice tournament with lots of fun, excitement and justified success.

Is there a version of this that does not rely on the DSL Reports test but could be run with two, sufficiently powerful hosts on a local network, perhaps with flent? Even with a 300 Mbps downstream connection, I can't push my x86 boxes hard enough.

Also, is the use of irqbalance being tracked?

Ok, I merged your suggested bits and maybe fixed a bug or two (at least when run in Bash) and or maybe added minor alterations and the like.

I think for the moment, it's fine as data collection goes, what we need now is basically a strategy for summarizing the results, and then a predictive model that uses the summaries to predict performance of each router.

@jeff

The script just collects data on what's going on, any load at all will work, such as iperf3 or flent or even a file transfer from a fileserver, just as long as it exercises the network during the data collection.

irqbalance isn't specifically being tracked, but it does get data on how much time each cpu is using, so in that sense it does handle multi-core / balanced loads.

also @jeff, even with the 300Mbps if it gets the idle time down below 50% or so extrapolation will probably work fine. one of the things we're doing here (in theory) is collecting data and then using it in a predictive model rather than just using the data points themselves as estimates of the max performance. In fact, the best kind of load would be one that varies bandwidth slowly, building up to max over 10 seconds or something, so we can see how bandwidth relates to idle time.

OK, cool. Just built an apu2c4 image and may poke at it later this week or weekend. There's been all kinds of statements around its limits that surprise me based on my experience running them under Debian, Ubuntu, and FreeBSD. "Stock" OpenWRT firewall with NAT on the WAN, right?

Edit: LuCI, SQM, SQM extras, LuCI SQM module and examine under fq_codel and CAKE. I can crank in any bandwidth I want into the test harness with FreeBSD's ipfw and dummynet traffic-shaping capabilities.

Well, in some sense, anything is data, but the idea here is to figure out shaping performance, so it'd be basically a "functional" install for your purposes + sqm set up so it "works" for your purposes (controls the bufferbloat sufficiently well).

1 Like

A really useful kind of test would be to do something like run a script like this (with appropriate servers on a second machine)

TOTBW=600
for i in 1 2 3 4; do
iperf3 -t $((5-i)*10)) -c myserver -b $((TOTBW/4))M &
sleep 10
done

so that you start out at 1/4 bandwidth, and every 10 seconds you're spawning a new iperf trying to get an additional 1/4 of the bandwidth. (assuming I didn't make errors in that script). This way during the test we'll see bandwidth stepping up and we'll see how increased bandwidth affects cpu usage.

Running it again in reverse mode (-R) could be useful too, as upload and download aren't symmetric on SQM due to the IFB. If you do that you'll maybe want to run the data collection script longer, say 100 seconds

I believe we should still add the output of "uname -srvmp":

user@ubuntu:~$ uname -srvmp
Linux 4.13.0-41-generic #46~16.04.1-Ubuntu SMP Thu May 3 10:06:43 UTC 2018 x86_64 x86_64

root@openwrt:~# uname -srvmp
Linux 4.9.102 #0 Sat May 26 17:42:28 2018 mips unknown

so that we know the kernel in use, and if it exist "cat /etc/os-release":

user@ubuntu:~$ cat /etc/os-release  
NAME="Ubuntu" 
VERSION="16.04.4 LTS (Xenial Xerus)" 
ID=ubuntu 
ID_LIKE=debian 
PRETTY_NAME="Ubuntu 16.04.4 LTS" 
VERSION_ID="16.04" 
HOME_URL="http://www.ubuntu.com/" 
SUPPORT_URL="http://help.ubuntu.com/" 
BUG_REPORT_URL="http://bugs.launchpad.net/ubuntu/" 
VERSION_CODENAME=xenial 
UBUNTU_CODENAME=xenial 

root@openwrt:~# cat /etc/os-release  
NAME="OpenWrt" 
VERSION="SNAPSHOT" 
ID="openwrt" 
ID_LIKE="lede openwrt" 
PRETTY_NAME="OpenWrt SNAPSHOT" 
VERSION_ID="snapshot" 
HOME_URL="http://lede-project.org/" 
BUG_URL="http://bugs.lede-project.org/" 
SUPPORT_URL="http://forum.openwrt.org/" 
BUILD_ID="r7046-5857088c5e" 
LEDE_BOARD="ar71xx/generic" 
LEDE_ARCH="mips_24kc" 
LEDE_TAINTS="no-all busybox" 
LEDE_DEVICE_MANUFACTURER="OpenWrt" 
LEDE_DEVICE_MANUFACTURER_URL="http://lede-project.org/" 
LEDE_DEVICE_PRODUCT="Generic" 
LEDE_DEVICE_REVISION="v0" 
LEDE_RELEASE="OpenWrt SNAPSHOT r7046-5857088c5e"

So that we have a better idea about the relevant software environment things were measured under.

Why did I not simply add them to collect-data.sh? Because I think we should first agree that this information is not leaking information that should be private. But I believe that this should be innocent enough for routine collection. If nobody disagrees, I will create a PR for those.

Best Regards

This is not really required, it was just a way for distributing test versions that never really went anywhere...

1 Like

One of the cool things that flent's RRUL test allows is to concurrently saturate both up and downlink which puts the heaviest load on the shapers. Oh, I also have a hunch the load will inversely scale with packet size (assuming still link saturation) so it might be worth at least documenting the packet size if small packets are used (the default of assuming ~MTU sized packets seems sane though).

2 Likes

I think those bits of data make good sense and are not very sensitive, wrap them in some macro so we can m4 process them into json format, as we do for the other bits.

Great will do, and then it is time to actually start looking at the json generation step as well as the actual plot generation in R....

I hacked at generating ping data json yesterday, and it's pushed to the repo, so at the moment we have both bandwidth and latency going to json.

Next is to provide a decent json object describing the test conditions: WAN and LAN interfaces, kernel version, and CPU characteristics etc. Once we can get those things, the R code can know which interfaces to look at and soforth without manually entering info into the script.

I've got a stack of ping latency, smoothed bandwidth estimate, and smoothed percent idle each vs time that I'm generating as one multi panel plot. It's pretty useful.

Okay, created a pull request for those and threw in a few more. I am not 100% sure about adding "brctl show", sure on openwrt this will reveal the members of the standard br-lan bridge that is the lan interface but for more complex setups it might reveal more than necessary...

BTW I will get back to this in the next couple days, my wife's been out of town at a conference and I've been doing full time daddy duty, not to mention World Cup :wink:

Great, I just realized something else I want to change: tc is actually capable of writing its output to json directly, so I will cut the middleman and have the two "tc -s qdisc" invocations dump directly to json ("tc -j -s qdisc"), one thing less to convert (and one more reason why selecting json as in intermediary was a good idea).

Regarding the world cup, no time yet to watch any game (well, the one game I wanted to watch, being a hopeless Mexico fan, was made impossible by stop-and-go-and-freeze streaming performance by the official channel for the games here; I guess somebody underestimated the number of cord-cutters ;))

And, yes, all I can say to explain my in-action is that ${dayjob} and family did not leave much time to "play".

Best Regards

The Mexico game was a good one. We just watched it last night, cord cutting in the US on Sling is a viable thing, esp. with a gigabit fiber and Enterprise class router shaper dscp tagging proxy :rofl:

EDIT: also it helps that we're watching everything on recordings rather than live. Around here games are at 5am, 8am and 11am, so recording is the most reasonable way to go.