Poor HTTP performance on R650 running unleashed 200.{15,16}

[delamonpansie]

I've been optimizing roaming by tuning tx power settings. At some point, I noticed I could no longer fetch files from NAS over HTTP at full speed. It hovered around a mere 10-20MiBytes/sec. Before that, speed was almost always over 95MiB/sec.

It took quite a while before I realized the WiFi link was fine. Testing with iperf3 or doing fetch with Netcat on a port other than 80 gave me the expected > 95MiB/sec. It is just that only HTTP traffic was affected.

To be extra sure, I re-ran tests from scratch. The system was running 200.14.6.1.203. I've done test on 200.14 and then upgraded it to 200.16.7.0.402 and ran test again. I didn't change any settings after the upgrade. As far as I can tell, the configuration is identical.

It is still having poor performance for traffic on port 80. It is more evident on R650, but even R750 has it too.

	v200.14, port 80	v200.16, port 80	v200.16, port 800
R650	108MB/s	19MB/s	103MB/s
R750	didn't test	82MB/s	107MB/s

It looks like AP is having CPU overload problem of sorts. As it doing some sort of packet inspection for traffic on port 80 and having hard time due to sheer packet load.

# laptop connected to R650 running 200.14.6.1.203
$ curl dotty.home.arpa/pub/10G  -o /dev/zero                                                                                                                                                           
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 10.0G  100 10.0G    0     0   108M      0  0:01:34  0:01:34 --:--:--  112M

# laptop connected to R650 running 200.16.7.0.402
$ curl dotty.home.arpa/pub/10G  -o /dev/zero                                                                                                                                                           
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 10.0G  100 10.0G    0     0  19.2M      0  0:08:52  0:08:52 --:--:-- 19.9M

# laptop connected to R650 running 200.16.7.0.402
$ curl dotty.home.arpa:800/pub/10G  -o /dev/zero                                                                                                                                                       
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 10.0G  100 10.0G    0     0   103M      0  0:01:38  0:01:38 --:--:--  112M

# laptop connected to R750 running 200.16.7.0.402
$ curl dotty.home.arpa/pub/10G  -o /dev/zero 
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 10.0G  100 10.0G    0     0  82.3M      0  0:02:04  0:02:04 --:--:-- 90.3M

# laptop connected to R750 running 200.16.7.0.402
$ curl dotty.home.arpa:800/pub/10G  -o /dev/zero                                                                                                                                                       
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 10.0G  100 10.0G    0     0   107M      0  0:01:35  0:01:35 --:--:--  112M

As it turned out, it is possible to inspect CPU usage on access point via rscli.

These are R650 cpu usage snapshots every 1s while fetching huge file on port 80:

rkscli: sysinfo
CPU     : 47.74%
Memory  : 307236 KB (free), 790656 KB (total)
OK
rkscli: sysinfo
CPU     : 47.76%
Memory  : 307028 KB (free), 790656 KB (total)
OK
rkscli: sysinfo
CPU     : 47.76%
Memory  : 307204 KB (free), 790656 KB (total)

And this is the same measurement done while fetching the same file on port 800:

rkscli: sysinfo
CPU     : 2.00%
Memory  : 306896 KB (free), 790656 KB (total)
OK
rkscli: sysinfo
CPU     : 3.96%
Memory  : 306960 KB (free), 790656 KB (total)
OK
rkscli: sysinfo
CPU     : 2.97%
Memory  : 306928 KB (free), 790656 KB (total)
OK

It kinda proofs theory that something is CPU bound inside AP. My best guess it is most related to the URL/traffic filtering facilites.  However, I do not have applicaion feature enabled on any of my WLANs. This leaves me without any other idea what is wrong here.

[lardassresponde]

I've been adjusting the tx power settings to maximize roaming. I eventually discovered that I was unable to retrieve files from NAS over HTTP at full speed. It was only about 10–20 MiBytes/sec. Prior to that, speed was nearly always greater than 95MiB/sec. @drift boss 2