[needs-packaging] Bufferbloat: Ubuntu doesn't come with bufferbloat fixes

*** This is an automated message ***

This bug is tagged needs-packaging which identifies it as a request for a new package in Ubuntu. As a part of the managing needs-packaging bug reports specification, https://wiki.ubuntu.com/QATeam/Specs/NeedsPackagingBugs, all needs-packaging bug reports have Wishlist importance. Subsequently, I'm setting this bug's status to Wishlist.

If someone tell me how, I can build a candidate package.

Status changed to 'Confirmed' because the bug affects multiple users.

Can you explain more about how this mitigates bufferbloat, I suspect I have this issue on Ubuntu 12.04 and it gets pretty bad sometimes.

Here is a little big introduction from the source itself:
http://www.bufferbloat.net/projects/bloat/wiki/Introduction

The bufferbloat problem arises from the fact that the delay in a TCP network under congestion is exponential to the size of the buffers. For that, in 1993 RED was devised: http://en.wikipedia.org/wiki/Random_early_detection
It tries to keep the size of the buffers small by randomly (and thus fairly) dropping packages .

The problem is that people didn't like to have their packages dropped and manufactures built routers and network cards with huge buffers and over time each layer of abstraction on the network added its own buffer. So today we have tons of buffers that we aren't even aware of.

This "fix" I sent mitigates the problem of buffer bloat by reducing the max size of the linux buffer. It does nothing about the underlying buffers on the hardware but simple testing shows that it reduces the delay in a local area network by a lot.

It probably also reduces the delay on the internet too, it is just harder to test.

Testing in a lan can be done by doing a SSH copy of a huge file from/to another pc in the lan in one terminal and in another terminal you do a ping to the same or to another address.

Changing the size of the txqueuelen of the network interface in use affects the delay.
For most fun, try doing it in a wireless network.

There is more stuff that can be done, according to this:
http://www.bufferbloat.net/projects/bloat/wiki/Linux_Tips

Also, TCP algorithm might also be at fault:
http://cero2.bufferbloat.net/cerowrt/tcp.html

I talked to DaveTaht at UDS and we tested the effects of turning on bufferbloat fixes on wifi on my laptop and I saw very encouraging results. He gave me access to a script that needs to be added to /etc/networking/if-up.d/ that will set up wifi wlanXXX. The script could be modified such that it works for wlanXXX and ethXXX devs.

This script could be modified slightly to handle wlan and eth devices.

Has this been discussed with kernel upstream? In the first instance, such changes should be applied in the kernel defaults, rather than overriding in userspace.

The bufferbloat project has been at the problem for a great deal of time now. As of Linux 3.6, a great deal of the bufferbloat problem on ethernet has been resolved, with the incorporation (and device driver support) of BQL (linux 3.3), fq_codel (linux 3.5), and TCP small queues (3.6)

At linux plumbers in august, there seemed to be consensus that one day fq_codel could replace pfifo_fast, but it needs a bit more work and testing in the field. Many of the core netdev people are running it, there are also tests going on within several large companies. Work on improving wifi further is beginning. So, seeking wider deployment, by default, will get us to where it could ultimately be turned on by default. Certainly the benefits of the code as it stands now are *quite* noticible under many workloads particularly on desktops and more than a few on servers.

Latency under load last year: http://www.teklibre.com/~d/bloat/pfifo_fast_vs_sfq_qfq_linear.png
Linux 3.6 PFIFO_FAST: https://github.com/tohojo/netperf-wrapper/blob/master/sample_plots/100m_pfifo_fast.png
Linux 3.6 FQ_CODEL: https://github.com/tohojo/netperf-wrapper/blob/master/sample_plots/100m_fq_codel.png

Kamal Mostafa, for example, did some testing at canonical earlier this year and described his reaction in a kernel meeting...

http://www.bufferbloat.net/news/38

I am maintaining a simple script that turns it on, that perhaps could be shipped by default for the desktop configuration of ubuntu.

https://github.com/dtaht/deBloat/blob/master/src/debloat.sh

Regardless I encourage people to try fq_codel out, run it day and day out, as we do, and see if they notice a difference. They will. Particularly gamers.

Some backing reading material for the work:

Van Jacobson's presentation on codel and fq_codel: http://recordings.conf.meetecho.com/Recordings/watch.jsp?recording=IETF84_TSVAREA&chapter=part_3
Linux Plumbers BQL: http://linuxplumbers.ubicast.tv/videos/byte-queue-limits/
Linux Plumbers fq_codel: http://linuxplumbers.ubicast.tv/videos/codel-and-fq_codel-fighting-the-delays/

TCP small queues in LWN: http://lwn.net/Articles/507065/

...

I note that one of my other "causes" at the moment, is that the enormous improvements in network performance require, as a base, BQL support in the underlying device driver. The majority of intel ethernet devices are BQL-enabled, arm is lacking...

...

For the ongoing work into fixing routers, and cpe, which suffer far worse than servers and desktops do - see the cerowrt project.

FYI, we're now shipping the new iproute package with fq_codel support bits in Ubuntu Raring as of iproute (20121001-1ubuntu1) I have also pushed that package to the bufferbloat PPA for 12.04 and 12.10: https://launchpad.net/~kamalmostafa/+archive/bufferbloat

A lot of comments here seem outdated, however things have moved on in a good-way!

Taking reference of https://www.bufferbloat.net/projects/bloat/wiki/Linux_Tips/ -- would now seem that many of the bufferbloat fixes HAVE been implemented --

Upstream kernel changes have now introduced tcp_sack tcp_dsack seemingly.
Systemd, interestingly-enough, now enables fq_codel as default qdisc.
[See LP #1436945 ].

The remaining piece of this bufferbloat puzzle on desktop distributions, seems to be
turning on tcp_ecn fully so that it actually gets used on outgoing TCP connections
(with adaptive fallback now supported).

See related Issue: #1773157 -- Please support this and mark as affecting you as you see fit!
Also: https://github.com/systemd/systemd/issues/9087

Hope that helps!

Launchpad issue link didn't come out -- LP #1773157
https://bugs.launchpad.net/ubuntu/+source/procps/+bug/1773157