Very slow (1MB/s) disk write performance on (Ubuntu) Linux 3.4.x-generic Linux kernel

We recently had a major performance issue on some systems, where disk write speed is extremely slow (~1 MB/s — where normal performance is 150+MB/s).

hdparm read performance was unaffected, and no obvious messages were found anywhere; after reading many online suggestions, disabling acpi or various filesystem tweaks didn’t fix anything (it seems that the issue persists whatever the hard drive model or filesystem).

In the end, it comes down to the system having more than 8 GB of RAM installed.

Note that this issue seems to be fixed in later releases (3.5.0+) and is a regression (doesn’t happen on e.g. 2.6.32); the issue being is completely silent, i hope this post will help someone.

Hardware which has the issue (we didn’t find any hardware which doesn’t):

• Intel H55 (Intel Corporation 5 Series/3400)
• Intel Q77 (Intel Corporation 7 Series/C210)
• Intel Q87 (Intel Corporation 8 Series/C220)

Kernel: Linux version 3.4.82-030482-generic (apw@gomeisa) (gcc version 4.6.3 (Ubuntu/Linaro 4.6.3-1ubuntu5) ) #201402221435 SMP Sat Feb 22 19:44:42 UTC 2014 (from http://kernel.ubuntu.com/~kernel-ppa/mainline/v3.4.82-quantal/)

EDIT: to solve this, either remove enough RAM, or add “mem=8G” as kernel boot parameter (e.g. in /etc/default/grub on Ubuntu — don’t forget to run update-grub !)

To see if your system is affected, the following test should show very poor figures:

With 16 GB RAM:
dd if=/dev/zero of=test bs=512k count=20
20+0 records in
20+0 records out
10485760 bytes (10 MB) copied, 8.89589 s, 1.2 MB/s

Now with with 8 GB of RAM:
dd if=/dev/zero of=test bs=512k count=200
200+0 records in
200+0 records out
104857600 bytes (105 MB) copied, 0.473313 s, 222 MB/s