Transparent HugePages not enabled in 11.04 kernel

Just noticed this myself. In the changelog for linux-image-2.6.38, I see:

  linux (2.6.38-1.27) natty; urgency=low

...

    * [Config] disable CONFIG_TRANSPARENT_HUGEPAGE to fix i386 boot crashes

Pretty disappointing that this is just turned off instead of debugging the crash, since it is a major speedup especially for virtualization.

Transparent huge pages are enabled in the Fedora Core 15 and work well. I think THP helps common workloads (browser, office suite). If they are not enabled in natty, I may have to switch to Fedora - or at least run some benchmarks, because a 10-15% performance gain is material for me.

THP is disabled before rebasing to 2.6.38 release. Is this bug still remaining?
If so, how about enabling THP on amd64 only?

This bug is missing log files that will aid in dianosing the problem. From a terminal window please run:

apport-collect 743688

and then change the status of the bug back to 'New'.

If, due to the nature of the issue you have encountered, you are unable to run this command, please add a comment stating that fact and change the bug status to 'Confirmed'.

This change has been made by an automated script, maintained by the Ubuntu Kernel Team.

Come on, this is automated triage gone insane. Why does anyone have to collect any logs for a report where it takes anyone interested a second or two to confirm that yes, the kernel config is missing the option reported as missing??

so maybe i'm not understanding this correctly, but in regards to

 * [Config] disable CONFIG_TRANSPARENT_HUGEPAGE to fix i386 boot crashes

... why isn't it only disabled for the i386 ??? ... it appears to be disabled for everything

i just installed linux-image-server-lts-backport-natty on Ubuntu 10.04.3 LTS x86_64 , and was looking forward to trying out TRANSPARENT HUGEPAGES

$ grep TRANSPARENT /boot/config-2.6.38-11-generic
# CONFIG_TRANSPARENT_HUGEPAGE is not set

Is there any hope of this being enabled on 11.10?

I spent a day compiling a custom kernel with "transparent huge pages" enabled and set to "always" use mode (not madvise). I have been using the kernel for the last 24 hours without any trouble.

I find my laptop more responsive now. An informal test is that Stellarium's CPU utilisation was 45% in the stock kernel, while it went down to 30% in the custom kernel.

For a more formal test, we can use oprofile with these commands:

sudo opcontrol --init
sudo opcontrol --setup --event DTLB_MISSES:1000 --no-vmlinux
sudo opcontrol --start
[... run some benchmark here ...]
sudo opcontrol --stop
sudo opcontrol --dump
sudo opreport

I am able to run the above commands in both stock and custom kernels and see some numbers. But I need to properly define the benchmark before reporting the numbers.

HRJ:

per https://lists.ubuntu.com/archives/ubuntu-devel/2011-September/034060.html, Oneiric will ship with
CONFIG_TRANSPARENT_HUGEPAGE=y
# CONFIG_TRANSPARENT_HUGEPAGE_ALWAYS is not set
CONFIG_TRANSPARENT_HUGEPAGE_MADVISE=y

Thanks Julian.

Too bad they made it "madvise" and not "always".

There is sysfs entry at "/sys/kernel/mm/transparent_hugepage/" which allows changing the policy at run time, but I am not sure if it can be turned on if the kernel config has "madvise".

According to Documentation/vm/transhuge.txt you can also set it as the boot parameter "transparent_hugepage=always". The problem with setting it via sysfs is that none of the processes started before you made the change will use huge pages.

Based on what we've been seeing maintaining Oracle Linux, there are many workloads for which THP results in a ~9-10% performance hit overall on the system, and this includes kernels up through 3.8.13 (which we're using as the baseline for a new kernel in that distro). It has to do with the fact that the memory allocator uses the slowpath almost all the time, rather than a faster path when memory fragmentation is low. It's not that THP is _enabled_ so much that it is _compiled into the kernel at all_; as a result, we're probably going to remove it for the time being, and re-enable it when the allocator is revamped.

However, there is also good reason for choosing madvise as the default. Transparent hugepages *cannot be swapped out*. Read that carefully: when you use them, the allocated memory is always in-core, as if it had been mlock()ed into place. While systems with plenty of memory shouldn't need to worry about this (hell, I run my systems without swap at all!), it's a consideration to keep in mind. Applications for which huge pages may provide proven benefit, but aren't required, should start to adopt MADV_HUGEPAGE as an advisory call; this might include databases, Web browsers, and other data-intensive applications. That way, you don't run the risk of hitting memory walls thanks to a bunch of smaller system processes that just happen to use hugepages because it's set to "always".

BTW, by "we're probably going to remove it for the time being", I'm referring to the kernels in the distro at my day job (OL). I use Ubuntu at home. ;)

@Todd
are you sure they are not swappable?
according to lwn they are:
http://lwn.net/Articles/423584/
"""
This scheme will increase the use of huge pages transparently, but it does not yet solve the whole problem. Huge pages must be swappable, lest the system run out of memory in a hurry. Rather than complicate the swapping code with an understanding of huge pages, Andrea simply splits a huge page back into its component small pages if that page needs to be reclaimed. Many other operations (mprotect(), mlock(), ...) will also result in the splitting of a page.
"""

also the performance regression might be fixed in 3.11;
http://kernel.opensuse.org/cgit/kernel/commit/?id=348944504362417205107e63cfe4821aa87ec1bb
mm: fix aio performance regression for database caused by THP

@Julian,

That isn't what the article is saying. Huge pages are not swappable at the moment. However Andrea's work splits huge pages into base pages for a number of reasons, one of those is memory pressure that would force part of the page to be swapped out.

I know non-anonymous huge pages, like those from hugetblfs, are currently not swappable.
But if I understand the article correctly _anonymous_ huge pages can be swapped.
You lose the benefits of huge pages in this case (as they are split) but the system does not regress in functionality.

Kernel EOL, closing this bug, please feel free to open a new one.