Monday 25 July 2016

Very High CPU usage, however no database/application processing on Linux platforms.

Very High CPU usage, however no database/application processing on Linux platforms.

The process khugepaged  has to be checked to find its  CPU usage. It might utilize 100% CPU.

Cause:

Many latest Linux distributions ship with Transparent Hugepages enabled by default. When Linux uses Transparent Hugepages, the kernel tries to allocate memory in large chunks (usually 2MB), rather than 4K. This can improve performance by reducing the number of pages the CPU must track. However, some applications still allocate memory based on 4K pages. This can cause noticeable perfomance problems when Linux tries to defrag 2MB pages.

Solutions:

If you are running Cassandra database  on this linux server:

1. A temporary fix: drop caches by entering:

sync && echo 3 > /proc/sys/vm/drop_caches

2. Efficient  solution: disable defrag for hugepages by entering:

echo never | sudo tee /sys/kernel/mm/transparent_hugepage/defrag

Another alternative: add -XX:+AlwaysPreTouch to the jvm.options file. This change should be tested carefully before being put into production.