Difference between revisions of "Linux memory arenas"

From Bisq Wiki
Jump to navigation Jump to search
Line 10: Line 10:
  
  
This setting reduced resident memory use by ~300 MB, and reduced virtual memory use by ~3.7 GB on an 8 core, 16 GB RAM machine running Ubuntu 18.  The trade off for reducing resident memory use in this manner is the added cost of the contention among threads for memory access among fewer arenas, but this was not perceptible during testing and profiling.   (The major bottlenecks in Bisq performance are related to Tor/P2P network I/O.)
+
This setting reduced resident memory use by ~300 MB, and reduced virtual memory use by ~3.7 GB on an 8 core, 16 GB RAM machine running Ubuntu 18.  The trade off for reducing resident memory use in this manner is the added cost of the contention between threads for memory access among fewer arenas, but this was not perceptible during testing and profiling.   (The major bottlenecks in Bisq performance are related to Tor/P2P network I/O.)
  
  
 
There are many sources of information on this and other tunable memory allocation parameters, starting with the Linux [http://man7.org/linux/man-pages/man3/mallopt.3.html mallopt] man page; additional research and troubleshooting tips are welcome.
 
There are many sources of information on this and other tunable memory allocation parameters, starting with the Linux [http://man7.org/linux/man-pages/man3/mallopt.3.html mallopt] man page; additional research and troubleshooting tips are welcome.

Revision as of 20:28, 5 April 2020

Java Threads and Linux Memory Arenas

With the intention of improving efficiency of memory access in multi-threaded applications, glibc may create more memory regions (known as arenas) than necessary.    In some cases, multi-threaded java applications allocating few and small objects can trigger creation of one arena for each of those threads, resulting in large growth of the java process' resident memory, no matter how well on-heap memory is managed by the application.

One way to limit arena creation in Bisq is to define the environment variable MALLOC_ARENA_MAX=4.

   export MALLOC_ARENA_MAX=4

If not set (default=0), a combination of applications may create as many arenas as the number of cpu cores * 8.


This setting reduced resident memory use by ~300 MB, and reduced virtual memory use by ~3.7 GB on an 8 core, 16 GB RAM machine running Ubuntu 18. The trade off for reducing resident memory use in this manner is the added cost of the contention between threads for memory access among fewer arenas, but this was not perceptible during testing and profiling.   (The major bottlenecks in Bisq performance are related to Tor/P2P network I/O.)


There are many sources of information on this and other tunable memory allocation parameters, starting with the Linux mallopt man page; additional research and troubleshooting tips are welcome.