How Redis uses different memory allocators to compare fragmentation rates

In the source code of zmalloc.c in Redis, we can see the following code:

/* Explicitly override malloc/free etc when using tcmalloc. */

#if defined(USE_TCMALLOC )

　#define malloc(size) tc_malloc(size)

tc_calloc(count,size) is the definition of calloc function

　#define realloc(ptr,size) tc_realloc( ptr,size)

　#define free(ptr) tc_free(ptr)

　#elif defined(USE_JEMALLOC)

　#define malloc(size) je_malloc(size)

The following sentence is rewritten as requested: The statement "#define calloc(count,size) je_calloc(count,size)" is used to replace the standard defined function "calloc" with a custom function named "je_calloc".

　#define realloc(ptr, size) je_realloc(ptr,size)

　#define free(ptr) je_free(ptr)

　#endif

　From the above code we can see that Redis is When compiling, it will first determine whether to use tcmalloc. If so, the function implementation in the standard libc will be replaced with the function corresponding to tcmalloc. After this, it will check whether jemalloc takes effect. If not, the memory management function in standard libc will be used.

In the latest version 2.4.4, jemalloc has been included in the source code package as part of the source code package, so it can be used directly. And if you want to use tcmalloc, you need to install it yourself.

Let’s briefly talk about how to install the tcmalloc package. tcmalloc is part of google-proftools, so we actually need to install google-proftools. When installing on a 64-bit machine, the required libunwind library must first be installed.

　wget http://download.savannah.gnu.org/releases/libunwind/libunwind-0.99-alpha.tar.gz

　tar zxvf libunwind-0.99-alpha.tar.gz

　cd libunwind-0.99-alpha/

　CFLAGS=-fPIC ./configure

　make CFLAGS=-fPIC

　make CFLAGS=-fPIC install

Comparison of fragmentation rates of how Redis uses different memory allocators

Then install google-preftools:

wget http://google-perftools.googlecode.com/files/ google-perftools-1.8.1.tar.gz

tar zxvf google-perftools-1.8.1.tar.gz

cd google-perftools-1.8.1/

./configure --disable-cpu-profiler --disable-heap-profiler --disable-heap-checker --disable-debugalloc --enable-minimal

make && make install

Sudo echo "/usr/local/lib" > /etc/ld.so.conf.d/usr_local_lib.conf #If you don’t have this file, create one yourself

sudo /sbin/ldconfig

　Then install Redis and specify the corresponding parameters when making to enable tcmalloc

　$ curl -O http://redis.googlecode.com/files/redis-2.4.4.tar. gz

　$ tar xzvf redis-2.4.4.tar.gz

　$ cd redis-2.4.4

　$ make USE_TCMALLOC=yes FORCE_LIBC_MALLOC=yes

　$ sudo make install

　After restarting Redis, you can see the memory allocator used through the info command.

Back to the topic of this article, this article discusses the three memory allocators corresponding to tcmalloc, jemalloc and libc. The following is a simple test result, taken from the Redis info information, designed to evaluate its performance and fragmentation rate. The tests used different allocators. We can see that the fragmentation rate is the lowest when using tcmalloc, which is 1.01, jemalloc is 1.02, and libc’s allocator fragmentation rate is 1.31, as shown below:

Used_memory:708391440

used_menory_human:675.57M

used_memory_rss:715169792

used_memory_peak:708814040

used_memory_peak_human:675.98M

mem_fragmentation_ratio:1.01

mem_ allocator:tcmalloc -1.7

used_memory:708381168

used_menory_human:675.56M

used_memory_rss:723587072

used_memory_peak:708803768

## used_memory_peak_human:675.97 M

mem_fragmentation_ratio:1.02

mem_allocator:jemalloc-2.2.1

used_memory:869000400

used_menory_human:828.74M

used_memory_rss:1136689152

Used_memory_peak:868992208

used_memory_peak_human:828.74M

mem_fragmentation_ratio:1.31

mem_allocator:libc

The above is the detailed content of How Redis uses different memory allocators to compare fragmentation rates. For more information, please follow other related articles on the PHP Chinese website!