Zend memory manager for php

                                           

Zend Memory Manager

Zend Memory Manager, often abbreviated to ZendMM or ZMM, is A C layer designed to provide the ability to allocate and free dynamic request-bound memory.

Pay attention to "request binding" in the above sentence.

ZendMM is not just a classic layer on top of libc's dynamic memory allocator, mainly represented by two API calls malloc()/free(). ZendMM is about the request-bound memory that PHP must allocate when processing a request.

Related learning recommendations: PHP programming from entry to proficiency

Two main dynamic memory pools in PHP

PHP It is a shared nothing architecture. Well, not at 100%. Let us explain. More information on steps and cycles.

PHP can handle hundreds or thousands of requests in the same process. By default, PHP will forget any information about the current request after it is completed.

The "forgot" message is interpreted as freeing any dynamic buffers allocated while processing the request. This means that you cannot use traditional libc calls to allocate dynamic memory while processing a request. This is perfectly valid, but you give the opportunity to forget to free the buffer.

ZendMM comes with an API that replaces libc's dynamic allocator by copying its API. Programmers must use this API instead of libc's allocator when handling requests.

For example, when PHP processes a request, it will parse the PHP file. For example, those will result in declarations of functions and classes. When the compiler starts compiling a PHP file, it will allocate some dynamic memory to store the classes and functions it discovers. However, PHP releases these at the end of the request. By default, PHP forgets

a lot of

information from one request to another.

However, there are some very rare pieces of information that you need to persist across multiple requests. But this is not common.

What can be kept unchanged by requests? What we call persistent

objects. Again: that's not a common situation. For example, the current PHP executable path does not change between requests. Its information is permanently allocated, which means it calls traditional libc's

malloc ()

to allocate it.

what else? some string. For example, the "_SERVER" string will be reused between requests because a $_SERVER

PHP array will be created for each request. So

"_SERVER" the string itself can be permanently allocated, since it will only be allocated once. You must remember:

When writing PHP core or extensions, there are two ways of dynamic memory allocation:

Request binding Determined dynamic allocation.

• Permanent dynamic allocation.

  • Request binding dynamic memory allocation
is only performed when PHP handles the request (not before or after).

• Should only be performed using the ZendMM dynamic memory allocation API.

  is very common in extension design. Basically 95% of dynamic allocations are request binding.
  • is tracked by ZendMM and will notify you about leaks.
  • Permanent dynamic memory allocation
should not be performed while PHP is processing a request (it's not forbidden, but it's a bad idea).

• will not be tracked by ZendMM and you will not be notified of leaks.

  should be rare in extensions.
  • Also, keep in mind that all PHP source code is based on this memory level. Therefore, many internal structures are allocated using the Zend memory manager. Most make calls to a "persistent" API which, when called, will result in a traditional libc allocation.
  This is a request-bound allocation zend_string:

zend_string *foo = zend_string_init("foo", strlen("foo"), 0);

This is a persistent allocation:

zend_string *foo = zend_string_init("foo", strlen("foo"), 1);

Same HashTable.

Request bound allocation:

zend_array ar;
zend_hash_init(&ar, 8, NULL, NULL, 0);

Durable allocation:

zend_array ar;
zend_hash_init(&ar, 8, NULL, NULL, 1);

It is always the same across all the different Zend APIs. Usually passed as the last parameter,
"0"

means "I want to allocate this structure using ZendMM, so request a binding", or

"1"

means "I want to use ZendMM to allocate this structure, so the binding is requested", or

"1" means "I want to use ZendMM to allocate this structure, so the binding is requested" Call traditional libc's malloc() to allocate this structure". Apparently these structures provide an API that remembers how it allocated the structure so that the correct release function is used when destroyed. So in code like this:

zend_string_release(foo);
zend_hash_destroy(&ar);

The API knows whether these structures are allocated using request binding or permanently allocated, in the first case it will be freed using

efree()

and in the second The first situation is libc's

free().

Zend Memory Manager API

The API is located in Zend/zend_alloc.h

The API is mainly C macros, not functions, so if you debug them and want to understand their working Principle, please be prepared. These APIs copy libc's functions, often with an "e" added to the function name; therefore, you should not make the mistake, there are not many details about the API.

Basically, the ones you will use most often are emalloc(size_t) and efree(void *).

Also provided is ecalloc(size_t nmemb, size_t size), which allocates a single nmemb of size size and zeroes out the region. If you are an experienced C programmer, then you should know that whenever possible, it is best to use ecalloc() over emalloc() because ecalloc () will zero out the memory area, which may be of great help in pointer error detection. Keep in mind that emalloc() works basically the same as libc malloc(): it will look for a large enough area in different pools and give you the most suitable Space. Therefore, you may end up with a garbage-collected pointer.

Then safe_emalloc(size_t nmemb, size_t size, size_t offset), which is emalloc(size * nmemb offset), but it will check for overflows for you . This API call should be used if the number that must be provided comes from an untrusted source (such as userland).

Regarding strings, estrdup(char *) and estrndup(char *, size_t len) allow copying string or binary strings.

No matter what happens, the pointer returned by ZendMM must be released by calling ZendMM's efree(), and not libc's free().

Note

Instructions on persistent allocation. Durable allocations remain valid between requests. You usually use the usual libc malloc/ free to do this, but ZendMM has some shortcuts for the libc allocator: the "persistent" API. The API starts with "p" letters and lets you choose between ZendMM allocation or persistent allocation. Therefore pemalloc(size_t, 1) is just malloc(), pefree(void *, 1) is free(),pestrdup(void *, 1) is strdup(). Just saying.

Zend Memory Manager Debug Shield

ZendMM provides the following features:

• Memory consumption management.
• Memory leak tracking and automatic release.
• Speed ​​up allocations by pre-allocating buffers of known sizes and keeping hot caches idle

Memory consumption management

ZendMM is a PHP userland Underlying the "memory_limit" function. Every single byte allocated using the ZendMM layer is counted and summed. You know what happens when the INI's memory_limit is reached. This also means that any allocations performed via ZendMM are reflected in memory_get_usage() in the PHP userland.

As an extension developer, this is a good thing because it helps keep track of the heap size of your PHP process.

If a memory limit error is initiated, the engine will release from the current code position into the capture block and then terminate gracefully. But it's not possible to get back to a code location that exceeds the limit. You have to be prepared for this.

Theoretically, this means that ZendMM cannot return a NULL pointer to you. If the allocation from the operating system fails, or the allocation produces a memory limit error, the code will run into a catch block and will not return to your allocation call.

If for any reason you need to bypass this protection, you must use traditional libc calls such as malloc(). At any rate please be careful and know what you are doing. If you use ZendMM, you may need to allocate a large amount of memory and may exceed PHP's memory_limit. So use another allocator (like libc), but beware: your extension will increase the current process heap size. memory_get_usage() cannot be seen in PHP, but the current heap can be analyzed by using OS facilities (such as /proc/{pid}/maps)

Note

If you need to completely disable ZendMM, you can use the USE_ZEND_ALLOC = 0 environment variable to start PHP. This way, every call to the ZendMM API (such as emalloc()) will be directed to the libc call, and ZendMM will be disabled. This is especially useful in the case of debugging memory.

Memory Leak Tracing

Remember the main rule of ZendMM: it starts when the request starts, and then expects you to call its API when you need dynamic memory to handle the request. When the current request ends, ZendMM shuts down.

By closing, it will browse all its live pointers and, if using PHP's debug build, it will warn you about memory leaks.

Let us explain it a little clearer: If at the end of the current request, ZendMM finds some active memory blocks, it means that these memory blocks are leaking. There shouldn't be any active blocks of memory on the ZendMM heap at the end of the request, because whoever allocated some memory should have freed it.

如果您忘记释放块，它们将全部显示在 stderr上。此内存泄漏报告进程仅在以下情况下有效：

• 你正在使用 PHP 的调试构建
• 在 php.ini 中具有 report_memleaks = On（默认）

这是一个简单泄漏到扩展中的示例：

PHP_RINIT_FUNCTION(example)
{
    void *foo = emalloc(128);
}

在启动该扩展的情况下启动 PHP，在调试版本上会在 stderr 上生成：

[Fri Jun 9 16:04:59 2017]  Script:  '/tmp/foobar.php'
/path/to/extension/file.c(123) : Freeing 0x00007fffeee65000 (128 bytes), script=/tmp/foobar.php
=== Total 1 memory leaks detected ===

当 Zend 内存管理器关闭时，在每个已处理请求的末尾，将生成这些行。

但是要当心：

• 显然，ZendMM 对持久分配或以不同于使用持久分配的方式执行的分配一无所知。因此，ZendMM 只能警告你有关它知道的分配信息，在这里不会报告每个传统的 libc 分配信息。
• 如果 PHP 以错误的方式关闭（我们称之为不正常关闭），ZendMM 将报告大量泄漏。这是因为引擎在错误关闭时会使用longjmp()调用 catch 块，防止清理所有内存的代码运行。因此，许多泄漏得到报告。尤其是在调用 PHP 的 exit()/ die()之后，或者在 PHP 的某些关键部分触发了致命错误时，就会发生这种情况。
• 如果你使用非调试版本的 PHP，则 stderr 上不会显示任何内容，ZendMM 是愚蠢的，但仍会清除程序员尚未明确释放的所有分配的请求绑定缓冲区

你必须记住的是 ZendMM 泄漏跟踪是一个不错的奖励工具，但它不能代替真正的 C 内存调试器。

ZendMM 内部设计

常见错误和错误

这是使用 ZendMM 时最常见的错误，以及你应该怎么做。

1. 不处理请求时使用 ZendMM。

获取有关 PHP 生命周期的信息，以了解在扩展中何时处理请求，何时不处理。如果在请求范围之外使用 ZendMM（例如在MINIT()中），在处理第一个请求之前，ZendMM 会静默清除分配，并且可能会使用after-after-free：根本没有。

1. 缓冲区上溢和下溢。

使用内存调试器。如果你在 ZendMM 返回的内存区域以下或过去写入内容，则将覆盖关键的 ZendMM 结构并触发崩溃。如果 ZendMM 能够为你检测到混乱，则可能会显示“zend_mm_heap损坏”的消息。堆栈追踪将显示从某些代码到某些 ZendMM 代码的崩溃。ZendMM 代码不会自行崩溃。如果你在 ZendMM 代码中间崩溃，那很可能意味着你在某个地方弄乱了指针。插入你喜欢的内存调试器，查找有罪的部分并进行修复。

1. 混合 API 调用

如果分配一个 ZendMM 指针(即emalloc())并使用 libc 释放它（free()），或相反的情况：你将崩溃。要严谨对待。另外，如果你将其不知道的任何指针传递给 ZendMM 的efree()：将会崩溃。

The above is the detailed content of Zend memory manager for php. For more information, please follow other related articles on the PHP Chinese website!