Learn about the file module and core module in Node in one article

This article will take you through the file module and core module in Node, and talk about the search for file modules, the compilation and execution of file modules, and the compilation and execution of JavaScript and C/C core modules. I hope Helpful for everyone!

When we use Nodejs for daily development, we often use require to import two types of modules. One is the module we write ourselves or use npm Installed third-party modules, this type of module is called file module in Node; the other type is the built-in module provided by Node for our use, such as os, fs and other modules, these modules are called core modules.

It should be noted that the difference between the file module and the core module lies not only in whether it is built-in by Node, but also in the file positioning, compilation and execution process of the module. There are obvious differences between the two. Not only that, file modules can also be subdivided into ordinary file modules, custom modules or C/C extension modules, etc. Different modules also have many details that differ in file positioning, compilation and other processes.

This article will address these issues and clarify the concepts of file modules and core modules as well as their specific processes and details that need to be paid attention to in file location, compilation or execution. I hope it will be helpful to you.

Let’s start with the file module.

File module

What is a file module?

In Node, modules required using module identifiers starting with ., .. or / (that is, using relative paths or absolute paths) will be treated as file modules. In addition, there is a special type of module. Although it does not contain a relative path or an absolute path, and it is not a core module, it points to a package. When Node locates this type of module, it will use Module Path to search for the module one by one. Modules, such modules are called custom modules.

Therefore, file modules include two types, one is ordinary file modules with paths, and the other is custom modules without paths.

The file module is dynamically loaded at runtime, which requires a complete file positioning, compilation and execution process, and is slower than the core module.

For file positioning, Node handles these two types of file modules differently. Let’s take a closer look at the search processes for these two types of file modules.

Searching for ordinary file modules

For ordinary file modules, since the path is carried and the direction is very clear, the search will not take long, so the search efficiency is higher than the following The custom modules introduced are a bit higher. However, there are still two points to note.

First, under normal circumstances, when using require to introduce a file module, the file extension is generally not specified, for example:

const math = require("math");

Because the extension is not specified, Node cannot yet determine the final file. In this case, Node will complete the extensions in the order of .js, .json, and .node, and try them one by one. This process is called file extension analysis.

Another thing to note is that in actual development, in addition to requiring a specific file, we usually also specify a directory, such as:

const axios = require("../network");

In this case, Node will First perform file extension analysis. If the corresponding file is not found, but a directory is obtained, Node will treat the directory as a package.

Specifically, Node will return the file pointed to by the main field of package.json in the directory as the search result. If the file pointed by main is wrong, or the package.json file does not exist at all, Node will use index as the default file name, and then use .js, .node Perform extension analysis and search for target files one by one. If not found, an error will be thrown.

(Of course, since Node has two types of module systems, CJS and ESM, in addition to finding the main field, Node will also use other methods. Since it is outside the scope of this article, I will not go into details.)

Searching for custom modules

As mentioned just now, Node will use the module path when searching for custom modules. So what is the module path?

Friends who are familiar with module parsing should know that the module path is an array composed of paths. The specific value can be seen in the following example:

// example.js
console.log(module.paths);

Print results:

As you can see, the module in Node has a module path array, which is stored in module.paths and is used to specify how Node finds the custom module referenced by the current module.

具体来讲，Node 会遍历模块路径数组，逐个尝试其中的路径，查找该路径对应的 node_modules 目录中是否有指定的自定义模块，如果没有就向上逐级递归，一直到根目录下的 node_modules 目录，直到找到目标模块为止，如果找不到的话就会抛出错误。

可以看出，逐级向上递归查找 node_modules 目录是 Node 查找自定义模块的策略，而模块路径便是这个策略的具体实现。

同时我们也得出一个结论，在查找自定义模块时，层级越深，相应的查找耗时就会越多。因此相比于核心模块和普通的文件模块，自定义模块的加载速度是最慢的。

当然，根据模块路径查找到的仅仅是一个目录，并不是一个具体的文件，在查找到目录后，同样地，Node 会根据上文所描述的包处理流程进行查找，具体过程不再赘述了。

以上是普通文件模块和自定义模块的文件定位的流程和需要注意的细节，接下来我们来看者两类模块是如何编译执行的。

文件模块的编译执行

当定位到 require 所指向的文件后，通常模块标识符都不带有扩展名，根据上文提到的文件扩展名分析我们可以知道，Node 支持三种扩展名文件的编译执行：

• JavaScript 文件。通过 fs 模块同步读取文件后编译执行。除了 .node 和 .json 文件，其他文件都会被当作 .js 文件载入。

• .node 文件，这是用 C/C++ 编写后编译生成的扩展文件，Node 通过 process.dlopen() 方法加载该文件。

• json 文件，通过 fs 模块同步读取文件后，使用 JSON.parse() 解析并返回结果。

在对文件模块进行编译执行之前，Node 会使用如下所示的模块封装器对其进行包装：

(function(exports, require, module, __filename, __dirname) {
    // 模块代码
});

可以看到，通过模块封装器，Node 将模块包装进函数作用域中，与其他作用域隔离，避免变量的命名冲突、污染全局作用域等问题，同时，通过传入 exports、require 参数，使该模块具备应有的导入与导出能力。这便是 Node 对模块的实现。

了解了模块封装器后，我们先来看 json 文件的编译执行流程。

json 文件的编译执行

json 文件的编译执行是最简单的。在通过 fs 模块同步读取 JSON 文件的内容后，Node 会使用 JSON.parse() 解析出 JavaScript 对象，然后将它赋给该模块的 exports 对象，最后再返回给引用它的模块，过程十分简单粗暴。

JavaScript 文件的编译执行

在使用模块包装器对 JavaScript 文件进行包装后，包装之后的代码会通过 vm 模块的 runInThisContext()（类似 eval） 方法执行，返回一个 function 对象。

然后，将该 JavaScript 模块的 exports、require、module 等参数传递给这个 function 执行，执行之后，模块的 exports 属性被返回给调用方，这就是 JavaScript 文件的编译执行过程。

C/C++ 扩展模块的编译执行

在讲解 C/C++ 扩展模块的编译执行之前，先介绍一下什么是 C/C++ 扩展模块。

C/C++ 扩展模块属于文件模块中的一类，顾名思义，这类模块由 C/C++ 编写，与 JavaScript 模块的区别在于其加载之后不需要编译，直接执行之后就可以被外部调用了，因此其加载速度比 JavaScript 模块略快。相比于用 JS 编写的文件模块，C/C++ 扩展模块明显更具有性能上的优势。对于 Node 核心模块中无法覆盖的功能或者有特定的性能需求，用户可以编写 C/C++ 扩展模块来达到目的。

那 .node 文件又是什么呢，它跟 C/C++ 扩展模块有什么关系？

事实上，编写好之后的 C/C++ 扩展模块经过编译之后就生成了 .node 文件。也就是说，作为模块的使用者，我们并不直接引入 C/C++ 扩展模块的源代码，而是引入 C/C++ 扩展模块经过编译之后的二进制文件。因此，.node 文件并不需要编译，Node 在查找到 .node 文件后，只需加载和执行该文件即可。在执行的过程中，模块的 exports 对象被填充，然后返回给调用者。

It is worth noting that the .node file generated by compiling the C/C extension module has different forms under different platforms: C/C extension module under the *nix system It is compiled into a dynamic link shared object file by compilers such as g/gcc, with the extension .so; under Windows, it is compiled into a dynamic link library file by the Visual C compiler, with the extension Named .dll. But the extension we use in actual use is .node. In fact, the extension of .node is just to look more natural. In fact, in Windows Under it is a .dll file, under *nix it is a .so file.

After Node finds the .node file to require, it will call the process.dlopen() method to load and execute the file. Since .node files have different file forms under different platforms, in order to achieve cross-platform, the dlopen() method is used in Windows and *nix There are different implementations under the platform, and then they are encapsulated through the libuv compatibility layer. The following figure shows the compilation and loading process of C/C extension modules under different platforms:

Core module

The core module is During the compilation process of Node source code, the binary executable file is compiled. When the Node process starts, some core modules are directly loaded into the memory. Therefore, when these core modules are introduced, the two steps of file location and compilation and execution can be omitted, and will be judged before the file module in the path analysis. So its loading speed is the fastest.

The core module is actually divided into two parts written in C/C and JavaScript. The C/C files are stored in the src directory of the Node project, and the JavaScript files are stored in the lib directory. Obviously, the compilation and execution processes of these two parts of modules are different.

Compilation and execution of JavaScript core module

For the compilation of JavaScript core module, during the compilation process of Node source code, Node will use the js2c.py tool that comes with V8 , convert all built-in JavaScript codes, including JavaScript core modules, into arrays in C, and the JavaScript codes are stored in the node namespace in the form of strings. When starting the Node process, the JavaScript code is loaded directly into memory.

When a JavaScript core module is introduced, Node will call process.binding() Locate its location in memory through module identifier analysis and take it out. After being taken out, the JavaScript core module will also be wrapped by the module wrapper, then executed, the exports object will be exported, and returned to the caller.

Compilation and execution of C/C core module

In the core module, some modules are all written in C/C, and some modules complete the core part in C/C , other parts are packaged or exported by JavaScript to meet performance requirements. Modules such as buffer, fs, os are all partially implemented through C/ Written in C. This C module implements the core inside the main part, and the JavaScript module implements the encapsulation outside the main part. This is a common way for Node to improve performance.

The parts of the core module written in pure C/C are called built-in modules, such as node_fs, node_os, etc. They are usually not called directly by users, but Is directly dependent on the JavaScript core module. Therefore, during the introduction of Node's core module, there is such a reference chain:

So how does the JavaScript core module load the built-in module?

Remember the process.binding() method? Node removes the JavaScript core module from memory by calling this method. This method also works with JavaScript core modules to assist in loading built-in modules.

Specific to the implementation of this method, when loading a built-in module, first create an exports empty object, and then call the get_builtin_module() method to take out the built-in module object, and execute register_func () Fill in the exports object and finally return it to the caller to complete the export. This is the loading and execution process of the built-in module.

Through the above analysis, for the introduction of a reference chain such as the core module, taking the os module as an example, the general process is as follows:

In summary, the introduction The process of the os module involves the introduction of JavaScript file modules, the loading and execution of JavaScript core modules, and the loading and execution of built-in modules. The process is very cumbersome and complicated, but for the caller of the module, because the underlying complex implementation and details are shielded, The entire module can be imported simply through require(), which is very simple. friendly.

Summary

This article introduces the basic concepts of file modules and core modules as well as their specific processes and details that need to be paid attention to in file location, compilation or execution. Specifically:

• File modules can be divided into ordinary file modules and custom modules according to the different file positioning processes. Ordinary file modules can be directly located because of their clear paths, sometimes involving the process of file extension analysis and directory analysis; custom modules will search based on the module path, and after successful search, the final file location will be performed through directory analysis.

• File modules can be divided into JavaScript modules and C/C extension modules according to different compilation and execution processes. After the JavaScript module is packaged by the module wrapper, it is executed through the runInThisContext method of the vm module; the C/C extension module can be executed directly because it is already an executable file generated after compilation. , returns the exported object to the caller.

• The core module is divided into JavaScript core module and built-in module. The JavaScript core module is loaded into memory when the Node process starts. It can be taken out and then executed through the process.binding() method; the compilation and execution of the built-in module will go through process.binding (), get_builtin_module() and register_func() processing.

In addition, we also got the reference chain for Node to introduce the core module, that is, the file module-->JavaScript core module-->built-in module, and also learned The C module is implemented internally to complete the core, and the JavaScript module is implemented externally to encapsulate the module writing method.

For more programming related knowledge, please visit: Programming Video! !

The above is the detailed content of Learn about the file module and core module in Node in one article. For more information, please follow other related articles on the PHP Chinese website!