Study Notes on Module Mechanism in Node.js_node.js

Since the birth of Javascript, no one has ever regarded it as a programming language. In the Web 1.0 era, this scripting language was mainly used for form validation and web page special effects. It wasn't until the Web 2.0 era, when front-end engineers used it to greatly improve the user experience on web pages, that JS was widely valued. As JS becomes more and more popular, it has roughly gone through changes in tool libraries, component libraries, front-end frameworks, and front-end applications. Javascript inherently lacks one feature: modules, and the emergence of the CommonJS specification makes up for this shortcoming. This article will introduce the CommonJS specification and Node’s module mechanism.

Among other high-level languages, Java has class files, Python has an import mechanism, and PHP has include and require. The way JS introduces code through the <script> tag appears messy. In the past, people had to use namespaces and other methods to artificially constrain the code. It was not until the emergence of the CommonJS specification that the front-end and back-end Javascript could be unified. Node draws on the Modules specification of CommonJS to implement a very easy-to-use module system. </script>

1. CommonJS module specification

The module specification of CommonJS is divided into 3 parts:

1). Module reference: introduce a module's API into the current context through the require() method and pass in a module identifier, such as var math = require('math');
2).Module definition: Export the methods or variables of the current module through the exports object. There is also a module object in the module, and exports are actually attributes of the module. In Node, a file is a module, and the "global variables" within the module are not visible to the outside world. Only the attributes mounted on exports are public, such as exports.add = function() {}; exports.PI = 3.1415926;
3). Module identification: It is actually the parameter passed to require(), such as the above 'math', which must be a string that conforms to the camel nomenclature, or a relative path starting with "." ".." or Absolute path, it can have no filename suffix ".js"

2. Node module implementation process

In Node, modules are divided into two categories: one is the core module provided by Node itself, and the other is the file module written by the user. Part of the core module is compiled into a binary file during the compilation process of Node source code. The core module is loaded directly into the memory when Node starts, so its loading speed is the fastest. The file module is dynamically loaded at runtime and requires three steps: path analysis, file location, and compilation and execution. Note that Node caches imported modules to reduce the cost of secondary introduction, and adopts the strategy of loading from the cache with the highest priority for secondary loading of the same module.

2.1 Path Analysis

Path analysis mainly analyzes the module identifiers mentioned above, which are mainly divided into the following categories:

1), core modules, such as http, fs, path, etc.
2) Relative path file module starting with, . or ..
3). Absolute path file module starting with /
4). Custom file module, which may be in the form of a file or package. Node will try to find the target file one by one according to the module path array module.paths. It usually searches for the directory named node_modules along the current directory step by step until the root directory, so this is the most time-consuming way to find it.

2.2 File Positioning

Based on path analysis, file location needs to pay attention to the following details:

1) File extension analysis: Since the CommonJS specification allows the module identifier to not fill in the extension, Node will fill in the extension in the order of .js, .json, and .node, and try in sequence
2) Directory analysis and packages: If the corresponding file is not found after the above file extension analysis, but a directory is obtained, Node will treat the directory as a package

2.3 Compilation and execution

After locating the specific file, Node will create a new module object, load and compile it according to the path. For different extensions, the loading method is different:

1), .js file: read the file synchronously through the fs module and compile and execute
2). .node file: This is an extension file written in C/C and loaded through the dlopen() method
3).json file: Read the file synchronously through the fs module, and use JSON.parse() to parse and return the result
4). Other extension files: are loaded as .js files

We know that each module file has three variables: require, exports, and module by default. Even in Node’s API documentation, we know that each module also has two variables: filename and dirname. They Where did it come from? How does Node's module ensure that the declared "global variables" do not actually contaminate other modules? In fact, Node will wrap the file contents head and tail during the compilation of JS modules. The following is an example of a JS file with head and tail packaging:

Copy code The code is as follows:

(function(exports, require, module, __filename, __dirname) {
/* The middle is the actual content of the JS file */
var math = require('math');
exports.area = function(radius) {
          return Math.PI * radius * radius;
};
/* The actual content of the JS file ends */
});

In this way, each module file is scope-isolated, and variables such as require, exports, and module are also injected into the context of the module. This is Node's implementation of the CommonJS module specification. The compilation process of C/C modules and Node core modules is relatively complicated and will not be described in detail.

3. Module call stack

It is necessary to clarify the calling relationship of various modules in Node, as shown in the figure below:

C/C built-in module is the lowest module and is a core module. It mainly provides APIs for Javascript core modules and third-party Javascript file modules to call. In practice, such modules are almost never exposed. The Javascript core module has two main responsibilities: one is to serve as the encapsulation layer and bridging layer of the C/C built-in module for file module calls, and the other is a purely functional module that does not need to deal with the bottom layer. File modules are usually written by third parties, including ordinary Javascript modules and C/C extension modules.

4. Packages and NPM

4.1 Package Structure

A package is essentially an archive file (usually .zip or .tar.gz), which can be decompressed and restored to a directory after installation. CommonJS's package specification consists of two parts: package structure and package description file. A package structure that fully conforms to the CommonJS specification should contain the following files:

1).package.json: package description file
2).bin: directory where executable binary files are stored
3).lib: Directory to store Javascript code
4).doc: Directory for storing documents
5).test: Directory to store unit test cases

4.2 Package Description File

The package description file is a JSON file - package.json, located in the root directory of the package. It is an important part of the package and is used to describe the general information of the package. All behaviors of NPM to be mentioned later are closely related to the fields of this file. The following uses the package.json file of the well-known Web framework express project as an example to illustrate the meaning of some common fields.

1).name: package name
2).description: package introduction
3).version: version number, which must comply with "semantic version control", refer to http://semver.org/
4).dependencies: List of packages that are required to use the current package. This attribute is very important. NPM will automatically load dependent packages through this attribute
5).repositories: List of locations where source code is hosted

For the usage of other fields, please refer to NPM package.json description

4.3 NPM common functions

NPM (node ​​package manager), usually called node package manager. Its main function is to manage node packages, including: installation, uninstallation, update, view, search, release, etc.

4.3.1 NPM package installation

There are two types of installation of Node packages: local installation and global installation. The difference between the two is as follows:

1). Local installation npm install : The package will be downloaded to the current directory and can only be used in the current directory.
2). Global installation npm install -g : The package will be downloaded to a specific system directory, and the installed package can be used in all directories.

4.3.2 NPM package management

The following takes grunt-cli (grunt command line tool) as an example to list commonly used package management commands:

1).npm install: Install all packages declared in the dependencies and devDependencies fields of the package.json file
2).npm install grunt-cli@0.1.9: Install a specific version of grunt-cli
3).npm install grunt-contrib-copy --save: Install grunt-contrib-copy and save the dependency to the package.json file
4).npm uninstall grunt-cli: Uninstall package
5).npm list: Check which packages are installed
6).npm publish ：Publish package