An in-depth analysis of the callback queue of Node.js

Queue is an important technology in Node.js for efficiently handling asynchronous operations. [Video tutorial recommendation: node js tutorial]

In this article, we will take a deep dive into queues in Node.js: what they are, how they work (via the event loop) and what they are type.

What are queues in Node.js?

Queue is a data structure used in Node.js to organize asynchronous operations. These operations exist in different forms, including HTTP requests, read or write file operations, streams, etc.

Handling asynchronous operations in Node.js is very challenging.

Unpredictable delays (or, worse, no results) may occur during HTTP requests, depending on network quality. There may also be delays when trying to read and write files with Node.js, depending on the size of the file.

Similar to timers and many other operations, the time when an asynchronous operation completes may also be uncertain.

With these different latency scenarios, Node.js needs to be able to handle all of these operations efficiently.

Node.js cannot handle operations based on first-start-first-handle (first-start-first-handle) or first-finish-first-handle (first-finish-first-handle).

One reason why this cannot be done is that an asynchronous operation may also contain another asynchronous operation.

Making room for the first asynchronous process means that the inner asynchronous process must complete before other asynchronous operations in the queue can be considered.

There are many situations to consider, so your best option is to create rules. This rule affects how event loops and queues work in Node.js.

Let’s take a brief look at how Node.js handles asynchronous operations.

Call stack, event loop and callback queue

The call stack is used to keep track of the currently executing function and where it started. When a function is about to be executed, it is added to the call stack. This helps JavaScript retrace its processing steps after executing a function.

The callback queue is a queue that saves the callback function as an asynchronous operation when the background operation is completed. They work in a first-in-first-out (FIFO) manner. We will introduce different types of callback queues later in this article.

Please note that Node.js is responsible for all asynchronous activities, as JavaScript can take advantage of its single-threaded nature to prevent new threads from being spawned.

After completing the background operation, it is also responsible for adding functions to the callback queue. JavaScript itself has nothing to do with callback queues. At the same time, the event loop will continuously check whether the call stack is empty so that a function can be extracted from the callback queue and added to the call stack. The event loop only checks the queue after all synchronization operations have been performed.

So, in what order does the event loop select callback functions from the queue?

First, let’s look at the five main types of callback queues.

Types of callback queue

IO queue (IO queue)

IO operations refer to operations involving external devices (such as computer hard disk, network card, etc.). Common operations include reading and writing file operations, network operations, etc. These operations should be asynchronous as they are left to Node.js to handle.

JavaScript cannot access the computer's internal devices. When such an operation is performed, JavaScript transfers it to Node.js for processing in the background.

After completion, they will be transferred to the IO callback queue for event loop to be transferred to the call stack for execution.

Timer queue

Every operation involving the Node.js timer function (such as setTimeout() and setInterval()) are all to be added to the timer queue.

Please note that the JavaScript language itself does not have a timer function. It uses the timer API provided by Node.js (including setTimeout) to perform time-related operations. So the timer operation is asynchronous. Whether it's 2 seconds or 0 seconds, JavaScript will hand off the time-related operation to Node.js, which will then complete it and add it to the timer queue.

For example:

setTimeout(function() {
        console.log('setTimeout');
    }, 0)
    console.log('yeah')


# 返回
yeah
setTimeout

When processing an asynchronous operation, JavaScript will continue to perform other operations. Only after all synchronization operations have been processed, the event loop will enter the callback queue.

Microtask queue (Microtask queue)

The queue is divided into two queues:

• The first queue contains the reason process.nextTick Function and delayed function.

Each iteration executed by the event loop is called a tick (time scale).

process.nextTick 是一个函数，它在下一个 tick （即事件循环的下一个迭代）执行一个函数。微任务队列需要存储此类函数，以便可以在下一个 tick 执行它们。

这意味着事件循环必须继续检查微任务队列中的此类函数，然后再进入其他队列。

• 第二个队列包含因 promises 而延迟的函数。

如你所见，在 IO 和计时器队列中，所有与异步操作有关的内容都被移交给了异步函数。

但是 promise 不同。在 promise 中，初始变量存储在 JavaScript 内存中（你可能已经注意到了fcfd9c63d2b5ae1697cab5937aad0a2f）。

异步操作完成后，Node.js 会将函数（附加到 Promise）放在微任务队列中。同时它用得到的结果来更新 JavaScript 内存中的变量，以使该函数不与 fcfd9c63d2b5ae1697cab5937aad0a2f 一起运行。

以下代码说明了 promise 是如何工作的：

let prom = new Promise(function (resolve, reject) {
        // 延迟执行
        setTimeout(function () {
            return resolve("hello");
        }, 2000);
    });
    console.log(prom);
    // Promise { <pending> }
    
    prom.then(function (response) {
        console.log(response);
    });
    // 在 2000ms 之后,输出
    // hello

关于微任务队列，需要注意一个重要功能，事件循环在进入其他队列之前要反复检查并执行微任务队列中的函数。例如，当微任务队列完成时，或者说计时器操作执行了 Promise 操作，事件循环将会在继续进入计时器队列中的其他函数之前参与该 Promise 操作。

因此，微任务队列比其他队列具有最高的优先级。

检查队列（Check queue）

检查队列也称为即时队列（immediate queue）。IO 队列中的所有回调函数均已执行完毕后，立即执行此队列中的回调函数。setImmediate 用于向该队列添加函数。

例如：

const fs = require('fs');
setImmediate(function() {
    console.log('setImmediate');
})
// 假设此操作需要 1ms
fs.readFile('path-to-file', function() {
    console.log('readFile')
})
// 假设此操作需要 3ms
do...while...

执行该程序时，Node.js 把 setImmediate 回调函数添加到检查队列。由于整个程序尚未准备完毕，因此事件循环不会检查任何队列。

因为 readFile 操作是异步的，所以会移交给 Node.js，之后程序将会继续执行。

do while  操作持续 3ms。在这段时间内，readFile 操作完成并被推送到 IO 队列。完成此操作后，事件循环将会开始检查队列。

尽管首先填充了检查队列，但只有在 IO 队列为空之后才考虑使用它。所以在 setImmediate 之前，将 readFile 输出到控制台。

关闭队列（Close queue）

此队列存储与关闭事件操作关联的函数。

包括以下内容：

• 流关闭事件，在关闭流时发出。它表示不再发出任何事件。
• http关闭事件，在服务器关闭时发出。

这些队列被认为是优先级最低的，因为此处的操作会在以后发生。

你肯sing不希望在处理 promise 函数之前在 close 事件中执行回调函数。当服务器已经关闭时，promise 函数会做些什么呢？

队列顺序

微任务队列具有最高优先级，其次是计时器队列，I/O队列，检查队列，最后是关闭队列。

回调队列的例子

让我们通过一个更复杂的例子来说明队列的类型和顺序：

const fs = require("fs");

// 假设此操作需要 2ms
fs.writeFile('./new-file.json', '...', function() {
    console.log('writeFile')
})

// 假设这需要 10ms 才能完成 
fs.readFile("./file.json", function(err, data) {
    console.log("readFile");
});

// 不需要假设，这实际上需要 1ms
setTimeout(function() {
    console.log("setTimeout");
}, 1000);

// 假设此操作需要 3ms
while(...) {
    ...
}

setImmediate(function() {
    console.log("setImmediate");
});

// 解决 promise 需要 4 ms
let promise = new Promise(function (resolve, reject) {
    setTimeout(function () {
        return resolve("promise");
    }, 4000);
});
promise.then(function(response) {
    console.log(response)
})

console.log("last line");

程序流程如下：

• 在 0 毫秒时，程序开始。
• 在 Node.js 将回调函数添加到 IO 队列之前，fs.writeFile 在后台花费 2 毫秒。

fs.readFile takes 10ms at the background before Node.js adds the callback function to the IO queue.

• 在 Node.js 将回调函数添加到 IO 队列之前，fs.readFile 在后台花费 10 毫秒。
• 在 Node.js 将回调函数添加到计时器队列之前，setTimeout 在后台花费 1ms。
• 现在，while 操作（同步）需要 3ms。在此期间，线程被阻止（请记住 JavaScript 是单线程的）。
• 同样在这段时间内，setTimeout 和 fs.writeFile 操作完成，并将它们的回调函数分别添加到计时器和 IO 队列中。

现在的队列是：

// queues
Timer = [
    function () {
        console.log("setTimeout");
    },
];
IO = [
    function () {
        console.log("writeFile");
    },
];

setImmediate 将回调函数添加到 Check 队列中：

js
// 队列
Timer...
IO...
Check = [
    function() {console.log("setImmediate")}
]

在将 promise 操作添加到微任务队列之前，需要花费 4ms 的时间在后台进行解析。

最后一行是同步的，因此将会立即执行：

# 返回
"last line"

因为所有同步活动都已完成，所以事件循环开始检查队列。由于微任务队列为空，因此它从计时器队列开始：

// 队列
Timer = [] // 现在是空的
IO...
Check...


# 返回
"last line"
"setTimeout"

当事件循环继续执行队列中的回调函数时，promise 操作完成并被添加到微任务队列中：

// 队列
    Timer = [];
    Microtask = [
        function (response) {
            console.log(response);
        },
    ];
    IO = []; // 当前是空的
    Check = []; // 当前是在 IO 的后面，为空


    # results
    "last line"
    "setTimeout"
    "writeFile"
    "setImmediate"

几秒钟后，readFile 操作完成，并添加到 IO 队列中：

// 队列
    Timer = [];
    Microtask = []; // 当前是空的
    IO = [
        function () {
            console.log("readFile");
        },
    ];
    Check = [];


    # results
    "last line"
    "setTimeout"
    "writeFile"
    "setImmediate"
    "promise"

最后，执行所有回调函数：

// 队列
    Timer = []
    Microtask = []
    IO = [] // 现在又是空的
    Check = [];


    # results
    "last line"
    "setTimeout"
    "writeFile"
    "setImmediate"
    "promise"
    "readFile"

这里要注意的三点：

• 异步操作取决于添加到队列之前的延迟时间。并不取决于它们在程序中的存放顺序。
• 事件循环在每次迭代之继续检查其他任务之前，会连续检查微任务队列。
• 即使在后台有另一个 IO 操作（readFile），事件循环也会执行检查队列中的函数。这样做的原因是此时 IO 队列为空。请记住，在执行 IO 队列中的所有的函数之后，将会立即运行检查队列回调。

总结

JavaScript 是单线程的。每个异步函数都由依赖操作系统内部函数工作的 Node.js 去处理。

Node.js 负责将回调函数（通过 JavaScript 附加到异步操作）添加到回调队列中。事件循环会确定将要在每次迭代中接下来要执行的回调函数。

了解队列如何在 Node.js 中工作，使你对其有了更好的了解，因为队列是环境的核心功能之一。 Node.js 最受欢迎的定义是 non-blocking（非阻塞），这意味着异步操作可以被正确的处理。都是因为有了事件循环和回调队列才能使此功能生效。

更多编程相关知识，可访问：编程教学！！

The above is the detailed content of An in-depth analysis of the callback queue of Node.js. For more information, please follow other related articles on the PHP Chinese website!