An in-depth analysis of NodeJs concurrent and asynchronous callback processing_node.js

It is not accurate to say concurrent and asynchronous. It should be said to be continuous asynchronous. The single-threaded asynchronous feature of NodeJs directly causes the callback to be unable to determine the final execution result when multiple asynchronous operations are performed at the same time. Give a simple example:

for(var i = 0; i < 5; i++) {
  fs.readFile('file', 'utf-8', function(error, data){});
} 

I initiated 5 asynchronous operations to read files in a row. It is very simple. So the question is, how can I make sure that all asynchronous operations have been completed? Because the subsequent operations can only be carried out after they are all executed. I believe that students with some experience will think of using counting methods, but how to ensure correct counting is another problem. Think carefully:

Callback is a function that adds 1 to the counter during each asynchronous operation and -1 when each asynchronous operation ends. Determine whether to execute the callback by judging whether the counter is 0. This logic is very simple. It requires a global variable relative to execution time and callback time as a counter, and it must perform a +1 operation when passed to the asynchronous method, and then return a function for callback, which is a bit convoluted, but see See the advanced usage of Js functions:

var pending = (function() {
  var count = 0;
  return function() {
    count++;
    return function() {
      count--;
      if (count === 0) {
        // 全部执行完毕
      }
    }
  }
}); 

When pending is called, it is pending(), for example:

var done = pending(); 

At this time, the counting variable count is initialized to 0, and the returned function is attached to done. If done() is executed at this time, what will happen? Is it to directly execute the first function returned by pending, that is: pending()()? What is this execution? First, the counting variable count+1 is returned, and a function is returned. This function is directly passed as a callback to the asynchronous method. , when executing this callback, first set the count variable count-1, and then determine whether the count is 0. If it is 0, it means that all asynchronous execution is completed, thus achieving continuous asynchronous operations with the same callback.

The key lies in the two returns. Simply put:

The first return function is count+1, and then returns the function that needs callback

The second return function is the function that needs a callback. If it is executed, it will count-1, and then determine whether all asynchronous execution is completed. If it is completed, it will call back

Look at a practical example, asynchronous callback for reading multiple files:

var fileName = ['1.html', '2.html', '3.html'];
var done = pending(function(fileData) {
  console.log('done');
  console.log(fielData);
});
for(var i = 0; i < fileName.lenght; i++) {
  fs.readFile(fileName[i], 'utf-8', done(fileName[i]));
}

The done one uses the pending method to wrap up the method we want to callback and execute. When the counter reaches 0, it will be executed. Then we have to improve the pending method:

var pending = (function(callback) {
  var count = 0;
  var returns = {};
  console.log(count);
  return function(key) {
    count++;
    console.log(count);
    return function(error, data) {
      count--;
      console.log(count);
      returns[key] = data;
      if (count === 0) {
        callback(returns);
      }
    }
  }
}); 

callback is our callback function. When var done = pending(callback), done is actually the first return function. It has a parameter that can be used as the subscript of the returned value, so in the loop body In done(fileName[i]), the file name is passed in. This done() is executed directly. After counting+1, it returns the callback function to be passed to the asynchronous method. As mentioned earlier, this callback function will determine whether to execute the callback function we want to execute based on the count variable. , and pass the contents of the file to it, that is, returns. Okay, let’s run it, I believe we can see the results accurately.

0
1
2
3
2
1
0
done
{"1.html": "xxx", "2.html": "xxx", "3.html": "xxx"}

It can be clearly seen from the count, from 0-3 to 0, and then our callback function outputs done and the contents of the file.

This problem is solved, we need to think about how to encapsulate and reuse such a method. Otherwise, wouldn’t it be unscientific to write pending every time?

Let’s take a look at the processing method of UnJs (one of my NodeJs-based web development frameworks), applied to sub-template operations in template parsing:

unjs.asyncSeries = function(task, func, callback) {
  var taskLen = task.length;
  if (taskLen <= 0) {
    return;
  }
  var done = unjs.pending(callback);
  for(var i = 0; i < taskLen; i++) {
    func(task[i], done);
  }
} 

asyncSeries has three parameters, meaning:

task: The object that needs to be processed, such as the file that needs to be read, it is a list, if it is not a list, or the length of the list is 0, it will not be executed

func: Asynchronous methods, such as fs.readFile, are passed in through it

callback: The method we want to callback

done is the same as before. It is passed to func, but it is not executed. Because we hope that the application side can control the parameters, we let the application side execute it.

Look at the operation when dealing with sub-templates:

var subTemplate = [];
var patt = /\{\% include \'(.+)\' \%\}/ig;
while(sub = patt.exec(data)) {
  var subs = sub;
  subTemplate.push([subs[0], subs[1]]);
}
unjs.asyncSeries(subTemplate, function(item, callback) {
  fs.readFile('./template/' + item[1], 'utf-8', callback(item[0]));
}, function(data) {
  for(var key in data) {
    html = html.replace(key, data[key]);
  }
}); 

The list of subTemplate is data generated based on the analysis of sub-templates. It is a two-dimensional array. The first value of each sub-item is the calling text of the sub-template, that is: {% include 'header.html ' %} such a string, the second parameter is the sub-template file name, that is: header.html

The second parameter of asyncSeries is callback, which is actually the third parameter, which is the pending callback method of the callback function we want to execute. As mentioned earlier, inside asyncSeries, it is not running. Instead, it is run here, that is: callback(item[0]), with parameters, because later the string calling the sub-template in the parent template will be replaced with the content of the corresponding sub-template based on this parameter.

In this way, as long as continuous asynchronous is needed, you can use the asyncSeries method to handle it. Because of the asynchronous relationship, the flow of the program is a bit convoluted, and it may not be easy to understand at first. Even if you are familiar with it, you may suddenly not understand it. It doesn't matter. For example, the callback in the second parameter is actually generated by the third parameter. You may be wondering, what exactly is this callback? There are also two returns in pending, which are not easy to understand and need to be thought about more.

Okay, the continuous asynchronous callback is completed using the advanced features of Js function. However, the asynchronous nature of NodeJs really makes the control of the program very problematic, such as continuous asynchronous operations that require value transfer, etc. These can all be achieved through this idea and changes.

The above content is the knowledge about concurrent and asynchronous callback processing in NodeJs shared by the editor. I hope you like it.