A brief discussion on Node.js+COW technology for process creation and file copying

This article will take you to understand COW (Copy-On-Write) technology and introduce the process creation and file copy application of COW technology Node.js. I hope it will be helpful to everyone!

COW is not a cow, it is the abbreviation of Copy-On-Write, which is a technology that is copying but not completely copying.

Generally speaking, copying is to create two identical copies. The two copies are independent:

However, sometimes copying does not work. No matter how necessary, you can reuse the previous one. At this time, you can just quote the previous one and copy the corresponding part of the content when writing the content. In this way, if the content is used for reading, copying is eliminated, but if writing is required, part of the content will actually be copied for modification.

This is called "copy-on-write", also known as Copy-On-Write.

The principle is very simple, but it is very common in the memory management and file system of the operating system. Node.js has also become "lazy" because of this technology.

In this article, we will explore the application of Copy-On-Write in process creation and file copying in Node.js. [Recommended learning: "nodejs tutorial"]

File copy

The most common idea for file copying is to write a completely identical copy file content to another location, but there are two problems with this:

• Write exactly the same content. If the same file is copied hundreds of times, then the same content will be created hundreds of times. Times? What a waste of hard disk space
• What if the power goes off halfway through writing? How to restore overwritten content?

How to do it? At this time, operating system designers thought of COW technology.

Using COW technology to realize file copying perfectly solves the above two problems:

• Copying just adds a reference to the previous content. If it is not modified, it will not actually be copied, only The corresponding data blocks are not actually copied until the content is modified for the first time, thus avoiding the waste of a large amount of hard disk space.
• When writing a file, modifications will be made to another free disk block first, and then copied to the target location after the modification is completed, so that there will be no problem of being unable to roll back after a power outage

You can use the Copy-On-Write mode in the fs.copyFile api of Node.js:

By default, copyFile will write to the target file, overwriting the original content

const fsPromises = require('fs').promises;

(async function() {
  try {
    await fsPromises.copyFile('source.txt', 'destination.txt');
  } catch(e) {
    console.log(e.message);
  }
})();

But You can specify the copy strategy through the third parameter:

const fs = require('fs');
const fsPromises = fs.promises;
const { COPYFILE_EXCL, COPYFILE_FICLONE, COPYFILE_FICLONE_FORCE} = fs.constants;

(async function() {
  try {
    await fsPromises.copyFile('source.txt', 'destination.txt', COPYFILE_FICLONE);
  } catch(e) {
    console.log(e.message);
  }
})();

There are 3 supported flags:

• COPYFILE_EXCL: If the target file already exists, an error will be reported (the default is to overwrite)
• COPYFILE_FICLONE: Copy in copy-on-write mode. If the operating system does not support it, it will switch to real copy (the default is direct copy)
• COPYFILE_FICLONE_FORCE: Copy in copy-on-write mode. If the operating system does not support it, an error will be reported.

These three constants are 1, 2, and 4. They can be combined by bitwise OR and passed in:

const flags = COPYFILE_FICLONE | COPYFILE_EXCL;
fsPromises.copyFile('source.txt', 'destination.txt', flags);

Node.js Supports the copy-on-write technology of the operating system, which can improve performance in some scenarios. It is recommended to use the COPYFILE_FICLONE method, which is better than the default method.

Process Creation

Fork is a common way to create a process, and its implementation is a copy-on-write technology.

We know that the process is divided into three parts in the memory: code segment, data segment, and stack segment:

• Code segment: stores the code to be executed
• Data segment: stores some global data
• Stack segment: stores execution status

If a new process is created based on this process, these three parts of memory must be copied. And if these three parts of memory have the same content, then memory space is wasted.

So fork does not actually copy the memory, but creates a new process and references the memory of the parent process. When the data is modified, this part of the memory will actually be copied.

This is why process creation is called fork, which is a fork, because it is not completely independent, but a certain part is forked into two parts, but Most of it is still the same.

But what if the code to be executed is different? At this time, you need to use exec, which will create a new code segment, data segment, stack segment, and execute new code.

The fork and exec APIs can also be used in Node.js:

fork:

const cluster = require('cluster');

if (cluster.isMaster) {
  console.log('I am master');
  cluster.fork();
  cluster.fork();
} else if (cluster.isWorker) {
  console.log(`I am worker #${cluster.worker.id}`);
}

exec:

const { exec } = require('child_process');
exec('my.bat', (err, stdout, stderr) => {
  if (err) {
    console.error(err);
    return;
  }
  console.log(stdout);
});

fork is the basis for Linux process creation , which shows how important copy-on-write technology is.

Summary

Copying multiple copies of the same content is undoubtedly a waste of space, so the operating system uses Copy-On- when copying files and copying memory during process creation. Write technology will only copy when it is actually modified.

Node.js supports the setting of flags of fs.copyFile. You can specify COPYFILE_FICLONE to use Copy-On-Write to copy files. It is also recommended that you use this method to save hard disk space and improve file copying. performance.

The fork of a process is also an implementation of Copy-On-Write. It does not directly copy the code segment, data segment, and stack segment of the process to new content, but refers to the previous ones and only when modifying them. Will do a real memory copy.

In addition, Copy-On-Write has many applications in the implementation of Immutable and in distributed read-write separation and other fields.

COW makes Node.js "lazy" but performs better.

Original address: https://juejin.cn/post/6999497362255118366

Author: zxg_God said there must be light

More programming For related knowledge, please visit: Introduction to Programming! !

The above is the detailed content of A brief discussion on Node.js+COW technology for process creation and file copying. For more information, please follow other related articles on the PHP Chinese website!