Detailed explanation of Node.js file operations_node.js

Node has a set of data stream APIs that can process files like network streams. It is very convenient to use, but it only allows sequential processing of files and cannot read and write files randomly. Therefore, some lower-level file system operations need to be used.

This chapter covers the basics of file processing, including how to open a file, read a portion of the file, write data, and close the file.

Many of Node’s file APIs are almost replicas of the corresponding file APIs in UNIX (POSIX). For example, the way to use file descriptors is just like in UNIX. The file descriptor in Node is also an integer number, representing an entity. Index into the process file descriptor table.

There are 3 special file descriptors - 1, 2 and 3. They represent standard input, standard output and standard error file descriptors respectively. Standard input, as the name suggests, is a read-only stream that processes use to read data from the console or process channel. Standard output and standard error are file descriptors used only to output data. They are often used to output data to the console, other processes, or files. Standard error is responsible for error information output, and standard output is responsible for ordinary process output.

Once the process is started, these file descriptors can be used. They do not actually have corresponding physical files. You can't read and write data at a random position, (Translator's Note: The original text is You can write to and read from specific positions within the file. Depending on the context, the author may have written less "not"), you can only operate the network like The data is read and output sequentially like a data stream, and the written data cannot be modified.

Ordinary files are not subject to this restriction. For example, in Node, you can create files that can only append data to the tail, and you can also create files that read and write random locations.

Almost all file-related operations involve processing file paths. This chapter will first introduce these tool functions, and then explain in depth file reading, writing and data operations

Processing file paths

File paths are divided into relative paths and absolute paths, which are used to represent specific files. You can merge file paths, extract file name information, and even detect whether a file exists.

In Node, you can use strings to process file paths, but that will complicate the problem. For example, if you want to connect different parts of the path, some parts end with "/" and some do not, and the path separators are different. The operating system may also be different, so when you connect them, the code will be very wordy and cumbersome.

Fortunately, Node has a module called path, which can help you standardize, connect, parse paths, convert from absolute paths to relative paths, extract various parts of information from the path, and detect whether the file exists. In general, the path module is actually just some string processing, and it does not go to the file system for verification (except for the path.exists function).

Standardization of paths

It's usually a good idea to normalize paths before storing or using them. For example, file paths obtained by user input or configuration files, or paths connected by two or more paths, should generally be standardized. You can use the normalize function of the path module to normalize a path, and it can also handle "..", "." and "//". For example:

Copy code The code is as follows:

var path = require('path');

path.normalize('/foo/bar//baz/asdf/quux/..');

// => '/foo/bar/baz/asdf'

Connection path

Using the path.join() function, you can connect any number of path strings. Just pass all the path strings to the join() function in sequence:

Copy code The code is as follows:

                var path = require('path');

path.join('/foo', 'bar', 'baz/asdf', 'quux', '..');

                                     // => '/foo/bar/baz/asdf'

As you can see, path.join() will automatically normalize the path internally.

Parse path

Use path.resolve() to resolve multiple paths into an absolute path. Its function is like performing a "cd" operation on these paths one by one. Unlike the parameters of the cd command, these paths can be files, and they do not have to actually exist - the path.resolve() method does not access the underlying file system. Determine if the path exists, it's just some string manipulation.

For example:

Copy code The code is as follows:

                var path = require('path');

path.resolve('/foo/bar', './baz');

                             // => /foo/bar/baz

path.resolve('/foo/bar', '/tmp/file/');

                                // => /tmp/file

If the parsing result is not an absolute path, path.resolve() will append the current working directory as the path to the front of the parsing result, for example:

Copy code The code is as follows:

Path.resolve('wwwroot', 'static_files/png/', '../gif/image.gif');
// If the current working directory is /home/myself/node,
will be returned // => /home/myself/node/wwwroot/static_files/gif/image.gif'

Calculate the relative path of two absolute paths

path.relative() can tell you how to jump from one absolute address to another, such as:

Copy code The code is as follows:

              var path = require('path');

                  path.relative('/data/orandea/test/aaa', '/data/orandea/impl/bbb');

                                    // => ../../impl/bbb

Extract data from path

Take the path "/foo/bar/myfile.txt" as an example. If you want to get all the contents of the parent directory (/foo/bar), or read other files in the same directory, for this, you must use path.dirname(filePath) gets the directory part of the file path, such as:

Copy code The code is as follows:

                var path = require('path');

path.dirname('/foo/bar/baz/asdf/quux.txt');

                                // => /foo/bar/baz/asdf

Or, if you want to get the file name from the file path, which is the last part of the file path, you can use the path.basename function:

Copy code The code is as follows:

                  var path = require('path');

path.basename('/foo/bar/baz/asdf/quux.html')

                                                                                                                                                                         

The file path may also contain the file extension, which is usually the part of the string after the last "." character in the file name.

path.basename can also accept an extension string as the second parameter, so that the returned file name will automatically remove the extension and only return the name part of the file:

Copy code The code is as follows:

                var path = require('path');

path.basename('/foo/bar/baz/asdf/quux.html', '.html');

                                                                                                                                           

To do this, you first need to know the file extension. You can use path.extname() to get the extension:

Copy code The code is as follows:

                var path = require('path');
path.extname('/a/b/index.html');

                                     // => '.html'

path.extname('/a/b.c/index');

                                    // => ''

path.extname('/a/b.c/.');

                                    // => ''

path.extname('/a/b.c/d.');

                                                                       // =>

Check if the path exists

So far, the path processing operations mentioned above have nothing to do with the underlying file system, they are just some string operations. However, sometimes you need to determine whether a file path exists. For example, sometimes you need to determine whether a file or directory exists, and create it if it does not exist. You can use path.exsits():

Copy code The code is as follows:

                var path = require('path');

path.exists('/etc/passwd', function(exists) {

console.log('exists:', exists);

// = & gt; true

             });

path.exists('/does_not_exist', function(exists) {

console.log('exists:', exists);

                                                                                                                                                                                                                 ​ 

             });

Note: Starting from Node 0.8 version, exists has been moved from the path module to the fs module and changed to fs.exists. Except for the namespace, nothing else has changed:

Copy code The code is as follows:

                  var fs = require('fs');

                                                         fs.exists('/does_not_exist', function(exists) {

console.log('exists:', exists);

                                                                                                                                                                                                                 ​ 

             });

path.exists() is an I/O operation, because it is asynchronous, so it requires a callback function. When the I/O operation returns, this callback function is called and the result is passed to it. You can also use its synchronous version path.existsSync(), which has exactly the same function, except that it does not call the callback function, but returns the result directly:

Copy code The code is as follows:

              var path = require('path');
path.existsSync('/etc/passwd');

                     // => true

fs module introduction

The fs module contains all related functions for file query and processing. With these functions, you can query file information, read, write and close files. Import the fs module like this:

Copy code The code is as follows:

        var fs = require(‘fs’)

Query file information

Sometimes you may need to know the file size, creation date or permissions and other file information. You can use the fs.stath function to query the meta-information of the file or directory:

Copy code The code is as follows:

                  var fs = require('fs');
                          fs.stat('/etc/passwd', function(err, stats) {

if (err) { throw err;}

console.log(stats);

             });

This code snippet will have output similar to the following

Copy code The code is as follows:

{ dev: 234881026,
ino: 95028917,

mode: 33188,

nlink: 1,

uid: 0,

gid: 0,

rdev: 0,

size: 5086,

blksize: 4096,

blocks: 0,

atime: Fri, 18 Nov 2011 22:44:47 GMT,

mtime: Thu, 08 Sep 2011 23:50:04 GMT,

ctime: Thu, 08 Sep 2011 23:50:04 GMT }

1. The fs.stat() call will pass an instance of the stats class as a parameter to its callback function. You can use the stats instance as follows:

2.stats.isFile() - Returns true if it is a standard file, not a directory, socket, symbolic link or device, otherwise false
3.stats.isDiretory() - Returns true if it is a directory, otherwise false
4.stats.isBlockDevice() - Returns true if it is a block device. In most UNIX systems, block devices are usually located in the /dev directory
5.stats.isChracterDevice() - Returns true if it is a character device
6.stats.isSymbolickLink() - Returns true if it is a file link
7.stats.isFifo() - Returns true if it is a FIFO (a special type of UNIX named pipe)
8.stats.isSocket() - if it is a UNIX socket (TODO: googe it)

Open file

Before reading or processing a file, you must first use the fs.open function to open the file. Then the callback function you provide will be called and get the file descriptor. You can use this file descriptor to read later. Write to this already open file:

Copy code The code is as follows:

                  var fs = require('fs');

                                                              fs.open('/path/to/file', 'r', function(err, fd) {

// got fd file descriptor

             });

The first parameter of fs.open is the file path, and the second parameter is some tags used to indicate the mode in which to open the file. These tags can be r, r, w, w, a or a. Below is a description of these tags (from the fopen page of the UNIX documentation)

1.r - Open the file in read-only mode, the initial position of the data stream is at the beginning of the file
2.r - Open the file in read-write mode, the initial position of the data stream is at the beginning of the file
3.w - If the file exists, the file length will be cleared to 0, that is, the file content will be lost. If it doesn't exist, try to create it. The initial position of the data stream is at the beginning of the file
4.w - Open the file in read-write mode. If the file does not exist, try to create it. If the file exists, set the file length to 0, that is, the file content will be lost. The initial position of the data stream is at the beginning of the file
5.a - Open the file in write-only mode. If the file does not exist, try to create it. The initial position of the data stream is at the end of the file. Each subsequent write operation will append the data to the end of the file.
6.a - Open the file for reading and writing. If the file does not exist, try to create it. The initial position of the data stream is at the end of the file. Each subsequent write operation will append the data to the end of the file.

Read file

Once the file is opened, you can start reading the file contents, but before you start, you must first create a buffer to place the data. This buffer object will be passed to the fs.read function as a parameter and filled with data by fs.read.

Copy code The code is as follows:

var fs = require('fs');

fs.open('./my_file.txt', 'r', function opened(err, fd) {

if (err) { throw err }

var readBuffer = new Buffer(1024),

bufferOffset = 0,

bufferLength = readBuffer.length,

filePosition = 100;

fs.read(fd,

         readBuffer,

         bufferOffset,

         bufferLength,

         filePosition,

         function read(err, readBytes) {

                   if (err) { throw err; }

                   console.log('just read ' readBytes ' bytes');

                   if (readBytes > 0) {

                            console.log(readBuffer.slice(0, readBytes));

                   }

});

});

上面代码尝试打开一个文件，当成功打开后(调用opened函数)，开始请求从文件流第100个字节开始读取随后1024个字节的数据（第11行）。

fs.read()的最后一个参数是个回调函数（第16行），当下面三种情况发生时，它会被调用：

1.有错误发生
2.成功读取了数据
3.没有数据可读

如果有错误发生，第一个参数（err）会为回调函数提供一个包含错误信息的对象，否则这个参数为null。如果成功读取了数据，第二个参数（readBytes）会指明被读到缓冲区里数据的大小，如果值是0，则表示到达了文件末尾。

注意：一旦把缓冲区对象传递给fs.open()，缓冲对象的控制权就转移给给了read命令，只有当回调函数被调用，缓冲区对象的控制权才会回到你手里。因此在这之前，不要读写或者让其它函数调用使用这个缓冲区对象；否则，你可能会读到不完整的数据，更糟的情况是，你可能会并发地往这个缓冲区对象里写数据。

写文件

通过传递给fs.write()传递一个包含数据的缓冲对象，来往一个已打开的文件里写数据：

复制代码 代码如下:

var fs = require('fs');

fs.open('./my_file.txt', 'a', function opened(err, fd) {

    if (err) { throw err; }

    var writeBuffer = new Buffer('writing this string'),

    bufferPosition = 0,

    bufferLength = writeBuffer.length, filePosition = null;

    fs.write( fd,

        writeBuffer,

        bufferPosition,

        bufferLength,

        filePosition,

        function wrote(err, written) {

           if (err) { throw err; }

           console.log('wrote ' written ' bytes');

        });

});

In this example, the 2nd line of code (Translator's Note: Original text is 3) attempts to open a file in append mode (a), and then the 7th line of code (Translator's Note: Original text is 9) writes data to the file . The buffer object needs to be accompanied by several pieces of information as parameters:

1. Buffer data
2. Where does the data to be written start from in the buffer
3. The length of the data to be written
4.Which location in the file is the data written to?
5. The callback function write

that is called when the operation is completed

In this example, the filePostion parameter is null, which means that the write function will write the data to the current location of the file pointer. Because the file is opened in append mode, the file pointer is at the end of the file.

Similar to the read operation, never use the incoming buffer object during the execution of fs.write. Once fs.write starts executing, it gains control of that buffer object. You can only wait until the callback function is called before you can use it again.

Close file

You may have noticed that none of the examples in this chapter so far have code to close the file. Because they are small and simple examples that are used only once, the operating system makes sure that all files are closed when the Node process ends.

However, in a real application, once you open a file you want to make sure you eventually close it. To do this, you need to keep track of all those open file descriptors and then eventually close them by calling fs.close(fd[,callback]) when they are no longer in use. It's easy to miss a file descriptor if you're not careful. The following example provides a function called openAndWriteToSystemLog, showing how to close the file carefully:

Copy code The code is as follows:

var fs = require('fs');
function openAndWriteToSystemLog(writeBuffer, callback){
fs.open('./my_file', 'a', function opened(err, fd) {
If (err) { return callback(err); }
function notifyError(err) {
                  fs.close(fd, function() {
                 callback(err);
            });
}
        var bufferOffset = 0,
        bufferLength = writeBuffer.length,
         filePosition = null;
             fs.write( fd, writeBuffer, bufferOffset, bufferLength, filePosition,
             function wrote(err, written) {
If (err) { return notifyError(err); }
                          fs.close(fd, function() {
                    callback(err);
                });
            }
);
});
}
openAndWriteToSystemLog(
new Buffer('writing this string'),
Function done(err) {
If (err) {
console.log("error while opening and writing:", err.message);
             return;
}
console.log('All done with no errors');
}
);

Here, a function called openAndWriteToSystemLog is provided, which accepts a buffer object containing the data to be written, and a callback function that is called after the operation is completed or an error occurs. If an error occurs, the first callback function The parameters will contain the error object.

Pay attention to the internal function notifyError, which will close the file and report the error that occurred.

Note: At this point, you know how to use low-level atomic operations to open, read, write and close files. However, Node also has a set of more advanced constructors that allow you to work with files in a simpler way.

For example, if you want to use a safe way to allow two or more write operations to append data to a file concurrently, then you can use WriteStream.

Also, if you want to read a certain area of ​​a file, consider using ReadStream. These two use cases will be introduced in Chapter 9 "Data Reading and Writing Streams".

Summary

When you use files, you need to process and extract file path information in most cases. By using the path module, you can concatenate paths, standardize paths, calculate path differences, and convert relative paths into absolute paths. You can extract the extension, file name, directory and other path components of the specified file path.

Node provides a set of low-level APIs in the fs module to access the file system. The low-level APIs use file descriptors to operate files. You can open a file with fs.open, write to a file with fs.write, read a file with fs.read, and close a file with fs.close.

You should always use proper error handling logic to close files when an error occurs - ensuring that open file descriptors are closed before the call returns.