How does Node.js handle streams? How to set stream encoding?

Node.js is an event-driven asynchronous I/O framework, which can be said to be the running environment of JavaScript on the server. It is widely used for its single-threaded, non-blocking I/O, event-driven and other characteristics. When processing file streams, in order to avoid problems such as garbled characters, encoding needs to be set.

Node.js processing streams

Node.js's stream (Stream) is an abstract interface that can be used to process large data collections. Streams implement data transmission in a manner similar to that of a pipeline, dividing the data into segments for processing, without having to wait for the entire data collection to appear before proceeding to the next step. Therefore, streams can significantly reduce memory footprint and processing time when processing large amounts of data or large files.

Node.js has four basic stream objects, namely Readable, Writable, Duplex, Transform. Among them, Readable and Writable streams are the most widely used.

Readable stream can be understood as an input stream, which reads data into Node.js for processing. A Writable stream is an output stream that outputs data to a target location. Two streams can be combined, such as a pipe, to handle the transmission of data.

During the Node.js stream operation process, the problem of Chinese garbled characters may occur. At this time we need to set the encoding to solve this problem.

Set stream encoding

In Node.js, the encoding of the stream is set using the setEncoding() method. Encoding settings can be made for readable and writable streams.

For example, set the encoding for a readable stream as follows:

const fs = require('fs');

const readableStream = fs.createReadStream('test.txt', 'utf8');
readableStream.setEncoding('utf8');

At this time, the readable stream will read the data in segments according to the encoding type, and divide the segments into Data output. Similarly, for writable streams, you can also use the setEncoding() method to set the encoding. For example:

const fs = require('fs');

const writableStream = fs.createWriteStream('output.txt', 'utf8');
writableStream.setEncoding('utf8');

It should be noted that before the Node.js V8.0.0 version, setting the stream encoding is only valid for strings. If you want to read and write binary data, you need to use BufferObject.

Therefore, if you want to use encoding when reading and writing non-string data, you need to serialize and deserialize the data when reading and writing, as follows:

const fs = require('fs');
const data = { name: 'Node.js' };
const str = JSON.stringify(data);

const readableStream = fs.createReadStream('test.txt', 'binary');
const writableStream = fs.createWriteStream('output.txt', 'binary');
writableStream.write(new Buffer(str, 'utf8'));

This way , we can use encoding to avoid problems such as garbled characters during the processing of Node.js streams.

Summary

Node.js stream processing is very suitable for processing large data sets and large files. When processing stream data, pay attention to setting the encoding to avoid problems such as Chinese garbled characters. Before Node.js V8.0.0, setting the encoding was only valid for strings. If you need to process non-string data, you need to use the Buffer object to serialize and deserialize the data.

The above is the detailed content of How does Node.js handle streams? How to set stream encoding?. For more information, please follow other related articles on the PHP Chinese website!