1, opening analysis
The so-called buffer Buffer means "temporary storage area", which is a section of memory that temporarily stores input and output data.
The JS language itself only has a string data type and no binary data type, so NodeJS provides a global constructor Buffer equivalent to String to provide operations on binary data. In addition to reading the file to obtain an instance of the Buffer, it can also be constructed directly, for example:
var buffer = new Buffer([ 0x68, 0x65, 0x6c, 0x6c, 0x6f ]) ;
Buffer is similar to a string. In addition to using the .length attribute to get the byte length, you can also use the [index] method to read the bytes at the specified position, for example:
buffer[0] ; // 0x68;
Buffer and string can be converted to each other. For example, binary data can be converted into string using specified encoding:
var str = buffer.toString("utf-8"); // hello
Convert the string to binary data in the specified encoding:
var buffer= new Buffer("hello", "utf-8") ; //
A little difference:
There is an important difference between Buffer and string. The string is read-only, and any modification to the string results in a new string, while the original string remains unchanged.
As for Buffer, it is more like a C language array that can perform pointer operations. For example, you can use the [index] method to directly modify the bytes at a certain position.
-------------------------------------------------- -------------------------------------------------- -------------------------------------------------- -------------------------------------------------- --------------------------------
Theslice method does not return a new Buffer, but rather returns a pointer to a position in the middle of the original Buffer, as shown below.
[ 0x68, 0x65, 0x6c, 0x6c, 0x6f ]
|
bin bin.slice(2)
Therefore, modifications to the Buffer returned by the slice method will affect the original Buffer, for example:
var sub = bin.slice(2) ;
sub[0] = 0x65;
console.log(buffer); //
This is similar to applying for a new memory and copying the data in the existing memory. Below is an example.
var dup = new Buffer(bin.length) ;
buffer.copy(dup) ;
dup[0] = 0x48 ;
console.log(buffer); //
console.log(dup) ; //
In short, Buffer extends the data processing capabilities of JS from strings to arbitrary binary data.
The above is a brief introduction to what Buffer is. Let’s talk about how to use it and the specific usage scenarios.
Two, let’s talk about Buffer
JavaScript is very friendly to string processing. Whether it is a wide byte or a single-byte string, it is considered a string. Node needs to process network protocols, operate databases, process pictures, file uploads, etc. It also needs to process a large amount of binary data. The built-in strings are far from meeting these requirements, so Buffer came into being.
Buffer structure
Buffer is a typical module that combines Javascript and C. The performance-related parts are implemented in C, and the non-performance-related parts are implemented in JavaScript.
Node has already loaded the Buffer into the memory when the process starts and puts it into the global object, so there is no need to require
Buffer object: similar to an array, its elements are two-digit hexadecimal digits.
Buffer memory allocation
The memory allocation of the Buffer object is not in the heap memory of V8. The memory application is implemented at the C level of Node.
In order to efficiently use the requested memory, Node adopts the slab allocation mechanism. Slab is a dynamic memory management mechanism that applies various *nix operating systems. Slab has three states:
(1) full: fully allocated status
(2) partial: partial allocation status
(3) empty: no assigned status
Buffer conversion
Buffer objects can be converted to and from strings. The supported encoding types are as follows:
ASCII, UTF-8, UTF-16LE/UCS-2, Base64, Binary, Hex
String to Buffer
new Buffer(str, [encoding]), default UTF-8
buf.write(string, [offset], [length], [encoding])
Buffer to string
buf.toString([encoding], [start], [end])
Encoding type not supported by Buffer
Determine whether it is supported by Buffer.isEncoding(encoding)
iconv-lite: pure JavaScript implementation, lighter weight, better performance without C to javascript conversion
iconv: Call C’s libiconv library to complete
Buffer splicing
Pay attention to "res.on('data', function(chunk) {})". The parameter chunk is a Buffer object. Direct splicing will automatically convert it into a string. For wide-byte characters, garbled characters may be generated. ,
Solution:
(1) Through the setEncoding() method in the readable stream, this method allows the data event to be transmitted no longer as a Buffer object, but as an encoded string, which uses the StringEncoder module internally.
(2) Temporarily store the Buffer object into an array, and finally assemble it into a large Buffer and then encode it into a string for output.
Buffer is widely used in file I/O and network I/O. Its performance is very important, and its performance is much higher than that of ordinary strings.
In addition to the performance loss in converting strings, the use of Buffer has a highWaterMark setting that is crucial to performance when reading files.
a. The highWaterMark setting has a certain impact on the allocation and use of Buffer memory.
b. Setting highWaterMark too small may result in too many system calls.
When to use buffer and when not to use it ------ Pure javascript supports unicode code but not binary. When dealing with TCP stream or file stream, it is necessary to process the stream. When we save non-utf-8 strings, binary and other formats, we must use "Buffer".
3. Introduction of examples
var buf = new Buffer("this is text concat test !") ,str = "this is text concat test !" ;
console.time("buffer concat test !");
var list = [] ;
var len = 100000 * buf.length ;
for(var i=0;i List.push(buf) ;
len = buf.length ;
}
var s1 = Buffer.concat(list, len).toString() ;
console.timeEnd("buffer concat test !") ;
console.time("string concat test !") ;
var list = [] ;
for (var i = 100000; i >= 0; i--) {
List.push(str) ;
}
var s2 = list.join("") ;
console.timeEnd("string concat test !") ;
The following are the results:
The reading speed of string is definitely faster, and the buffer also needs toString() operation. So when we save a string, we should still use string. Even if a large string is spliced into a string, it will not be slower than the buffer.
So when do we need to use buffer again? When there is no other way, when we save non-utf-8 strings, binary and other formats, we must use it.
Four, summary
(1), JavaScript is suitable for processing Unicode encoded data, but it is not friendly to the processing of binary data.
(2), so when processing TCP streams or file systems, it is necessary to process octet streams.
(3), Node has several methods for processing, creating and consuming octet streams.
(4) The original data is stored in a Buffer instance. A Buffer is similar to an integer array, but its memory is allocated outside the V8 stack. The size of a Buffer cannot be changed.
(5), the encoding types processed are: ascii, utf8, utf16le, ucs2 (alias of utf16le), base64, binary, hex.
(6), Buffer is a global element, and you can get a Buffer instance directly by using new Buffer().

JavaScript runs in browsers and Node.js environments and relies on the JavaScript engine to parse and execute code. 1) Generate abstract syntax tree (AST) in the parsing stage; 2) convert AST into bytecode or machine code in the compilation stage; 3) execute the compiled code in the execution stage.

The future trends of Python and JavaScript include: 1. Python will consolidate its position in the fields of scientific computing and AI, 2. JavaScript will promote the development of web technology, 3. Cross-platform development will become a hot topic, and 4. Performance optimization will be the focus. Both will continue to expand application scenarios in their respective fields and make more breakthroughs in performance.

Both Python and JavaScript's choices in development environments are important. 1) Python's development environment includes PyCharm, JupyterNotebook and Anaconda, which are suitable for data science and rapid prototyping. 2) The development environment of JavaScript includes Node.js, VSCode and Webpack, which are suitable for front-end and back-end development. Choosing the right tools according to project needs can improve development efficiency and project success rate.

Yes, the engine core of JavaScript is written in C. 1) The C language provides efficient performance and underlying control, which is suitable for the development of JavaScript engine. 2) Taking the V8 engine as an example, its core is written in C, combining the efficiency and object-oriented characteristics of C. 3) The working principle of the JavaScript engine includes parsing, compiling and execution, and the C language plays a key role in these processes.

JavaScript is at the heart of modern websites because it enhances the interactivity and dynamicity of web pages. 1) It allows to change content without refreshing the page, 2) manipulate web pages through DOMAPI, 3) support complex interactive effects such as animation and drag-and-drop, 4) optimize performance and best practices to improve user experience.

C and JavaScript achieve interoperability through WebAssembly. 1) C code is compiled into WebAssembly module and introduced into JavaScript environment to enhance computing power. 2) In game development, C handles physics engines and graphics rendering, and JavaScript is responsible for game logic and user interface.

JavaScript is widely used in websites, mobile applications, desktop applications and server-side programming. 1) In website development, JavaScript operates DOM together with HTML and CSS to achieve dynamic effects and supports frameworks such as jQuery and React. 2) Through ReactNative and Ionic, JavaScript is used to develop cross-platform mobile applications. 3) The Electron framework enables JavaScript to build desktop applications. 4) Node.js allows JavaScript to run on the server side and supports high concurrent requests.

Python is more suitable for data science and automation, while JavaScript is more suitable for front-end and full-stack development. 1. Python performs well in data science and machine learning, using libraries such as NumPy and Pandas for data processing and modeling. 2. Python is concise and efficient in automation and scripting. 3. JavaScript is indispensable in front-end development and is used to build dynamic web pages and single-page applications. 4. JavaScript plays a role in back-end development through Node.js and supports full-stack development.


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

WebStorm Mac version
Useful JavaScript development tools

MantisBT
Mantis is an easy-to-deploy web-based defect tracking tool designed to aid in product defect tracking. It requires PHP, MySQL and a web server. Check out our demo and hosting services.

ZendStudio 13.5.1 Mac
Powerful PHP integrated development environment

SublimeText3 Chinese version
Chinese version, very easy to use

PhpStorm Mac version
The latest (2018.2.1) professional PHP integrated development tool
