


Detailed examples of several ideas for deduplication of JavaScript arrays
The demand for data deduplication is actually that tool libraries such as lodash have mature and complete implementations and can be maturely used in production environments. But this does not prevent us from the perspective of thinking expansion to see how duplication removal can be achieved using several ideas. This article mainly shares with you several ideas for deduplication of JavaScript arrays.
The first is the implementation of the conventional double-layer circular comparison idea
function doubleLoopUniq(arr) { let result = []; for (let i = 0, len = arr.length, isExist; i <p>Use the built-in indexOf of js to remove duplicates</p><pre class="brush:php;toolbar:false">function indexOfUniq(arr) { let result = []; for (let i = 0, len = arr.length; i <p>After sorting, compare before and after to remove duplicates</p> <pre class="brush:php;toolbar:false">function sortUniq(arr) { let result = [], last; // 这里解构是为了不对原数组产生副作用 [ ...arr ].sort().forEach(item => { if (item != last) { result.push(item); last = item; } }); return result; }
Deduplication through hashTable
function hashUniq(arr) { let hashTable = arr.reduce((result, curr, index, array) => { result[curr] = true; return result; }, {}) return Object.keys(hashTable).map(item => parseInt(item, 10)); }
ES6 SET one line of code to achieve deduplication
function toSetUniq(arr) { return Array.from(new Set(arr)); }
splice deduplication (directly operates the array itself, with side effects)
function inPlaceUniq(arr) { let idx = 0; while (idx <p>Finally in Run a simple test under nodejs to see which one is more efficient~</p><pre class="brush:php;toolbar:false">let data = []; for (var i = 0; i <p>The results are as follows</p><pre class="brush:php;toolbar:false">hashTable 168ms sortUniq 332ms toSetUniq 80ms indexOfUniq 4280ms doubleLoopUniq 13303ms inPlaceUniq 9977ms
Extended thinking: If the elements in the array are objects, how to remove duplicates?
Since it is a reference type, deepEqual will inevitably be used. Although this idea can answer this problem, it is inevitably not efficient enough.
It can also be seen from the above test that deduplication through new Set and hashTable is the most efficient.
So there is no doubt that we need to transform based on these two methods. I want to use hashTable.
On the other hand, in order to reduce the time-consuming caused by deep comparison, I try to use JSON.stringify to reference The type is converted to a basic type.
function collectionUniq(collection) { let hashTable = {}; collection.forEach(item => { hashTable[JSON.stringify(item)] = true; }) return Object.keys(hashTable).map(item => JSON.parse(item)) }
Then here comes the problem. We all know that the attributes of objects are unordered. If the data is like this, then it is GG.
let collection = [ { a: 1, b: 2, c: 3 }, { b: 2, c: 3, a: 1 } ]
There is a toHash idea, After a basic deduplication of this array, in order to ensure accuracy,
first traverse the JSON string=>
Pass charCodeAt() gets the unicode encoding of each string =>
and adds them to get a total number. Finally, compare them in pairs. Those with equal values are duplicates, thus achieving the effect of deduplication.
function toHash(obj) { let power = 1; let res = 0; const string = JSON.stringify(obj, null, 2); for (let i = 0, l = string.length; i <p>This is just a basic idea for implementation, and there is a lot of room for improvement. In order to reduce the possibility of hash collision, the weight of some special characters can be increased or decreased. </p><p><strong>The key point is to ensure that the probability of collision is smaller than winning the jackpot. </strong></p><p class="comments-box-content">Related recommendations: <br></p><p class="comments-box-content"><a href="http://www.php.cn/js-tutorial-386521.html" target="_self"> Sharing several methods of deduplication in JavaScript arrays</a></p><p class="comments-box-content"><a href="http://www.php.cn/php-weizijiaocheng-386230.html" target="_self"> Implementing array deduplication in PHP Method code</a></p><p class="comments-box-content"><a href="http://www.php.cn/js-tutorial-384672.html" target="_self">JS simple implementation of array deduplication method analysis</a></p>
The above is the detailed content of Detailed examples of several ideas for deduplication of JavaScript arrays. For more information, please follow other related articles on the PHP Chinese website!

Yes, the engine core of JavaScript is written in C. 1) The C language provides efficient performance and underlying control, which is suitable for the development of JavaScript engine. 2) Taking the V8 engine as an example, its core is written in C, combining the efficiency and object-oriented characteristics of C. 3) The working principle of the JavaScript engine includes parsing, compiling and execution, and the C language plays a key role in these processes.

JavaScript is at the heart of modern websites because it enhances the interactivity and dynamicity of web pages. 1) It allows to change content without refreshing the page, 2) manipulate web pages through DOMAPI, 3) support complex interactive effects such as animation and drag-and-drop, 4) optimize performance and best practices to improve user experience.

C and JavaScript achieve interoperability through WebAssembly. 1) C code is compiled into WebAssembly module and introduced into JavaScript environment to enhance computing power. 2) In game development, C handles physics engines and graphics rendering, and JavaScript is responsible for game logic and user interface.

JavaScript is widely used in websites, mobile applications, desktop applications and server-side programming. 1) In website development, JavaScript operates DOM together with HTML and CSS to achieve dynamic effects and supports frameworks such as jQuery and React. 2) Through ReactNative and Ionic, JavaScript is used to develop cross-platform mobile applications. 3) The Electron framework enables JavaScript to build desktop applications. 4) Node.js allows JavaScript to run on the server side and supports high concurrent requests.

Python is more suitable for data science and automation, while JavaScript is more suitable for front-end and full-stack development. 1. Python performs well in data science and machine learning, using libraries such as NumPy and Pandas for data processing and modeling. 2. Python is concise and efficient in automation and scripting. 3. JavaScript is indispensable in front-end development and is used to build dynamic web pages and single-page applications. 4. JavaScript plays a role in back-end development through Node.js and supports full-stack development.

C and C play a vital role in the JavaScript engine, mainly used to implement interpreters and JIT compilers. 1) C is used to parse JavaScript source code and generate an abstract syntax tree. 2) C is responsible for generating and executing bytecode. 3) C implements the JIT compiler, optimizes and compiles hot-spot code at runtime, and significantly improves the execution efficiency of JavaScript.

JavaScript's application in the real world includes front-end and back-end development. 1) Display front-end applications by building a TODO list application, involving DOM operations and event processing. 2) Build RESTfulAPI through Node.js and Express to demonstrate back-end applications.

The main uses of JavaScript in web development include client interaction, form verification and asynchronous communication. 1) Dynamic content update and user interaction through DOM operations; 2) Client verification is carried out before the user submits data to improve the user experience; 3) Refreshless communication with the server is achieved through AJAX technology.


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

SecLists
SecLists is the ultimate security tester's companion. It is a collection of various types of lists that are frequently used during security assessments, all in one place. SecLists helps make security testing more efficient and productive by conveniently providing all the lists a security tester might need. List types include usernames, passwords, URLs, fuzzing payloads, sensitive data patterns, web shells, and more. The tester can simply pull this repository onto a new test machine and he will have access to every type of list he needs.

ZendStudio 13.5.1 Mac
Powerful PHP integrated development environment

Zend Studio 13.0.1
Powerful PHP integrated development environment

MinGW - Minimalist GNU for Windows
This project is in the process of being migrated to osdn.net/projects/mingw, you can continue to follow us there. MinGW: A native Windows port of the GNU Compiler Collection (GCC), freely distributable import libraries and header files for building native Windows applications; includes extensions to the MSVC runtime to support C99 functionality. All MinGW software can run on 64-bit Windows platforms.

WebStorm Mac version
Useful JavaScript development tools
