Preface
List deduplication is a common problem when writing Python scripts, because no matter where the source data comes from, when we convert it into a list, there may be expected The result is not our final result. The most common thing is that there are duplicate elements in the list. At this time, the first thing we need to do is deduplication.
Let’s take the simplest way first, using Python’s built-in data type set.
Suppose our list data is like this:
level_names = [ u'Second Level', u'Second Level', u'Second Level', u'First Level', u'First Level' ]
Because the elements of the set cannot be repeated, so When converting a list into a set, duplicate elements will be automatically removed. This is the basic principle. The code is as follows:
>>> the_list = set(level_names) >>> print(the_list) set([u'Second Level', u'First Level'])
The disadvantage of this method is that it is converted into a list again. It is impossible to save the previous list order. If there is no such requirement, this method is the simplest answer. Maybe some friends think it is too simple. There is no technical content in this. Yes, so in general interview questions, you are asked to list. The heavy words are usually written like this:
Please write down the method to remove the heavy words from the list (set cannot be used)
People have stated that set cannot be used. , so, sometimes this trick doesn’t work, so of course it doesn’t trouble us, we have other methods.
We all know that lists can be traversed, and the problem of traversing is simple. We then define an empty list, then traverse the list with data, and add a judgment when traversing. If it is not in the empty list, add Enter it and throw it away if you have it. The code is as follows:
the_list = [] for level in level_names: if level not in the_list: the_list.append(level) print(the_list)
Do you think this method is okay, but this method is suitable for ordinary small lists. No problem, but if you encounter a super large list, you will not be able to do what you want, because the list in the_list becomes very large, which will affect the efficiency when making judgments, because the list is searched in index order, and when the amount of data is large, it will Slow down.
Maybe you want to ask, what should I do if I encounter a large list? Is there a more awesome way? Of course, let's continue. Since using lists when judging will affect efficiency, let's change our thinking. We use sets. Then you may ask, are sets faster? Yes, because the hash function used by set When searching for values, although the set is unordered, the position is fixed. It only takes one time to check whether a specific element exists. Someone on the Internet did a comparison of searching for elements in lists and sets. Under the same data conditions, it took 16 minutes to use list. , it takes 52 seconds to use set. You can see the effect from this comparison. Not much else to say, just post the code:
the_list = [] the_set = set() for level in level_names: if level not in the_set: the_set.add(level) the_list.append(level) print(the_list)
More you For articles related to Python list deduplication methods that you should know, please pay attention to the PHP Chinese website!

The reasons why Python scripts cannot run on Unix systems include: 1) Insufficient permissions, using chmod xyour_script.py to grant execution permissions; 2) Shebang line is incorrect or missing, you should use #!/usr/bin/envpython; 3) The environment variables are not set properly, and you can print os.environ debugging; 4) Using the wrong Python version, you can specify the version on the Shebang line or the command line; 5) Dependency problems, using virtual environment to isolate dependencies; 6) Syntax errors, using python-mpy_compileyour_script.py to detect.

Using Python arrays is more suitable for processing large amounts of numerical data than lists. 1) Arrays save more memory, 2) Arrays are faster to operate by numerical values, 3) Arrays force type consistency, 4) Arrays are compatible with C arrays, but are not as flexible and convenient as lists.

Listsare Better ForeflexibilityandMixdatatatypes, Whilearraysares Superior Sumerical Computation Sand Larged Datasets.1) Unselable List Xibility, MixedDatatypes, andfrequent elementchanges.2) Usarray's sensory -sensical operations, Largedatasets, AndwhenMemoryEfficiency

NumPymanagesmemoryforlargearraysefficientlyusingviews,copies,andmemory-mappedfiles.1)Viewsallowslicingwithoutcopying,directlymodifyingtheoriginalarray.2)Copiescanbecreatedwiththecopy()methodforpreservingdata.3)Memory-mappedfileshandlemassivedatasetsb

ListsinPythondonotrequireimportingamodule,whilearraysfromthearraymoduledoneedanimport.1)Listsarebuilt-in,versatile,andcanholdmixeddatatypes.2)Arraysaremorememory-efficientfornumericdatabutlessflexible,requiringallelementstobeofthesametype.

Pythonlistscanstoreanydatatype,arraymodulearraysstoreonetype,andNumPyarraysarefornumericalcomputations.1)Listsareversatilebutlessmemory-efficient.2)Arraymodulearraysarememory-efficientforhomogeneousdata.3)NumPyarraysareoptimizedforperformanceinscient

WhenyouattempttostoreavalueofthewrongdatatypeinaPythonarray,you'llencounteraTypeError.Thisisduetothearraymodule'sstricttypeenforcement,whichrequiresallelementstobeofthesametypeasspecifiedbythetypecode.Forperformancereasons,arraysaremoreefficientthanl

Pythonlistsarepartofthestandardlibrary,whilearraysarenot.Listsarebuilt-in,versatile,andusedforstoringcollections,whereasarraysareprovidedbythearraymoduleandlesscommonlyusedduetolimitedfunctionality.


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

SublimeText3 English version
Recommended: Win version, supports code prompts!

ZendStudio 13.5.1 Mac
Powerful PHP integrated development environment

Safe Exam Browser
Safe Exam Browser is a secure browser environment for taking online exams securely. This software turns any computer into a secure workstation. It controls access to any utility and prevents students from using unauthorized resources.

SublimeText3 Chinese version
Chinese version, very easy to use

EditPlus Chinese cracked version
Small size, syntax highlighting, does not support code prompt function
