How to Eliminate Duplicate Dictionaries in a Python List?

Removing Duplicates from a List of Dictionaries

Duplication in a data collection can be a hindrance to efficient data processing. In Python programming, lists of dictionaries are commonly used to store tabular data. However, there may be instances where you need to remove duplicate dictionaries from such a list.

Consider the following list of dictionaries:

[
    {'id': 1, 'name': 'john', 'age': 34},
    {'id': 1, 'name': 'john', 'age': 34},
    {'id': 2, 'name': 'hanna', 'age': 30},
]

The goal is to obtain a list with only unique dictionaries, excluding the duplicates. To achieve this, we can employ a straightforward approach:

Creating a Temporary Dictionary with ID as Key

1. Create a temporary dictionary using a list comprehension, where the key for each dictionary is its 'id' field.
2. This step essentially maps each unique 'id' value to a specific dictionary.

Extracting Unique Dictionaries from Values

1. Obtain the values of the temporary dictionary using the values() method.
2. The result is a list of unique dictionaries, with duplicates removed.

Python Implementation

Here's how to implement this approach in Python:

<code class="python">def remove_duplicates_from_dicts(dict_list):
    dict_id_mapping = {v['id']: v for v in dict_list}
    return list(dict_id_mapping.values())

sample_list = [
    {'id': 1, 'name': 'john', 'age': 34},
    {'id': 1, 'name': 'john', 'age': 34},
    {'id': 2, 'name': 'hanna', 'age': 30},
]
print(remove_duplicates_from_dicts(sample_list))</code>

This code will produce the following output:

[{'id': 1, 'name': 'john', 'age': 34}, {'id': 2, 'name': 'hanna', 'age': 30}]

By employing this strategy, you can effectively remove duplicate dictionaries from a list and obtain a new list with only unique elements.

The above is the detailed content of How to Eliminate Duplicate Dictionaries in a Python List?. For more information, please follow other related articles on the PHP Chinese website!