How to find duplicate elements in a stream in Java

Finding duplicate elements in a data stream is one of the common questions in Java interviews and even in many student exams. Java provides a variety of methods for finding duplicate elements. We mainly focus on two methods: the first is to use the Set of the Java Collection Framework, and the other is to use the built-in method Collections.Frequency() of the stream.

Java program to find duplicate elements in a stream

Before discussing the different methods of obtaining duplicates from a data collection, it is necessary to discuss the filter() method. It will be an important part of the sample program.

filter()

It allows us to filter the elements of the stream based on specified conditions. It is part of a higher-order function that applies some behavior to the flow term. This method takes a predicate as a parameter and returns a list of elements matching the predicate.

grammar

                
filter(predicate);

Usage of Java collection framework

It is a sub-interface of Java Collection Interface and does not allow duplicate values. It is very similar to mathematical sets. We can use the add() method which adds only those dissimilar elements to the set. To use the properties of the Set interface, we need to use the HashSet class that implements this interface.

Example

The following example illustrates how to use the Set interface to find duplicate elements from a stream.

method

• Use the Arrays.asList() method to create a list to store a fixed-size list.

• Then, use the HashSet class to define a Set to store only dissimilar elements.

• Now, use the filter() method along with stream() and forEach() to filter out only duplicate items. Here, stream() specifies the input in the form of a stream and we will use forEach() to iterate and print the repeated elements.

import java.util.*;
public class Duplicate {
   public static void main(String []args) {
      // create a list with duplicate items
      List<Integer> itemsList = Arrays.asList(10, 12, 10, 33, 40, 40, 61, 61);
      // declaring a new Set 
      Set<Integer> newitemSet = new HashSet<>();
      System.out.println("The list of duplicate Items: ");
      itemsList.stream() // converting list to stream
         .filter(nums -> !newitemSet.add(nums)) // to filter out the elementsthat are not added to the set
         .forEach(System.out::println); // print the duplicates
   }
}

Output

The list of duplicate Items: 
10
40
61

Use Collections.Frequency() method

Another simplest way to filter duplicate elements from a stream or collection is to use the Collections.Frequency() method of the "java.util" package, which is used to return the total number of elements in the specified collection.

Grammar

Collections.frequency(nameOfCollection, obj);

it's here,

nameOfCollection represents the stream, and obj represents the element whose frequency needs to be determined.

Example

In the following example, we will use the Collections.Frequency() method to count the occurrences of each element in the stream, and then return elements that appear more than once. We will print the entire list of occurrences of repeated elements along with the count.

import java.util.*;
public class FindDuplicates {
   public static void main(String[] args) {
      // create a list with duplicate items
      List<Integer> itemslist = Arrays.asList(10, 12, 10, 10, 33, 40, 40, 61, 61);
      System.out.println("The list of duplicate Items with frequency: ");
      itemslist.stream() // converting list to stream 
         .filter(itr -> Collections.frequency(itemslist, itr) > 1) // checking the frequency of duplicate items
         .forEach(System.out::println); // printing the frequency of duplicate items
      System.out.println("Count of duplicate items: ");    
      // to count the duplicate items    
      System.out.println(itemslist.stream()
       .filter(itr -> Collections.frequency(itemslist, itr) > 1)
       .count());
    }
}

Output

The list of duplicate Items with frequency: 
10
10
10
40
40
61
61
Count of duplicate items: 
7

Example

Here is another example where we will use both Set Interface and Collections.Frequency() methods to get only duplicate elements. The Collections.Frequency() method will count the occurrences of each element in the stream and then collect elements with a count greater than 1 into a Set to remove duplicates. The resulting Set will contain only repeated elements from the stream.

import java.util.stream.*;
import java.util.*;
public class FindDuplicates {
   public static void main(String[] args) {
      // create a list with duplicate items
      List<Integer> itemslist = Arrays.asList(10, 12, 10, 10, 33,40, 40, 61, 61);
      // set to store duplicate items
      Set<Integer> duplicates = itemslist.stream()
         .filter(itr -> Collections.frequency(itemslist, itr) > 1) // checking the frequency of duplicate items
         .collect(Collectors.toSet()); // adding only duplicate items to set
      // printing the duplicate items    
      System.out.println("The list of duplicate Items:" + duplicates); 
   }
}

Output

The list of duplicate Items:[40, 10, 61]

in conclusion

In this section, we will conclude our discussion with some key points from the above examples and concepts. We can use the filter() method to filter out specific types of elements from the data collection. It works behind the scenes by applying a predicate to each element. Set Interface's ability to store only distinct elements makes it an excellent choice for a given task.

The above is the detailed content of How to find duplicate elements in a stream in Java. For more information, please follow other related articles on the PHP Chinese website!