How to use CompletableFuture to ensure the order consistency of batch interface request results?

Efficiently concurrent processing of batch interface requests: Ensure the order of results is consistent

Concurrent calls to multiple third-party interfaces can significantly improve efficiency when processing large amounts of data efficiently. However, simple multithreaded concurrency may cause the order of return results to be inconsistent and does not correspond to the original data list. This article will introduce how to solve this problem using Java's CompletableFuture to ensure that the interface call results are exactly the same as the original data order.

question:

Suppose that more than 1000 third-party interfaces need to be called concurrently and processed to return results. If multiple threads are started using a simple for loop, the order of interface calls cannot be guaranteed, and the order of the final results does not match the original data list. Some sample code uses CompletableFuture.runAsync to perform asynchronous tasks, but ignores the collection of results and the maintenance of sequentiality.

Solution:

In order to ensure that the order of results is consistent with the original data list, the key is to use CompletableFuture.supplyAsync instead of CompletableFuture.runAsync . supplyAsync method can return a result, while runAsync does not return a value. Return the results of each interface call through supplyAsync , and then collect the results into the list by streaming processing to ensure that the order of the results is consistent with the original data list.

Improved code:

 public static void main(String[] args) {
    List<string> dataList = new ArrayList(); // Original data list// ... Initialize dataList ...

    ExecutorService executorService = new ThreadPoolExecutor(
            //The number of core threads Runtime.getRuntime().availableProcessors(),
            //Maximum number of threads Runtime.getRuntime().availableProcessors() * 2,
            //Thread survival time 60L, TimeUnit.SECONDS,
            new LinkedBlockingQueue(),
            new ThreadPoolExecutor.CallerRunsPolicy());

    List <completablefuture> > futures = new ArrayList();
    for (String data : dataList) {
        futures.add(CompletableFuture.supplyAsync(() -> {
            logger.info("Start execution of asynchronous thread->>" data);
            // Call the interface and pass in data
            //Judge whether data matches based on the interface return value//Return the processed result return processData(data); // Process data and return the result}, executorService)); // Use a custom thread pool}

    //After processing logic of all requests completed, CompletableFuture.allOf(futures.toArray(new CompletableFuture[0])).thenRun(() -> {
        List<string> results = futures.stream()
                .map(CompletableFuture::join)
                .collect(Collectors.toList());
        logger.info("Thread execution is completed: {}", JSON.toJSONString(results));
        // Call to send SMS}).thenRun(() -> executorService.shutdown());
}

// Method of processing data, modify private static String processData(String data) {
    // ... Interface call and data processing logic...
    return data "processed result";
}</string></completablefuture></string>

By storing the results of each CompletableFuture in the futures list and collecting the results at the end using futures.stream().map(CompletableFuture::join).collect(Collectors.toList()) , the result order is ensured to be consistent with the original data list. CompletableFuture::join method blocks until the result of CompletableFuture is obtained. This effectively solves the problem of inconsistent results in the original code.

The above is the detailed content of How to use CompletableFuture to ensure the order consistency of batch interface request results?. For more information, please follow other related articles on the PHP Chinese website!