java - 分布式环境下，定时任务或异步处理如何保持幂等性？

Answer 1

Impotent generally means that the method does not change the business status, so it can ensure that the effect of repeated calls is the same as the effect of a single call.

Looking at your description, your scheduled tasks and asynchronous processing are likely to change the business status (such as inserting data). It is very likely that your method itself is not idempotent. If so, there is basically no solution. .

I think the actual idea of ​​your question may be: In a distributed environment, how to ensure that when sending repeated requests for scheduled tasks and asynchronous processing, the actual business logic is only executed once?

If so, you can use a centralized storage (such as redis) to save the caller's request record. After receiving the request, the server uses atomic query and save operations (such as redis's setnx command) , to ensure that only one request will be saved successfully. This can achieve the effect of only executing the actual business once.

boolean setSuccess = redis.setnx(request.serializeToString(),"");//原子操作
if(setSuccess){
  doBusiness(); //执行业务
}else{
  doNothing(); //什么都不做
}

Answer 2

I personally don’t have any experience in this area, but the company’s approach is to use IP to make judgments, and only the designated IP can execute this scheduled task.

The following is a personal fantasy without practical experience:
Build a public application specifically to handle scheduled tasks, and then provide a message interface for specific applications to call.

Answer 3

Use transactions to implement it, distributed transactions, or message queues

Answer 4

1. Make the operation itself idempotent, such as changing the +1 operation to the = operation

2. Give each operation an ID and record each execution. Before execution, check whether the operation with this ID has been executed before

Answer 5

You can consider a message queue with an ack mechanism, such as RabbitMQ, etc., which not only ensures that a task is only assigned to one worker, but also ensures the integrity of the success of the task

Answer 6

We implement it through zk scheduling. In a distributed environment, a task can only be executed by one machine at most. ZK can realize this function very well

Answer 7

All answers so far are wrong, including the accepted one. The reason is that scheduled tasks or asynchronous processing have nothing to do with idempotence.

Answer 8

redis + token is fine

Answer 9

Use a queue. Once you take it, it will be gone. It will only be executed once. If the execution fails, it will be thrown back to the queue and wait for the next time.