Reliably Send an HTTP Request as a User Leaves a Page

In many cases, I need to send an HTTP request containing some data to record what the user does, such as navigating to another page or submitting a form. Consider this example, which sends some information to an external service when clicking on a link:

 <a href="https://www.php.cn/link/3cbfb2330b21840b385a45c958602663">Go to Page</a>

document.getElementById('link').addEventListener('click', (e) => {
  fetch("/log", {
    method: "POST",
    headers: {
      "Content-Type": "application/json"
    }, 
    body: JSON.stringify({
      Some: "data"
    })
  });
});

There is nothing particularly complicated here. The link works as usual (I didn't use e.preventDefault() ), but before that behavior occurs, a POST request is triggered. No need to wait for any response. I just want to send it to any service I'm visiting.

At first glance, you might think that the scheduling of the request is synchronized, after which we will continue to navigate from the page, and other servers successfully process the request. But it turns out that this is not always the case.

The browser does not guarantee that open HTTP requests will be retained

When certain events occur to terminate a page in the browser, there is no guarantee that the HTTP request being processed will be successful (more on the lifecycle of the page "termination" and other states). The reliability of these requests may depend on a number of factors—the network connection, application performance, and even the configuration of external services itself.

Therefore, sending data at these moments can be far from reliable, and if you rely on these logs to make data-sensitive business decisions, there can be a potentially significant problem.

To illustrate this unreliability, I set up a small Express application with a page using the code above. When the link is clicked, the browser navigates to /other , but before this happens, a POST request is issued.

While everything is happening, I open the "Network" tab of my browser and am using the "Slow 3G" connection speed. After the page loads, I cleared the logs and everything looks quiet:

But once you click on the link, things go wrong. When navigation occurs, the request is cancelled.

This leaves us almost unable to be sure that external services can actually handle the request. To verify this behavior, this also happens when we use window.location to navigate programmatically:

 document.getElementById('link').addEventListener('click', (e) => {
  e.preventDefault();

  // The request has been queued but was cancelled immediately after the navigation occurs.
  fetch("/log", {
    method: "POST",
    headers: {
      "Content-Type": "application/json"
    },
    body: JSON.stringify({
      Some: 'data'
    }),
  });

  window.location = e.target.href;
});

These unfinished requests may be abandoned regardless of how or when the navigation occurs and when the active page terminates.

Why was it cancelled?

The root cause of the problem is that by default, XHR requests (via fetch or XMLHttpRequest ) are asynchronous and non-blocking. Once the request is queued, the actual work of the request is passed to the browser-level API in the background.

This is great in terms of performance - you don't want the request to take up the main thread. But this also means that when pages enter the "termination" state, they are at risk of being abandoned and there is no guarantee that any background work can be completed. Here is Google's summary of that particular lifecycle state:

Once the page starts uninstalling and is cleared from memory by the browser, it is in a terminated state. In this state, no new tasks cannot be started and may be terminated if the ongoing task runs for too long.

In short, the browser's design assumption is that when a page is closed, there is no need to continue processing any background processes it queues.

So, what are our choices?

The most obvious way to avoid this problem is to delay the user action as much as possible until the request returns a response. In the past, this was done wrongly by using the synchronization flags supported in XMLHttpRequest . But using it completely blocks the main thread, causing many performance issues - I've written a few things about this in the past - so this idea shouldn't even be considered. In fact, it is about to exit the platform (Chrome v80 has already removed it).

Instead, if you are going to take this approach, it is better to wait for the Promise to resolve to the returned response. Once you return, you can safely perform the behavior. Using our previous code snippet, this might look like this:

 document.getElementById('link').addEventListener('click', async (e) => {
  e.preventDefault();

  // Wait for the response to return...
  await fetch("/log", {
    method: "POST",
    headers: {
      "Content-Type": "application/json"
    },
    body: JSON.stringify({
      Some: 'data'
    }),
  });

  // …and then navigate to leave.
  window.location = e.target.href;
});

This can do the job, but has some non-trivial drawbacks.

First, it affects the user experience by delaying the occurrence of required behavior. Collecting analytics data is certainly beneficial to the business (and to future users), but that is far from ideal because it makes current users pay for achieving these benefits. Not to mention, as an external dependency, any latency or other performance issues in the service itself will be reflected to the user. If a timeout from your analytics service causes the customer to fail to complete a high-value operation, everyone will lose.

Second, this approach is not as reliable as it initially sounds, as some termination behavior cannot be delayed programmatically. For example, e.preventDefault() is useless in delaying the user to close the browser tab. So, at best, it will only cover the collection of data from certain user actions, but not enough to fully trust it.

Instruct the browser to retain unfinished requests

Thankfully, most browsers have built-in options to retain unfinished HTTP requests without compromising the user experience.

Use Fetch's keepalive logo

If the keepalive flag is set to true when using fetch() , the corresponding request remains open even if the page that initiated the request has been terminated. Using our initial example, this will make the implementation look like this:

 <a href="https://www.php.cn/link/3cbfb2330b21840b385a45c958602663">Go to Page</a>

document.getElementById('link').addEventListener('click', (e) => {
  fetch("/log", {
    method: "POST",
    headers: {
      "Content-Type": "application/json"
    },
    body: JSON.stringify({
      Some: "data"
    }),
    keepalive: true
  });
});

When clicking on the link and page navigation occurs, no request cancellation occurs:

Instead, we get a (unknown) state simply because the active page never waits for any response to be received.

Such single-line code is easy to fix, especially if it is part of a common browser API. However, if you are looking for a more centralized option with a simpler interface, there is another way to do that with nearly the same browser support.

Use Navigator.sendBeacon()

Navigator.sendBeacon() function is specifically used to send one-way requests (beacons). A basic implementation is as follows, which sends a POST with a stringified JSON and a "text/plain" Content-Type :

 navigator.sendBeacon('/log', JSON.stringify({
  Some: "data"
}));

But this API does not allow you to send custom headers. So in order to send our data as "application/json", we need to make a small tweak and use the Blob:

 <a href="https://www.php.cn/link/3cbfb2330b21840b385a45c958602663">Go to Page</a>

document.getElementById('link').addEventListener('click', (e) => {
  const blob = new Blob([JSON.stringify({ some: "data" })], { type: 'application/json; charset=UTF-8' });
  navigator.sendBeacon('/log', blob);
});

Ultimately, we got the same result – the request was completed even after the page was navigated. But there are some things that are happening, which may make it better than fetch() : beacons are sent at low priority.

For demonstration, here is what is displayed in the Network tab when using both fetch() and sendBeacon() with keepalive :

By default, fetch() gets a "high" priority, while beacons (noted as "ping" type above) have a "lowest" priority. This is a good thing for requests that are not important to the page's functionality. Directly from the Beacon Specification:

This specification defines an interface that […] minimizes resource competition with other time-critical operations while ensuring that such requests are still processed and delivered to the destination.

In other words, sendBeacon() ensures that its requests do not hinder requests that are really important to your application and user experience.

Honorary mention of ping attributes

It is worth mentioning that more and more browsers support ping attributes. When attached to the link, it issues a small POST request:

<a href="https://www.php.cn/link/fef56cae0dfbabedeadb64bf881ab64f" ping="http://localhost:3000/log">
  Go to Other Page
</a>

These request headers will contain the page where the link is clicked (ping-from), and the href value of the link (ping-to):

 <code>headers: { 'ping-from': 'http://localhost:3000/', 'ping-to': 'https://www.php.cn/link/fef56cae0dfbabedeadb64bf881ab64f' 'content-type': 'text/ping' // ...其他标头},</code>

It is technically similar to sending beacons, but has some notable limitations:

1. It is strictly limited to use for links, which would be a bad choice if you need to track data related to other interactions such as button clicks or form submissions.
2. The browser supports well, but not great. At the time of writing, Firefox specifically does not enable it by default.
3. You cannot send any custom data with the request. As mentioned before, you can only get a few ping-* headers at most, and any other headers.

All in all, ping is a great tool if you only send simple requests and don't want to write any custom JavaScript. However, if you need to send more content, it may not be the best option.

So, which one should I choose?

There is certainly a trade-off when sending the last-second request using fetch() or sendBeacon() with keepalive . To help tell which approach is best for different situations, here are some things to consider:

You might choose fetch() keepalive in the following cases:

• You need to pass custom headers easily using requests.
• You want to issue a GET request to the service, not a POST request.
• You are supporting older browsers (such as IE) and have fetch polyfill loaded.

However, sendBeacon() might be a better option in the following cases:

• You are making simple service requests that don't require much customization.
• You prefer a more concise and elegant API.
• You want to make sure that your request does not compete with other high priority requests sent in the application.

Avoid repeating the same mistakes

There is a reason why I chose to dig deep into the way the browser handles in-process requests when the page terminates. Not long ago, after we started firing requests immediately when we submitted the form, our team discovered that the frequency of specific types of analytic logs suddenly changed. This change is sudden and significant – down about 30% from what we have seen in history.

Digging into the cause of this problem and the available tools to avoid reappearing the problem saved the day. So if anything, I hope to understand the nuances of these challenges can help someone avoid some of the pain we encounter. Happy record!

The above is the detailed content of Reliably Send an HTTP Request as a User Leaves a Page. For more information, please follow other related articles on the PHP Chinese website!