Using Background Processing to Speed Up Page Load Times

Core points

• Background processing significantly improves page loading speed: Transfer heavy tasks to background processing, releases main thread resources, and focuses on page loading, thereby improving user browsing experience.
• The two core components of background processing: Task queue and worker process. The application creates pending tasks, while the worker processes extract tasks one by one from the queue for processing. Multiple worker processes can be processed in parallel to speed up.
• Technical stack selection: Use Beanstalkd task queue to store tasks, Symfony Console component to implement worker processes (as console commands), and Supervisor manages worker processes.
• Image library optimization: Avoid image scaling on the first request (which increases the initial load burden), but instead render thumbnails asynchronously after the gallery is created. This task can be moved to the background to improve user experience and scalability.
• Potential Challenges and Responses: Make sure that the tasks are successfully completed in the correct order and that errors in the background tasks are properly handled. This can be mitigated by using task queues and robust error handling and logging mechanisms.

This article is part of a series of articles about building sample applications (Multi-Picture Blog) for performance benchmarking and optimization. (View the code base here)

In previous articles, we added on-demand image scaling feature. Images are scaled and cached for later use when requested for the first time. This is convenient, but also increases the overhead of first loading; the system must render the thumbnails dynamically and "block" the first user's page rendering before image rendering is completed.

The optimization solution is to render thumbnails after creating the gallery. You might be thinking, "Okay, but this will block the user who created the gallery?" This not only leads to a bad user experience, but it is also not a scalable solution. Users will be confused about longer load times, or worse, if the image is too large to process, you will encounter a timeout and/or error. The best solution is to move these heavy tasks to the background.

Background Tasks

Background tasks are the best way to handle any heavy tasks. We can immediately notify the user that we have received their request and arrange for processing. The same is true for YouTube to upload videos: the video is not accessible after uploading. Users need to wait until the video is fully processed before previewing or sharing.

Processing or generating files, sending emails, or any other non-critical tasks should be done in the background.

How to work in backend processing

The background processing method has two key components: task queue and worker process. The application creates tasks that need to be processed, while the worker waits and gets one task from the queue at a time.

You can create multiple worker processes (processes) to speed up processing, break down large tasks into smaller chunks and process them simultaneously. You can organize and manage backend processing according to your needs, but be aware that parallel processing is not an easy task: you should pay attention to potential race conditions and handle failed tasks gracefully.

Our technology stack

We use the Beanstalkd task queue to store tasks, use the Symfony Console component to implement the worker process as a console command, and use Supervisor to manage the worker process.

If you use Homestead Improved, Beanstalkd and Supervisor are installed, so you can skip the installation instructions below.

Instalkd

Beanstalkd is a fast job queue with a common interface that was originally designed to reduce latency in page browsing in high-traffic web applications by running time-consuming tasks asynchronously.

You can use many available client libraries. In our project, we are using Pheanstalk.

To install Beanstalkd on your Ubuntu or Debian server, just run sudo apt-get install beanstalkd. Check out the official download page to learn how to install Beanstalkd on other operating systems.

After installation, Beanstalkd starts as a daemon, waiting for the client to connect and create (or process) the job:

<code>/etc/init.d/beanstalkd
Usage: /etc/init.d/beanstalkd {start|stop|force-stop|restart|force-reload|status}</code>

Install Pheanstalk as a dependency by running composer require pda/pheanstalk.

The queue will be used to create and fetch jobs, so we centralize job creation in one factory service: JobQueueFactory

<code class="language-php"><?php
namespace App\Service;

use Pheanstalk\Pheanstalk;

class JobQueueFactory
{
    private $host = 'localhost';
    private $port = '11300';

    const QUEUE_IMAGE_RESIZE = 'resize';

    public function createQueue(): Pheanstalk
    {
        return new Pheanstalk($this->host, $this->port);
    }
}</code>

Now, we can inject factory services as needed to interact with the Beanstalkd queue. We define the queue name as a constant and reference it when putting the job into the queue or monitoring the queue in the worker process.

Installing Supervisor

According to the official page, Supervisor is a client/server system that allows its users to monitor and control many processes on the Unix operating system.

We will use it to start, restart, expand and monitor worker processes.

Install Supervisor on your Ubuntu/Debian server by running

. After installation, Supervisor will run in the background as a daemon. Use sudo apt-get install supervisor to control the Supervisor process: supervisorctl

<code>$ sudo supervisorctl help

default commands (type help <topic>):
</topic>=====================================
add    exit      open  reload  restart   start   tail
avail  fg        pid   remove  shutdown  status  update
clear  maintail  quit  reread  signal    stop    version</code>

To use Supervisor to control a process, we first have to write a configuration file and describe how we want to control our process. The configuration is stored in

. A simple Supervisor configuration for resizing worker processes is as follows: /etc/supervisor/conf.d/

<code>[program:resize-worker]
process_name=%(program_name)s_%(process_num)02d
command=php PATH-TO-YOUR-APP/bin/console app:resize-image-worker
autostart=true
autorestart=true
numprocs=5
stderr_logfile = PATH-TO-YOUR-APP/var/log/resize-worker-stderr.log
stdout_logfile = PATH-TO-YOUR-APP/var/log/resize-worker-stdout.log</code>

We tell Supervisor how to name the generated process, the path to the commands to be run, the process automatically started and restarted, how many processes we want, and where to record the output. Learn more about Supervisor configuration here.

Resize the image in the background

Once our infrastructure is set up (i.e., Beanstalkd and Supervisor are installed), we can modify our application to resize the image in the background after creating the gallery. To do this, we need:

• Update the image service logic in ImageController
Implement the resized worker process as a console command
Create Supervisor configuration for our worker process
Update the device and resize the image in the device class.

Update picture service logic

So far, we have been resizing the image on the first request: if the image file requested does not exist, it will be created dynamically.

ImageControllerWe will now modify

to return the requested image response only if the resized image file exists (i.e. the image has been resized).

If it does not exist, the application will return a common placeholder image response indicating that the image is being resized. Note that the placeholder image response has different cache control headers because we don't want to cache the placeholder image; we want the image to be rendered immediately after the resizing process is complete.

GalleryCreatedEventWe will create a simple event called UploadController with the payload as the Gallery ID. After the gallery is successfully created, this event will be dispatched in

:

<code>/etc/init.d/beanstalkd
Usage: /etc/init.d/beanstalkd {start|stop|force-stop|restart|force-reload|status}</code>

In addition, we will update the flash message using "Images are now being processed."

so that users know that we need to do some processing on their images before we are ready.

GalleryEventSubscriberWe will create GalleryCreatedEvent event subscriber, which will react to

and request resize jobs for each image in the newly created gallery:

<code class="language-php"><?php
namespace App\Service;

use Pheanstalk\Pheanstalk;

class JobQueueFactory
{
    private $host = 'localhost';
    private $port = '11300';

    const QUEUE_IMAGE_RESIZE = 'resize';

    public function createQueue(): Pheanstalk
    {
        return new Pheanstalk($this->host, $this->port);
    }
}</code>

Now, when the user successfully creates the gallery, the application will render the gallery page, but the thumbnails of some images are still not ready, so they will not be displayed:

Once the worker process has completed resizing, the complete gallery page should be rendered next refresh.

Implement the resized worker process as a console command

$queue->reserve()The worker process is a simple process that performs the same job for each job taken from the queue. The execution of the worker process is blocked at the

call until the worker keeps the job, or a timeout occurs.

Only one worker process can obtain and process jobs. Jobs usually contain payloads, such as strings or serialized arrays/objects. In our example, it will be the UUID of the created gallery.

A simple worker looks like this:

<code>/etc/init.d/beanstalkd
Usage: /etc/init.d/beanstalkd {start|stop|force-stop|restart|force-reload|status}</code>

You may have noticed that the worker process will exit after a defined timeout or after a job is processed. We can wrap the worker process logic in an infinite loop and make it repeat its jobs indefinitely, but this can cause problems such as database connection timeout after long inactivity and making deployment more difficult. To prevent this, our worker process lifecycle will end after a single task is completed. Supervisor then restarts the worker process as a new process.

View ResizeImageWorkerCommand to understand the structure of Worker commands. Working processes implemented in this way can also be manually started as the Symfony console command: ./bin/console app:resize-image-worker.

Create Supervisor configuration

We want our worker process to start automatically, so we will set the autostart=true directive in the configuration. Since the worker process must be restarted after a timeout or successful task processing, we will also set the autorestart=true directive.

The best part of backend processing is the ease of parallel processing. We can set the numprocs=5 directive, and Supervisor will generate five worker process instances. They will wait for jobs and process them independently, allowing us to easily scale the system. As the system evolves, you may need to increase the number of processes. Since we will have multiple processes running, we need to define the structure of the process name, so we set the process_name=%(program_name)s_%(process_num)02d directive.

Last but not least, we want to store the output of worker processes so that they can be analyzed and debugged when problems arise. We will define the stderr_logfile and stdout_logfile paths.

The full Supervisor configuration for our resize worker process is as follows:

<code class="language-php"><?php
namespace App\Service;

use Pheanstalk\Pheanstalk;

class JobQueueFactory
{
    private $host = 'localhost';
    private $port = '11300';

    const QUEUE_IMAGE_RESIZE = 'resize';

    public function createQueue(): Pheanstalk
    {
        return new Pheanstalk($this->host, $this->port);
    }
}</code>

After creating (or updating) a configuration file located in the /etc/supervisor/conf.d/ directory, you must tell Supervisor to reread and update its configuration by executing the following command:

<code>$ sudo supervisorctl help

default commands (type help <topic>):
</topic>=====================================
add    exit      open  reload  restart   start   tail
avail  fg        pid   remove  shutdown  status  update
clear  maintail  quit  reread  signal    stop    version</code>

If you use Homestead Improved (you should use!), you can use scripts/setup-supervisor.sh to generate a Supervisor configuration for the project: sudo ./scripts/setup-supervisor.sh.

Update device

Image thumbnails will no longer be rendered on the first request, so when we load the device in the LoadGalleriesData device class, we need to explicitly request rendering for each image:

<code>[program:resize-worker]
process_name=%(program_name)s_%(process_num)02d
command=php PATH-TO-YOUR-APP/bin/console app:resize-image-worker
autostart=true
autorestart=true
numprocs=5
stderr_logfile = PATH-TO-YOUR-APP/var/log/resize-worker-stderr.log
stdout_logfile = PATH-TO-YOUR-APP/var/log/resize-worker-stdout.log</code>

Now you should feel the device loading slower, which is why we moved it to the background instead of forcing the user to wait for it to finish!

Tips and Tips

The worker process runs in the background, so even if you deploy a new version of the application, you will still have an outdated worker process running before the first restart.

In our case, we have to wait for all worker processes to complete their tasks or time out (5 minutes) until we are sure that all worker processes have been updated. Be aware of this when creating the deployment process!

FAQs on using background processing to speed up page loading time (FAQ)

What role does background processing play in speeding up page loading?

Background processing plays a crucial role in improving page loading speed. It allows certain tasks to be performed in the background, thereby freeing up the main thread resources and focusing on page loading. This means that users don't have to wait for these tasks to complete to load the page, resulting in a faster and smoother browsing experience.

How does Symfony Process components assist in background processing?

The Symfony Process component is a powerful tool that allows you to execute commands in child processes. It provides a simple object-oriented API for running system commands and managing its output. This is especially useful for background processing, as it allows you to run tasks in separate processes without blocking the main thread.

What are some common use cases for backend processing?

Background processing is usually used in situations where tasks can be executed without regard to the main thread. This includes tasks such as sending emails, processing pictures, running complex calculations, and more. By running these tasks in the background, you can improve the performance of your application and provide a better user experience.

How to execute background processes in PHP?

The exec() function can be used to execute background processes in PHP. This function allows you to run commands in a child process and then continue to execute the rest of the script without waiting for the command to complete. Here is a simple example:

exec("php background_task.php > /dev/null &");

In this example, background_task.php is the script you want to run in the background.

What is the Symfony Messenger component and what does it have to do with background processing?

The Symfony Messenger component is a message bus system that can be used to asynchronously dispatch messages to handlers. This means you can send messages to the bus and then continue executing the script without waiting for the message to be processed. This is a form of background processing, because the processing of messages can be done in a separate process.

How to use backend processing to improve the performance of your website?

By unloading the task to the background, you can free up the main thread resource and focus on loading the page. This can significantly improve the performance of your website, especially if you have time-consuming or resource-intensive tasks. Some common tasks that can be offloaded to the background include sending emails, processing pictures, and running complex calculations.

What are the potential challenges of backend processing and how to mitigate these challenges?

One of the main challenges of backend processing is to ensure that the tasks are successful and completed in the correct order. This can be mitigated by using a task queue, which ensures that tasks are executed in the order they are added. Another challenge is handling errors in background tasks. This can be solved by implementing robust error handling and logging mechanisms.

Can background processing be used in combination with other performance optimization techniques?

Yes, background processing can be used in conjunction with other performance optimization techniques. For example, you can use caches to store the results of expensive operations and then use background processing to update the cache periodically. This allows you to provide the latest data without slowing down the application.

How to monitor the progress of background tasks?

A variety of tools and techniques can be used to monitor the progress of backend tasks. A common approach is to use logging to record the status of each task. You can also use tools like Symfony's Messenger component, which provides built-in support for monitoring and debugging backend tasks.

Are there any security precautions when using background processing?

Yes, there are some security precautions when using background processing. For example, you need to make sure that background tasks do not leak sensitive information and are not subject to injection attacks. You should also make sure your tasks run in a secure environment and that they don't have more permissions than they need to do their job.

The above is the detailed content of Using Background Processing to Speed Up Page Load Times. For more information, please follow other related articles on the PHP Chinese website!