A brief discussion on large-scale web system architecture

Dynamic applications refer to network application software developed in server-side languages ​​such as c/c++, php, Java, perl, .net, etc., such as forums, online photo albums, dating, BLOG and other common applications . Dynamic application systems are usually inseparable from database systems, cache systems, distributed storage systems, etc.

　 Large-scale dynamic application system platform is mainly aimed at the underlying system architecture established for large-traffic, high-concurrency websites. The operation of large-scale websites requires the support of a reliable, secure, scalable, and easy-to-maintain application system platform to ensure the smooth operation of website applications.

Large-scale dynamic application systems can be divided into several subsystems:

　1) Web front-end system

　2) Load balancing system

　3) Database cluster system

　4) Caching system

　5) Distributed storage system

　 6) Distributed server management system

7) Code distribution system

Web front-end system

Structure diagram:

In order to achieve server sharing for different applications, avoid single points of failure, and centralized management, For the purpose of unified configuration and other purposes, servers are not divided by applications, but all servers are used uniformly. Each server can provide services for multiple applications. When the number of visits to certain applications increases, the entire server cluster can be reached by adding server nodes. The performance improvement will also benefit other applications. The web front-end system is based on virtual host platforms such as Apache/Lighttpd/Eginx, etc., and provides a PHP program running environment. The server is transparent to developers and does not require developers to intervene in server management

Load balancing system

Load balancing systems are divided into two types: hardware and software. Hardware load balancing is highly efficient but expensive, such as F5 and so on. Software load balancing systems are less expensive or free and less efficient than hardware load balancing systems, but they are sufficient for websites with average or slightly larger traffic, such as lvs and nginx. Most websites use a combination of hardware and software load balancing systems.

　 Database cluster system

Structural diagram:

Since the Web front-end adopts a load balancing cluster structure to improve the effectiveness and scalability of the service, the database must also be highly reliable to ensure the high reliability of the entire service system. How to build a highly reliable database system that can provide large-scale concurrent processing?

　We can adopt the solution as shown in the picture above:

　1) Use the MySQL database. Considering the characteristics of Web application databases that read more and write less, we mainly optimize the reading database and provide dedicated reading Database and write database, read operations and write operations are implemented in the application to access different databases respectively.

　2) Use the MySQL Replication mechanism to quickly copy the database of the master database (writing database) to the slave database (reading database). One master database corresponds to multiple slave databases, and data from the master database is synchronized to the slave databases in real time. Dumpling machine www.yjlmj.com Organized and released

　　3) There are multiple writing databases, each of which can be used by multiple applications, which can solve the performance bottleneck problem and single point of failure problem of writing database.

　　4) There are multiple reading databases, and load balancing is achieved through load balancing equipment, thereby achieving high performance, high reliability and high scalability of the reading database.

　5) The database server and application server are separated.

　6) Use BigIP for load balancing from the database.

　Caching system

　Caching is divided into file cache, memory cache and database cache. The most commonly used and most efficient method in large-scale web applications is memory caching. The most commonly used memory caching tool is Memcached. Using the correct caching system can achieve the following goals:

　1. Using the caching system can improve access efficiency, increase server throughput, and improve user experience.

　2. Reduce the access pressure on the database and save set server.

　 3. There are multiple Memcached servers to avoid single points of failure, provide high reliability and scalability, and improve performance.

　Distributed storage system

　Structural diagram:

Web system platformThe storage requirements have the following two characteristics:

　1) The amount of storage is large, often reaching a level that cannot be provided by a single server scale, such as photo albums, videos and other applications. Therefore, professional large-scale storage systems are needed.

　2) Each node in the load balancing cluster may access any data object, and the data processing of each node can also be shared by other nodes. Therefore, the data to be operated by these nodes can only be logically one The whole, not independent data resources.

　Therefore, a high-performance distributed storage system is a very important part for large-scale website applications. (A brief introduction to a distributed storage system needs to be added here.)

Distributed server management system

Structure diagram:

With the continuous increase in website access traffic, most network services are based on Load balancing clusters are used to provide services to the outside world. With the expansion of the cluster size, the original server management model based on a single machine can no longer meet our needs. The new needs must be able to centralize, group, batch, and automatically manage servers. Manage and execute planned tasks in batches.

　There are some excellent software among distributed server management system software, and one of the more ideal ones is Cfengine. It can group servers, and different groups can customize system configuration files, scheduled tasks and other configurations. It is based on the C/S structure. All server configuration and management script programs are saved on the Cfengine Server, and the managed server runs the Cfengine Client program. The Cfengine Client regularly sends requests to the server through an SSL encrypted connection. Get the latest configuration files and management commands, scripts, patch installation and other tasks.

With Cfengine, a centralized server management tool, we can efficiently implement large-scale server cluster management. The managed server and Cfengine Server can be distributed in any location. As long as the network can be connected, rapid and automated management can be achieved. .

　 Code release system

Structural diagram:

With the continuous increase in website access traffic, most network services are provided externally in the form of load balancing clusters. With the expansion of the cluster scale, in order to meet the needs of the cluster For batch distribution and update of program code in the environment, we also need a program code release system.

　This publishing system can help us achieve the following goals:

　1) The server in the production environment provides services in the form of a virtual host, which does not require developers to intervene in maintenance and direct operation. The publishing system can be used to implement the server without logging in. The program is distributed to the target server.

　2) We need to realize the management of the four development stages of internal development, internal testing, production environment testing, and production environment release. The release system can intervene in the code release at each stage.

　3) We need to implement source code management and version control, and SVN can achieve this requirement.

　 Here you can use the commonly used tool Rsync to achieve code synchronization distribution between server clusters by developing corresponding script tools.

The above has introduced a brief discussion of large-scale web system architecture, including various aspects. I hope it will be helpful to friends who are interested in PHP tutorials.