Comprehensive analysis of memcached – 5. memcached applications and compatible programs_PHP tutorial

Original link: http://gihyo.jp/dev/feature/01/memcached/0005
 href="http://gihyo.jp/dev/feature/01/memcached/0005">

The link to this series of articles is at Here:

• The 1st time: http://www.phpchina.com/html/29/n-35329.html
• The 2nd time: http://www. phpchina.com/html/30/n-35330.html
• The 3rd time: http://www.phpchina.com/html/31/n-35331.html
• The 4th time The fifth time: http://www.phpchina.com/html/32/n-35332.html href="http://www.phpchina.com/html/32/n-35332.html">
• The fifth time: http://www.phpchina.com/html/32/n- 35333.html href="http://www.phpchina.com/html/32/n-35333.html"> href="http://www.phpchina.com/html/32/n-35332.html">

I am Nagano from Mixi. The serialization of memcached is finally coming to an end. Until last time, we have introduced topics directly related to memcached. This time we will introduce some mixi cases and practical application topics, and introduce some programs compatible with memcached.  href="http://tech.idv2.com/2008/07/24/memcached-004/">

 id="more-614">

• mixi case study
  • Server configuration and number href="#content_2_1">
  • memcached process href="#content_2_2">
  • memcached usage Methods and clients
    • Maintain connection through Cache::Memcached::Fast
    • Public data processing and rehash
• memcached application experience
  • Start via daemontools
  • Monitor
  • memcached performance
• Compatible applications
  • Tokyo Tyrant case
• Summary

 class="list1">

mixi case study

mixi uses memcached in the early stages of providing services. With the rapid increase in website visits, simply adding a slave to the database can no longer meet the needs, so memcached was introduced. In addition, we have also verified from the aspect of increasing scalability, proving that the speed and stability of memcached can meet the needs. Now, memcached has become a very important part of the mixi service.

Figure 1 Current system components

Server configuration and number

mixi uses many servers, such as database server, application server, picture server, reflection server To the proxy server, etc. Memcached alone has nearly 200 servers running. The typical configuration of the memcached server is as follows:

• CPU: Intel Pentium 4 2.8GHz
• Memory: 4GB
• Hard disk: 146GB SCSI
• Operating system :Linux (x86_64)

These servers were previously used for database servers etc. As CPU performance improves and memory prices fall, we actively replace database servers and application servers with servers with more powerful performance and more memory. In this way, the sharp increase in the number of servers used by mixi as a whole can be suppressed and management costs can be reduced. Since the memcached server takes up almost no CPU, the replaced server is used as the memcached server.

memcached process

Each memcached server starts only one memcached process. The memory allocated to memcached is 3GB, and the startup parameters are as follows:

/usr/bin/memcached -p 11211 -u nobody -m 3000 -c 30720

Due to the x86_64 operating system, more than 2GB of memory can be allocated. In 32-bit operating systems, each process can only use up to 2GB of memory. I have also considered starting multiple processes that allocate less than 2GB of memory, but this will double the number of TCP connections on one server and make management complicated, so mixi uses a 64-bit operating system.

In addition, although the server has 4GB of memory, only 3GB is allocated. This is because memory allocation exceeding this value may cause memory swap (swap). In the second serialization, Naka Maesaka explained the memory storage "slab allocator" of memcached. At that time, he said that the memory allocation amount specified when memcached starts is the amount used by memcached to save data, and does not include the memory occupied by the "slab allocator" itself. , and a management space set up to save data. Therefore, the actual memory allocation of the memcached process is larger than the specified capacity, which should be noted.

Most of the data mixedi saves in memcached is relatively small. In this way, the size of the process is much larger than the specified capacity. Therefore, we repeatedly changed the memory allocation amount to verify, and confirmed that the 3GB size will not cause swap. This is the value currently applied.

Memcached usage methods and clients

Now, mixi’s service uses about 200 memcached servers as a pool. The capacity of each server is 3GB, so the entire server has a huge memory database of nearly 600GB. The client library uses Cache::Memcached::Fast, which is mentioned many times in this series, to interact with the server. Of course, the distributed algorithm of the cache uses the Consistent Hashing algorithm introduced in the fourth time.

• Cache::Memcached::Fast - search.cpan.org

应用层上memcached的使用方法由开发应用程序的工程师自行决定并实现。 但是，为了防止车轮再造、防止Cache::Memcached::Fast上的教训再次发生， 我们提供了Cache::Memcached::Fast的wrap模块并使用。

通过Cache::Memcached::Fast维持连接

Cache::Memcached的情况下，与memcached的连接（文件句柄）保存在Cache::Memcached包内的类变量中。 在mod_perl和FastCGI等环境下，包内的变量不会像CGI那样随时重新启动， 而是在进程中一直保持。其结果就是不会断开与memcached的连接， 减少了TCP连接建立时的开销，同时也能防止短时间内反复进行TCP连接、断开 而导致的TCP端口资源枯竭。

但是，Cache::Memcached::Fast没有这个功能，所以需要在模块之外 将Cache::Memcached::Fast对象保持在类变量中，以保证持久连接。

package Gihyo::Memcached; use strict; use warnings; use Cache::Memcached::Fast; my @server_list = qw/192.168.1.1:11211 192.168.1.1:11211/; my $fast; ## 用于保持对象 sub new { my $self = bless {}, shift; if ( !$fast ) { $fast = Cache::Memcached::Fast->new({ servers => \@server_list }); } $self->{_fast} = $fast; return $self; } sub get { my $self = shift; $self->{_fast}->get(@_); }

上面的例子中，Cache::Memcached::Fast对象保存到类变量$fast中。

公共数据的处理和rehash

诸如mixi的主页上的新闻这样的所有用户共享的缓存数据、设置信息等数据， 会占用许多页，访问次数也非常多。在这种条件下，访问很容易集中到某台memcached服务器上。 访问集中本身并不是问题，但是一旦访问集中的那台服务器发生故障导致memcached无法连接， 就会产生巨大的问题。

连载的第4次 中提到，Cache::Memcached拥有rehash功能，即在无法连接保存数据的服务器的情况下， 会再次计算hash值，连接其他的服务器。

但是，Cache::Memcached::Fast没有这个功能。不过，它能够在连接服务器失败时， 短时间内不再连接该服务器的功能。

my $fast = Cache::Memcached::Fast->new({ max_failures => 3, failure_timeout => 1 });

在failure_timeout秒内发生max_failures以上次连接失败，就不再连接该memcached服务器。 我们的设置是1秒钟3次以上。

此外，mixi还为所有用户共享的缓存数据的键名设置命名规则， 符合命名规则的数据会自动保存到多台memcached服务器中， 取得时从中仅选取一台服务器。创建该函数库后，就可以使memcached服务器故障 不再产生其他影响。

memcached应用经验

到此为止介绍了memcached内部构造和函数库，接下来介绍一些其他的应用经验。

通过daemontools启动

通常情况下memcached运行得相当稳定，但mixi现在使用的最新版1.2.5 曾经发生过几次memcached进程死掉的情况。架构上保证了即使有几台memcached故障 也不会影响服务，不过对于memcached进程死掉的服务器，只要重新启动memcached， 就可以正常运行，所以采用了监视memcached进程并自动启动的方法。 于是使用了daemontools。

daemontools是qmail的作者DJB开发的UNIX服务管理工具集， 其中名为supervise的程序可用于服务启动、停止的服务重启等。

• daemontools href="http://cr.yp.to/daemontools.html">

这里不介绍daemontools的安装了。mixi使用了以下的run脚本来启动memcached。

#!/bin/sh if [ -f /etc/sysconfig/memcached ];then . /etc/sysconfig/memcached fi exec 2>&1 exec /usr/bin/memcached -p $PORT -u $USER -m $CACHESIZE -c $MAXCONN $OPTIONS

监视

mixi使用了名为“nagios”的开源监视软件来监视memcached。

• Nagios: Home

在nagios中可以简单地开发插件，可以详细地监视memcached的get、add等动作。 不过mixi仅通过stats命令来确认memcached的运行状态。

define command { command_name check_memcached command_line $USER1$/check_tcp -H $HOSTADDRESS$ -p 11211 -t 5 -E -s 'stats\r\nquit\r\n' -e 'uptime' -M crit }

此外，mixi将stats目录的结果通过rrdtool转化成图形，进行性能监视， 并将每天的内存使用量做成报表，通过邮件与开发者共享。

memcached的性能

连载中已介绍过，memcached的性能十分优秀。我们来看看mixi的实际案例。 这里介绍的图表是服务所使用的访问最为集中的memcached服务器。



图2 请求数

 src="http://tech.idv2.com/wp-content/uploads/2008/07/Comprehensive analysis of memcached – 5. memcached applications and compatible programs_PHP tutorial">

图3 流量

 src="http://tech.idv2.com/wp-content/uploads/2008/07/Comprehensive analysis of memcached – 5. memcached applications and compatible programs_PHP tutorial">

图4 TCP连接数

From top to bottom are the number of requests, traffic and TCP connections. The maximum number of requests is 15,000qps, the traffic reaches 400Mbps, and the number of connections at this time has exceeded 10,000. This server has no special hardware and is just the ordinary memcached server introduced at the beginning. The CPU utilization at this time is:



Figure 5 CPU utilization

It can be seen that there is still an idle part. Therefore, memcached's performance is very high and can be used as a safe place for web application developers to save temporary or cached data.

Compatible Applications

The implementation and protocol of memcached are very simple, so there are many memcached-compatible implementations. Some powerful extensions can write memcached memory data to disk to achieve data persistence and redundancy. As introduced in the third serialization, the future memcached storage layer will become pluggable and gradually support these functions.

Here are several applications compatible with memcached.

repcached

A patch that provides replication function for memcached.

Flared

is stored in QDBM. At the same time, functions such as asynchronous replication and fail over are implemented.

memcachedb

is stored in BerkleyDB. A message queue is also implemented.

Tokyo Tyrant

Store data to Tokyo Cabinet. Not only is it compatible with the memcached protocol, it can also be accessed via HTTP.

Tokyo Tyrant Case

mixi uses Tokyo Tyrant from the above compatible apps. Tokyo Tyrant is a network interface for Tokyo Cabinet DBM developed by Hirabayashi. It has its own protocol, but it also has a memcached-compatible protocol and can also exchange data over HTTP. Although Tokyo Cabinet is an implementation that writes data to disk, it is quite fast.

mixi does not use Tokyo Tyrant as a cache server, but as a DBMS that saves key-value pair combinations. Mainly used as a database to store the user's last access time. It is related to almost all mixi services, and the data needs to be updated every time the user visits the page, so the load is quite high. MySQL processing is very cumbersome, and using memcached alone to save data may cause data loss, so Tokyo Tyrant was introduced. But there is no need to re-develop the client, you can just use Cache::Memcached::Fast unchanged, which is also one of the advantages. For detailed information about Tokyo Tyrant, please refer to our company's development blog.

• mixi Engineers' Blog - Tokyo Tyrant's high-load-resistant DB construction 🎜>Summary
• So far, the "Comprehensive Analysis of Memcached" series is over. We introduced the foundation, internal structure, decentralized algorithm and application of memcached. It would be our honor if you become interested in memcached after reading this. For information about mixi’s system and applications, please refer to our company’s development blog. Thank you for reading.

Copyright Statement

: You can
reprint

at will, but the original author charlee, the original link and this statement must be noted when reprinting.

 id="content_2_5"> id="content_2_3"> id="content_2_2">http://www.bkjia.com/PHPjc/735130.html

 class="list1">www.bkjia.com

truehttp: //www.bkjia.com/PHPjc/735130.htmlTechArticleOriginal link: http://gihyo.jp/dev/feature/01/memcached/0005 This series of articles The link is here: The 1st time: http://www.phpchina.com/html/29/n-35329.html The 2nd time: http://www.php...