search

Home >Backend Development >PHP Tutorial >nginx+tomcat+memcached configuration+script

nginx+tomcat+memcached configuration+script

WBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWB
WBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOriginal
2016-08-08 09:29:211081browse

研究了下nginx+tomcat+memcached 的配置，练习了下脚本的书写，参考了好几篇文章，在此记录一下，有不对的地方敬请指正。

系统环境：
RHEL6.5 x64
iptables -F & selinux is disabled

主机角色：
node1 :192.168.122.101 :nginx tomcat memcached
node2 :192.168.122.102 :tomcat memcached

https://code.google.com/memcached-session-manager
memcached的session管理

nginx做反向代理两台tomcat，用memcached同步session，防止数据丢失

Tomcat1将session存储在memcacted2上.tomcat和Memcached使用交叉存储,只有当M2不可用时T1才将数据存储在M1上(M1是T1的failoverNode),使用这种方式可以避免单点故障.从而实现应用的高可用性.

注意:两台node上的tomcat应完全一致

1.在两台node主机上配置tomcat环境

<code><span>#./jdk-6u32-linux-x64.bin</span><span># mv jdk1.6.0_32/ /usr/local/lnmp/jdk</span><span>#vim /etc/profile              编写环境变量</span>export JAVA_HOME=/usr/local/lnmp/jdk

export CLASSPATH=:$JAVA_HOME/lib:$JAVA_HOME/jre/lib

export PATH=$PATH:$JAVA_HOME/bin

<span>#source /etc/profile</span>测试java能否正常工作
<span>#vim test.java</span><span>public</span><span><span>class</span><span>test</span>{</span><span>public</span><span>static</span><span>void</span> main(String[] args)

{System.out.println(<span>"Hello!"</span>);  }

}

<span>#javac test.java          编译</span><span>#java test               执行后出现Hello!说明java环境配置好</span></code>

安装tomcat服务器

<code><span>#tar zxf apache-tomcat-7.0.37.tar.gz      解压就能用,不需要编译</span><span>#mv apache-tomcat-7.0.37 tomcat</span><span>#/TOMCAT_ROOT_DIR/tomcat/webapps/ROOT                tomcat的默认发布目录</span><span>#/TOMCAT_ROOT_DIR/tomcat/bin/startup.sh(shutdown.sh)    tomcat默认的启动和关闭脚本</span></code>

tomcat默认开启8080端口,

测试http://192.168.122.101:8080 访问到tomcat默认的测试页

<code><span>#cd tomcat/webapps/ROOT</span><span>#cat test.jsp           #测试页</span><span>this</span> time is: new java.util.Date()%></code>

测试访问;http://192.168.122.101:8080 显示当前时间

tomcat访问8080端口,使用nginx反向代理。

使用nginx发布jsp动态网页:

nginx配置文件:

cat /usr/local/nginx/conf/nginx.conf

<code><span>#user  nginx nginx;</span>
worker_processes  <span>4</span>;

<span>#error_log  logs/error.log;</span><span>#error_log  logs/error.log  notice;</span><span>#error_log  logs/error.log  info;</span><span>#pid        logs/nginx.pid;</span>events {
    use epoll;
    worker_connections  <span>1024</span>;
}


http {
    upstream tomcat {  <span>#两台tomcat负载均衡</span>
    sticky;               <span>#session同步，为nginx的模块</span><span>server</span><span>192.168</span><span>.0</span><span>.1</span>:<span>8080</span>;
        <span>server</span><span>192.168</span><span>.0</span><span>.2</span>:<span>8080</span>;
    }

    include       mime.types;
    default_type  application/octet-stream;

    <span>#log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '</span><span>#                  '$status $body_bytes_sent "$http_referer" '</span><span>#access_log  logs/access.log  main;</span>    sendfile        on;
    <span>#tcp_nopush     on;</span><span>#keepalive_timeout  0;</span>
    keepalive_timeout  <span>65</span>;

    <span>#gzip  on;</span><span>server</span> {
        listen       <span>80</span>;
        server_name  localhost;

        <span>#charset koi8-r;</span><span>#access_log  logs/host.access.log  main;</span>        location / {
            root   html;
            proxy_pass   http:<span>//tomcat;</span><span>index</span><span>index</span>.html <span>index</span>.htm;
        }

        <span>#error_page  404              /404.html;</span><span># redirect server error pages to the static page /50x.html</span>
        error_page   <span>500</span><span>502</span><span>503</span><span>504</span>  /<span>50</span>x.html;
        location = /<span>50</span>x.html {
            root   html;
        }

        location ~ \.jsp$ {
            proxy_pass   http:<span>//tomcat;</span>
        }
}

nginx -t && nginx -s reload</code>

访问:http://192.168.122.101/test.jsp 测试

同步两台tomcat的内容，并且修改java环境变量/etc/profile

测试：http://192.168.122.102:8080/test.jsp

两台node(tomcat)准备好了

2、在node1上的nginx环境中发布网页

使用nginx的负载均衡功能,

体现在nginx.conf中：

<code><span>upstream</span> tomcat-lb {

　　<span>server</span><span>192.168.122.101:8080</span>;

　　<span>server</span><span>192.168.122.102:8080</span>;

}

<span>location</span><span>~ \.jsp$</span> {

　　<span>proxy_pass</span><span>http://tomcat-lb</span>;

}

<span>nginx</span> -t && nginx -s reload</code>

测试：http：//192.168.122.101/test.jsp
实现两台主机的负载均衡(访问nginx所在的node)

问题：在后台实现了nginx的负载均衡，但是当一个用户刷新数据时，数据总是变化的，试想，在动态页面，如果用户提交数据刷新后并没有提交到服务器上，并且必须要重新填写表格，这样会造成糟糕的用户体验。

解决:给nginx增加一个sticky模块.(需要重新编译nginx)

重新在nginx在进行模块化编译：

nginx-sticky-modules.tar.gz(解压就行)

<code><span>#tar zxf nginx-sticky-modules.tar.gz -C /root/nginx-1.4.2</span><span>#cd nginx-1.4.2/</span><span>#make clean</span><span>#./configure --prefix=/usr/local/nginx --add-module=$NGINX_PKG_DIR/$DIR/nginx-sticky-module-1.0 --with-http_ssl_module --with-http_stub_status_module</span><span>#make && make install</span><span>#vim nginx.conf</span>upstream linux {
　　sticky;   #装了nginx-sticky后的功能
　　server <span>192.168</span><span>.122</span><span>.101</span>:<span>8080</span> ;
　　server <span>192.168</span><span>.122</span><span>.102</span>:<span>8080</span> ;
}</code>

测试：http://192.168.122.101/test.jsp 后发现刷新不会来回负载(每个用户看到的其实不是一个tomcat上的数据,但是对用户是透明的)

3.nginx负载tomcat的jsp时,需要解决session共享:

使用memcache进行缓存(用户)后端数据,但是又要想到解决单点故障问题,因而可以采用两台memcache作为后端负载.

memcached默认端口11211，后端使用交叉存储(tomcat会将session同步，session自动寻找存储的memcached，但是默认是交叉存储，当一个memcached服务器坏掉，tomcat都会存到存活的memcached服务器上)

只要tomcat不宕掉,一切数据都还存在

但是当memcached宕掉,tomcat会向存活的memcached上存取

session 的序列化方案官方推荐的有 4 种:

  1. java serialization

  2. msm-kryo-serializer

  3. msm-javolution-serializer

  4. msm-xstream-serializer

其中性能最好的是Kryo,我们使用kryo来做

mecached服务器node1 and node2

<code><span>#yum install memcached -y</span><span>#/etc/init.d/memcached start</span></code>

memcached默认开启11211端口

测试访问:telnet localhost 11211

stats 查看状态
set user 0 0 3 存储新值(add replace)
get user 获取值

配置两台memcached服务器(tomcat),下载jar包(必须支持相关的java程序)
提供测试环境的包，解压即可用：
链接: http://pan.baidu.com/s/1mgIF9NU 密码: n5wu

asm-3.2.jar
couchbase-client-1.2.2.jar
kryo-1.03.jar
kryo-serializers-0.11.jar
memcached-session-manager-1.6.5.jar
memcached-session-manager-tc7-1.6.5.jar
minlog-1.2.jar
msm-kryo-serializer-1.6.5.jar
reflectasm-0.9.jar
spymemcached-2.10.3.jar

node1 and node2 同步内容

<code><span>#cd tomcat/lib/</span><span>#mget jar/*    下载tomcat的session共享管理包</span><span>#cd tomcat/conf/</span><span># vim context.xml</span><manager classname="<span">"de.javakaffee.web.msm.MemcachedBackupSessionManager"
memcachedNodes=<span>"n1:192.168.122.101:11211,n2:192.168.122.102:11211"</span>
failoverNodes=<span>"n1"</span>    #tomcat2需要写成n2
requestUriIgnorePattern=<span>".*\.(ico|png|gif|jpg|css|js)$"</span> transcoderFactoryClass=<span>"de.javakaffee.web.msm.serializer.kryo.KryoTranscoderFactory"</span>
/>


<span>#tomcat/bin/shutdown.sh #重启tomcat,以识别memcached-session-manager</span><span># tail -f logs/catalina.out    #默认日志</span>INFO: MemcachedSessionService finished initialization, sticky <span>true</span>, operation timeout <span>1000</span>, with node ids [n2] and failover node ids [n1]</manager></code>

正常启动

编辑测试文件:(提交用户信息的jsp测试页)

<code># vim tomcat/webapps/ROOT/test.jsp             

<span>"text/html; charset=GBK"</span> %><span>"java.util.*"</span> %><span>html</span>><span>head</span>><span>title</span>>Cluster App Test<span><span>title</span>></span><span><span>head</span>></span><span>body</span>>Server Info:

<span>request</span>.getLocalAddr() + <span>" : "</span> + <span>request</span>.getLocalPort()+<span>"<br>"</span>);%><span>"<br> ID "</span> + session.getId()+<span>"<br>"</span>);

<span>String</span> dataName = <span>request</span>.getParameter(<span>"dataName"</span>);

<span>if</span> (dataName != <span>null</span> && dataName.length() > <span>0</span>) {

<span>String</span> dataValue = <span>request</span>.getParameter(<span>"dataValue"</span>);

session.setAttribute(dataName, dataValue);

}

out.print(<span>"<b>Session list</b>"</span>);

Enumeration e = session.getAttributeNames();

<span>while</span> (e.hasMoreElements()) {

<span>String</span> name = (<span>String</span>)e.nextElement();

<span>String</span> value = session.getAttribute(name).toString();

out.println( name + <span>" = "</span> + value+<span>"<br>"</span>);

System.out.println( name + <span>" = "</span> + value);

}

%><span>form</span><span>action</span>=<span>"test.jsp"</span><span>method</span>=<span>"POST"</span>>name:<span>input</span><span>type</span>=<span>text</span><span>size</span>=<span>20</span><span>name</span>=<span>"dataName"</span>><span>br</span>>key:<span>input</span><span>type</span>=<span>text</span><span>size</span>=<span>20</span><span>name</span>=<span>"dataValue"</span>><span>br</span>><span>input</span><span>type</span>=<span>submit</span>><span><span>form</span>></span><span><span>body</span>></span><span><span>html</span>></span></code>

session共享完成:两台tomcat和memcached做相同的配置(jdk,memcache,tomcat)

测试:

http://192.168.122.101/test.jsp
使用一台tomcat和另外一台的memcached进行session共享

任何一台tomcat或者memcached挂了都无所谓

session会记录并且保持用户的数据信息

我同时写了一键安装的脚本，还有一些问题，也分享出来，希望有大神能指点～～

第一个，在有nginx的机器上运行：

<code><span>#!/bin/bash
</span>
setenforce <span>0</span> > /dev/null
iptables -F > /dev/null
sed -i <span>'s/^SELINUX=enforcing/SELINUX=disabled/g'</span> /etc/selinux/config

<span>###############  nginx + tomcat + memcached</span>DIR_NOW=`<span>pwd</span>`

IPADDR_NTM=<span>'192.168.122.101'</span>
IPADDR_TM=<span>'192.168.122.102'</span>NGINX_PKG_DIR=<span>'/root/one_key_install'</span>
NGINX_PKG_NAME=<span>'nginx-1.6.1.tar.gz'</span>
NGINX_DIR=<span>'/usr/local/nginx'</span>TOMCAT_PKG_DIR=<span>'/root/one_key_install'</span>
TOMCAT_PKG_NAME=<span>'apache-tomcat-7.0.37.tar.gz'</span>
TOMCAT_DIR=<span>'/usr/local/tomcat'</span>STICKY_PKG_DIR=<span>'/root/one_key_install'</span>
STICKY_PKG_NAME=<span>'nginx-sticky-module-1.0.tar.gz'</span>JDK_BIN_DIR=<span>'/root/one_key_install'</span>
JDK_BIN_NAME=<span>'jdk-6u32-linux-x64.bin'</span><span>###################  nginx + sticky install</span><span>cd</span><span>$NGINX_PKG_DIR</span>
tar zxf <span>$NGINX_PKG_NAME</span>
DIR=`ls -F | grep /$ | grep nginx | awk -F <span>'/'</span><span>'{print $1}'</span>`
tar zxf <span>$STICKY_PKG_DIR</span>/<span>$STICKY_PKG_NAME</span> -C <span>$NGINX_PKG_DIR</span>/<span>$DIR</span>sed -i <span>'s/^CFLAGS="$CFLAGS -g"/#CFLAGS="$CFLAGS -g"/g'</span> /<span>$NGINX_PKG_DIR</span>/<span>$DIR</span>/auto/cc/gcc
sed -i <span>'s/^#define NGINX_VER          \"nginx\/\" NGINX_VERSION/#define NGINX_VER          \"nginx\/\"/g'</span> /<span>$NGINX_PKG_DIR</span>/<span>$DIR</span>/src/core/nginx.h

yum install gcc pcre-devel openssl-devel -y

<span>cd</span><span>$NGINX_PKG_DIR</span>/<span>$DIR</span>
./configure --prefix=/usr/local/nginx --add-module=<span>$NGINX_PKG_DIR</span>/<span>$DIR</span>/nginx-sticky-module-<span>1.0</span> --with-http_ssl_module --with-http_stub_status_module
make && make install

ln <span>-s</span> /usr/local/nginx/sbin/nginx /usr/local/sbin/
useradd -M <span>-d</span> /usr/local/nginx/ <span>-s</span> /sbin/nologin nginx

<span>cd</span><span>$DIR_NOW</span>
rm <span>-f</span> /usr/local/nginx/conf/nginx.conf
cp nginx.conf.exp /usr/local/nginx/conf/nginx.conf
sed -i <span>"s/server 192.168.0.1:8080;/server <span>${IPADDR_NTM}</span>:8080;/g"</span> /usr/local/nginx/conf/nginx.conf
sed -i <span>"s/server 192.168.0.2:8080;/server <span>${IPADDR_TM}</span>:8080;/g"</span> /usr/local/nginx/conf/nginx.conf

<span>#####################  tomcat install</span><span>cd</span><span>$JDK_BIN_DIR</span>
sh <span>$JDK_BIN_NAME</span>
DIR=`ls -F | grep /$ | grep jdk | awk -F <span>'/'</span><span>'{print $1}'</span>`
mv <span>$DIR</span> /usr/local/
mv /usr/local/<span>$DIR</span> /usr/local/jdk
<span>echo</span><span>"export JAVA_HOME=/usr/local/jdk
export CLASSPATH=:\$JAVA_HOME/lib
export PATH=\$PATH:\$JAVA_HOME/bin
"</span>>>/etc/profile

<span>cd</span><span>$TOMCAT_PKG_DIR</span>
tar zxf <span>$TOMCAT_PKG_NAME</span> -C /usr/local
<span>cd</span> /usr/local
mv /usr/local/`ls | grep tomcat` /usr/local/tomcat
ln <span>-s</span> /usr/local/tomcat/bin/startup.sh /usr/local/sbin/tomcat-start
ln <span>-s</span> /usr/local/tomcat/bin/shutdown.sh /usr/local/sbin/tomcat-stop

<span>cd</span><span>$DIR_NOW</span>
tar zxf kryo_pkgs.tar.gz
<span>cd</span> kryo_pkgs
cp * /usr/local/tomcat/lib

<span>cd</span><span>$DIR_NOW</span>
rm <span>-f</span> /usr/local/tomcat/conf/context.xml
cp context.xml.exp /usr/local/tomcat/conf/context.xml
 <span>cd</span><span>$DIR_NOW</span>
rm <span>-f</span> /usr/local/tomcat/conf/context.xml
cp context.xml.exp /usr/local/tomcat/conf/context.xml
sed -i <span>"s#memcachedNodes=\"n1:192.168.0.1:11211,n2:192.168.0.2:11211\"#memcachedNodes=\"n1:<span>${IPADDR_NTM}</span>:11211,n2:<span>${IPADDR_TM}</span>:11211\"#"</span> /usr/local/tomcat/conf/context.xml

<span>####################  memcached install</span>yum install memcached -y

<span>####################  start services</span>
/etc/init.d/memcached start
<span>source</span> /etc/profile
tomcat-start
nginx</code>

第二个，在只有tomcat和memcached的机器上运行

<code><span>#!/bin/bash
</span>
setenforce <span>0</span> > /dev/null
iptables -F > /dev/null
sed -i <span>'s/^SELINUX=enforcing/SELINUX=disabled/g'</span> /etc/selinux/config

DIR_NOW=`<span>pwd</span>`

IPADDR_NTM=<span>'192.168.122.101'</span>
IPADDR_TM=<span>'192.168.122.102'</span>TOMCAT_PKG_DIR=<span>'/root/no_ngx'</span>
TOMCAT_PKG_NAME=<span>'apache-tomcat-7.0.37.tar.gz'</span>
TOMCAT_DIR=<span>'/usr/local/tomcat'</span>STICKY_PKG_DIR=<span>'/root/no_ngx'</span>
STICKY_PKG_NAME=<span>'nginx-sticky-module-1.0.tar.gz'</span>JDK_BIN_DIR=<span>'/root/no_ngx'</span>
JDK_BIN_NAME=<span>'jdk-6u32-linux-x64.bin'</span><span>cd</span><span>$JDK_BIN_DIR</span>
sh <span>$JDK_BIN_NAME</span>DIR=`ls -F | grep /$ | grep jdk | awk -F <span>'/'</span><span>'{print $1}'</span>`
mv <span>$DIR</span> /usr/local/
mv /usr/local/<span>$DIR</span> /usr/local/jdk
<span>echo</span><span>"export JAVA_HOME=/usr/local/jdk
export CLASSPATH=:\$JAVA_HOME/lib
export PATH=\$PATH:\$JAVA_HOME/bin
"</span>>>/etc/profile

<span>cd</span><span>$TOMCAT_PKG_DIR</span>
tar zxf <span>$TOMCAT_PKG_NAME</span> -C /usr/local/
<span>cd</span> /usr/local
mv /usr/local/`ls | grep tomcat` /usr/local/tomcat


<span>cd</span><span>$DIR_NOW</span>
tar zxf kryo_pkgs.tar.gz
<span>cd</span> kryo_pkgs
cp * /usr/local/tomcat/lib

<span>cd</span><span>$DIR_NOW</span>
rm <span>-f</span> /usr/local/tomcat/conf/context.xml
cp context.xml.exp /usr/local/tomcat/conf/
mv /usr/local/tomcat/conf/context.xml.exp /usr/local/tomcat/conf/context.xml
sed -i <span>"s#memcachedNodes=\"n1:192.168.0.1:11211,n2:192.168.0.2:11211\"#memcachedNodes=\"n1:<span>${IPADDR_NTM}</span>:11211,n2:<span>${IPADDR_TM}</span>:11211\"#"</span> /usr/local/tomcat/conf/context.xml
sed -i <span>'s/failoverNodes="n1"/failoverNodes="n2"/'</span> /usr/local/tomcat/conf/context.xml

yum install memcached -y

/etc/init.d/memcached start

<span>source</span> /etc/profile
ln <span>-s</span> /usr/local/tomcat/bin/startup.sh /usr/local/sbin/tomcat-start
ln <span>-s</span> /usr/local/tomcat/bin/shutdown.sh /usr/local/sbin/tomcat-stop

tomcat-start</code>

这两个脚本存在同样的问题，执行完毕之后发现

<code><span>source</span> /etc/profile</code>

这一句没有执行，必须手动执行，一直百思不得其解，希望有谁能解答，多谢啦～～

The above introduces the nginx+tomcat+memcached configuration + script, including the relevant content. I hope it will be helpful to friends who are interested in PHP tutorials.

Statement：
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Previous article：Common nginx extension installationNext article：Common nginx extension installation

Related articles

See more