search
HomeOperation and MaintenanceNginxnginx ip blacklist dynamic ban method
nginx ip blacklist dynamic ban methodMay 17, 2023 pm 05:58 PM
nginxip

1.方案

黑名单持久化到mysql (常见的方案是redis,但不利于控制,如:不同的ip设置不同的有效期、ip的crud、统计等等);

通过lua-nginx-module,在nginx中开辟一块内存(lua_shared_dict),lua将黑名单定期从mysql全量刷新至lua_shared_dict;

所有请求,都要到与lua_shared_dict中的ip check一下。

2.安装

2.1 安装luajit

cd luajit-2.0.5
make
make install prefix=/usr/local/luajit

2.2.安装nginx时,将lua模块编译进去

export luajit_lib=/usr/local/luajit/lib
export luajit_inc=/usr/local/luajit/include/luajit-2.1
 
./configure --prefix=/nginx \
--with-ld-opt="-wl,-rpath,/usr/local/luajit/lib" \
--add-module=/opt/ngx_devel_kit-0.3.1rc1 \
--add-module=/opt/lua-nginx-module-0.10.14rc3
 
make -j2
make install
ln -s /nginx/sbin/nginx /usr/sbin/nginx

3.配置

3.1 nginx配置

http {
  server_tokens off;
  lua_package_path "/usr/local/lib/lua/?.lua;;";
  lua_shared_dict ip_blacklist 4m;
}
 
server {
  set $real_ip $remote_addr;
  if ( $http_x_forwarded_for ~ "^(\d+\.\d+\.\d+\.\d+)" ) {
    set $real_ip $1;
  }
 
  # 管理信息,访问该url可以查看nginx中的ip黑名单信息
  location /get-ipblacklist-info {
    access_by_lua_file conf/lua/get_ipblacklist_info.lua;
  }
 
  # 同步url,通过定时任务调用该url,实现ip黑名单从mysql到nginx的定时刷新
  location /sync-ipblacklist {
   access_by_lua_file conf/lua/sync_ipblacklist.lua;
  }
 
  # 生产域名配置,所有需要ip黑名单控制的location,都要包含以下语句
  location / {
   access_by_lua_file conf/lua/check_realip.lua;
  }
 
}

nginx服务器配置以下crrontab

* * * * * /usr/bin/curl -o /dev/null -s http://127.0.0.1/sync-ipblacklist > /dev/null 2>&1

3.2 lua脚本

sync_ipblacklist.lua

local mysql_host = "ip of mysql server"
local mysql_port = 3306
local database = "dbname"
local username = "user"
local password = "password"
 
-- update ip_blacklist from mysql once every cache_ttl seconds
local cache_ttl  = 1
local mysql_connection_timeout = 1000
 
local client_ip  = ngx.var.real_ip
local ip_blacklist  = ngx.shared.ip_blacklist
local last_update_time = ip_blacklist:get("last_update_time");
 
if last_update_time == nil or last_update_time < ( ngx.now() - cache_ttl ) then
 
 local mysql = require "resty.mysql";
 local red = mysql:new();
 
 red:set_timeout(mysql_connect_timeout);
 
 local ok, err, errcode, sqlstate = red:connect{
     host = mysql_host,
     port = mysql_port,
     database = database,
     user = username,
     password = password,
     charset = "utf8",
     max_packet_size = 1024 * 1024,
    }
 if not ok then
 ngx.log(ngx.err, "mysql connection error while retrieving ip_blacklist: " .. err);
 else
 new_ip_blacklist, err, errcode, sqlstate = red:query("select ip_addr from ip_blacklist where status = 0 order by create_time desc limit 10000", 100)
 if not new_ip_blacklist then
  ngx.log(ngx.err, "bad result. errcode: " .. errcode .. " sqlstate: " .. sqlstate .. " err: " .. err);
  return
 end
 
 ip_blacklist:flush_all();
 for k1, v1 in pairs(new_ip_blacklist) do
  for k2, v2 in pairs(v1) do
  ip_blacklist:set(v2,true);
  end
 end
 
 ip_blacklist:set("last_update_time", ngx.now());
 end
end
 
ngx.say("sync successful");

get_ipblacklist_info.lua

-- 调用url查看黑名单信息
-- 1万ip消耗不到1.5m ngx.shared内存
-- 获取所有key会堵塞别的正常请求对ngx.shared内存的访问,因此只能取少数key展示
require "resty.core.shdict"
ngx.say("total space: " .. ngx.shared.ip_blacklist:capacity() .. "<br/>");
ngx.say("free space: " .. ngx.shared.ip_blacklist:free_space() .. "<br/>");
ngx.say("last update time: " .. os.date("%y%m%d_%h:%m:%s",ngx.shared.ip_blacklist:get("last_update_time")) .. "<br/>");
ngx.say("first 100 keys: <br/>");
ngx.say("--------------------------<br/>");
ip_blacklist = ngx.shared.ip_blacklist:get_keys(100);
for key, value in pairs(ip_blacklist) do
 ngx.say(key .. ": " .. value .. "<br/>");
end

check_realip.lua

if ngx.shared.ip_blacklist:get(ngx.var.real_ip) then
 return ngx.exit(ngx.http_forbidden);
end

3.3 数据库设计

create table `ip_blacklist` (
 `id` int(11) not null auto_increment,
 `ip_addr` varchar(15) collate utf8mb4_bin default null,
 `status` int(11) default &#39;0&#39; comment &#39;0: valid 有效, 1: invalid 失效&#39;,
 `effective_hour` decimal(11,2) default &#39;24&#39; comment &#39;有效期,单位:小时&#39;,
 `ip_source` varchar(255) collate utf8mb4_bin default null comment &#39;黑名单来源&#39;,
 `create_time` datetime default current_timestamp,
 `modify_time` datetime default current_timestamp on update current_timestamp,
 `remark` varchar(255) collate utf8mb4_bin default null comment &#39;备注&#39;,
 primary key (`id`)
) engine=innodb default charset=utf8mb4 collate=utf8mb4_bin;
 
 
create procedure proc_ip_blacklist_status_update()
-- 将过期的ip状态改为失效
begin 
 update ip_blacklist
 set status=1
 where date_add(create_time,interval effective_hour hour) < now();
 commit;
end;
 
 
create event job_ip_blacklist_status_update
on schedule every 1 minute
on completion preserve
enable
do
call proc_ip_blacklist_status_update();

4 crud

黑名单产生有手工的方式,也有自动的方式,或者两者兼有。

自动的方式有通过python分析elk日志,将恶意ip自动写入mysql,这是个大话题,这里不涉及。

手工的方式可以人肉查看elk请求日志,发现恶意ip,手工填入mysql,这里推荐一个开源的crud工具,用户体验很nice(比直接navicat好多了),当然也可以自己写……

项目的强大之处在于,所有表都帮你生成菜单,然后这些表的crud就直接用了。

具体操作见官方说明,就不赘述了。

nginx ip黑名单动态封禁的方法

The above is the detailed content of nginx ip blacklist dynamic ban method. For more information, please follow other related articles on the PHP Chinese website!

Statement
This article is reproduced at:亿速云. If there is any infringement, please contact admin@php.cn delete
内存飙升!记一次nginx拦截爬虫内存飙升!记一次nginx拦截爬虫Mar 30, 2023 pm 04:35 PM

本篇文章给大家带来了关于nginx的相关知识,其中主要介绍了nginx拦截爬虫相关的,感兴趣的朋友下面一起来看一下吧,希望对大家有帮助。

nginx限流模块源码分析nginx限流模块源码分析May 11, 2023 pm 06:16 PM

高并发系统有三把利器:缓存、降级和限流;限流的目的是通过对并发访问/请求进行限速来保护系统,一旦达到限制速率则可以拒绝服务(定向到错误页)、排队等待(秒杀)、降级(返回兜底数据或默认数据);高并发系统常见的限流有:限制总并发数(数据库连接池)、限制瞬时并发数(如nginx的limit_conn模块,用来限制瞬时并发连接数)、限制时间窗口内的平均速率(nginx的limit_req模块,用来限制每秒的平均速率);另外还可以根据网络连接数、网络流量、cpu或内存负载等来限流。1.限流算法最简单粗暴的

nginx+rsync+inotify怎么配置实现负载均衡nginx+rsync+inotify怎么配置实现负载均衡May 11, 2023 pm 03:37 PM

实验环境前端nginx:ip192.168.6.242,对后端的wordpress网站做反向代理实现复杂均衡后端nginx:ip192.168.6.36,192.168.6.205都部署wordpress,并使用相同的数据库1、在后端的两个wordpress上配置rsync+inotify,两服务器都开启rsync服务,并且通过inotify分别向对方同步数据下面配置192.168.6.205这台服务器vim/etc/rsyncd.confuid=nginxgid=nginxport=873ho

nginx php403错误怎么解决nginx php403错误怎么解决Nov 23, 2022 am 09:59 AM

nginx php403错误的解决办法:1、修改文件权限或开启selinux;2、修改php-fpm.conf,加入需要的文件扩展名;3、修改php.ini内容为“cgi.fix_pathinfo = 0”;4、重启php-fpm即可。

如何解决跨域?常见解决方案浅析如何解决跨域?常见解决方案浅析Apr 25, 2023 pm 07:57 PM

跨域是开发中经常会遇到的一个场景,也是面试中经常会讨论的一个问题。掌握常见的跨域解决方案及其背后的原理,不仅可以提高我们的开发效率,还能在面试中表现的更加

nginx怎么禁止访问phpnginx怎么禁止访问phpNov 22, 2022 am 09:52 AM

nginx禁止访问php的方法:1、配置nginx,禁止解析指定目录下的指定程序;2、将“location ~^/images/.*\.(php|php5|sh|pl|py)${deny all...}”语句放置在server标签内即可。

nginx部署react刷新404怎么办nginx部署react刷新404怎么办Jan 03, 2023 pm 01:41 PM

nginx部署react刷新404的解决办法:1、修改Nginx配置为“server {listen 80;server_name https://www.xxx.com;location / {root xxx;index index.html index.htm;...}”;2、刷新路由,按当前路径去nginx加载页面即可。

Linux系统下如何为Nginx安装多版本PHPLinux系统下如何为Nginx安装多版本PHPMay 11, 2023 pm 07:34 PM

linux版本:64位centos6.4nginx版本:nginx1.8.0php版本:php5.5.28&php5.4.44注意假如php5.5是主版本已经安装在/usr/local/php目录下,那么再安装其他版本的php再指定不同安装目录即可。安装php#wgethttp://cn2.php.net/get/php-5.4.44.tar.gz/from/this/mirror#tarzxvfphp-5.4.44.tar.gz#cdphp-5.4.44#./configure--pr

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
2 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
Repo: How To Revive Teammates
4 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
Hello Kitty Island Adventure: How To Get Giant Seeds
3 weeks agoBy尊渡假赌尊渡假赌尊渡假赌

Hot Tools

SublimeText3 Linux new version

SublimeText3 Linux new version

SublimeText3 Linux latest version

Notepad++7.3.1

Notepad++7.3.1

Easy-to-use and free code editor

Atom editor mac version download

Atom editor mac version download

The most popular open source editor

WebStorm Mac version

WebStorm Mac version

Useful JavaScript development tools

ZendStudio 13.5.1 Mac

ZendStudio 13.5.1 Mac

Powerful PHP integrated development environment