Home > Article > Operation and Maintenance > How to solve the problem of setting up file upload and download service with nginx+lua
How to solve the problem of setting up file upload and download service with nginx+lua
- PHPzforward
- 2023-05-11 20:52:041562browse
Main logic
Upload
The front end requests the nginx service, and nginx calls the upload script. The script By searching the configuration, find the corresponding logical storage path and the IP and port of the agent of the physical storage machine, send the package to the corresponding agent through tcp, and the agent deployed on the corresponding machine receives the data and writes it to a local file.
Download
http download request nginx, nginx calls the download script, the script parses the link parameters, finds the corresponding agent address according to the parameters, the request returns the file binary content, the script accepts The data returned by the agent is returned to the requesting end.
Configuring nginx+lua
Next, we will mainly talk about the nginx installation configuration (this includes lua's binary stream processing lpack, md5 calculation, mysql operation, json operation)
1. Install nginx
Download
Unziptar -xvf nginx-1.10.3.tar.gz
2. Install luajit（ Lightweight lua)
Modify the installation path in the makefile export prefix= /usr/local/luajit
Then install make &make install
3. Install nginx_lua_module
Download
Unzip
4. Install ngx_devel_kit (ndk provides functions and macros to handle some basic tasks, reducing the amount of code for third-party module development)
Download
5. Install and compile, import
export luajit_lib=/usr/local/luajit/lib export luajit_inc=/usr/local/luajit/include/luajit-2.0 ./configure --prefix=/usr/local/nginx --with-http_stub_status_module --with-http_ssl_module --add-module=/home/oicq/jeffzhuang/ngx_devel_kit-0.3.0 --add-module=/home/oicq/jeffzhuang/lua-nginx-module-0.10. make -j2 make install
Start /usr/local/nginx/sbin/nginx Restart command` usr/local/nginx/sbin/nginx -s reload v
If an error is reported and the luajit library cannot be found ln -s /usr/local/lib/libluajit-5.1.so.2 /lib64/libluajit-5.1.so.2
Test nginx and open the browser directly You can see the welcome interface at http:10.x.x.x:8080
6. Configure conf/nginx.conf to run the lua script
Add the search path of the lua library lua_package_path, lua_package_cpath
7. Add mysql.lua download and copy it to the lua_package_path directory.
8. Add csjon
Modify the makefile. prefix=/usr/local/luajit is the installation path of luajit. After make, copy the generated cjson.so to the
lua_package_cpath directory
9. To install lpack, you can use the ready-made lpack.lua Copy to lua_package_path or use https://github.com/luadist/lpack to compile and generate lpack.so. Copy to lua_package_cpath. For 64-bit, you need to add the compilation command -fpic
10, upload.lua download
11. md5 download
Main code
1. Front-end upload page code
<!doctype html> <html> <head> <title>file upload example</title> </head> <body> <form action="emer_upload/order_system_storage" method="post" enctype="multipart/form-data"> <input type="file" name="testfilename"/> <input type="submit" name="upload" value="upload" /> </form> </body> </html>
2. Upload upload code. This module parses the file upload request. In the process, it is mainly implemented using a simple algorithm similar to a finite state machine, and different states are processed by corresponding handlers.
--文件下载服务写到 saverootpath .."/" .. filename 下面
function download()
local chunk_size = 4096
local form,err=upload:new(chunk_size)
if not form then
ngx.log(ngx.err, "failed to new upload: ", err)
ngx.exit(ngx.http_internal_server_error)
end
form:set_timeout(100000)
while true do
local typ,res,err=form:read()
if not typ then
errormsg="failed to read :"..err
return 1
end
if typ =="header" then
local key=res[1]
local value=res[2]
if key =="content-disposition" then
local kvlist=string.split(value,';')
for _, kv in ipairs(kvlist) do
local seg = string.trim(kv)
if seg:find("filename") then
local kvfile = string.split(seg, "=")
filename = string.sub(kvfile[2], 2, -2)
if filename then
--获取文件后缀名字
fileextension=getextension(filename)
local linuxtime=tostring(os.time())
filepath=saverootpath .."/" ..linuxtime..filename
filetosave,errmsg = io.open(filepath, "w+")
--存储的文件路径
--ngx.say("failed to open file ", filepath)
if not filetosave then
--ngx.say("failed to open file ", filepath .. errmsg)
errormsg="打开文件失败"..filepath .. errmsg
return 1
end
else
errormsg="请求参数找不到文件名字"
return 1
end
--跳出循环
break
end
end
end
elseif typ =="body" then
if filetosave then
filetosave:write(res)
filemd5:update(res)
end
elseif typ =="part_end" then
if filetosave then
local md5_sum=filemd5:final()
--ngx.say("md5: ", str.to_hex(md5_sum))
filemd532=str.to_hex(md5_sum)
filetosave:close()
filetosave = nil
end
elseif typ =="eof" then
break
else
ngx.log(ngx.info, "do other things")
end
end
return 0
end3. TCP receives binary data
-- 读取byte function readint8(tcp) local next, val = string.unpack(tcp:receive(1), "b") return tonumber(val); end -- 读取int16 function readint16(tcp) local next, val = string.unpack(tcp:receive(2), "h"); return tonumber(val); end -- 读取int32 function readint32(tcp) local next, val = string.unpack(tcp:receive(4), ">i"); return tonumber(val); end -- 读取字符串 function readstring(tcp,len) return tostring(tcp:receive(len)); end
4. TCP writes binary data. The communication protocol with the agent here is: start flag + packet length + json string + end flag, so The parameter corresponding to pack is biab, > is converted into big endian
jsondata["filename"]=filemd532 .. "." .. fileextension
jsondata["cmd"]="write"
jsondata["filesize"]=tostring(filelen)
jsondata["path"]=system.."/"..storagedate
local jsonstr=cjson.encode(jsondata)
local uilen=string.len(jsonstr)
senddata=bpack(">b1iab",startindex,uilen,jsonstr,endindex)
socket:send(senddata)5. When there is a download error, redirect is used to jump directly to the error page to facilitate the output of error information. In fact, you can also be a user here token verification
local errorurl="/downloaderror.html" errormsg="url 参数解析有问题 "..index return ngx.redirect(errorurl.."?msg="..errormsg,``` ngx.http_moved_temporarily)
The above is the detailed content of How to solve the problem of setting up file upload and download service with nginx+lua. For more information, please follow other related articles on the PHP Chinese website!