SHELL实例之日志时间转换-Mysql Tutorial-php.cn

Home

Database

Mysql Tutorial

SHELL实例之日志时间转换

WBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWBOYWB

Jun 07, 2016 pm 03:11 PM

apacheshellExamplelogtimeConvert

想把Apache已经生成的access.log文件中的[01/Aug/2008:14:06:30 +0800] 部分，转换成UTC时间的形式，然后输出为一个新的日志文件，如何用Shell实现

一下是access.log中记录的一个例子：
192.168.10.174 - - [01/Aug/2008:14:06:30 +0800] "GET / HTTP/1.1" 302 659 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; GoogleT5; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506)"
192.168.10.174 - - [01/Aug/2008:14:06:31 +0800] "GET /index.jsp HTTP/1.1" 200 16533 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; GoogleT5; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506)"
192.168.10.174 - - [01/Aug/2008:14:06:32 +0800] "GET /ShowImageServlet?id=1&lang=ja&bean=SolutionBean HTTP/1.1" 200 5083 "http://192.168.10.253/index.jsp" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; GoogleT5; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506)"
192.168.10.174 - - [01/Aug/2008:14:06:32 +0800] "GET /ShowImageServlet?id=2&lang=ja&bean=SolutionBean HTTP/1.1" 200 5148 "http://192.168.10.253/index.jsp" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; GoogleT5; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506)"
192.168.10.174 - - [01/Aug/2008:14:06:32 +0800] "GET /images/favicon.ico HTTP/1.1" 200 5 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; GoogleT5; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506)"
192.168.10.174 - - [01/Aug/2008:14:06:32 +0800] "GET /images/w3c_xhtml1.0.gif HTTP/1.1" 200 2333 "http://192.168.10.253/index.jsp" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; GoogleT5; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506)"
192.168.10.174 - - [01/Aug/2008:14:06:32 +0800] "GET /images/zh/title.gif HTTP/1.1" 200 6606 "http://192.168.10.253/index.jsp" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; GoogleT5; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506)"
192.168.10.174 - - [01/Aug/2008:14:06:32 +0800] "GET /include/MatrixOne.swf HTTP/1.1" 200 4073 "http://192.168.10.253/index.jsp" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; GoogleT5; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506)"
192.168.10.174 - - [01/Aug/2008:14:06:32 +0800] "GET /script/Select.htc HTTP/1.1" 200 13800 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; GoogleT5; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506)"
192.168.10.174 - - [01/Aug/2008:14:06:32 +0800] "GET /script/Select.htc HTTP/1.1" 200 13800 "-" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; GoogleT5; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506)"
192.168.10.174 - - [01/Aug/2008:14:06:32 +0800] "GET /script/check.js HTTP/1.1" 200 3619 "http://192.168.10.253/index.jsp" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; GoogleT5; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506)"
192.168.10.174 - - [01/Aug/2008:14:06:32 +0800] "GET /script/initMenu.js HTTP/1.1" 200 17565 "http://192.168.10.253/index.jsp" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; GoogleT5; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506)"
192.168.10.174 - - [01/Aug/2008:14:06:32 +0800] "GET /script/ja/souceXML.xml HTTP/1.1" 200 2604 "http://192.168.10.253/index.jsp" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; GoogleT5; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506)"
192.168.10.174 - - [01/Aug/2008:14:06:32 +0800] "GET /script/styleSwitch.js HTTP/1.1" 200 1660 "http://192.168.10.253/index.jsp" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; GoogleT5; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506)"
192.168.10.174 - - [01/Aug/2008:14:06:32 +0800] "GET /style/gray%20window/bgImg.gif HTTP/1.1" 200 97 "http://192.168.10.253/index.jsp" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; GoogleT5; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506)"
192.168.10.174 - - [01/Aug/2008:14:06:32 +0800] "GET /style/gray%20window/bottom_bg.jpg HTTP/1.1" 200 4657 "http://192.168.10.253/index.jsp" "Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; GoogleT5; SLCC1; .NET CLR 2.0.50727; .NET CLR 3.0.04506)"

try:
$ awk 'BEGIN { FS="[][]"; OFS="" } {gsub(/\//,"-",$2);sub(/:/," ",$2); "date -ud \""$2"\""|getline a;$2="["a"]";print }' access.log

或者用gsed s/...//e

整个地球分为二十四时区，每个时区都有自己的本地时间。在国际无线电通信场合，为了统一起见，使用一个统一的时间，称为通用协调时(UTC, Universal Time Coordinated)。

例如我的例子中的[01/Aug/2008:14:06:30 +0800]是本地时间（即北京时间）
UTC + 时区差＝本地时间

时区差东为正，西为负。在此，把东八区时区差记为 +0800，

UTC + (＋0800) = 本地（北京）时间 (1)

那么，UTC = 本地时间（北京时间)）- 0800 (2)

14:06:30 - 08:00:00 = 06:06:30 这个就是UTC时间，不过有的时候会得到前一天的时间
这个也是个问题。。。

Statement

The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn

What are stored procedures in MySQL?May 01, 2025 am 12:27 AM

Stored procedures are precompiled SQL statements in MySQL for improving performance and simplifying complex operations. 1. Improve performance: After the first compilation, subsequent calls do not need to be recompiled. 2. Improve security: Restrict data table access through permission control. 3. Simplify complex operations: combine multiple SQL statements to simplify application layer logic.

How does query caching work in MySQL?May 01, 2025 am 12:26 AM

The working principle of MySQL query cache is to store the results of SELECT query, and when the same query is executed again, the cached results are directly returned. 1) Query cache improves database reading performance and finds cached results through hash values. 2) Simple configuration, set query_cache_type and query_cache_size in MySQL configuration file. 3) Use the SQL_NO_CACHE keyword to disable the cache of specific queries. 4) In high-frequency update environments, query cache may cause performance bottlenecks and needs to be optimized for use through monitoring and adjustment of parameters.

What are the advantages of using MySQL over other relational databases?May 01, 2025 am 12:18 AM

The reasons why MySQL is widely used in various projects include: 1. High performance and scalability, supporting multiple storage engines; 2. Easy to use and maintain, simple configuration and rich tools; 3. Rich ecosystem, attracting a large number of community and third-party tool support; 4. Cross-platform support, suitable for multiple operating systems.

How do you handle database upgrades in MySQL?Apr 30, 2025 am 12:28 AM

The steps for upgrading MySQL database include: 1. Backup the database, 2. Stop the current MySQL service, 3. Install the new version of MySQL, 4. Start the new version of MySQL service, 5. Recover the database. Compatibility issues are required during the upgrade process, and advanced tools such as PerconaToolkit can be used for testing and optimization.

What are the different backup strategies you can use for MySQL?Apr 30, 2025 am 12:28 AM

MySQL backup policies include logical backup, physical backup, incremental backup, replication-based backup, and cloud backup. 1. Logical backup uses mysqldump to export database structure and data, which is suitable for small databases and version migrations. 2. Physical backups are fast and comprehensive by copying data files, but require database consistency. 3. Incremental backup uses binary logging to record changes, which is suitable for large databases. 4. Replication-based backup reduces the impact on the production system by backing up from the server. 5. Cloud backups such as AmazonRDS provide automation solutions, but costs and control need to be considered. When selecting a policy, database size, downtime tolerance, recovery time, and recovery point goals should be considered.

What is MySQL clustering?Apr 30, 2025 am 12:28 AM

MySQLclusteringenhancesdatabaserobustnessandscalabilitybydistributingdataacrossmultiplenodes.ItusestheNDBenginefordatareplicationandfaulttolerance,ensuringhighavailability.Setupinvolvesconfiguringmanagement,data,andSQLnodes,withcarefulmonitoringandpe

How do you optimize database schema design for performance in MySQL?Apr 30, 2025 am 12:27 AM

Optimizing database schema design in MySQL can improve performance through the following steps: 1. Index optimization: Create indexes on common query columns, balancing the overhead of query and inserting updates. 2. Table structure optimization: Reduce data redundancy through normalization or anti-normalization and improve access efficiency. 3. Data type selection: Use appropriate data types, such as INT instead of VARCHAR, to reduce storage space. 4. Partitioning and sub-table: For large data volumes, use partitioning and sub-table to disperse data to improve query and maintenance efficiency.

How can you optimize MySQL performance?Apr 30, 2025 am 12:26 AM

TooptimizeMySQLperformance,followthesesteps:1)Implementproperindexingtospeedupqueries,2)UseEXPLAINtoanalyzeandoptimizequeryperformance,3)Adjustserverconfigurationsettingslikeinnodb_buffer_pool_sizeandmax_connections,4)Usepartitioningforlargetablestoi

See all articles