search
HomeDatabaseMysql TutorialMySQL导入GeoIP的CSV数据

MySQL导入GeoIP的CSV数据 一、缘由 因为需要使用IP到城市的映射库,从知乎上进行搜索,了解了目前大概弄了个列表如下所示: 1)腾讯-腾讯IP分享计划_IP分享 链接网址:http://ip.qq.com 2)新浪 链接网址:http://int.dpool.sina.com.cn/iplookup/iplookup.

MySQL导入GeoIP的CSV数据

一、缘由

        因为需要使用IP到城市的映射库,从知乎上进行搜索,了解了目前大概弄了个列表如下所示:

  • 1)腾讯-腾讯IP分享计划_IP分享

    链接网址:http://ip.qq.com

  • 2)新浪

    链接网址:http://int.dpool.sina.com.cn/iplookup/iplookup.php?ip=8.8.8.8&format=js

  • 3)百度-Web服务Geocoding API-百度地图

    链接网址:http://developer.baidu.com/map/ip-location-api.htm

  • 4)淘宝-淘宝IP地址库

    链接网址:http://ip.taobao.com/accurancy.php

  • 5)geoip-MaxMind-IP地理定位和在线欺诈预防

    链接网址:http://www.maxmind.com

  • 6)纯真

    链接网址:http://www.cz88.net

  • 7)17mon-IP归属地数据库下载_17MON网络工具集

    链接网址:http://tool.17mon.cn/ipdb.html


        在以上所罗列的表项中,主要是了解了geoip的免费的数据库。这个数据可以自行管理更新,也可以通过官网更新。本次所遇到的主要是在将geoip的CSV数据导入mysql时遇到了问题。


二、导入数据流程

        1)下载geopip的IP地址CSV数据,下载地址如下所示:http://dev.maxmind.com/geoip/legacy/geolite/

        2)解压缩数据。

        3)根据官网提供的数据表结构进行数据库设计,官网中对CSV数据的格式描述连接为:http://dev.maxmind.com/geoip/legacy/csv/

        3.1)数据库的设计,此处只是做了一个粗略的数据映射,关于数据类型与约束关系等还可以再进行细化,脚本如下所示

create database geoip;
use geoip;

--    1.1 具体的位置信息表
create table location(
    locId bigint primary key,
    country varchar(2),
    region char(2),
    city varchar(255),
    postal varchar(8),
    latitude decimal,
    longtitude decimal,
    metroCode int,
    areaCode char(3)
);

--    1.2 IP地址对应的位信息表
create table block(
    startIpNum int,
    endIpNum int,
    locId bigint
);

        3.2)导入到本地mysql数据库,shell脚本如下所示:

#    2.1 
mysql -u${MYSQL_USER} -p${PASSWORD} geoip --local-infile=1 -e 'load data local infile "${LOCATION_CSV_FILE}" replace into table location fields terminated by "," OPTIONALLY ENCLOSED BY "\"" lines terminated by "\n";'
#    2.2
mysql -u${MYSQL_USER} -p${PASSWORD} geoip --local-infile=1 -e 'load data local infile "${BLOCKS_CSV_FILE}"  replace into table block fields terminated by "," OPTIONALLY ENCLOSED BY "\"" lines terminated by "\n";'
意:
  • (1)其中MYSQL_USER是数据库用户,PASSWORD是相应的密码,geoip是3.1中创建的数据名,--local-infile=1是允许在导入数据时使用本地绝对路径,-e后跟的是SQL语句。
  • (2)在-e后的SQL语句中,LOCATION_CSV_FILE是geoip中下载解压后的一个location数据文件名,BLOCKS_CSV_FILE是geoip中下载解压后的一个blocks数据文件名。


        3.3)数据查询,脚本如下所示:

SELECT * 
FROM block as b left outer join location as l on(l.locId = b.locId)
WHERE
INET_ATON('174.36.207.186') BETWEEN b.startIpNum AND b.endIpNum
LIMIT 1;
注意:
        以上的查询应该很浅显易懂,其中INET_ATON是MYSQL提供的,它的功能是将点分十进制的IP表示转换成INT型数据它是网络字节序的(即大端序),同样也有一个将INT型数据转换为点分十进制的函数INET_NTOA

        4)通过3中的几个步骤就可以成功的将数据导入到本地的MySQL库中。

三、小结


        通过数据导入到数据库中,这样在上层就可以编程应用。存在的问题:还需要后期对IP地址数据库进行更新与维护。对于其它几类的IP地址库的使用效果,在后续还需要进行对比研究,以选择效果更好的IP地址库


参考文献

  • http://www.zhihu.com/question/19584593
  • http://dev.maxmind.com/geoip/legacy/csv/
  • http://dev.mysql.com/doc/refman/5.6/en/miscellaneous-functions.html#function_inet-aton
  • http://dev.mysql.com/doc/refman/5.1/zh/sql-syntax.html#load-data
Statement
The content of this article is voluntarily contributed by netizens, and the copyright belongs to the original author. This site does not assume corresponding legal responsibility. If you find any content suspected of plagiarism or infringement, please contact admin@php.cn
Explain the ACID properties (Atomicity, Consistency, Isolation, Durability).Explain the ACID properties (Atomicity, Consistency, Isolation, Durability).Apr 16, 2025 am 12:20 AM

ACID attributes include atomicity, consistency, isolation and durability, and are the cornerstone of database design. 1. Atomicity ensures that the transaction is either completely successful or completely failed. 2. Consistency ensures that the database remains consistent before and after a transaction. 3. Isolation ensures that transactions do not interfere with each other. 4. Persistence ensures that data is permanently saved after transaction submission.

MySQL: Database Management System vs. Programming LanguageMySQL: Database Management System vs. Programming LanguageApr 16, 2025 am 12:19 AM

MySQL is not only a database management system (DBMS) but also closely related to programming languages. 1) As a DBMS, MySQL is used to store, organize and retrieve data, and optimizing indexes can improve query performance. 2) Combining SQL with programming languages, embedded in Python, using ORM tools such as SQLAlchemy can simplify operations. 3) Performance optimization includes indexing, querying, caching, library and table division and transaction management.

MySQL: Managing Data with SQL CommandsMySQL: Managing Data with SQL CommandsApr 16, 2025 am 12:19 AM

MySQL uses SQL commands to manage data. 1. Basic commands include SELECT, INSERT, UPDATE and DELETE. 2. Advanced usage involves JOIN, subquery and aggregate functions. 3. Common errors include syntax, logic and performance issues. 4. Optimization tips include using indexes, avoiding SELECT* and using LIMIT.

MySQL's Purpose: Storing and Managing Data EffectivelyMySQL's Purpose: Storing and Managing Data EffectivelyApr 16, 2025 am 12:16 AM

MySQL is an efficient relational database management system suitable for storing and managing data. Its advantages include high-performance queries, flexible transaction processing and rich data types. In practical applications, MySQL is often used in e-commerce platforms, social networks and content management systems, but attention should be paid to performance optimization, data security and scalability.

SQL and MySQL: Understanding the RelationshipSQL and MySQL: Understanding the RelationshipApr 16, 2025 am 12:14 AM

The relationship between SQL and MySQL is the relationship between standard languages ​​and specific implementations. 1.SQL is a standard language used to manage and operate relational databases, allowing data addition, deletion, modification and query. 2.MySQL is a specific database management system that uses SQL as its operating language and provides efficient data storage and management.

Explain the role of InnoDB redo logs and undo logs.Explain the role of InnoDB redo logs and undo logs.Apr 15, 2025 am 12:16 AM

InnoDB uses redologs and undologs to ensure data consistency and reliability. 1.redologs record data page modification to ensure crash recovery and transaction persistence. 2.undologs records the original data value and supports transaction rollback and MVCC.

What are the key metrics to look for in an EXPLAIN output (type, key, rows, Extra)?What are the key metrics to look for in an EXPLAIN output (type, key, rows, Extra)?Apr 15, 2025 am 12:15 AM

Key metrics for EXPLAIN commands include type, key, rows, and Extra. 1) The type reflects the access type of the query. The higher the value, the higher the efficiency, such as const is better than ALL. 2) The key displays the index used, and NULL indicates no index. 3) rows estimates the number of scanned rows, affecting query performance. 4) Extra provides additional information, such as Usingfilesort prompts that it needs to be optimized.

What is the Using temporary status in EXPLAIN and how to avoid it?What is the Using temporary status in EXPLAIN and how to avoid it?Apr 15, 2025 am 12:14 AM

Usingtemporary indicates that the need to create temporary tables in MySQL queries, which are commonly found in ORDERBY using DISTINCT, GROUPBY, or non-indexed columns. You can avoid the occurrence of indexes and rewrite queries and improve query performance. Specifically, when Usingtemporary appears in EXPLAIN output, it means that MySQL needs to create temporary tables to handle queries. This usually occurs when: 1) deduplication or grouping when using DISTINCT or GROUPBY; 2) sort when ORDERBY contains non-index columns; 3) use complex subquery or join operations. Optimization methods include: 1) ORDERBY and GROUPB

See all articles

Hot AI Tools

Undresser.AI Undress

Undresser.AI Undress

AI-powered app for creating realistic nude photos

AI Clothes Remover

AI Clothes Remover

Online AI tool for removing clothes from photos.

Undress AI Tool

Undress AI Tool

Undress images for free

Clothoff.io

Clothoff.io

AI clothes remover

AI Hentai Generator

AI Hentai Generator

Generate AI Hentai for free.

Hot Article

R.E.P.O. Energy Crystals Explained and What They Do (Yellow Crystal)
4 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Best Graphic Settings
4 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. How to Fix Audio if You Can't Hear Anyone
4 weeks agoBy尊渡假赌尊渡假赌尊渡假赌
R.E.P.O. Chat Commands and How to Use Them
4 weeks agoBy尊渡假赌尊渡假赌尊渡假赌

Hot Tools

Dreamweaver Mac version

Dreamweaver Mac version

Visual web development tools

EditPlus Chinese cracked version

EditPlus Chinese cracked version

Small size, syntax highlighting, does not support code prompt function

Atom editor mac version download

Atom editor mac version download

The most popular open source editor

VSCode Windows 64-bit Download

VSCode Windows 64-bit Download

A free and powerful IDE editor launched by Microsoft

SublimeText3 Mac version

SublimeText3 Mac version

God-level code editing software (SublimeText3)