mongodb sharding基本概念 这里先介绍sharding的架构和几个基本概念术语。 shard server :shard server可以使一个mongod实例,也可以是replica set。 config sever:为了将指定collection存储在多个shard中,那么就需要个key来进行分割,config server存储
mongodb sharding基本概念
这里先介绍sharding的架构和几个基本概念术语。
shard server :shard server可以使一个mongod实例,也可以是replica set。
config sever:为了将指定collection存储在多个shard中,那么就需要个key来进行分割,config server存储各个节点的配置信息。shard key的范围,以及分布情况。
route process:由此介入客户端,通过询问config server,确定到那个shard上面查询,在连接相应的shard操作,不保存数据和配置信息。
由于资源限制,在一台机子上做一下实验
Shard Server 1:30000
Shard Server 2:30001
Config Server :40000
Route Process:50000
步骤:
启动shard server 1和2
[mongo@172_16_3_216 mongo]$ mkdir -p /mongo/shard/data0
[mongo@172_16_3_216 mongo]$ mkdir -p /mongo/shard/data1
[mongo@172_16_3_216 mongo]$ touch shard.log
[mongo@172_16_3_216 mongo]$ mongod --shardsvr --port 30000 --dbpath /mongo/shard/data0 --fork --logpath shard.log --directoryperdb
[mongo@172_16_3_216 mongo]$ touch shard1.log
[mongo@172_16_3_216 mongo]$ mongod --shardsvr --port 30001 --dbpath /mongo/shard/data1 --fork --logpath shard1.log --directoryperdb
启动config server
[mongo@172_16_3_216 mongo]$ mkdir -p /mongo/shard/config
[mongo@172_16_3_216 mongo]$ touch config.log
[mongo@172_16_3_216 mongo]$ mongod --configsvr --port 40000 --dbpath /mongo/shard/config --fork --logpath config.log --directoryperdb
启动route process
[mongo@172_16_3_216 mongo]$ touch route.log
[mongo@172_16_3_216 mongo]$ mongos --port 50000 --configdb localhost:40000 --fork --logpath route.log --chunkSize 2
初始化sharding
mongo admin --port 50000
MongoDB shell version: 1.8.4
connecting to: 127.0.0.1:50000/admin
> db.runCommand({addshard:"localhost:30000"}) ----添加shard1
{ "shardAdded" : "shard0000", "ok" : 1 }
> db.runCommand({addshard:"localhost:30001"}) -----添加shard2
{ "shardAdded" : "shard0001", "ok" : 1 }
> db.runCommand({enablesharding:"test"}) ---对数据库test分片
{ "ok" : 1 }
> db.runCommand({shardcollection:"test.tb1",key:{_id:1}}) ---对数据库test中tb1按_id作为key
{ "collectionsharded" : "test.tb1", "ok" : 1 }
验证sharding
> for (var i=1;i
> db.tb1.stats()
{
"sharded" : true,
"ns" : "test.tb1",
"count" : 50000,
"size" : 3600016,
"avgObjSize" : 72.00032,
"storageSize" : 13975552,
"nindexes" : 1,
"nchunks" : 4,
"shards" : {
"shard0000" : {
"ns" : "test.tb1",
"count" : 17888,
"size" : 1287944,
"avgObjSize" : 72.00044722719142,
"storageSize" : 2793472,
"numExtents" : 5,
"nindexes" : 1,
"lastExtentSize" : 2097152,
"paddingFactor" : 1,
"flags" : 1,
"totalIndexSize" : 753664,
"indexSizes" : {
"_id_" : 753664
},
"ok" : 1
},
"shard0001" : {
"ns" : "test.tb1",
"count" : 32112,
"size" : 2312072,
"avgObjSize" : 72.00024912805182,
"storageSize" : 11182080,
"numExtents" : 6,
"nindexes" : 1,
"lastExtentSize" : 8388608,
"paddingFactor" : 1,
"flags" : 1,
"totalIndexSize" : 1343488,
"indexSizes" : {
"_id_" : 1343488
},
"ok" : 1
}
},
"ok" : 1
}
查看sharding信息:
> db.runCommand({listshards:1})
{
"shards" : [
{
"_id" : "shard0000",
"host" : "localhost:30000"
},
{
"_id" : "shard0001",
"host" : "localhost:30001"
}
],
"ok" : 1
}
新增shard server:
[mongo@172_16_3_216 mongo]$ mkdir -p /mongo/shard/data2
[mongo@172_16_3_216 mongo]$ touch shard2.log
[mongo@172_16_3_216 mongo]$ mongod --shardsvr --port 30002 --dbpath /mongo/shard/data2 --fork --logpath shard2.log --directoryperdb
> db.runCommand({ addshard:"localhost:30002" })
{ "shardAdded" : "shard0002", "ok" : 1 }
> db.runCommand({listshards:1})
{
"shards" : [
{
"_id" : "shard0000",
"host" : "localhost:30000"
},
{
"_id" : "shard0001",
"host" : "localhost:30001"
},
{
"_id" : "shard0002",
"host" : "localhost:30002"
}
],
"ok" : 1
}
如果分片的表继续有插入数据,那么数据就会分配到新加的片上,而且会根据sharding key进行数据的迁移,和重新分布。
所以建议在添加删除节点的时候,建议避开高峰期,在业务最低谷的时候操作。
删除shard server:
> use admin
switched to db admin
> db.runCommand({"removeshard" : "localhost:30002"});
{
"msg" : "draining started successfully",
"state" : "started",
"shard" : "shard0002",
"ok" : 1
}
很简单,remove就可以了,原来的数据会按照key分配到剩下的shard server上。
最后> db.printShardingStatus()可以查看sharding的信息。注意:操作都是在route process上面,不要登录到shard server操作。

MySQL processes data replication through three modes: asynchronous, semi-synchronous and group replication. 1) Asynchronous replication performance is high but data may be lost. 2) Semi-synchronous replication improves data security but increases latency. 3) Group replication supports multi-master replication and failover, suitable for high availability requirements.

The EXPLAIN statement can be used to analyze and improve SQL query performance. 1. Execute the EXPLAIN statement to view the query plan. 2. Analyze the output results, pay attention to access type, index usage and JOIN order. 3. Create or adjust indexes based on the analysis results, optimize JOIN operations, and avoid full table scanning to improve query efficiency.

Using mysqldump for logical backup and MySQLEnterpriseBackup for hot backup are effective ways to back up MySQL databases. 1. Use mysqldump to back up the database: mysqldump-uroot-pmydatabase>mydatabase_backup.sql. 2. Use MySQLEnterpriseBackup for hot backup: mysqlbackup--user=root-password=password--backup-dir=/path/to/backupbackup. When recovering, use the corresponding life

The main reasons for slow MySQL query include missing or improper use of indexes, query complexity, excessive data volume and insufficient hardware resources. Optimization suggestions include: 1. Create appropriate indexes; 2. Optimize query statements; 3. Use table partitioning technology; 4. Appropriately upgrade hardware.

MySQL view is a virtual table based on SQL query results and does not store data. 1) Views simplify complex queries, 2) Enhance data security, and 3) Maintain data consistency. Views are stored queries in databases that can be used like tables, but data is generated dynamically.

MySQLdiffersfromotherSQLdialectsinsyntaxforLIMIT,auto-increment,stringcomparison,subqueries,andperformanceanalysis.1)MySQLusesLIMIT,whileSQLServerusesTOPandOracleusesROWNUM.2)MySQL'sAUTO_INCREMENTcontrastswithPostgreSQL'sSERIALandOracle'ssequenceandt

MySQL partitioning improves performance and simplifies maintenance. 1) Divide large tables into small pieces by specific criteria (such as date ranges), 2) physically divide data into independent files, 3) MySQL can focus on related partitions when querying, 4) Query optimizer can skip unrelated partitions, 5) Choosing the right partition strategy and maintaining it regularly is key.

How to grant and revoke permissions in MySQL? 1. Use the GRANT statement to grant permissions, such as GRANTALLPRIVILEGESONdatabase_name.TO'username'@'host'; 2. Use the REVOKE statement to revoke permissions, such as REVOKEALLPRIVILEGESONdatabase_name.FROM'username'@'host' to ensure timely communication of permission changes.


Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

SublimeText3 English version
Recommended: Win version, supports code prompts!

ZendStudio 13.5.1 Mac
Powerful PHP integrated development environment

Safe Exam Browser
Safe Exam Browser is a secure browser environment for taking online exams securely. This software turns any computer into a secure workstation. It controls access to any utility and prevents students from using unauthorized resources.

SublimeText3 Chinese version
Chinese version, very easy to use

EditPlus Chinese cracked version
Small size, syntax highlighting, does not support code prompt function
