MongoDB Sharding
Sharding
There is another type of cluster in Mongodb, which is sharding technology, which can meet the needs of MongoDB's massive growth in data volume.
When MongoDB stores massive amounts of data, one machine may not be enough to store the data or provide acceptable read and write throughput. At this time, we can split the data on multiple machines so that the database system can store and process more data.
Why use sharding
Copy all write operations to the primary node
Delayed sensitive data Will query on the primary node
A single replica set is limited to 12 nodes
When the request volume is huge, insufficient memory will occur.
Insufficient local disk
Vertical expansion is expensive
MongoDB sharding
The following figure shows the distribution of the sharded cluster structure in MongoDB:
The above figure mainly has three main components as follows:
Shard:
is used to store actual data blocks. In the actual production environment, a shard server role can be assumed by several machines and a relica set. Preventing a single point of host failure
Config Server:
mongod instance stores the entire ClusterMetadata, including chunk information.
Query Routers:
Front-end routing, the client accesses from this, and makes the entire cluster look like a single database, and the front-end application can Use transparently.
Sharding instance
The port distribution of the sharding structure is as follows:
Shard Server 1：27020 Shard Server 2：27021 Shard Server 3：27022 Shard Server 4：27023 Config Server ：27100 Route Process：40000
Step 1: Start Shard Server
[root@100 /]# mkdir -p /www/mongoDB/shard/s0 [root@100 /]# mkdir -p /www/mongoDB/shard/s1 [root@100 /]# mkdir -p /www/mongoDB/shard/s2 [root@100 /]# mkdir -p /www/mongoDB/shard/s3 [root@100 /]# mkdir -p /www/mongoDB/shard/log [root@100 /]# /usr/local/mongoDB/bin/mongod --port 27020 --dbpath=/www/mongoDB/shard/s0 --logpath=/www/mongoDB/shard/log/s0.log --logappend --fork .... [root@100 /]# /usr/local/mongoDB/bin/mongod --port 27023 --dbpath=/www/mongoDB/shard/s3 --logpath=/www/mongoDB/shard/log/s3.log --logappend --fork
Step 2: Start Config Server
[root@100 /]# mkdir -p /www/mongoDB/shard/config [root@100 /]# /usr/local/mongoDB/bin/mongod --port 27100 --dbpath=/www/mongoDB/shard/config --logpath=/www/mongoDB/shard/log/config.log --logappend --fork
Note:Here we can start it exactly like starting the ordinary mongodb service, without adding the -shardsvr and configsvr parameters. Because the function of these two parameters is to change the startup port, we can just specify the port ourselves.
Step three: Start the Route Process
/usr/local/mongoDB/bin/mongos --port 40000 --configdb localhost:27100 --fork --logpath=/www/mongoDB/shard/log/route.log --chunkSize 500
In the mongos startup parameters, the chunkSize item is used to specify the size of the chunk. The unit is MB. The default size is 200MB.
Step 4: Configuring Sharding
Next, we use MongoDB Shell to log in to mongos and add Shard node
[root@100 shard]# /usr/local/mongoDB/bin/mongo admin --port 40000
MongoDB shell version: 2.0.7
connecting to: 127.0.0.1:40000/admin
mongos> db.runCommand({ addshard:"localhost:27020" })
{ "shardAdded" : "shard0000", "ok" : 1 }
......
mongos> db.runCommand({ addshard:"localhost:27029" })
{ "shardAdded" : "shard0009", "ok" : 1 }
mongos> db.runCommand({ enablesharding:"test" }) #设置分片存储的数据库
{ "ok" : 1 }
mongos> db.runCommand({ shardcollection: "test.log", key: { id:1,time:1}})
{ "collectionsharded" : "test.log", "ok" : 1 }Step 5: There is no need to change much in the program code. Just connect the database to the interface 40000
