Setting up sharding on a MongoDB collection isn't as complicated as it sounds. In fact, it usually is an overkill. However, once you decide to take the plunge, here's a basic primer on how to go about it. Will be adding the caveats at a later point in time.
Create database
direct: mongos] test> use departments;
switched to db departments
Enable sharding on the database
[direct: mongos] departments> sh.enableSharding('departments')
{
ok: 1,
'$clusterTime': {
clusterTime: Timestamp({ t: 1633085931, i: 3 }),
signature: {
hash: Binary(Buffer.from("0000000000000000000000000000000000000000", "hex"), 0),
keyId: Long("0")
}
},
operationTime: Timestamp({ t: 1633085931, i: 2 })
}
Create collection in the current database
[direct: mongos] departments> db.createCollection('tech')
{ ok: 1 }
Create index, hashed on single column
[direct: mongos] departments> db.tech.createIndex({_id: "hashed"})
_id_hashed
Now, shard the collection
[direct: mongos] departments> sh.shardCollection('departments.tech', {_id: "hashed"})
{
collectionsharded: 'departments.tech',
ok: 1,
'$clusterTime': {
clusterTime: Timestamp({ t: 1633086030, i: 32 }),
signature: {
hash: Binary(Buffer.from("0000000000000000000000000000000000000000", "hex"), 0),
keyId: Long("0")
}
},
operationTime: Timestamp({ t: 1633086030, i: 28 })
}
Insert data, 10-15 documents
[direct: mongos] departments> db.tech.insertOne({name: 'shazazzzm2', empID: 2898})
{
acknowledged: true,
insertedId: ObjectId("6156ea70d336dd9789acaec6")
}
(Recommended) Before running the method, use the flushRouterConfig command to refresh the cached routing table to avoid returning stale distribution information for the collection.
[direct: mongos] departments> db.adminCommand( { flushRouterConfig: "departments.tech" } );
{
flushed: true,
ok: 1,
'$clusterTime': {
clusterTime: Timestamp({ t: 1633087540, i: 1 }),
signature: {
hash: Binary(Buffer.from("0000000000000000000000000000000000000000", "hex"), 0),
keyId: Long("0")
}
},
operationTime: Timestamp({ t: 1633087540, i: 1 })
}
mongos> db.tech.getShardDistribution();
Shard ShardReplSet2 at ShardReplSet2/ip-172-31-22-236:27018
{
data: '0B',
docs: 0,
chunks: 2,
'estimated data per chunk': '0B',
'estimated docs per chunk': 0
}
---
Shard ShardReplSet at ShardReplSet/ip-172-31-27-182:27018
{
data: '0B',
docs: 0,
chunks: 2,
'estimated data per chunk': '0B',
'estimated docs per chunk': 0
}
---
Totals
{
data: '0B',
docs: 0,
chunks: 4,
'Shard ShardReplSet2': [ '0 % data', '0 % docs in cluster', '0B avg obj size on shard' ],
'Shard ShardReplSet': [ '0 % data', '0 % docs in cluster', '0B avg obj size on shard' ]
}
Replica sets
Remember: Run rs.initiate() only on the primary node.
Check replication status
rs.printSecondaryReplicationInfo()
Command quickref
# Used to determine which shard the queried data was found on
db.users.find({userId:123}).explain().queryPlanner.winningPlan
# Check if replica set member is primary/secondary
db.hello()
Log Rotation
Configure logging in the daemon conf
sudo nano /etc/mongod.conf
....
# where to write logging data.
systemLog:
destination: file
logAppend: true
path: /var/log/mongodb/mongod.log
logRotate: reopen
....
Set up the logrotate script. (Note: On mongos, change 'pidof mongod' to 'pidof mongos', and 'create 640 mongodb mongodb' to 'create 640 root root', for some reason this was set to run as root, may need looking at)
sudo nano /etc/logrotate.d/mongod.conf
/var/log/mongodb/mongod.log {
daily
size 100M
rotate 5
missingok
#compress
#delaycompress
#notifempty
create 640 mongodb mongodb
sharedscripts
postrotate
/bin/kill -SIGUSR1 `pidof mongod` >/dev/null 2>&1
endscript
}
Test the rotation
sudo logrotate -f /etc/logrotate.d/mongod.conf
ubuntu@ip-172-31-44-33:~$ ls -lh /var/log/mongodb/
total 48M
-rw-r----- 1 mongodb mongodb 44M Jan 10 10:18 mongod.log
-rw------- 1 mongodb mongodb 4.0M Jan 10 08:18 mongod.log.1
Log rotation can also be triggered from within the shell, mongosh, by running
db.adminCommand( { logRotate : 1 } )
This post first appeared on Guides, Tutorials, A Few Tips And Stories Around Technology, please read the originial post: here