What is MongoDB

MongoDB is an open source NoSQL database which provides high availability, high performance and auto scaling. It is basically a document oriented database program. This can support faster queries and also can include keys from embedded documents and arrays.One of the attractive feature in MongoDB called Replica Set, which made them widely used in DB replication. The replica set provides automatic failover and data redundancy. MongoDB stores data records as BSON documents, it is basically the binary representation of the Json documents.

What is sharding and its importance

It is a MongoDB process to store data set across different systems. It helps to partition all the data across independent instance and also allows to add more machines based on data growth to your stack. MongoDB uses sharding to support deployments with very large data sets and high throughput operations.

Components for Sharding

The main three sharding components are below,

• Shard : This server is mainly used to store all the data. If this is a production environment each shard is a replica sets and thus it provides high data availability and consistency.
• Config server : This server is for storing cluster metadata, mappings of cluster data set and shards. The mongo query server will use this data to deliver different operations in the production environment. In a production environment we recommend to use at least a minimum of 3 instances.
• Mongos/Query Router : This is where mongo instances runs as application interfaces.The application will make requests to the ‘mongos’ instance, and then ‘mongos’ will deliver the requests using shard key to the shards replica sets.

Mongo Shard with Ansible

To set up this infrastructure I will be using 7 servers. Two config servers, Four Shard servers and one Mongo/query router.

The servers details are as below

Config

• config1.mongo.sysally
• config2.mongo.sysally

Shard

• shard1.mongo.sysally
• shard2.mongo.sysally
• shard3.mongo.sysally
• shard4.mongo.sysally

Mongos/Query Router

• mongo.sysally

We are taking advantage of the Ansible to deploy the Sharded MongoDB clustering.

→ SSH into your Ansible server and edit /root/ansible_playbooks/hosts file.

Create an Ansible inventory file that contains the IP addresses of the machines to which hosts and Mongodb are to be installed. Either hostnames or IPs can be specified.

For eg; vi /root/ansible_playbooks/hosts

→Install mongodb-org package and copy hosts file to all machines using ansible script.
vi mongo_install.yaml
And add the below ansible script.

Run the ansible playbook

ansible-playbook -i inventory mongo_install.yaml

This may take some time to complete, please stand up and walk a round 🙂

Create Config Server Replica Set

ssh to config1 server and config2 server and make the changes as below.

systemctl stop mongod

Edit sections bindip, replication and sharding from vi /etc/mongod.conf

This configuration is to be made in config2 server also just changing the ‘bind IP’

Start mongo service

mongod –config /etc/mongod.conf

Now we need to Initiate replica set name.

—-> SSH to config1

mongo –host config1.mongo.sysally –port 27017

initiate replica set
rs.initiate(
{
_id: “config1”,
configsvr: true,
members: [
{ _id : 0, host : “config1.mongo.sysally:27017” },
{ _id : 1, host : “config2.mongo.sysally:27017” }
]
}
)

Run the following command in mongo shell to see the status,

rs.status()

Create the Shard Replica Sets

Here we are going to create shard replica sets. For that, we select two servers each and set a replica name
→shard1.mongo.sysally’ and ‘shard2.mongo.sysally’ with replica set name: ‘shardA’
→shard3.mongo.sysally’ and ‘shard4.mongo.sysally’ with replica set name: ‘shardB’
In shard1 and shard2 server, change the relevant values in mongod.conf

vi /etc/mongod.conf

Then change bindIP to the server’s IP(corresponding IP of shard1.mongo.sysally/ shard2.mongo.sysally) on which mongo service should run and, the following is to be changed to the values below,

replication:
replSetName: “shardA”
sharding:
clusterRole: shardsvr 

Similarly, in shard3 and shard4 server, change the relevant values in mongodb.conf along with the the servers IP on which mongo service should run

replication:
replSetName: “shardB”
sharding:
clusterRole: shardsvr

Now SSH into the Shard1 and connect to Mongo

→ mongo –host shard1.mongo.sysally –port 27017

rs.initiate(
{
_id : “shardA”,
members: [
{ _id : 0, host : “shard1.mongo.sysally:27017” },
{ _id : 1, host : “shard2.mongo.sysally:27017” }
]
}
)

Again, Now SSH into shard 3 server for shard replica ShardB

rs.initiate(
{
_id : “shardB”,
members: [
{ _id : 0, host : “shard3.mongo.sysally:27017” },
{ _id : 1, host : “shard4.mongo.sysally:27017” }
]
}
)

The following command will show the status of current replica set that we are working on.

rs.status()

How to Configure mongos/Query Router

→ SSH in to Mongo Query Router

systemctl stop mongod

mongos –configdb “config1/config1.mongo.sysally:27017,config2.mongo.sysally:27017” –bind_ip

Open a duplicate shell of the same server

For ‘shardA’ instances:

sh.addShard( “shardA/shard1.mongo.sysally:27017”)

sh.addShard( “shardA/shard2.mongo.sysally:27017”)

For ‘shardB’ instances:

sh.addShard( “shardB/shard3.mongo.sysally:27017”)

sh.addShard( “shardB/shard4.mongo.sysally:27017”)

To check the status of the Sharding please use the below command,

sh.status()

That’s it.

Now you know where to start with, when the application demands more resources and requires high throughput or when you expect fast data growth.

Happy Sharding!

Thanks for dropping by. Ready for the next blog?

Understanding LVE usage and OOM killer in Cloud Linux on Plesk

The post Deploy sharded MongoDB cluster using Ansible appeared first on Sysally.