Solution:

The data set for the client was few hundred meg but needed to be up and running with disaster recovery option. So we opt for replicaSet design and setting up servers in two different AWS Availability Zones. Here are few things you should know before starting a replicaSet.

MongoDB config (mongod.conf) is actually a YAML document. YAML is a superset of JSON and it uses indentation (spaces, not TAB) and string quoted in inverted commas. Tricky part is, you can run mongodb with default config and it works perfectly but as soon as you start the replicaSet of mongod, it will start throwing error. You need to treat it like a YAML file.

Setting up DNS-resolvable hostname is really recommended. You just need to create a private VPC domain in Route53 and point the private IP of server to desired hostname.

It’s time to setup the replicaSet:

1- Create a directory /data/db or you can leave it too /var/lib/mongodb

2- Update /etc/mongod.conf it should look like as follows:

$cat /etc/mongod.conf # mongod.conf # for documentation of all options, see: # http://docs.mongodb.org/manual/reference/configuration-options/ # Where and how to store data. storage: dbPath: "/data/db/" journal: enabled: true # engine: # mmapv1: # wiredTiger: # where to write logging data. systemLog: destination: file logAppend: true path: "/var/log/mongodb/mongod.log" # network interfaces net: port: 27017 bindIp: 0.0.0.0 # how the process runs processManagement: timeZoneInfo: "/usr/share/zoneinfo"

#security: #operationProfiling:

replication: oplogSizeMB: 1024 replSetName: "<replsetname>"

#sharding: ## Enterprise-Only Options: #auditLog: #snmp:

You may find few resources saying ‘bindIP should be comment out’ NO! mongo cannot connect with other resources. Either you put static IPs or host names or you leave it to 0.0.0.0 for anyone to connect.

3- you need to make mondb owner of the /var/log/mongodb/mongod.log and /data/db/ to make it work smooth.

4- Perform task 1–3 in all instances being included in replicaSet.

The architecture follows like this:

App →Mongo ReplicaSet (Primary, Secondary and Arbiter)

OR

App →Mongo ReplicaSet (Primary, Secondary and Secondary)

The second option gives a lot more resiliency and help us scaling over multi-AZ fail-over option.

5- Now start the mongod with new config:

$ sudo service mongod restart or $mongod –config /etc/mongod.conf

6- Open Mongo shell on any instance by typing $mongo

7- Create Replica process by creating a Primary

>cfg = {“_id” : “[replication_set_name]”, “members” : [{“_id” : 0,”host” : “[Primary_Host_IP]:27017”}]}

8- Initiate the Replica Set procedure:

Primary> rs.initiate(cfg);

9- Add as many fail overs as you need:

Primary> rs.add(“[Failover_Host_IP]:27017”)

10- Only if you wish to have an arbiter, it’s not necessary

Primary>rs.addArb(“:27017”)

11- To check the status of your replica set.

Primary>rs.status()

12- If you need to restore mongo with dump, do it on the current Primary.