As I recently created a guide on how to setup 5-node MongoDB Replica Set Cluster on Oracle Linux I would like to add notes about setting the same setup on FreeBSD system.

I Will only post the differences between Oracle Linux and FreeBSD setup here, check original post for MongoDB instructions.

To distinguish commands I type on the host system and mongoX node I use two different prompts, this way it should be obvious what command to execute and where.

Command on the host system.

host # command

Command on the mongoX node.

root@mongoX:/ # command

VirtualBox

The machines are generally the same, here is their list.

For FreeBSD setup we would still use VirtualBox NAT Network connectivity for the virtual machines communication.

FreeBSD Install

The FreeBSD install would be generally the same as in the FreeBSD Desktop – Part 2 – Install guide. To not do the same thing 5 times I installed the first node, then after reboot I connected it to the Internet with dhclient em0 command, then added the MongoDB 3.6 package with pkg install -y mongodb36 command and then I cloned that machine into the remaining nodes. Remember to regenerate the MAC address in VirtualBox interfaces for these machines because when two or more machines share the same MAC address funny things will happen 🙂

The table below lists all MongoDB nodes and their IP addresses and roles that we will use.

NODE ADDRESS ROLE mongo0 10.0.10.10/24 DATA mongo1 10.0.10.11/24 DATA mongo2 10.0.10.12/24 DATA mongo3 10.0.10.13/24 DATA mongo4 10.0.10.14/24 ARBITER (does not contain data)

The ‘last’ mongo4 node will be have the ARBITER role while mongo0 to mongo3 nodes will have DATA role.

After FreeBSD installation is finished the tuned /etc/rc.conf file on mongo0 – mongo4 machines looks as follows where X is the node number from 0 to 4.

Main FreeBSD configuration file.

root@mongoX:~ # cat /etc/rc.conf # NETWORK hostname=mongoX ifconfig_em0="inet 10.0.10.1X/24 up" # DAEMONS zfs_enable=YES sshd_enable=YES moused_enable=YES syslogd_flags="-ss" sendmail_enable=NONE # OTHER clear_tmp_enable=YES dumpdev=NO

Boot parameters.

root@mongoX:~ # cat /boot/loader.conf # INSTALLER kern.geom.label.disk_ident.enable=0 kern.geom.label.gptid.enable=0 vfs.zfs.min_auto_ashift=12 zfs_load=YES

Parameters to be set at run time.

root@mongoX:~ # cat /etc/sysctl.conf # INSTALLER security.bsd.see_other_uids=0 security.bsd.see_other_gids=0 security.bsd.unprivileged_read_msgbuf=0 security.bsd.unprivileged_proc_debug=0 kern.randompid=50X0 security.bsd.stack_guard_page=1

FreeBSD Limits

MongoDB prefers that number of files limit should be at least 231471 and number of processes limit should be at least 0.5 times number of files limit. Below is the warning that will be displayed by MongoDB.

2018-04-19T17:12:43.465+0200 I CONTROL [initandlisten] ** WARNING: soft rlimits too low. rlimits set to 12085 processes, 231471 files. Number of processes should be at least 115736 : 0.5 times number of files.

To make numbers aligned we will use 256 * 1024 for files and 128 * 1024 for processes.

host # expr 256 \* 1024 262144

host # expr 128 \* 1024 131072

Set the boot parameters.

host # for I in 0 1 2 3 4; do ssh -p 220${I} root@localhost 'cat >> /boot/loader.conf << __EOF # FASTBOOT autoboot_delay=1 # MONGODB LIMITS kern.maxproc=131072 __EOF' done

… and the run time parameters.

host # for I in 0 1 2 3 4; do ssh -p 220${I} root@localhost 'cat >> /etc/sysctl.conf << __EOF # MONGODB LIMITS kern.maxfiles=262144 kern.maxfilesperproc=262144 kern.maxprocperuid=131072 __EOF' done host #

Reboot to make the boot parameters take effect.

host # for I in 0 1 2 3 4; do ssh -p 220${I} root@localhost reboot; done

MongoDB Startup Script

In order to MongoDB work correctly on FreeBSD one parameter needs to be added to the MongoDB – mongod – startup scipt. The parameter --setParameter=disabledSecureAllocatorDomains=\* needs to be added to the mongod_flags parameter. You may modify the /usr/local/etc/rc.d/mongod script or You can specify ‘custom’ mongod_flags parameter in /etc/rc.conf file which we will do here.

In case You would want to patch the /usr/local/etc/rc.d/mongod script here is the diff(1) .

root@mongo0:~ # diff -u /root/mongod.ORG /usr/local/etc/rc.d/mongod --- /root/mongod.ORG 2018-04-25 20:50:04.326644000 +0200 +++ /usr/local/etc/rc.d/mongod 2018-04-25 20:50:14.019943000 +0200 @@ -30,7 +30,7 @@ : ${mongod_enable="NO"} : ${mongod_limits="NO"} : ${mongod_dbpath="/var/db/mongodb"} -: ${mongod_flags="--logpath ${mongod_dbpath}/mongod.log --logappend"} +: ${mongod_flags="--logpath ${mongod_dbpath}/mongod.log --logappend --setParameter=disabledSecureAllocatorDomains=\*"} : ${mongod_user="mongodb"} : ${mongod_group="mongodb"} : ${mongod_config="/usr/local/etc/mongodb.conf"}

To put the changes together, here is the /etc/rc.conf file after needed changes on all MongoDB nodes.

root@mongoX:~ # cat /etc/rc.conf # NETWORK hostname=mongoX ifconfig_em0="inet 10.0.10.1X/24 up" # DAEMONS zfs_enable=YES sshd_enable=YES moused_enable=YES syslogd_flags="-ss" sendmail_enable=NONE # OTHER clear_tmp_enable=YES dumpdev=NO # MONGODB mongod_enable=YES mongod_flags="--logpath /var/db/mongodb/mongod.log --logappend --setParameter=disabledSecureAllocatorDomains=\*"

Time Daemon

As with every cluster we need to synchronize time between cluster nodes, so we will enable ntpd(8) here.

root@mongoX:~ # sysrc ntpd_enable=YES ntpd_enable: NO -> YES

root@mongoX:~ # service ntpd start Starting ntpd.

Here is final /etc/rc.conf file after adding the ntpd(8) time daemon.

root@mongoX:~ # cat /etc/rc.conf # NETWORK hostname=mongoX ifconfig_em0="inet 10.0.10.1X/24 up" # DAEMONS zfs_enable=YES sshd_enable=YES ntpd_enable=YES moused_enable=YES syslogd_flags="-ss" sendmail_enable=NONE # OTHER clear_tmp_enable=YES dumpdev=NO # MONGODB mongod_enable=YES mongod_flags="--logpath /var/db/mongodb/mongod.log --logappend --setParameter=disabledSecureAllocatorDomains=\*"

… and thats about the changes, You may now start the MongoDB instances on each node and create the cluster as specified in original article.

EOF