Archive for: ‘July 2013’

Deploying DRBD on Linux

July 6, 2013 Posted by admin

As you probably know RHEL 6 / CentOS 6 does not have DRBD on any of the yum repository. You need a support contract with RHEL to get DRBD (they partnered with Linbit after it was decided to not support DRBD in EL6 because DRBD didn’t get into the mainline kernel until 2.6.33, and EL6 has 2.6.32. However you can alternatively install it from source or rpm packages or use the yum ELRepo repository (http://elrepo.org).

Option 1 – RPM Packages
# wget http://dl.atrpms.net/el6-`arch`/atrpms/stable/drbd-kmdl-`uname -r`-8.3.8.1-30.el6.`arch`.rpm
# wget http://dl.atrpms.net/el6-x86_64/atrpms/stable/drbd-8.3.8.1-30.el6.`arch`.rpm
# rpm –ivh drbd-kmdl-`uname -r`-8.3.8.1-30.el6.`arch`.rpm drbd-8.3.8.1-30.el6.`arch`.rpm

Option 2 – ELRepo
# rpm -Uvh http://elrepo.org/elrepo-release-6-4.el6.elrepo.noarch.rpm
# vi /etc/yum.repos.d/elrepo.repo
enabled=0
# yum –enablerepo=elrepo install drbd83-utils kmod-drbd83

Lets now set up our DRBD device. We will create a 40GB LVM to be used by DRBD.
First of all, verify that on both servers ntpd is running and that the time is equal.

[root@c6server1 ~]# service ntpd status
ntpd (pid 1072) is running…
[root@c6server1 ~]# date
Mon Aug 15 10:48:14 PDT 2011
[root@c6server2 ~]# service ntpd status
ntpd (pid 1088) is running…
[root@c6server2 ~]# date
Mon Aug 15 10:48:14 PDT 2011

Create the Logical Volume on both servers:
[root@c6server1 ~]# lvcreate -L 40GB -n drbd-main vg_c6server1
[root@c6server2 ~]# lvcreate -L 40GB -n drbd-main vg_c6server2

~ Notice that each of my servers have different Volume Groups (vg_c6server1 and vg_c6server2). If you copy and past that you will need to make sure that you change them to whatever you have named.
!!!!! VERY IMPORTANT – DON’T FORMAT AND MOUNT THE VOLUME !!!!!
Save the original global configuration file on both servers:
mv /etc/drbd.d/global_common.conf /etc/drbd.d/global_common.sample
and create a new file:

# vi /etc/drbd.d/global_common.conf

global { usage-count no; }
common {
syncer { rate 10M; }
}

Create a new resource file that I called “main”, on both servers they have to be equal:
# vi /etc/drbd.d/main.res
resource main {
protocol C;
startup { wfc-timeout 0; degr-wfc-timeout 120; }
disk { on-io-error detach; }
on c6server1 {
device /dev/drbd0;
disk /dev/vg_c6server1/drbd-main;
meta-disk internal;
address 10.0.0.1:7788;
}
on centos6-2 {
device /dev/drbd0;
disk /dev/vg_c6server2/drbd-main;
meta-disk internal;
address 10.0.0.2:7788;
}
}

Now we have to create the metadata on both servers:

[root@c6server1]# drbdadm create-md main

[root@c6server2]# drbdadm create-md main

and we can now start DRBD on both servers at the same time:

[root@c6server1 drbd.d]# service drbd start
Starting DRBD resources: [ d(main) s(main) n(main) ].

[root@c6server2 drbd.d]# service drbd start
Starting DRBD resources: [ d(main) s(main) n(main) ].

You have two ways that I know to verify that DRBD is running properly:
# service drbd status
# cat /proc/drbd
version: 8.3.11 (api:88/proto:86-96)
GIT-hash: 0de839cee13a4160eed6037c4bddd066645e23c5 build by dag@Build64R6, 2011-08-08 08:54:05
0: cs:Connected ro:Secondary/Secondary ds:Inconsistent/Inconsistent C r—–
ns:0 nr:0 dw:0 dr:0 al:0 bm:0 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:41941724

As you can see they say that it is connected but ro is Secondary/Secondary meaning that we haven’t told the system which one is the Primary server (master) that contains the block to be replicated. Once we tell the system who is the master it will start the synchronization.

We will tell DRBD that server1 is the Primary server:
On server1:

# drbdsetup /dev/drbd0 primary –o

# cat /proc/drbd
version: 8.3.11 (api:88/proto:86-96)
GIT-hash: 0de839cee13a4160eed6037c4bddd066645e23c5 build by dag@Build64R6, 2011-08-08 08:54:05
0: cs:SyncSource ro:Primary/Secondary ds:UpToDate/Inconsistent C r—–
ns:503808 nr:0 dw:0 dr:504472 al:0 bm:30 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:41437916
[>………………..] sync’ed: 1.3% (40464/40956)M
finish: 1:06:40 speed: 10,340 (10,280) K/sec

The synchronization started and it will take a little while to be completed. Please wait until it is done and move to the next step.

0: cs:Connected ro:Primary/Secondary ds:UpToDate/UpToDate C r—–
ns:41941724 nr:0 dw:0 dr:41942388 al:0 bm:2560 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0

Now that our servers are in sync we can create format our /dev/drbd0 with our preferred file system. In my case I will use ext4.

On server1:
# mkfs.ext4 /dev/drbd0