Jun 192011

As I’ve been written on my previous post about Maildir Replication using ChironFS in postfix, I will explain step by step it can also be done by using drbd and ocfs2.

Compiling & Installing

note: if your Linux distribution is shipped with a kernel older than 2.6.33 you have to install a kernel module package and packages for the user land code. If your distribution contains a Linux-2.6.33 kernel or newer you only need to install the user land code.

In my case i have my linux distribution older than 2.6.33, so i will also compile drbd kernel module.

Download drbd source and create Binary RPMS packages

$ wget http://oss.linbit.com/drbd/8.3/drbd-8.3.10.tar.gz
$ tar xvzf drbd-8.3.10.tar.gz
$ cd drbd-8.3.10
$ ./configure --enable-spec --with-km
$ cp ../drbd*.tar.gz `rpm -E %_sourcedir`
$ rpmbuild -bb drbd.spec
$ rpmbuild -bb drbd-km.spec
$ sudo rpm -ivh /path/to/RPMS/drbd-*

I’ll be using loop files for this setup since I don’t have access to raw partitions. but if you have raw block device available you can subtitute this part:

resource r0 {
	meta-disk internal;
	device /dev/drbd0;
	disk /dev/loop0;


resource r0 {
	meta-disk internal;
	device /dev/drbd0;
	disk /dev/sdxx;

sdxx can be sda1, sdb1 sdb2 or what ever your raw disk device called

# dd if=/dev/zero of=/drbd-postfix.img bs=1M count=5000
# losetup /dev/loop0 /drbd-postfix.img

Place this DRBD resource file in /etc/drbd.d/r0.res. Be sure to adjust the server names and IP addresses for your servers.

resource r0 {
	meta-disk internal;
	device /dev/drbd0;
	disk /dev/loop0;

	syncer { rate 1000M; }
        net {
                after-sb-0pri discard-zero-changes;
                after-sb-1pri discard-secondary;
                after-sb-2pri disconnect;
	startup { become-primary-on both; }

	on postfix1 { address; }
	on postfix2 { address; }

important: Do this only with a clustered filesystem. If you do this with a non-clustered filesystem like ext2/ext3/ext4 or reiserfs, you will have data corruption.

Now we can create the volume and start DRBD:

# drbdadm create-md r0
# /etc/init.d/drbd start
# chkconfig drbd on

Run the following command on the primary node only:

# drbdsetup /dev/drbd0 primary -o

If nothings gone wrong you should see something like this when you run cat /proc/drbd on both node

[root@postfix1] # cat /proc/drbd
version: 8.3.10 (api:88/proto:86-96)
GIT-hash: 5c0b0469666682443d4785d90a2c603378f9017b build by postfix1, 2011-06-16 15:40:42

 1: cs:Connected ro:Primary/Primary ds:UpToDate/UpToDate C r-----
    ns:18 nr:22 dw:40 dr:1068 al:2 bm:3 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0


[root@postfix2]# cat /proc/drbd
version: 8.3.10 (api:88/proto:86-96)
GIT-hash: 5c0b0469666682443d4785d90a2c603378f9017b build by postfix2, 2011-06-16 20:25:22

 1: cs:Connected ro:Primary/Primary ds:UpToDate/UpToDate C r-----
    ns:27 nr:91 dw:119 dr:1817 al:4 bm:3 lo:0 pe:0 ua:0 ap:0 ep:1 wo:b oos:0

If the secondary node not a primary yet, make it a primary node as well:

# drbdadm disconnect r0
# drbdadm primary r0
# drbdadm connect r0

If everythings run as expected you will see these on both node

[root@postfix1]# drbdadm role r0
[root@postfix2]# drbdadm role r0

We’re now ready to configuring OCFS2 part. Only one package is needed:

# yum -y install ocfs2-tools

Create the /etc/ocfs2 directory and place the following configuration in /etc/ocfs2/cluster.conf

	node_count = 2
	name = postfix

	ip_port = 7788
	ip_address =
	number = 1
	name = postfix1
	cluster = postfix

	ip_port = 7788
	ip_address =
	number = 2
	name = postfix2
	cluster = postfix

Now it’s time to configure OCFS2. Run service o2cb configure and follow the prompts. Use the defaults for all of the responses except for two questions:

  • Answer “y” to “Load O2CB driver on boot”
  • Answer “postfix” to “Cluster to start on boot”

Start OCFS2 and enable it at boot up:

# chkconfig o2cb on
# chkconfig ocfs2 on
# /etc/init.d/o2cb start
# /etc/init.d/ocfs2 start

Create an OCFS2 partition on the primary node only:

# mkfs.ocfs2 -L "postfix" /dev/drbd0

On both node add this line to /etc/fstab

/dev/drbd0  /mnt/postfix  ocfs2  noauto,noatime  0 0

On both node run this commands

# mkdir /mnt/postfix
# chown -R postfix:postfix /mnt/postfix
# mount /dev/drbd0

At this point, you should be all done with drbd and ocfs2 part. If you want to test OCFS2, copy a file into your /mnt/postfix on one node and check if it appears on the other node. If you remove it, it should be gone instantly on both nodes.


# maildirmake /mnt/postfix/test
# ls /mnt/postfix/test
cur  new  tmp

postfix2, we will see same maildir structure as postfix1

# ls /mnt/postfix/test
cur  new  tmp

Now, on postfix part it’s just add/modify this line

virtual_mailbox_base = /mnt/postfix

One of possible schemes:


Postfix drbd ocfs2

Postfix drbd ocfs2

DNS part

example.com.                 MX      10 postfix1.example.com.
example.com.                 MX      10 postfix2.example.com.
postfix1             IN      A       202.xxx.xxx.1
postfix1             IN      A       202.xxx.xxx.2

postfix1.example.com and postfix2.example.com not behave like a primary and backup mx commonly used by people. but they have exactly the same configuration, every email received on each server will be forwarded directly to maildir.

Mysql part

Mysql server could be on a different server, separated from both postfix server. In this scheme I described the mysql server resides on a server postfix1, and listen on the private interface ip address, so postfix2 can also access the mysql server.

On both postfix1 and postfix2 mysql_virtual_mailbox_maps.cf

user = user
password = password
hosts =
dbname = postfixdb
query = SELECT maildir FROM mailbox WHERE username='%s' AND active = 1

Complete virtual mailbox setting on both nodes

virtual_mailbox_maps = mysql:/etc/postfix/mysql_virtual_mailbox_maps.cf
virtual_mailbox_base = /mnt/postfix

As usual, Use extreme caution if you find yourself in this scenario.


 Leave a Reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>