May 222011
 

In The last article “Postfix Randomizing Outgoing IP Using TCP_TABLE And Perl“, i was writing about my experiment randomizing outbound ip. someone asked if the script can be mimicked as in the iptables statistic module. indeed, when using the previous script, the results obtained will really random. although that is what the script was created.

I found this wiki explaining about Weighted Round-Robin Scheduling.

Supposing that there is a server set S = {S0, S1, …, Sn-1};
W(Si) indicates the weight of Si;
i indicates the server selected last time, and i is initialized with -1;
cw is the current weight in scheduling, and cw is initialized with zero;
max(S) is the maximum weight of all the servers in S;
gcd(S) is the greatest common divisor of all server weights in S;

while (true) {
    i = (i + 1) mod n;
    if (i == 0) {
        cw = cw - gcd(S);
        if (cw <= 0) {
            cw = max(S);
            if (cw == 0)
            return NULL;
        }
    }
    if (W(Si) >= cw)
        return Si;
}

anyway, I do not have much time to implement that algorithm into the script, so I use an existing perl module List:: util:: WeightedRoundRobin

I’ve modified the script a bit. This script is not exactly imitate iptables stastitic module, but if we set the right weight value, we’re going to get interesting patterns and results.

#!/usr/bin/perl -w
# author: Hari Hendaryanto <hari.h -at- csmcom.com>
use strict;
use warnings;
use Sys::Syslog qw(:DEFAULT setlogsock);
use List::Util::WeightedRoundRobin;
use Storable;

# $count variable latest value stored in file.hash for next script execution
# reference http://www.perlmonks.org/?node_id=510202
my $hashfile="/tmp/file.hash";
store {}, $hashfile unless -r $hashfile;

#
# our transports lists, we will define this in master.cf as transport services
# Queued using Weighted Round-Robin Scheduling
#
my $list = [
    {
        name    => 'smtp1:',
        weight  => 4,
    },
    {
        name    => 'smtp2:',
        weight  => 2,
    },
    {
        name    => 'smtp3:',
        weight  => 2,
    },
    {
        name    => 'smtp4:',
        weight  => 2,
    },
    {
        name    => 'smtp5:',
        weight  => 3,
    },
    {
        name    => 'smtp6:',
        weight  => 2,
    },
    {
        name    => 'smtp7:',
        weight  => 1,
    },
    {
        name    => 'smtp8:',
        weight  => 2,
    },
    {
        name    => 'smtp9:',
        weight  => 2,
    },
    {
        name    => 'smtp10:',
        weight  => 2,
    },

  ];

my $WeightedList = List::Util::WeightedRoundRobin->new();
my $weighted_list = $WeightedList->create_weighted_list( $list );

# $maxinqueue max number of queue in smtp list
my $maxinqueue = scalar(@{$weighted_list});

#
# Initalize and open syslog.
#
openlog('postfix/randomizer','pid','mail');
#
# Autoflush standard output.
#
select STDOUT; $|++;

while (<>) {
	chomp;
	my $count;
	my $hash=retrieve($hashfile);

	# patched by Heartless Mofo <mofoheartless -at- gmail.com>
	# in order to achieve true round robin
	if (time() - $hash->{"skipper"} <= 1)
	{
        	$hash->{"index"}=$hash->{"index"};
	}
	elsif (time() - $hash->{"skipper"} > 1)
	{
        	$hash->{"index"}++;
        	$hash->{"skipper"} = time();
	}
	# end of patch

	if (!defined $hash->{"index"})
	{
        	$count = 0;
	} else {
        	$count = $hash->{"index"};
	}

	if ($count >= $maxinqueue)
	{
        	$hash->{"index"} = 0;
        	$count = 0;
	}

	$hash->{"index"}++;
	store $hash, $hashfile;
	my $random_smtp = ${$weighted_list}[$count];
	if (/^get\s(.+)$/i) {
		print "200 $random_smtp\n";
		syslog("info","Using: %s Transport Service", $random_smtp);
		next;
	}
	print "200 smtp:\n";
}

we can set the script to rotate / queueing output in this manner.

smtp1: smtp2: smtp3: smtp4: smtp5: smtp6: smtp7: smtp8: smtp9: smtp10:

Simply by setting all weight with the same value. we can also set different weight to get a different pattern.

    {
        name    => 'smtp1:',
        weight  => 4,
    },
.....
.....
.....

This simple script bellow also performing weighted random choice. This script require List::Util::WeightedChoice module. the result will be random, but one element can be selected more often by setting weight value greater than others.

#!/usr/bin/perl -w
# author: Hari Hendaryanto <hari.h -at- csmcom.com>
use strict;
use warnings;
use Sys::Syslog qw(:DEFAULT setlogsock);
use List::Util::WeightedChoice qw( choose_weighted );

my $smtp_lists =
[
	'smtp1:',
	'smtp2:',
	'smtp3:',
	'smtp4:',
	'smtp5:',
	'smtp6:',
	'smtp7:',
	'smtp8:',
	'smtp9:',
	'smtp10:'
];
my $weights =
[
	50,
	25,
	1,
	30,
	10,
	5,
	25,
	35,
	45,
	15
];

#
# Initalize and open syslog.
#
openlog('postfix/randomizer','pid','mail');
#
# Autoflush standard output.
#
select STDOUT; $|++;

while (<>) {
	chomp;
	my $random_smtp = choose_weighted( $smtp_lists, $weights );
	if (/^get\s(.+)$/i) {
		print "200 $random_smtp\n";
		syslog("info","Using: %s Transport Service", $random_smtp);
		next;
	}
	print "200 smtp:\n";
}

good luck.

  12 Responses to “Postfix Rotating Outgoing IP Using TCP_TABLE And Perl”

Comments (12)
  1. Incoming mail rejecting…. Log is appended here

    Dec 24 07:30:51 host postfix/smtpd[6219]: 989408406FB: client=mail-ve0-f174.google.com[209.85.128.174]
    Dec 24 07:30:51 host postfix/randomizer[6225]: Using: smtp16: Transport Service
    Dec 24 07:30:51 host postfix/randomizer[6225]: Using: smtp17: Transport Service
    Dec 24 07:30:51 host postfix/smtpd[6219]: 989408406FB: reject: RCPT from mail-ve0-f174.google.com[209.85.128.174]: 550 5.1.1 : Recipient address rejected: User unknown in local recipient table; from= to= proto=ESMTP helo=
    Dec 24 07:30:51 host postfix/cleanup[6226]: 989408406FB: message-id=
    Dec 24 07:30:51 host opendkim[2281]: 989408406FB: mail-ve0-f174.google.com [209.85.128.174] not internal
    Dec 24 07:30:51 host opendkim[2281]: 989408406FB: not authenticated
    Dec 24 07:30:51 host opendkim[2281]: 989408406FB: DKIM verification successful
    Dec 24 07:30:51 host opendkim[2281]: 989408406FB: s=20120113 d=gmail.com SSL
    Dec 24 07:30:51 host postfix/qmgr[6216]: 989408406FB: from=, size=2010, nrcpt=2 (queue active)
    Dec 24 07:30:51 host postfix/randomizer[6225]: Using: smtp18: Transport Service
    Dec 24 07:30:51 host postfix/randomizer[6225]: Using: smtp1: Transport Service
    Dec 24 07:30:51 host postfix/smtpd[6219]: disconnect from mail-ve0-f174.google.com[209.85.128.174]
    Dec 24 07:30:51 host postfix-rotate1/smtp[6227]: 989408406FB: to=, relay=none, delay=0.34, delays=0.18/0.01/0.14/0, dsn=5.4.6, status=bounced (mail for amritanews.com loops back to myself)
    Dec 24 07:30:51 host postfix-rotate1/smtp[6228]: 989408406FB: to=, relay=none, delay=0.34, delays=0.18/0.01/0.14/0, dsn=5.4.6, status=bounced (mail for amritanews.com loops back to myself)
    Dec 24 07:30:51 host postfix/cleanup[6226]: E8CF38406FD: message-id=
    Dec 24 07:30:51 host postfix/qmgr[6216]: E8CF38406FD: from=, size=4273, nrcpt=1 (queue active)
    Dec 24 07:30:51 host postfix/randomizer[6225]: Using: smtp2: Transport Service
    Dec 24 07:30:51 host postfix/bounce[6230]: 989408406FB: sender non-delivery notification: E8CF38406FD
    Dec 24 07:30:51 host postfix/qmgr[6216]: 989408406FB: removed
    Dec 24 07:30:52 host postfix-rotate1/smtp[6231]: connect to gmail-smtp-in.l.google.com[2607:f8b0:400d:c02::1a]:25: Network is unreachable
    Dec 24 07:30:52 host postfix-rotate1/smtp[6231]: E8CF38406FD: to=, relay=gmail-smtp-in.l.google.com[173.194.68.26]:25, delay=0.99, delays=0.01/0.01/0.59/0.38, dsn=2.0.0, status=sent (250 2.0.0 OK 1387888253 j7si17727126qab.167 – gsmtp)
    Dec 24 07:30:52 host postfix/qmgr[6216]: E8CF38406FD: removed

  2. First of all congratulation for you work and thank you for sharing.
    I build a test server with few public IPs, used you script and everything works great up to the point where I’m trying to randomize slow transport maps.
    Tried this suggestion http://www.kutukupret.com/2010/12/06/postfix-randomizing-outgoing-ip-using-tcp_table-and-perl/#comment-1026
    but the message matching the regex goes through that specific transport and then to the default IP bypassing the randomizer.

    $hash->{“index”}++;
    store $hash, $hashfile;
    my $random_smtp = ${$weighted_list}[$count];

    if (/^get\s+(.+@+exampledomain+\..{2,6})/) {
    print “200 slowest:\n”;
    next;
    }

    if (/^get\s(.+)$/i) {
    my $rcpt_domain = $1;
    print “200 $random_smtp\n”;
    syslog(“info”,”using transport: service for: “, $random_smtp, $rcpt_domain);
    next;
    }

    Unfortunately your script was my first interaction with perl and it’s difficult for me to make substantial modifications to it.

    Any suggestions?

    Thanks
    Mike

  3. my original pearl script was not involving any weighted feature , if you follow that, that should be works fine.

  4. Hi, I modified your original script, kept the custom transport maps settings in master.cf and settings in main.cf.
    When sending an email I get the same behavior as with weighted script, it’s passed through that transport and then relayed through default IP.
    Should I replicate the random smtp client services in master.cf and add the settings specific to that transport?

    Mike

  5. Do you think it would be possible to fall-back from IPv6 to IPv4 address when receiving smtp server doesn’t support it?

    David

  6. As M. Thaha K pointed out, this script will prevent your postfix server from receiving email.

    I modified the while loop to allow receiving of email.

    In the regex where it says @mydomain\.com just substitute for your domain — actually the domain you have set in the mydomain variable of main.cf.

    Also it now prints in the logs what request it is reacting for.

    while () {
    	chomp;
    	my $dest = ($_);
    	my $count;
    	my $hash=retrieve($hashfile);
    
    	if (!defined $hash->{"index"})
    	{
    		$count = 0;
    	} else {
    		$count = $hash->{"index"};
    	}
    
    	if ($count >= $maxinqueue)
    	{
    		$hash->{"index"} = 0;
    		$count = 0;
    	}
    
    	$hash->{"index"}++;
    	store $hash, $hashfile;
    	my $random_smtp = ${$weighted_list}[$count];
    
    	if (/^get\s(.+)\@mydomain\.com$/i) {
    		print "200 local\n";
    		next;
    	}
    
    	#else f (/^get\s(.+)$/i) {
    	elsif (/^get\s(.+)\@(.+)$/i) {
    		print "200 $random_smtp\n";
    		syslog("info","Using: %s Transport Service for %s", $random_smtp, $dest);
    		next;
    	}
    	print "200 smtp:\n";
    }
    
  7. I also noticed in the mail logs that I will connect to send an email and postfix will make two requests for the same transport.

    e.g. mofoheartless@gmail.com – it will request a transport twice for that address, say smtp10 and smtp11, but only smtp11 will be used to actualy deliver mail.

    To make matters worse postfix gets a transport twice from the same randomizer process (same PID) so it is useless to write the tcp table request to the hash file and use that to compare with the the following request and tell postfix to use the same transport for the same address.

    Because it happens almost instantly, in the same process!

    So smtp10 is simply ‘skipped’ and the point of rotation kind of breaks because you are using only half of the transports to deliver. It wrecks the point of round robin.

    So I read http://www.kutukupret.com/2011/11/15/postfix-changing-outgoing-ip-by-time-interval-using-tcp_table-and-perl/ and I had this idea:

    Put this code before ” if (!defined $hash->{“index”}) ” :

    if (time() – $hash->{“skipper”} {“index”}=$hash->{“index”};
    }

    elsif (time() – $hash->{“skipper”} > 1)
    {
    $hash->{“index”}++;
    $hash->{“skipper”} = time();
    }

    Now all our transports are used in true round robin fashion!! Wonderful!

    It may use the same IP if there are many parallel deliveries but that is a OK trade-off if our deliveries are now distributed with the exact weights we chose.

    5kthx leenoux! Your scripts rock!!! You’re a life saver for the postfix community.

    But I might give haraka a try some of these days.

    Hey if you’re into PERL you should hang around everything2.com and read some crazy stuff sometimes, maybe write.

    That site was into the real cradle of PERL together with slashdot and it mostly predates any other 2.0 that I know of.

    Almost every crazy thing I learned, I learned from e2 🙂

  8. That should be:

    if (time() – $hash->{“skipper”} {“index”}=$hash->{“index”};
    }

    elsif (time() – $hash->{“skipper”} > 1)
    {
    $hash->{“index”}++;
    $hash->{“skipper”} = time();
    }

    I have no idea what ate the code 🙂

  9. Oh I get it, it’s opening a HTML tag…

    I will use the amp HTML code:

        if (time() - $hash->{"skipper"} <= 1)
        {
               $hash->{"index"}=$hash->{"index"};
        }
        elsif (time() - $hash->{"skipper"} > 1)
        {
                $hash->{"index"}++;
                $hash->{"skipper"} = time();
        }
    

    < is the “less than” sign in case this doesn’t render properly.

    Damn.

  10. yes i noticede that too, less than, greater than was not rendered properly. and about the code, i appreciated your fix. i’ll try it later.

    edited:
    iran some quick test on your patch, i think it is works as intended 🙂

    # ./rr.pl 
    get a@example.com
    200 smtp2:
    get b@example.com
    200 smtp5:
    get c@example.com
    200 smtp6:
    get d@example.com
    200 smtp8:
    get d@example
    200 smtp10:
    

    fyi, i’ve never used this rotating script in real production system, i’m using randomizer one http://www.kutukupret.com/2010/12/06/postfix-randomizing-outgoing-ip-using-tcp_table-and-perl/. this is much more simple 🙂

  11. Thanks forever, leenoux!!!

 Leave a Reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

(required)

(required)

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.