May 222011
 

In The last article “Postfix Randomizing Outgoing IP Using TCP_TABLE And Perl“, i was writing about my experiment randomizing outbound ip. someone asked if the script can be mimicked as in the iptables statistic module. indeed, when using the previous script, the results obtained will really random. although that is what the script was created.

I found this wiki explaining about Weighted Round-Robin Scheduling.

Supposing that there is a server set S = {S0, S1, …, Sn-1};
W(Si) indicates the weight of Si;
i indicates the server selected last time, and i is initialized with -1;
cw is the current weight in scheduling, and cw is initialized with zero;
max(S) is the maximum weight of all the servers in S;
gcd(S) is the greatest common divisor of all server weights in S;

while (true) {
    i = (i + 1) mod n;
    if (i == 0) {
        cw = cw - gcd(S);
        if (cw <= 0) {
            cw = max(S);
            if (cw == 0)
            return NULL;
        }
    }
    if (W(Si) >= cw)
        return Si;
}

anyway, I do not have much time to implement that algorithm into the script, so I use an existing perl module List:: util:: WeightedRoundRobin

I’ve modified the script a bit. This script is not exactly imitate iptables stastitic module, but if we set the right weight value, we’re going to get interesting patterns and results.

#!/usr/bin/perl -w
# author: Hari Hendaryanto <hari.h -at- csmcom.com>
use strict;
use warnings;
use Sys::Syslog qw(:DEFAULT setlogsock);
use List::Util::WeightedRoundRobin;
use Storable;

# $count variable latest value stored in file.hash for next script execution
# reference http://www.perlmonks.org/?node_id=510202
my $hashfile="/tmp/file.hash";
store {}, $hashfile unless -r $hashfile;

#
# our transports lists, we will define this in master.cf as transport services
# Queued using Weighted Round-Robin Scheduling
#
my $list = [
    {
        name    => 'smtp1:',
        weight  => 4,
    },
    {
        name    => 'smtp2:',
        weight  => 2,
    },
    {
        name    => 'smtp3:',
        weight  => 2,
    },
    {
        name    => 'smtp4:',
        weight  => 2,
    },
    {
        name    => 'smtp5:',
        weight  => 3,
    },
    {
        name    => 'smtp6:',
        weight  => 2,
    },
    {
        name    => 'smtp7:',
        weight  => 1,
    },
    {
        name    => 'smtp8:',
        weight  => 2,
    },
    {
        name    => 'smtp9:',
        weight  => 2,
    },
    {
        name    => 'smtp10:',
        weight  => 2,
    },

  ];

my $WeightedList = List::Util::WeightedRoundRobin->new();
my $weighted_list = $WeightedList->create_weighted_list( $list );

# $maxinqueue max number of queue in smtp list
my $maxinqueue = scalar(@{$weighted_list});

#
# Initalize and open syslog.
#
openlog('postfix/randomizer','pid','mail');
#
# Autoflush standard output.
#
select STDOUT; $|++;

while (<>) {
	chomp;
	my $count;
	my $hash=retrieve($hashfile);

	# patched by Heartless Mofo <mofoheartless -at- gmail.com>
	# in order to achieve true round robin
	if (time() - $hash->{"skipper"} <= 1)
	{
        	$hash->{"index"}=$hash->{"index"};
	}
	elsif (time() - $hash->{"skipper"} > 1)
	{
        	$hash->{"index"}++;
        	$hash->{"skipper"} = time();
	}
	# end of patch

	if (!defined $hash->{"index"})
	{
        	$count = 0;
	} else {
        	$count = $hash->{"index"};
	}

	if ($count >= $maxinqueue)
	{
        	$hash->{"index"} = 0;
        	$count = 0;
	}

	$hash->{"index"}++;
	store $hash, $hashfile;
	my $random_smtp = ${$weighted_list}[$count];
	if (/^get\s(.+)$/i) {
		print "200 $random_smtp\n";
		syslog("info","Using: %s Transport Service", $random_smtp);
		next;
	}
	print "200 smtp:\n";
}

we can set the script to rotate / queueing output in this manner.

smtp1: smtp2: smtp3: smtp4: smtp5: smtp6: smtp7: smtp8: smtp9: smtp10:

Simply by setting all weight with the same value. we can also set different weight to get a different pattern.

    {
        name    => 'smtp1:',
        weight  => 4,
    },
.....
.....
.....

This simple script bellow also performing weighted random choice. This script require List::Util::WeightedChoice module. the result will be random, but one element can be selected more often by setting weight value greater than others.

#!/usr/bin/perl -w
# author: Hari Hendaryanto <hari.h -at- csmcom.com>
use strict;
use warnings;
use Sys::Syslog qw(:DEFAULT setlogsock);
use List::Util::WeightedChoice qw( choose_weighted );

my $smtp_lists =
[
	'smtp1:',
	'smtp2:',
	'smtp3:',
	'smtp4:',
	'smtp5:',
	'smtp6:',
	'smtp7:',
	'smtp8:',
	'smtp9:',
	'smtp10:'
];
my $weights =
[
	50,
	25,
	1,
	30,
	10,
	5,
	25,
	35,
	45,
	15
];

#
# Initalize and open syslog.
#
openlog('postfix/randomizer','pid','mail');
#
# Autoflush standard output.
#
select STDOUT; $|++;

while (<>) {
	chomp;
	my $random_smtp = choose_weighted( $smtp_lists, $weights );
	if (/^get\s(.+)$/i) {
		print "200 $random_smtp\n";
		syslog("info","Using: %s Transport Service", $random_smtp);
		next;
	}
	print "200 smtp:\n";
}

good luck.

  12 Responses to “Postfix Rotating Outgoing IP Using TCP_TABLE And Perl”

Comments (12)
  1. yes i noticede that too, less than, greater than was not rendered properly. and about the code, i appreciated your fix. i’ll try it later.

    edited:
    iran some quick test on your patch, i think it is works as intended 🙂

    # ./rr.pl 
    get a@example.com
    200 smtp2:
    get b@example.com
    200 smtp5:
    get c@example.com
    200 smtp6:
    get d@example.com
    200 smtp8:
    get d@example
    200 smtp10:
    

    fyi, i’ve never used this rotating script in real production system, i’m using randomizer one http://www.kutukupret.com/2010/12/06/postfix-randomizing-outgoing-ip-using-tcp_table-and-perl/. this is much more simple 🙂

  2. Thanks forever, leenoux!!!

 Leave a Reply

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>

(required)

(required)


*

%d bloggers like this: