This time i’ll show you how to randomize your smtp outbound’s IP addresses. This can be done via transport map. But, since ordinary Postfix lookup tables store information as (key, value) pairs. it will provide static value only. we need someting that can manipulate the value (right hand side) of a lookup table. In order to answer random transport value.
first come to mind was tcp_tables, tcp_tables lookup table gives some flexibility for us to execute our tiny perl script that will randomizing transport. that’s the basic idea.
Ok, here’s the first part, create perl script call random.pl, anyway this script only provide answer in “catch-all” manner. so it will randomized, all outgoing mail.
# cd /etc/postfix # vi random.pl
#!/usr/bin/perl -w
# author: Hari Hendaryanto <hari.h -at- csmcom.com>
use strict;
use warnings;
use Sys::Syslog qw(:DEFAULT setlogsock);
#
# our transports array, we will define this in master.cf as transport services
#
our @array = (
'rotate1:',
'rotate2:',
'rotate3:',
'rotate4:',
'rotate5:'
);
#
# Initalize and open syslog.
#
openlog('postfix/randomizer','pid','mail');
#
# Autoflush standard output.
#
select STDOUT; $|++;
while (<>) {
chomp;
# randomizing transports array
my $random_smtp = int(rand(scalar(@array)));
if (/^get\s(.+)$/i) {
print "200 $array[$random_smtp]\n";
syslog("info","Using: %s Transport Service", $random_smtp);
next;
}
print "200 smtp:";
}
Make it executable
# chmod 755 random.pl
master.cf parts
Run the scripts via postfix spawn daemon service.
127.0.0.1:2527 inet n n n - 0 spawn
user=nobody argv=/etc/postfix/random.pl
add 5 smtp client services called rotate1, rotate2, rotate3, rotate4, rotate5, that bind to its own ip
address and has uniq syslog/helo name.
# random smtp
rotate1 unix - - n - - smtp
-o syslog_name=postfix-rotate1
-o smtp_helo_name=smtp1.example.com
-o smtp_bind_address=1.2.3.1
rotate2 unix - - n - - smtp
-o syslog_name=postfix-rotate2
-o smtp_helo_name=smtp2.example.com
-o smtp_bind_address=1.2.3.2
rotate3 unix - - n - - smtp
-o syslog_name=postfix-rotate3
-o smtp_helo_name=smtp3.example.com
-o smtp_bind_address=1.2.3.3
rotate4 unix - - n - - smtp
-o syslog_name=postfix-rotate4
-o smtp_helo_name=smtp4.example.com
-o smtp_bind_address=1.2.3.4
rotate5 unix - - n - - smtp
-o syslog_name=postfix-rotate5
-o smtp_helo_name=smtp5.example.com
-o smtp_bind_address=1.2.3.5
Before we actually implement our randomize transport, let’s make sure that the setting actually work.
Reload postfix
# postfix reload
Run this query fiew times, and you’ll see the perl script will return “random answer” transport
# postmap -q "whatever" tcp:127.0.0.1:2527 rotate1:
# postmap -q "whatever" tcp:127.0.0.1:2527 rotate5:
And so on..
Note on “whatever”, since the script acted in “catch-all” mode as i’ve mentioned earlier, what ever postfix transport_maps client asked. it will be answered with random values such as rotate1, rotate2, rotate3, rotate4, rotate5 in randomized fashion.
main.cf parts
Add these lines
transport_maps = tcp:[127.0.0.1]:2527 127.0.0.1:2527_time_limit = 3600s
Reload postfix
that’s it. example log would be like these and that’s indicate that randomizer is working.
Month date 12:26:53 host postfix-rotate1/smtp[4252]: A1CA68480A4: to=<xxx@example.com>, relay=mx.example.com.com[xx.xx.xxx.xx]:25], delay=3.6, delays=0.69/0.01/0.81/2, dsn=2.0.0, status=sent (250 ok dirdel) --snip-- Month date 12:27:06 host postfix-rotate5/smtp[4253]: 41C2E8480A4: to=<xxx@example.net>, relay=mx.example.net[xx.xxx.xxx.xxx]:25], delay=6, delays=0.14/0.01/0.85/5, dsn=2.0.0, status=sent (250 ok dirdel) --snip-- Month date 12:27:22 host postfix-rotate3/smtp[4277]: 4BA9F8480A4: to=<xxx@example.org>, relay=mx.example.org[xx.xxx.xx.xxx]:25], delay=7.9, delays=0.85/0.02/0.61/6.4, dsn=2.0.0, status=sent (250 ok dirdel)
disclaimer:
I’m not taking any responsible if the reader “misuse” this tutorial.the tutorial is provide as-is for experimental purposes.
You may also want to read these posts:
- Postfix Rotating Outgoing IP Using TCP_TABLE And Perl
- Postfix Changing Outgoing IP By Time Interval Using TCP_TABLE And Perl
- Postfix Bind Sender Outgoing IP, Based On GeoIP Location
- Postfix Bind Sender Domain To Dedicated Outgoing IP Address
- postfix, integrating memcache as a lookup table using tcp_table
- Postfix Create Blackhole For Authenticated User’s Outgoing Submission (revised)
- Postfix smtp outgoing load balancing
- Measure Response Time Of SMTP Connections Using Perl
- Postfix Smtp Outgoing IP Rotator using iptables
- Postfix GeoIP Based Rejections
I like your new tut, would probably perform better with more IPs than your IPtables tut… What do you think would be the max of IPs a setup like this could handle to rotate?
it depend on hardware resource, i think. as usual, this tutorial is just a prototype, but it work.
I will try it out at the beginning of the year and let you know how it performs
Thank you, this is very neat.
Do not forget to add best_mx_transport = local of you may face the dreaded “mail loops back to myself”.
Also, I modified your script to add “weight” to the SMTP clients, so that randomness is not truly random. This is useful if you add a new IP address to the pool, so it does not get blacklisted as soon as it starts sending emails.
my %pool = (
“smtp1″ => 20,
“smtp2″ => 20,
“smtp3″ => 20,
“smtp4″ => 40
);
Last but not least, how do you maintain specific transport for specific domains? Eg some SMTP servers require a “slow transport” such as what is described in http://www.pubbs.net/postfix/200909/98526/
You would need to process the request in order to check the target domain, to see if you return a random SMTP or a specific transport.
Regards,
Amaury
that’s cool with weighted feature you’ve added. fyi, the reason i’ve wrote this script was not for doing spammy things. this just for fun in my spare time
i work in legal ISP
if you want to filter specific domain/destination to certain transport. you have just to “hardcoded” in the script.
create conditional filter before default action/randomize. ie
if (/^get\s+(yahoo\.com) { print "200 to_slow_yahoo:\n"; next; fi .... .... if (/^get\s+(.+)$/i) { my $rcpt_domain = $1; print "200 $random_smtp\n"; syslog("info","using transport: <%s> service for: <%s>", $random_smtp, $rcpt_domain); next; }to_slow_yahoo: can be randomize too with slow transport specific settings and specific ip pools
says, …to_slow_yahoo1: to_slow_yahoo2: to_slow_yahoo3: etc…
this untested yet
Regarding specific transports I came up with a similar solution, although it is used the other way around:
- normal mail go through a specific transport name “standard”
- mail sent to mailservers using conservative SMTP rates (such as RIM SMTP relays or Orange MXs) go through the randomizer. This way only the emails which need to be randomized are randomized, and they can be sent without much delay to their recipient.
This is not very efficient as you lose the “db” format which is favored by Postfix for this matter.
Last but not least you should remember that this technique (known as “snowshoeing” in the email deliverability community) is frowned upon and that it should not be abused, even to perform legit operations…
about postfix “db”, it’s not quite true that you’re loosing them. you can still access them through perl
little example, for “hash” lookup table
#!/usr/bin/perl use Fcntl; use DB_File; my %tab; my $null=chr(0); tie %tab,'DB_File','testmap.db',O_RDONLY,0400,$DB_HASH; # Sample query my $key='foo'; my $value=$tab{$key.$null}; chop $value; # chop null byte print $key." = ".$value."\n";hash file
i know this technique similar to snowshoeing
. that’s why i put disclaimer at the bottom of my posting.
however, you can tune your outgoing not too greedy with concurency recipients, messages, or connection, set your dns reverse.
if your traffic are legit, it’s like you own 100 machine in single box if you have 100 ip
Hi,
could you please post your modificated version with “weight” implemented? I try to implement it, but it is not working… I am not a very good programmer
Thank you
Lukas
“Do not forget to add best_mx_transport = local of you may face the dreaded “mail loops back to myself””
If you use postfix with virtuals users :
best_mx_transport = virtual
Regards,
Cidou
since i used it just for smtp relay, in experiment, no local/virtual user, so it doesn’t really matter.
but thanks for noticing anyway
I’m finally implementing this for some testing, but I have stumbled across something.
I already have a transport_maps entry pointing:
transport_maps = hash:/etc/postfix/transport
In that file I have entries for specific domains that will use another relay server, e.g.
ch***.com smtp:***.***.211.74
Any ideas on how I can combine my existing transport_maps with yours?
try this
Works great
Do you have an idea on how to implement this based on sending domain?
E.g. mails from domain1.com get a random in the perl from rotate1 – rotate3, domain2.com get random in the perl from rotate4-rotate9, domain3.net get random in the perol from rotate10-rotate12 – and the master contains the IP, helo_name etc
Basically what I am looking for, is a way to tell the perl from which array it should take a random based on the senderdomain.
that cannot be done in transport_maps, maybe you can use sender_dependent_relayhost_maps. i’ve never try it, though.
Hello,
Thanks for your great tutorials.
With reference to:
Amaury says: january 22, 2011 at 2:14 am
my %pool = (
“smtp1″ => 20,
“smtp2″ => 20,
“smtp3″ => 20,
“smtp4″ => 40
);
I am a newbie and I would like to add that feature.
Could you please add this feature to your tutorial?
I do not know how or where.
i think it’s not as simple by adding %pool hash in original code. that’s only variable. but basically that refer to “weight value” you can access with:
$pool{smtp1} $pool{smtp2} $pool{smtp3} $pool{smtp4}you should add “round robin weighted” algorithm rather than randomness to the script.
http://kb.linuxvirtualserver.org/wiki/Weighted_Round-Robin_Scheduling.
this example might not what you want, but it’s simple, you can play with this code.
require List::Util::WeightedRoundRobin module
#!/usr/bin/perl use strict; use warnings; use List::Util::WeightedRoundRobin; my $list = [ { name => 'smtp1:', weight => 6, }, { name => 'smtp2:', weight => 2, }, { name => 'smtp3:', weight => 2, }, { name => 'smtp4:', weight => 3, }, ]; my $WeightedList = List::Util::WeightedRoundRobin->new(); my $weighted_list = $WeightedList->create_weighted_list( $list ); my $random_index = rand @{$weighted_list}; print "${$weighted_list}[$random_index]\n";This code still randomize but not as random as original script.
Hello,
Is there a way to change rotator every XX emails sent with
this method like in “postfix-smtp-outgoing-ip-rotator-using-iptables”?
I like the iptables method and it works very well but
it does not rotate the hostname, so there is bad inboxing rate.
Is there a way to rotate the hostname with the iptables method?
try this:
http://www.kutukupret.com/2011/05/22/postfix-rotating-outgoing-ip-using-tcp_table-and-perl/