Advice on best way to set up multi-route NAT for lots of IPs

Advice on best way to set up multi-route NAT for lots of IPs

[Anton Melser]

Hi,
I am very new to iptables but have been trying hard to learn as much
as I can... I have a reasonably simple need but performance might
quickly become an issue so would like some advice on the best way to
go forward.
So, I have around 1600 public IPs in 4 blocks (3 x /23 + /25 on
different ISPs). I have a certain number of machines (somewhere from 3
to 8, needs to be variable and changeable without FW reconfiguration),
and each one needs to be able to send email from each external IP (and
needs to be able to do this deterministically). The only traffic
should be to port 25 on the external destination IPs - the machines
are only sending email, never receiving, so AFAICT everything can be
closed inbound (at least for NEW).
I thought that the best way to go would be to set up NAT using blocks
in the 10.0.0.0 range. So say for each external IP I would have a /24,
giving me up to 250-odd potential internal machines. So 10.1.1.1,
10.1.1.2, 10.1.1.3, etc. would map to 1.1.1.1; 10.1.2.1, 10.1.2.2,
10.1.2.3, etc. would map to 1.1.1.2, etc.
I have been reading as many sites as I can but I can't work out the
best way to go forward.

AFAICT the best way to do this is with iptables SNAT - is that the
case? It's not 1 to 1 so it needs to be stateful, and can't be done
with just iproute2 stuff - am I correct in my understanding?

There seem to be many different ways I could do this in terms of
routing - at least by source IP, TOS, and fwmark. Is one of these
preferable? Am I absolutely going to need a rule for every external
IP? I wouldn't have thought so, but can't work out how to do it... I
did some testing and was able to successfully send via several default
routes following
http://kindlund.wordpress.com/2007/11/19/configuring-multiple-default-routes-in-linux/
but that was when I was sending from the local machine without NAT...
If I do need a rule for every IP, is performance going to be an issue?
Would setting up some hashing like that explained in
http://lartc.org/lartc.html#LARTC.ADV-FILTER.HASHING be the best way
to mitigate these issues?

Any help or suggestions most welcome.
Thanks.
Anton

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Lloyd Standish]

On Sun, 01 Jan 2012 10:10:51 -0600, Anton Melser <anton@linux.com> wrote:

> So, I have around 1600 public IPs in 4 blocks (3 x /23 + /25 on
> different ISPs). I have a certain number of machines (somewhere from 3
> to 8, needs to be variable and changeable without FW reconfiguration),
> and each one needs to be able to send email from each external IP (and
> needs to be able to do this deterministically). The only traffic
> should be to port 25 on the external destination IPs - the machines
> are only sending email, never receiving, so AFAICT everything can be
> closed inbound (at least for NEW).
> I thought that the best way to go would be to set up NAT using blocks
> in the 10.0.0.0 range. So say for each external IP I would have a /24,
> giving me up to 250-odd potential internal machines. So 10.1.1.1,
> 10.1.1.2, 10.1.1.3, etc. would map to 1.1.1.1; 10.1.2.1, 10.1.2.2,
> 10.1.2.3, etc. would map to 1.1.1.2, etc.
> I have been reading as many sites as I can but I can't work out the
> best way to go forward.

Hi,
I am new to this list and I have little experience with netfilter, but I think I can help you. However, I need some clarification:

When you say your machines need to be able to send email from each of those 1600 public IPs, do you mean your 3-8 machines serve as SMTP relays for 1600 hosts, each with a public IP?  Do you mean that you are *not* the ISP, and are providing only smtp service for the hosts?

-- 
Lloyd

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Anton Melser]

...
> Hi,
> I am new to this list and I have little experience with netfilter, but I
> think I can help you. However, I need some clarification:
>
> When you say your machines need to be able to send email from each of those
> 1600 public IPs, do you mean your 3-8 machines serve as SMTP relays for 1600
> hosts, each with a public IP?  Do you mean that you are *not* the ISP, and
> are providing only smtp service for the hosts?

ESP. Think Mailchimp just a little smaller. Lots of clients need lots
of IPs (it's a reputation thing, and quite an interesting computing
problem, see http://blog.mailchimp.com/should-you-send-from-a-dedicated-ip-address/
or just search for "email marketing dedicated ip" for an intro).
A

-- 
echo '16i[q]sa[ln0=aln100%Pln100/snlbx]sbA0D4D465452snlbxq' | dc
This will help you for 99.9% of your problems ...

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Anton Melser]

...
> ESP. Think Mailchimp just a little smaller.
Actually, Mailchimp aren't that well known, just think IBM
http://www.unica.com/products/on-demand-interactive-marketing.htm.
A

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Lloyd Standish]

On Sun, 01 Jan 2012 14:41:24 -0600, Anton Melser <melser.anton@gmail.com> wrote:

>> I am new to this list and I have little experience with netfilter, but I
>> think I can help you. However, I need some clarification:
>>
>> When you say your machines need to be able to send email from each of those
>> 1600 public IPs, do you mean your 3-8 machines serve as SMTP relays for 1600
>> hosts, each with a public IP?  Do you mean that you are *not* the ISP, and
>> are providing only smtp service for the hosts?
>ESP. Think Mailchimp just a little smaller. Lots of clients need lots
> of IPs (it's a reputation thing, and quite an interesting computing
> problem, see http://blog.mailchimp.com/should-you-send-from-a-dedicated-ip-address/
> or just search for "email marketing dedicated ip" for an intro).
> A

So, I understand you are setting up 3-8 mail servers that will send out bulk email for 1600 hosts, so that the sender IP in the mails will have your 3-8 "reputable" IPs rather than one of the 1600 "unknown" IPs.

This would not be a regular email relay, since that would put the sender IP in the mail headers.  Are you thinking to use NAT to try to hide the sender IP? That's not the way to do it.

Frankly, this looks to me like bulk-email-laundering.  That is, it's a way to convey email "reputation" from one of 3-8 "trusted" IPs to the 1600 "unknown" ones.

Sorry, a have a personal issue with spam, and anything that could be used (if not by you, then by someone else) to get spam delivered. I think the email reputation of a public IP address should be earned, and it *should* take time to earn it.

-- 
Lloyd

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Anton Melser]

> So, I understand you are setting up 3-8 mail servers that will send out bulk
> email for 1600 hosts, so that the sender IP in the mails will have your 3-8
> "reputable" IPs rather than one of the 1600 "unknown" IPs.
>
> This would not be a regular email relay, since that would put the sender IP
> in the mail headers.  Are you thinking to use NAT to try to hide the sender
> IP? That's not the way to do it.
>
> Frankly, this looks to me like bulk-email-laundering.  That is, it's a way
> to convey email "reputation" from one of 3-8 "trusted" IPs to the 1600
> "unknown" ones.
>
> Sorry, a have a personal issue with spam, and anything that could be used
> (if not by you, then by someone else) to get spam delivered. I think the
> email reputation of a public IP address should be earned, and it *should*
> take time to earn it.

No, you misunderstand. (At least with IPv4) Reputation ~= IP. The goal
is 1 client = (at least) 1 IP. Reputation DOES take time to build up,
and the best way to build it up is by sending relevant, permissioned
newsletters from an IP that is used by only ONE client. We agree, this
is how it should be. But lots of clients = lots of IPs...
But this is *WAY* OT...
A

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Lloyd Standish]

On Mon, 02 Jan 2012 03:00:23 -0600, Anton Melser <melser.anton@gmail.com> wrote:

> No, you misunderstand. (At least with IPv4) Reputation ~= IP. The goal
> is 1 client = (at least) 1 IP. Reputation DOES take time to build up,
> and the best way to build it up is by sending relevant, permissioned
> newsletters from an IP that is used by only ONE client. We agree, this
> is how it should be. But lots of clients = lots of IPs...
> But this is *WAY* OT...

I have considerable experience running SMTP servers (Postfix), and I see no problem with having 1600 hosts.  I understand that these hosts would be spread out among your 3-8 mail servers, so the number of hosts served might be about 200 per SMTP server.  What might be a worry is the fact that they normally send bulk email.  You may want to ask your customers to send their bulk mail at certain times, to avoid overload.

If you set up your machines as relays, Postfix and other MTAs will write the public IP of the sender into each email in the first "Received" line.  Then the receiving ISPs can check the reputation of each sender IP.  If that's your goal ("...newsletters from an IP that is used by only ONE client.  We agree, this is how it should be."), that's the proper way to do it.  Your customers would each require only an email client running on a regular PC, not a full-fledged mail server as you imply.  I'm sure that plenty of email clients suitable for sending bulk email are available.

However, I don't think you want those public IPs in the bulk emails, based on the link you sent (http://blog.mailchimp.com/should-you-send-from-a-dedicated-ip-address/)  The whole idea of that service, which you pointed out as an example of your own, is to circumvent the time-consuming process of building an IP's email reputation by sending it from another, "trusted" IP.  Isn't that what you're trying to do?

Of course, this mailing list is not the place to debate this.

Since SNAT is done in the POSTROUTING chain, you can't use SNAT to try to remove evidence of your customers' public IPs from mail sent on the *same* machine that does the SNAT.  Even if you use a NATting router to FORWARD the email to mail servers running on other machines, the MTAs will know the true origin IP and will ignore the NAT IP when they write the mail header.

Hiding the true sender IP is a violation of protocol.  One way to violate protocol is to do something like remove the Received header that contains your sender's public IP.  That can easily be done, but I won't go into details.

As for limiting access to the spoofing mail server to your network ranges, that's not necessary since your relaying mail servers will require authentication.  However:

Allow NEW port 25 connections from each of your IP ranges:
iptables -A INPUT -p tcp -m state --state NEW -s x.x.x.x/23 -j ALLOW
etc.

I think you will need this (one rule only) to allow email negotiation:
iptables -A INPUT -p tcp -m state --state ESTABLISHED,RELATED -j ALLOW

Then set the default policy:
iptables -P INPUT DROP

-- 
Lloyd

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Anton Melser]

...
> I have considerable experience running SMTP servers (Postfix), and I see no
> problem with having 1600 hosts.  I understand that these hosts would be
> spread out among your 3-8 mail servers, so the number of hosts served might
> be about 200 per SMTP server.  What might be a worry is the fact that they
> normally send bulk email.  You may want to ask your customers to send their
> bulk mail at certain times, to avoid overload.

This is what may be causing the confusion. The point of the exercise
is actually for each client to be able to send with their own IP 24/7
with no downtime. If a public IP is tied to a physical MTA machine
then if you want to take that machine offline for maintenance (or it
goes offline without you wanting!), the IP can no longer send. If you
have 200 customers on an MTA, then that's 200 customers that can't
send while you are doing maintenance. And if all 200 customers decide
to send at the same time then our poor MTA will not be happy, and
neither will the client whose newsletter takes 15 hours to send! Yes,
it is possible to move IPs, but there are ARP cache problems and this
is most definitely not HA (you have to move queues, etc.). There is
also bonding or any number of other HA solutions but I'll bet trying
to do that with MTA software and many machines is much more
problematic than doing it with just one or two failover FW machines.
It seems relatively easy to cluster iptables for HA with conntrack and
other tools, so FW uptime can be assured easily (and cheaply) that
way. It is also vastly preferable not to have machines directly
accessible from the internet, meaning there will be some sort of
firewall (transparent proxy) anyway. NATing seemed to me to be a
pretty good way of being able to do maintenance at reasonable times of
the day (like a Monday morning at 10am instead of Monday morning at
2am) and add/remove capacity, etc. Clients never, ever like downtime,
that is a given I think. They also don't like being told "you need to
send starting from 1am" - their research has shown them that their
subscribers want their emails at 10:30am (or whatever), so if we can't
send at that time then a competitor will... The idea is to provide a
secure, robust, flexible platform with (almost) no downtime and
without costing many hundreds of thousands of $$$.
So nothing really nefarious here except liking sleep and wanting to do
things with FOSS as much as possible (and saving money)...
In a nutshell, one client = one IP = one reputation = many MTAs (for
redundancy and capacity and for no other reason) is the goal.

> If you set up your machines as relays, Postfix and other MTAs will write the
> public IP of the sender into each email in the first "Received" line.  Then
> the receiving ISPs can check the reputation of each sender IP.  If that's
> your goal ("...newsletters from an IP that is used by only ONE client.  We
> agree, this is how it should be."), that's the proper way to do it.  Your
> customers would each require only an email client running on a regular PC,
> not a full-fledged mail server as you imply.  I'm sure that plenty of email
> clients suitable for sending bulk email are available.
>
> However, I don't think you want those public IPs in the bulk emails, based
> on the link you sent
> (http://blog.mailchimp.com/should-you-send-from-a-dedicated-ip-address/)
>  The whole idea of that service, which you pointed out as an example of your
> own, is to circumvent the time-consuming process of building an IP's email
> reputation by sending it from another, "trusted" IP.  Isn't that what you're
> trying to do?

Nope, see above.

> Of course, this mailing list is not the place to debate this.
>
> Since SNAT is done in the POSTROUTING chain, you can't use SNAT to try to
> remove evidence of your customers' public IPs from mail sent on the *same*
> machine that does the SNAT.  Even if you use a NATting router to FORWARD the
> email to mail servers running on other machines, the MTAs will know the true
> origin IP and will ignore the NAT IP when they write the mail header.
>
> Hiding the true sender IP is a violation of protocol.  One way to violate
> protocol is to do something like remove the Received header that contains
> your sender's public IP.  That can easily be done, but I won't go into
> details.
:-). That is most certainly not the idea at all! The idea IS to have a
big black box that sends newsletters - no one cares whether the actual
physical machine that generates an email is the one that queues it for
sending, or whether an IP points to a firewall machine or a physical
MTA. The "true" sender IPs are fully referenced in the whois database
with full contact details, including physical company address and
contact phone numbers. No one wants to hide anything. Email addresses
included in the whois databases are regularly checked and any
complaints are dealt with promptly and seriously. Any of our clients
who do not comply with the various anti-spam laws or our much stricter
terms of service are immediately cut off and contracts nulled. That is
the only way to have good working relationships with the ISPs - we get
called by postmasters "client X is causing complaints, get rid of
them", not blocked like what happens to some irresponsible industry
players. Transparancy and openness is the only way forward for email
marketing, and the only way to maintain good relations with the
receivers. ISPs don't want a sender (one "end client" of ours) using
lots of different IPs. They also prefer to have a single sender on an
IP if possible - that makes it much easier for them to classify and
filter if necessary. The best way to make sure newsletters get
accepted into subscribers' inboxes is to do what the ISPs/MSPs want.
ISPs/MSPs want their users to be happy, which means receiving the
newsletters they subscribe to in their inboxes and putting the spam in
the spam folder (or not at all). That is what we want to - it is a
sustainable and responsible business and has a future.

> As for limiting access to the spoofing mail server to your network ranges,
> that's not necessary since your relaying mail servers will require
> authentication.  However:
>
> Allow NEW port 25 connections from each of your IP ranges:
> iptables -A INPUT -p tcp -m state --state NEW -s x.x.x.x/23 -j ALLOW
> etc.
>
> I think you will need this (one rule only) to allow email negotiation:
> iptables -A INPUT -p tcp -m state --state ESTABLISHED,RELATED -j ALLOW
>
> Then set the default policy:
> iptables -P INPUT DROP

Thanks for the tips, particularly on the LAN-local sending IP in the
headers, I had forgotten about that... and it will need to be replaced
with the clients' dedicated public IPs when it leaves the FW. That
might be a job for a netfilter module?
Anton

ps. Sorry about the dancing email "anton at linux dot com" and the
gmail. I thought I'd finally use my linux.com address (support the
cause!) but gmail defaults to my default address for replies and I
missed it. What I don't understand is why the list accepted my gmail
address when I subscribed with @linux.com...

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Lloyd Standish]

On Mon, 02 Jan 2012 16:14:56 -0600, Anton Melser <melser.anton@gmail.com> wrote:

> In a nutshell, one client = one IP = one reputation = many MTAs (for
> redundancy and capacity and for no other reason) is the goal.
>

One client - one IP is NOT the method promoted by mailchimp, the service you said is an example of what you are doing.

 From the mailchimp.com page you referred me to (http://blog.mailchimp.com/should-you-send-from-a-dedicated-ip-address/):

"[ISPs' spam filters give low ranking to new IPs that suddenly send a large volume of email.] To try to offset the high volume from this new IP, we take huge chunks of their campaign and distribute those across our shared IPs. Only a small fraction of this customer’s email is actually being sent from this dedicated IP during the break-in period. But as you can see, for some filters, it’s still risky looking. And so you get delivery problems for a while.

So long as the volume stays somewhat consistent, and so long as spam complaints stay within acceptable thresholds, their dedicated IP will make its way to “Trusted.” Actually, it’ll go to “neutral” for a while, then trusted."

I think the above makes it very clear why I understood that your service seeks to send out email for customers with source IPs *other* than the customers' own IP, at least during the IPs "break-in period".

Now you explain what you are really trying to do is provide mail server redundancy.  You can do that easily and cheaply with DNS failover.  But that is off-topic.

I must not understand the solution you have in mind, because I can't see how NAT could be of any help.

--
Lloyd

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Anton Melser]

> I think the above makes it very clear why I understood that your service
> seeks to send out email for customers with source IPs *other* than the
> customers' own IP, at least during the IPs "break-in period".
The link was meant to provide an introduction to some of the issues.
You are obviously focusing exclusively on possible ways the current
situation can be gamed or abused. Unfortunately there are not 15
different ways to "warm up" IPs. It takes time and so costs money, as
most people are still doing this at least semi-manually. You need to
build up slowly, that is how the ISPs/MSPs require it. In any case, it
takes weeks for an IP to earn a reputation and all that can be
destroyed in a single send (couple of hours). When an IP has a
reputation then you can dedicate it to a customer and the customer
then becomes responsible for their own reputation. Again, this is the
goal and there is only really one way to get there. If there were some
magic program that all ISPs/MSPs adhered to and required a large bond
to be posted in terms of guarantee, we would jump on it. If we could
just say, "here are many thousands of dollars, if anyone sends
anything dodgy from this IP then it is forfeit" that would save lots
of time and hassle. It doesn't exist. Even whitelisting services like
Return Path SenderScore certification require a minimum of 3mths on a
dedicated IP before they will *consider* accepting an IP in the
program! We have multi-year contracts with our clients - we are not at
all interested in customers that come, send crap for a week or two and
leave. That is not possible with our infrastructure because we have an
involved acceptance process where databases are analysed, marketing
programs reviewed, etc, and you sign a (absolute minimum) 12-mth
contract. Sign up costs are also significant, and spammers need for
things to be cheap. I recently read that on average 12.5 million spam
messages need to be sent for $100 of "viagra" to be sold. You would be
losing a LOT of money sending these messages on any reputable email
service provider!

> Now you explain what you are really trying to do is provide mail server
> redundancy.  You can do that easily and cheaply with DNS failover.  But that
> is off-topic.
Sending and receiving email are two quite different needs. I would be
very, very surprised if, say, Yahoo! did sending and receiving on the
same machines. The various SMTP standards never suggest that email
should only be sent from machines in the MX. The ISPs and MSPs don't
care if you use machines referenced in the MX records. I know because
anti-abuse masters have told me so. Sure, you need to provide robust
infrastructure for dealing with bounces and any complaints (to
postmaster@, abuse@, etc.) but that has nothing to do with sending
infrastructure. You should also provide rDNS but again, that has very
little to do with reputation indexes based on IP address. DNS failover
isn't an option for providing MTA sending uptime from a particular IP.
What I am trying to do is DNS failover for IPs - so having a public
token (in DNS it's a name, for me it's an IP) that is translated to
one or many internal values (IP for DNS, LAN local IP for me). Isn't
this NAT?
I am all up for alternative means for making sure a particular IP can
be available for sending 24/7 cheaply, if there are any. (Don't
mistake cheap for provider as cheap for sender though!) I thought
iptables/netfilter would be a good way of doing it but I might be
wrong...
Cheers
Anton

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Anton Melser]

...
> I am all up for alternative means for making sure a particular IP can
> be available for sending 24/7 cheaply, if there are any. (Don't
> mistake cheap for provider as cheap for sender though!) I thought
> iptables/netfilter would be a good way of doing it but I might be
> wrong...

In the hope that one final example shows the fact that what I want to
do is completely legitimate. Here are the headers of a *LINUX
FOUNDATION* newsletter I received today:

Received: from email-gaia.pd27.com (email-gaia.pd27.com. [208.43.21.70])
        by mx.google.com with ESMTP id gj7si35742626qab.7.2012.01.04.06.13.51;
        Wed, 04 Jan 2012 06:13:52 -0800 (PST)
Received-SPF: pass (google.com: domain of
undelivered+6342+158668917@pd25.com designates 208.43.21.70 as
permitted sender) client-ip=208.43.21.70;
Authentication-Results: mx.google.com; spf=pass (google.com: domain of
undelivered+6342+158668917@pd25.com designates 208.43.21.70 as
permitted sender) smtp.mail=undelivered+6342+158668917@pd25.com
Received: by email-gaia.pd27.com id h0hfa00oaq85 for
<my_email@gmail.com>; Wed, 4 Jan 2012 09:09:16 -0500 (envelope-from
<undelivered+6342+158668917@pd25.com>)
Return-Path: <undelivered+6342+158668917@pd25.com>
Message-ID: <1325686156.4f045d8cb7f92@swift.generated>
Date: Wed, 04 Jan 2012 09:09:16 -0500
Subject: CFP Deadline This Friday To Speak at Android Builders Summit and
 Embedded Linux Conference in February
From: Linux Foundation Events <no-reply@linuxfoundation.org>

208.43.21.70 is an IP (that seems to be, see the whois) registered to
Pardot, an online marketing (including email) infrastructure provider,
NOT the Linux Foundation. All the newsletters from the Linux
foundation that I have received since 2011-10-11 have come from the
same IP (hurrah, so it's a dedicated IP!). Pardot have declared many
thousands of IPs as valid IPs that receivers might receive email from
on their behalf (nslookup -type txt pd25.com, then keep digging into
the SPF records to get the actual IPs). They almost certainly have
thousands of clients.
If Pardot have the same issues as we do (they do, they are a
competitor for some products, including email sending), and the Linux
Foundation are a client of theirs and use their email marketing
services, then... the Linux Foundation are SPAMMERS! Yikes! We are all
doomed... :-).

So in other news - does anyone have any suggestions or advice on the
best way to do NAT + multi-routing via several gateways using
netfilter/iptables with 1600+ IPs?
Thanks,
Anton

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Andrew Beverley]

On Mon, 2012-01-02 at 23:14 +0100, Anton Melser wrote:
> What I don't understand is why the list accepted my gmail
> address when I subscribed with @linux.com...

I think all the vger lists are configured to accept from any address, in
order to allow bug reports to be submitted by anyone.

Andy

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Pete]

[-- Attachment #1: Type: text/plain, Size: 1224 bytes --]

On Sun, Jan 01, 2012 at 04:11:02PM -0600, Lloyd Standish wrote:
> On Sun, 01 Jan 2012 14:41:24 -0600, Anton Melser
> <melser.anton@gmail.com> wrote:
> 
> Think Mailchimp just a little smaller. Lots of clients need lots
> > of IPs (it's a reputation thing, and quite an interesting computing
> > problem, see
> > http://blog.mailchimp.com/should-you-send-from-a-dedicated-ip-address/
> > or just search for "email marketing dedicated ip" for an intro).

[..]

> 
> Frankly, this looks to me like bulk-email-laundering.  That is, it's a
> way to convey email "reputation" from one of 3-8 "trusted" IPs to the
> 1600 "unknown" ones.

This discussion is very intriguing to myself, no matter how OT. I'm
quite sure I'm not the only one.

It sounds to me like someone needs help on how to hide a botnet using an
iptables script at first glance. It can't be that of course so why are
1600 hosts wanting to send bulk email ?

spam has given email marketing such a bad reputation that I'd really
like to know why there are 1600 hosts that need to send
business/marketing email. Why 1600 ?

Sorry I'm new to the list and I realise I am contributing to the
OT-ishness of this thread.

Regards,

Pete.

[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 3952 bytes --]

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Anton Melser]

>> Frankly, this looks to me like bulk-email-laundering.  That is, it's a
>> way to convey email "reputation" from one of 3-8 "trusted" IPs to the
>> 1600 "unknown" ones.
>
> This discussion is very intriguing to myself, no matter how OT. I'm
> quite sure I'm not the only one.
>
> It sounds to me like someone needs help on how to hide a botnet using an
> iptables script at first glance. It can't be that of course so why are
> 1600 hosts wanting to send bulk email ?
>
> spam has given email marketing such a bad reputation that I'd really
> like to know why there are 1600 hosts that need to send
> business/marketing email. Why 1600 ?
>
> Sorry I'm new to the list and I realise I am contributing to the
> OT-ishness of this thread.

I said to myself "don't mention port 25, you'll get a barrage of
insults..."! But I realise it is intriguing to many people, and it's
very easy to jump to conclusions. I suppose the simplest way to
explain why 1600 is the following. If we accept that it is valid for a
client to have an IP and this client will send their newsletters from
only this IP and build reputation on this IP, then it is trivial: 1600
IPs = 1600 clients. If an intern for company X makes a booboo
(something like
http://it.slashdot.org/story/11/12/28/1929232/new-york-times-hacked
for example!) then company Y shouldn't suffer, should they? They will
need different IPs then.  Mailchimp claims (or claimed at one point)
to have 100,000 clients (I am not involved with Mailchimp in any way,
they are one of the biggest in the industry so I'm picking on them).
So 100000 IPs? It's more complicated than that unfortunately, as MSPs
and ISPs require certain minimum levels of traffic. The blog link
above in the thread mentions why you might not want a dedicated IP if
you don't send enough. The problem being that if you don't send for a
while, and then suddenly start sending again, everything gets
completely blocked or put in the spam folder. The vast majority of
companies can't warrant employing someone who spends their days
researching the latest requirements for sending newsletters, so they
sub-contract that to an application service provider (an Email Service
Provider). So we need lots of IPs and we need to manage them
efficiently on behalf of our clients.
Actually we don't use nearly 1600, as currently IP management is not
as optimised as it would be with a NAT (or similar) solution. We have
a lot of ad agencies as clients though, and they are only working for
their clients (white label or not), and we have many, many more than
1600 "end clients". We also definitely have clients that want to have
redundancy on connection providers (going over different backbones,
etc.), and bandwidth is cheaper on X but more reliable on Y, etc. so
we need different providers. My solution needs to be able to support
more than we could ever throw at it, so I want it to be able to
support 1600 IPs from the start.
Cheers
Anton
ps. Also see my response to Lloyd's last post.

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Ed W]

On 01/01/2012 16:10, Anton Melser wrote:
> Hi,
> I am very new to iptables but have been trying hard to learn as much
> as I can... I have a reasonably simple need but performance might
> quickly become an issue so would like some advice on the best way to
> go forward.
> So, I have around 1600 public IPs in 4 blocks (3 x /23 + /25 on
> different ISPs). I have a certain number of machines (somewhere from 3
> to 8, needs to be variable and changeable without FW reconfiguration),
> and each one needs to be able to send email from each external IP (and
> needs to be able to do this deterministically). The only traffic
> should be to port 25 on the external destination IPs - the machines
> are only sending email, never receiving, so AFAICT everything can be
> closed inbound (at least for NEW).
>

Although NAT would seem to be the most flexible solution (seems like you 
just need to read up on SNAT? Probably also some network stack tuning 
needed for such a large amount of NAT..?), you can probably also do this 
by adding the public IPs to your mailserver?  Eg with Postfix you can 
either lightly overload settings per transport in master.cf ( 
http://www.postfix.org/master.5.html ), or if you need something which 
more closely emulates a virtual machine then see the multi-instance 
stuff ( http://www.postfix.org/MULTI_INSTANCE_README.html ). I see no 
theoretical reason you couldn't have a (very) multihomed machine with 
the IPs on the servers themselves? The benefit might be that mailservers 
under high load will normally have a lot of connections open (hence high 
NAT requirements)

Postfix also has some interesting options to add connection caching and 
some other tricks which are helpful for larger installations and large 
outbound queue volumes.

You should probably spend some time on followup questions covering why 
you aren't a spam sender. Many technical folks will jump to the 
conclusion that anyone asking for help pumping large volumes of mail is 
likely to be up to no good.  Just saying how it is...

Good luck

Ed W

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Anton Melser]

...
> Although NAT would seem to be the most flexible solution (seems like you
> just need to read up on SNAT?
I have been doing that but thought I'd ask here for the advice from
the experts... There are many bad ways to skin a cat and I just wanted
to make sure I was using a reasonable way.

> Probably also some network stack tuning needed
> for such a large amount of NAT..?),
That was what I was hoping to avoid...

> you can probably also do this by adding
> the public IPs to your mailserver?
Definitely, makes load shifting very complicated though...

> Eg with Postfix you can either lightly
> overload settings per transport in master.cf (
> http://www.postfix.org/master.5.html ), or if you need something which more
> closely emulates a virtual machine then see the multi-instance stuff (
> http://www.postfix.org/MULTI_INSTANCE_README.html ). I see no theoretical
> reason you couldn't have a (very) multihomed machine with the IPs on the
> servers themselves? The benefit might be that mailservers under high load
> will normally have a lot of connections open (hence high NAT requirements)
>
> Postfix also has some interesting options to add connection caching and some
> other tricks which are helpful for larger installations and large outbound
> queue volumes.
Postfix wasn't really designed for sending newsletters for lots of
companies efficiently, and it doesn't do a very good job compared to
some highly targeted products (PowerMTA, Message Systems, etc.)

> You should probably spend some time on followup questions covering why you
> aren't a spam sender. Many technical folks will jump to the conclusion that
> anyone asking for help pumping large volumes of mail is likely to be up to
> no good.  Just saying how it is...

:-). I was hoping to avoid that but you are right. Funnily enough,
pretty much no one sends bulk newsletters with their own servers any
more, and we have spammers to thank for that! Probably something like
90% of fortune 500s use specialist providers, hence why IBM and other
megacorps decided they needed in. The problem being that "technical
folks" usually don't have the time or patience to properly take care
of the "marketing folks" - mail servers need to be set up with sending
newsletters in mind because if they aren't then the MSPs (Mailbox
Service Providers, like Hotmail, Yahoo, GMail) or ISPs (like Comcast,
etc.) will just block and say "this is probably spam". Most
postmasters don't know or care (or do but don't have the time) about
this, so tell the marketing people to send them from elsewhere, hence
the development of an email broadcasting outsourcing sector. Receivers
set up http://www.maawg.org/, and have welcomed in broadcasters (and
senders) so there can be a forum for them to tell us how to send to
them. People DO sign up for newsletters, and that means they want
them, so ISPs can't (and don't) just block everything. ISPs and MSPs
WANT individual clients to have dedicated IPs, so they can more easily
identify and whitelist/throttle/trash/block. That means if you have
many thousands of clients, you need many thousands of IPs... But you
don't need many thousands of machines (save the planet! :-)) -
particularly if you can set up SNAT efficiently!
Cheers
Anton

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Ed W]

On 02/01/2012 13:17, Anton Melser wrote:
>> you can probably also do this by adding
>> the public IPs to your mailserver?
> Definitely, makes load shifting very complicated though...

OK, so if you want an external "load balancer" then your problem reduces 
to *indicating* the desired mapped source address.

If the NAT is on an external box then you can't use fwmarks.  You can 
use either source port or dest port.  You could also add all IPs to all 
servers, but that seems rather tricky to make work in practice.  I think 
your best bet might be a hack, to use dest port as the indicator for 
"source IP".  Set your DNAT to map some range of dest ports to change 
the source to the IP and the dest port to 25. This will allow all 
machines to send and masquerade as any source ip...

I haven't quite thought this through, but I think it will work?


Good luck

Ed W

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Andrew Beverley]

On Sun, 2012-01-01 at 17:10 +0100, Anton Melser wrote:
> Hi,
> I am very new to iptables but have been trying hard to learn as much
> as I can... I have a reasonably simple need but performance might
> quickly become an issue so would like some advice on the best way to
> go forward.
> So, I have around 1600 public IPs in 4 blocks (3 x /23 + /25 on
> different ISPs). I have a certain number of machines (somewhere from 3
> to 8, needs to be variable and changeable without FW reconfiguration),
> and each one needs to be able to send email from each external IP (and
> needs to be able to do this deterministically). The only traffic
> should be to port 25 on the external destination IPs - the machines
> are only sending email, never receiving, so AFAICT everything can be
> closed inbound (at least for NEW).
> I thought that the best way to go would be to set up NAT using blocks
> in the 10.0.0.0 range. So say for each external IP I would have a /24,
> giving me up to 250-odd potential internal machines. So 10.1.1.1,
> 10.1.1.2, 10.1.1.3, etc. would map to 1.1.1.1; 10.1.2.1, 10.1.2.2,
> 10.1.2.3, etc. would map to 1.1.1.2, etc.
> I have been reading as many sites as I can but I can't work out the
> best way to go forward.

So you have something like:

Server A ----|
             |
Server B ----|
             |-----> Linux router ----> Internet
Server C ----|
             |
Server D ----|

Correct? And it's the Linux router you're asking about?

> 
> AFAICT the best way to do this is with iptables SNAT - is that the
> case?

I think the main question is: how does the Linux router know which IP
address that the mail should be sent from? Server A/B/C/D somehow need
to pass this information on. This can't be done with fwmarks, because
they aren't retained between on packets between servers.

>  It's not 1 to 1 so it needs to be stateful, and can't be done
> with just iproute2 stuff - am I correct in my understanding?

You might be able to do this with iproute2, but depends on answer to
above.

> 
> There seem to be many different ways I could do this in terms of
> routing - at least by source IP, TOS, and fwmark.

I'm going to guess that source IP is the only option. So can you set the
source IP from each server depending on its eventual external IP
address?

Andy

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Anton Melser]

...
> So you have something like:
>
> Server A ----|
>             |
> Server B ----|
>             |-----> Linux router ----> Internet
> Server C ----|
>             |
> Server D ----|
>
> Correct? And it's the Linux router you're asking about?

That is exactly right. I thought it might be useful to do part of the
routing on the servers (A-D) but that has the disadvantage of meaning
Windows can't be used (Windows doesn't do policy-based routing). Not
that the idea is to use Windows but I like choice...

>> AFAICT the best way to do this is with iptables SNAT - is that the
>> case?
>
> I think the main question is: how does the Linux router know which IP
> address that the mail should be sent from? Server A/B/C/D somehow need
> to pass this information on. This can't be done with fwmarks, because
> they aren't retained between on packets between servers.

My idea was to communicate the external/public IP that should be used
by the router by associating an internal network to each external IP.
So if an internal machine presents a packet from their address in
network X, the router knows that it should use public IP X. What I had
in mind was just taking the standard case where you have one publicly
available IP and lots of internal machines that need to access the
'net, and multiplying that by all the external IPs. So if we have 1600
external IPs then we'll have 1600 internal networks, each with N
hosts.

>>  It's not 1 to 1 so it needs to be stateful, and can't be done
>> with just iproute2 stuff - am I correct in my understanding?
>
> You might be able to do this with iproute2, but depends on answer to
> above.

My understanding was that iproute2 doesn't do stateful, and that if we
have many : 1 then we need stateful. Is that right?

>>
>> There seem to be many different ways I could do this in terms of
>> routing - at least by source IP, TOS, and fwmark.
>
> I'm going to guess that source IP is the only option. So can you set the
> source IP from each server depending on its eventual external IP
> address?

I was thinking that when the packets *arrive* on the router they could
be marked for ToS or fwmark from their source IPs. The ToS or fwmark
could then be used for routing decisions. On the surface of it there
is no benefit - if you can use source address for routing decisions
then why bother adding a step for marking? ToS and fwmark looked a
little simpler in the examples, but I'm a noob, so don't really know!
In any case, source IP seemed to be the best option, so it looks like
you are confirming my original suspicions.
Thanks for your input.
Anton

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Andrew Beverley]

On Thu, 2012-01-05 at 09:15 +0100, Anton Melser wrote:
> ...
> > So you have something like:
> >
> > Server A ----|
> >             |
> > Server B ----|
> >             |-----> Linux router ----> Internet
> > Server C ----|
> >             |
> > Server D ----|
> >
> > Correct? And it's the Linux router you're asking about?
> 
> That is exactly right. I thought it might be useful to do part of the
> routing on the servers (A-D) but that has the disadvantage of meaning
> Windows can't be used (Windows doesn't do policy-based routing). Not
> that the idea is to use Windows but I like choice...
> 
> >> AFAICT the best way to do this is with iptables SNAT - is that the
> >> case?
> >
> > I think the main question is: how does the Linux router know which IP
> > address that the mail should be sent from? Server A/B/C/D somehow need
> > to pass this information on. This can't be done with fwmarks, because
> > they aren't retained between on packets between servers.
> 
> My idea was to communicate the external/public IP that should be used
> by the router by associating an internal network to each external IP.
> So if an internal machine presents a packet from their address in
> network X, the router knows that it should use public IP X. What I had
> in mind was just taking the standard case where you have one publicly
> available IP and lots of internal machines that need to access the
> 'net, and multiplying that by all the external IPs. So if we have 1600
> external IPs then we'll have 1600 internal networks, each with N
> hosts.

Okay, I'm still a bit confused. Do the A, B, C, D servers above
represent physical machines, each of which is dedicated to a single
customer with single external IP address? I assume not, but that's how
I've read your statement above.

Surely you want several customers on each server, each of which binds to
a different internal IP address? Each internal IP address is then
individually mapped to an external IP address?

> 
> >>  It's not 1 to 1 so it needs to be stateful, and can't be done
> >> with just iproute2 stuff - am I correct in my understanding?
> >
> > You might be able to do this with iproute2, but depends on answer to
> > above.
> 
> My understanding was that iproute2 doesn't do stateful, and that if we
> have many : 1 then we need stateful. Is that right?

Again, depends on my understanding of your problem, but you could maybe
do stateless NAT using iproute2:

http://linux-ip.net/html/nat-stateless.html

Funnily enough, that website actually uses an SMTP example...

> 
> >>
> >> There seem to be many different ways I could do this in terms of
> >> routing - at least by source IP, TOS, and fwmark.
> >
> > I'm going to guess that source IP is the only option. So can you set the
> > source IP from each server depending on its eventual external IP
> > address?
> 
> I was thinking that when the packets *arrive* on the router they could
> be marked for ToS or fwmark from their source IPs. The ToS or fwmark
> could then be used for routing decisions. On the surface of it there
> is no benefit - if you can use source address for routing decisions
> then why bother adding a step for marking?

Agree. I don't see any reason to add a mark to a packet in this
scenario. Of course, TOS marks will transit between servers, but you're
not going to get 1600 unique ones :)

Andy

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Rob Sterenborg (Lists)]

On Thu, 2012-01-05 at 09:15 +0100, Anton Melser wrote:
> I was thinking that when the packets *arrive* on the router they could
> be marked for ToS or fwmark from their source IPs. The ToS or fwmark

You could mark them with a TOS value, but since (I understand that) you
want to NAT private subnets using 1600 public IP's, you'd need to be
able to check 1600 different TOS values otherwise I don't see how you
would be able to differentiate. That's not possible as the TOS field is
8 bit according to 'man iptables' (F15's 1.4.10, yes I have to look it
up too :-))..

There's also DSCP; the man page it has superseded TOS, and that there
can be 64 DSCP values (0-63), so that would also be a no-go AFAICS.

IIRC fwmark only exists on the localhost, not in the header of the IP
packet, so if I'm right then keep in mind that you can only use it at
the localhost. The man page says that the mark value is 32bits wide
which would make it usable here.

But I don't think all of this is going to help you.

> could then be used for routing decisions. On the surface of it there
> is no benefit - if you can use source address for routing decisions
> then why bother adding a step for marking? ToS and fwmark looked a
> little simpler in the examples, but I'm a noob, so don't really know!
> In any case, source IP seemed to be the best option, so it looks like
> you are confirming my original suspicions.

Since it seems you want to map private subnets to 1 public IP and do
that 1600 or so times, I don't see a way to do it easier then matching
the source address and SNAT it accordingly.
Yes, that would mean a lot of rules to create and maintain but I just
don't see any other way.


--
Rob

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Anton Melser]

...
> On Thu, 2012-01-05 at 09:15 +0100, Anton Melser wrote:
>> I was thinking that when the packets *arrive* on the router they could
>> be marked for ToS or fwmark from their source IPs. The ToS or fwmark
>
> You could mark them with a TOS value, but since (I understand that) you
> want to NAT private subnets using 1600 public IP's, you'd need to be
> able to check 1600 different TOS values otherwise I don't see how you
> would be able to differentiate. That's not possible as the TOS field is
> 8 bit according to 'man iptables' (F15's 1.4.10, yes I have to look it
> up too :-))..
>
> There's also DSCP; the man page it has superseded TOS, and that there
> can be 64 DSCP values (0-63), so that would also be a no-go AFAICS.
>
> IIRC fwmark only exists on the localhost, not in the header of the IP
> packet, so if I'm right then keep in mind that you can only use it at
> the localhost. The man page says that the mark value is 32bits wide
> which would make it usable here.
>
> But I don't think all of this is going to help you.

Maybe it will, see below!

>> could then be used for routing decisions. On the surface of it there
>> is no benefit - if you can use source address for routing decisions
>> then why bother adding a step for marking? ToS and fwmark looked a
>> little simpler in the examples, but I'm a noob, so don't really know!
>> In any case, source IP seemed to be the best option, so it looks like
>> you are confirming my original suspicions.
>
> Since it seems you want to map private subnets to 1 public IP and do
> that 1600 or so times, I don't see a way to do it easier then matching
> the source address and SNAT it accordingly.
> Yes, that would mean a lot of rules to create and maintain but I just
> don't see any other way.
...
Sorry to everyone for my explanation not being clear - I suppose that
is just a function of my lack of experience/understanding. You have it
right Rob - I want to map private subnets to different public IPs 1600
times. If the only way to do the NAT is with 1600 rules then I'll stop
looking elsewhere, thanks!
There is also the matter of routing though. I agree that this question
is more an iproute2 issue, and could/should be better asked on the
iproute2 list. In my mind marking the packets for ToS or fwmark was
actually for use at the routing level. The public IPs don't all belong
to a single subnet, and so there are actually 4 different gateways via
which the packets need to go (3 /23 and one /25 networks with 4
different gateways).
If people confirm that there is no better way they can think of for
achieving what I want to do, I shall thank you all and go and bother
the iproute2 people for the routing part!
Thank you all for your patience and help.
Cheers
Anton
ps. I'll do a blog post when I get a coherent config set up and post
back here for reference and your comments. It will need failover using
connection tracking so could end up being a nice little article.

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Andrew Beverley]

On Fri, 2012-01-06 at 06:15 +0100, Anton Melser wrote:
> If the only way to do the NAT is with 1600 rules then I'll stop
> looking elsewhere, thanks!

I think it probably is the only option from what you've said, especially
given the variety of different networks you have. I can't comment on the
performance though, which was one of your original questions.

> There is also the matter of routing though. I agree that this question
> is more an iproute2 issue, and could/should be better asked on the
> iproute2 list.

Well, there isn't really an iproute2 list as such...

There's netdev and LARTC, both also hosted at VGER, but by all means try
your question here if you'd like.

>  In my mind marking the packets for ToS or fwmark was
> actually for use at the routing level.

Sounds like the way to go. Gives you plenty of flexibility.

> ps. I'll do a blog post when I get a coherent config set up and post
> back here for reference and your comments. It will need failover using
> connection tracking so could end up being a nice little article.

That would be excellent. The more "real life" examples there are, the
better.

Andy

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Rob Sterenborg (lists)]

On Sun, 2012-01-01 at 17:10 +0100, Anton Melser wrote:
> I thought that the best way to go would be to set up NAT using blocks
> in the 10.0.0.0 range. So say for each external IP I would have a /24,
> giving me up to 250-odd potential internal machines. So 10.1.1.1,
> 10.1.1.2, 10.1.1.3, etc. would map to 1.1.1.1; 10.1.2.1, 10.1.2.2,
> 10.1.2.3, etc. would map to 1.1.1.2, etc.
> I have been reading as many sites as I can but I can't work out the
> best way to go forward.

So, I think I understand that you want to SNAT a complete private subnet
to a corresponding public subnet. Is the NETMAP target usable for you,
or am I misunderstanding you completely?
Something like:

iptables -t nat -A POSTROUTING -s ${private_subnet} -j NETMAP --to
${public_subnet}


(http://www.frozentux.net/iptables-tutorial/iptables-tutorial.html#NETMAPTARGET)


--
Rob

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Anton Melser]

On 5 January 2012 09:59, Rob Sterenborg (lists) <lists@sterenborg.info> wrote:
> On Sun, 2012-01-01 at 17:10 +0100, Anton Melser wrote:
>> I thought that the best way to go would be to set up NAT using blocks
>> in the 10.0.0.0 range. So say for each external IP I would have a /24,
>> giving me up to 250-odd potential internal machines. So 10.1.1.1,
>> 10.1.1.2, 10.1.1.3, etc. would map to 1.1.1.1; 10.1.2.1, 10.1.2.2,
>> 10.1.2.3, etc. would map to 1.1.1.2, etc.
>> I have been reading as many sites as I can but I can't work out the
>> best way to go forward.
>
> So, I think I understand that you want to SNAT a complete private subnet
> to a corresponding public subnet. Is the NETMAP target usable for you,
> or am I misunderstanding you completely?
> Something like:
>
> iptables -t nat -A POSTROUTING -s ${private_subnet} -j NETMAP --to
> ${public_subnet}

Thanks for the suggestion. It appears that NETMAP does 1:1 and both
SNAT and DNAT. I need to do many:1 lots of times (so (many:1)*n), and
I don't need (or want actually) DNAT. Is it possible to use NETMAP to
do this?
Thanks.
Anton

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Rob Sterenborg (lists)]

On Thu, 2012-01-05 at 12:59 +0100, Anton Melser wrote:
> > So, I think I understand that you want to SNAT a complete private subnet
> > to a corresponding public subnet. Is the NETMAP target usable for you,
> > or am I misunderstanding you completely?
> > Something like:
> >
> > iptables -t nat -A POSTROUTING -s ${private_subnet} -j NETMAP --to
> > ${public_subnet}
> 
> Thanks for the suggestion. It appears that NETMAP does 1:1 and both
> SNAT and DNAT. I need to do many:1 lots of times (so (many:1)*n), and
> I don't need (or want actually) DNAT. Is it possible to use NETMAP to
> do this?

According to this article, NETMAP does SNAT when used in POSTROUTING and
DNAT in PREROUTING, which sounds logical to me.
https://capcorne.wordpress.com/2009/03/24/natting-a-network-range-with-netmapiptables/

If you want to do many:1 NAT then that's SNAT, and when reading your
original email again that seems to be what you want (on a large scale).

>> So, I have around 1600 public IPs in 4 blocks (3 x /23 + /25 on
>> different ISPs).

>> So say for each external IP I would have a /24,
>> giving me up to 250-odd potential internal machines

So, each public IP services a /24 subnet and you have 1600 public IP's.
That would be a lot of rules to create because for each public IP you'd
need an SNAT rule, each matching a private subnet. Sorry, I don't know
of an easier solution for what you want.


--
Rob

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Andrew Beverley]

On Thu, 2012-01-05 at 12:59 +0100, Anton Melser wrote:
> On 5 January 2012 09:59, Rob Sterenborg (lists) <lists@sterenborg.info> wrote:
> > On Sun, 2012-01-01 at 17:10 +0100, Anton Melser wrote:
> >> I thought that the best way to go would be to set up NAT using blocks
> >> in the 10.0.0.0 range. So say for each external IP I would have a /24,
> >> giving me up to 250-odd potential internal machines. So 10.1.1.1,
> >> 10.1.1.2, 10.1.1.3, etc. would map to 1.1.1.1; 10.1.2.1, 10.1.2.2,
> >> 10.1.2.3, etc. would map to 1.1.1.2, etc.
> >> I have been reading as many sites as I can but I can't work out the
> >> best way to go forward.
> >
> > So, I think I understand that you want to SNAT a complete private subnet
> > to a corresponding public subnet. Is the NETMAP target usable for you,
> > or am I misunderstanding you completely?
> > Something like:
> >
> > iptables -t nat -A POSTROUTING -s ${private_subnet} -j NETMAP --to
> > ${public_subnet}
> 
> Thanks for the suggestion. It appears that NETMAP does 1:1 and both
> SNAT and DNAT. I need to do many:1 lots of times (so (many:1)*n),

Are you sure? Remember: we're talking IP addresses here (not physical
devices), and I thought you actually wanted to do one IP address from
the internal network to one external IP address. The IP address on the
internal network stipulating which external address to use.

So, I've never used NETMAP, but it sounds like it would work for you.

>  and
> I don't need (or want actually) DNAT.

Especially, if as Rob says, it'll do SNAT when used in POSTROUTING.

Andy

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Rob Sterenborg (lists)]

On Thu, 2012-01-05 at 16:59 +0000, Andrew Beverley wrote:
> On Thu, 2012-01-05 at 12:59 +0100, Anton Melser wrote:
> > On 5 January 2012 09:59, Rob Sterenborg (lists) <lists@sterenborg.info> wrote:
> > > On Sun, 2012-01-01 at 17:10 +0100, Anton Melser wrote:
> > >> I thought that the best way to go would be to set up NAT using blocks
> > >> in the 10.0.0.0 range. So say for each external IP I would have a /24,
> > >> giving me up to 250-odd potential internal machines. So 10.1.1.1,
> > >> 10.1.1.2, 10.1.1.3, etc. would map to 1.1.1.1; 10.1.2.1, 10.1.2.2,
> > >> 10.1.2.3, etc. would map to 1.1.1.2, etc.
> > >> I have been reading as many sites as I can but I can't work out the
> > >> best way to go forward.
> > >
> > > So, I think I understand that you want to SNAT a complete private subnet
> > > to a corresponding public subnet. Is the NETMAP target usable for you,
> > > or am I misunderstanding you completely?
> > > Something like:
> > >
> > > iptables -t nat -A POSTROUTING -s ${private_subnet} -j NETMAP --to
> > > ${public_subnet}
> > 
> > Thanks for the suggestion. It appears that NETMAP does 1:1 and both
> > SNAT and DNAT. I need to do many:1 lots of times (so (many:1)*n),
> 
> Are you sure? Remember: we're talking IP addresses here (not physical
> devices), and I thought you actually wanted to do one IP address from
> the internal network to one external IP address. The IP address on the
> internal network stipulating which external address to use.
> 
> So, I've never used NETMAP, but it sounds like it would work for you.
> 
> >  and
> > I don't need (or want actually) DNAT.
> 
> Especially, if as Rob says, it'll do SNAT when used in POSTROUTING.

Except if the OP wants to NAT, say, a /24 to each of his public IP's;
then it's not going to work with NETMAP. And that is what I understood
when I re-read his first post. NETMAP will only do a 1:1 NAT (each
private IP to a corresponding public IP) for networks.


--
Rob

Re: Advice on best way to set up multi-route NAT for lots of IPs

[Andrew Beverley]

On Thu, 2012-01-05 at 18:08 +0100, Rob Sterenborg (lists) wrote:
> Except if the OP wants to NAT, say, a /24 to each of his public IP's;
> then it's not going to work with NETMAP. And that is what I understood
> when I re-read his first post. NETMAP will only do a 1:1 NAT (each
> private IP to a corresponding public IP) for networks.

Ah, got you. As per my other (later) post, I'm not entirely sure I
understand the exact network configuration here.

I was assuming that the OP could send on the internal network from a
suitable internal IP address (per customer) and then map that 1:1 to an
external address.

Andy