TCP 2MSL on loopback

TCP 2MSL on loopback

[Howard Chu]

Why is the Maximum Segment Lifetime a global parameter? Surely the 
maximum possible lifetime of a particular TCP segment depends on the 
actual connection. At the very least, it would be useful to be able to 
set it on a per-interface basis. E.g., in the case of the loopback 
interface, it would be useful to be able to set it to a very small duration.

As I note in this draft 
http://www.ietf.org/internet-drafts/draft-chu-ldap-ldapi-00.txt
when doing a connection soak test of OpenLDAP using clients connected 
through localhost, the entire port range is exhausted in well under a 
second, at which point the test stalls until a port comes out of 
TIME_WAIT state so the next connection can be opened.

These days it's not uncommon for an OpenLDAP slapd server to handle tens 
of thousands of connections per second in real use (e.g., at Google, or 
at various telcos). While the LDAP server is fast enough to saturate 
even 10gbit ethernet using contemporary CPUs, we have to resort to 
multiple virtual interfaces just to make sure we have enough port 
numbers available.

Ideally the 2MSL parameter would be dynamically adjusted based on the 
route to the destination and the weights associated with those routes. 
In the simplest case, connections between machines on the same subnet 
(i.e., no router hops involved) should have a much smaller default value 
than connections that traverse any routers. I'd settle for a two-level 
setting - with no router hops, use the small value; with any router hops 
use the large value.
-- 
   -- Howard Chu
   Chief Architect, Symas Corp.  http://www.symas.com
   Director, Highland Sun        http://highlandsun.com/hyc
   Chief Architect, OpenLDAP     http://www.openldap.org/project/

Re: TCP 2MSL on loopback

[Eric Dumazet]

On Monday 05 March 2007 12:20, Howard Chu wrote:
> Why is the Maximum Segment Lifetime a global parameter? Surely the
> maximum possible lifetime of a particular TCP segment depends on the
> actual connection. At the very least, it would be useful to be able to
> set it on a per-interface basis. E.g., in the case of the loopback
> interface, it would be useful to be able to set it to a very small
> duration.

Hi Howard

I think you should address these questions on netdev instead of linux-kernel.

>
> As I note in this draft
> http://www.ietf.org/internet-drafts/draft-chu-ldap-ldapi-00.txt
> when doing a connection soak test of OpenLDAP using clients connected
> through localhost, the entire port range is exhausted in well under a
> second, at which point the test stalls until a port comes out of
> TIME_WAIT state so the next connection can be opened.
>
> These days it's not uncommon for an OpenLDAP slapd server to handle tens
> of thousands of connections per second in real use (e.g., at Google, or
> at various telcos). While the LDAP server is fast enough to saturate
> even 10gbit ethernet using contemporary CPUs, we have to resort to
> multiple virtual interfaces just to make sure we have enough port
> numbers available.
>

I dont uderstand... doesnt slapd server listen for connections on a given 
port, like http ? Or is it doing connections like a ftp server ?

Of course, if you want to open more than 60.000 concurrent connections, using 
127.0.0.1 address, you might have a problem...

> Ideally the 2MSL parameter would be dynamically adjusted based on the
> route to the destination and the weights associated with those routes.
> In the simplest case, connections between machines on the same subnet
> (i.e., no router hops involved) should have a much smaller default value
> than connections that traverse any routers. I'd settle for a two-level
> setting - with no router hops, use the small value; with any router hops
> use the large value.

Well, is it really a MSL problem ?

I did a small test (linux-2.6.21-rc1) and was able to get 1.000.000 
connections on localhost on my dual proc machine in one minute, without an 
error.

Re: TCP 2MSL on loopback

[David Miller]

You might want to post networking questions to the networking
developer mailing list, netdev@vger.kernel.org, instead of
linux-kernel which most of the networking developers do not
read.

Thank you.