LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Eric Dumazet <eric.dumazet@gmail.com>
To: PK <runningdoglackey@yahoo.com>, David Miller <davem@davemloft.net>
Cc: linux-kernel@vger.kernel.org, netdev <netdev@vger.kernel.org>,
	Tom Herbert <therbert@google.com>
Subject: Re: Problems with /proc/net/tcp6  - possible bug - ipv6
Date: Sat, 22 Jan 2011 16:15:44 +0100	[thread overview]
Message-ID: <1295709344.2651.55.camel@edumazet-laptop> (raw)
In-Reply-To: <1295686781.2609.37.camel@edumazet-laptop>

Le samedi 22 janvier 2011 à 09:59 +0100, Eric Dumazet a écrit : 
> Le vendredi 21 janvier 2011 à 22:30 -0800, PK a écrit :
> > Creating many ipv6 connections hits a ceiling on connections/fds ; okay, fine.
> > 
> > But in my case I'm seeing millions of entries spring up within a few seconds and 
> > then vanish within a few minutes, in /proc/net/tcp6 (vanish due to garbage 
> > collection?)
> > 
> > Furthermore I can trigger this easily on vanilla kernels from 2.6.36 to 
> > 2.6.38-rc1-next-20110121  inside a ubuntu 10.10 amd64 vm, causing the kernel to 
> > spew warnings.  There is also some corruption in the logs (see kernel-sample.log 
> > line 296), but that may be unrelated.
> > 
> > More explanation, kernel config of the primary machine I saw this on, sample 
> > ruby script to reproduce (inside the ubuntu VMs I apt-get and use ruby-1.9.1), 
> > are located at
> > https://github.com/runningdogx/net6-bug
> > 
> > Seems to only affect 64-bit.  So far I have not been able to reproduce on 32-bit 
> > ubuntu VMs of any kernel version.
> > Seems to only affect IPv6.  So far I have not been able to reproduce using IPv4 
> > connections (and watching /proc/net/tcp of course).
> > Does not trigger the bug if the connections are made to ::1.  Only externally 
> > routable local and global IPv6 addresses seem to cause problems.
> > Seems to have been introduced between 2.6.35 and 2.6.36 (see README on github 
> > for more kernels I've tried)
> > 
> > All the tested Ubuntu VMs are stock 10.10 userland, with vanilla kernels (the 
> > latest ubuntu kernel is 2.6.35-something, and my initial test didn't show it 
> > suffering from this problem)
> > 
> > Originally noticed on separate Gentoo 64-bit non-vm system when doing web 
> > benchmarking.
> > 
> > not subscribed, so please keep me in cc although I'll try to follow the thread
> > 
> > 


I had some incidents, after hours of testing...

After following patch, I could not reproduce it.

I cant believe this bug was not noticed before today.

Thanks !

[PATCH] tcp: fix bug in listening_get_next()

commit a8b690f98baf9fb19 (tcp: Fix slowness in read /proc/net/tcp)
introduced a bug in handling of SYN_RECV sockets.

st->offset represents number of sockets found since beginning of
listening_hash[st->bucket].

We should not reset st->offset when iterating through
syn_table[st->sbucket], or else if more than ~25 sockets (if
PAGE_SIZE=4096) are in SYN_RECV state, we exit from listening_get_next()
with a too small st->offset

Next time we enter tcp_seek_last_pos(), we are not able to seek past
already found sockets.

Reported-by: PK <runningdoglackey@yahoo.com>
CC: Tom Herbert <therbert@google.com>
Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
---
 net/ipv4/tcp_ipv4.c |    1 -
 1 file changed, 1 deletion(-)

diff --git a/net/ipv4/tcp_ipv4.c b/net/ipv4/tcp_ipv4.c
index 856f684..02f583b 100644
--- a/net/ipv4/tcp_ipv4.c
+++ b/net/ipv4/tcp_ipv4.c
@@ -1994,7 +1994,6 @@ static void *listening_get_next(struct seq_file *seq, void *cur)
 				}
 				req = req->dl_next;
 			}
-			st->offset = 0;
 			if (++st->sbucket >= icsk->icsk_accept_queue.listen_opt->nr_table_entries)
 				break;
 get_req:



  reply	other threads:[~2011-01-22 15:15 UTC|newest]

Thread overview: 11+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-01-22  6:30 PK
2011-01-22  8:59 ` Eric Dumazet
2011-01-22 15:15   ` Eric Dumazet [this message]
2011-01-22 19:42     ` PK
2011-01-22 21:20       ` Eric Dumazet
2011-01-22 21:40         ` Eric Dumazet
2011-01-24 22:31           ` David Miller
2011-01-24 22:40           ` David Miller
2011-01-25  0:02       ` David Miller
2011-01-24 22:42     ` David Miller
2011-01-31 22:51 PK

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1295709344.2651.55.camel@edumazet-laptop \
    --to=eric.dumazet@gmail.com \
    --cc=davem@davemloft.net \
    --cc=linux-kernel@vger.kernel.org \
    --cc=netdev@vger.kernel.org \
    --cc=runningdoglackey@yahoo.com \
    --cc=therbert@google.com \
    --subject='Re: Problems with /proc/net/tcp6  - possible bug - ipv6' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).