Netdev Archive on lore.kernel.org
help / color / mirror / Atom feed
From: David Laight <David.Laight@ACULAB.COM>
To: 'Ben Greear' <greearb@candelatech.com>,
Neal Cardwell <ncardwell@google.com>
Cc: netdev <netdev@vger.kernel.org>
Subject: RE: Debugging stuck tcp connection across localhost
Date: Mon, 10 Jan 2022 22:16:50 +0000 [thread overview]
Message-ID: <e8e6693695c04bd6a679ddd43733703b@AcuMS.aculab.com> (raw)
In-Reply-To: <9330e1c7-f7a2-0f1e-0ede-c9e5353060e3@candelatech.com>
From: Ben Greear <greearb@candelatech.com>
> Sent: 10 January 2022 18:10
...
> From my own looking at things, it seems that the sniffer fails to get frames near when the problem
> starts happening. I am baffled as to how that can happen, especially since it seems to stop getting
> packets from multiple different TCP connections (the sniffer filter would pick up some other loop-back
> related connections to the same IP port).
>
> And, if I interpret the ss output properly, after the problem happens, the sockets still think they
> are
> sending data. I didn't check closely enough to see if the peer side thought it received it.
>
> We are going to try to reproduce w/out wifi, but not sure we'll have any luck with that.
> We did test w/out VRF (using lots of ip rules instead), and similar problem was seen according to my
> test team (I did not debug it in detail).
>
> Do you have any suggestions for how to debug this further? I am happy to hack stuff into the
> kernel if you have some suggested places to add debugging...
Sounds like all transmit traffic on the loopback interface is being discarded
before the point where the frames get fed to tsmdump.
Possibly you could use ftrace to trace function entry+exit of a few
functions that happen in the transmit path and then isolate the point
where the discard is happening.
You can't afford to trace everything - slows things down too much.
But a few traces on each send path should be ok.
David
-
Registered Address Lakeside, Bramley Road, Mount Farm, Milton Keynes, MK1 1PT, UK
Registration No: 1397386 (Wales)
next prev parent reply other threads:[~2022-01-10 22:17 UTC|newest]
Thread overview: 19+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-01-06 14:59 Ben Greear
2022-01-06 15:20 ` Neal Cardwell
2022-01-06 15:39 ` Ben Greear
2022-01-06 16:16 ` Neal Cardwell
2022-01-06 19:05 ` Ben Greear
2022-01-06 20:04 ` Neal Cardwell
2022-01-06 20:20 ` Ben Greear
2022-01-06 22:26 ` Ben Greear
2022-01-10 18:10 ` Ben Greear
2022-01-10 22:16 ` David Laight [this message]
2022-01-11 10:46 ` Eric Dumazet
2022-01-11 21:35 ` Ben Greear
2022-01-12 7:41 ` Eric Dumazet
2022-01-12 14:52 ` Ben Greear
2022-01-12 17:12 ` Eric Dumazet
2022-01-12 18:01 ` Debugging stuck tcp connection across localhost [snip] Ben Greear
2022-01-12 18:44 ` Ben Greear
2022-01-12 18:47 ` Eric Dumazet
2022-01-12 18:54 ` Ben Greear
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=e8e6693695c04bd6a679ddd43733703b@AcuMS.aculab.com \
--to=david.laight@aculab.com \
--cc=greearb@candelatech.com \
--cc=ncardwell@google.com \
--cc=netdev@vger.kernel.org \
--subject='RE: Debugging stuck tcp connection across localhost' \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).