Netdev Archive on
help / color / mirror / Atom feed
From: "Björn Töpel" <>
To: "Li,Rongqing" <>,
	"Björn Töpel" <>
Cc: Netdev <>,
	intel-wired-lan <>,
	"Karlsson, Magnus" <>,
	bpf <>,
	Maciej Fijalkowski <>,
	Piotr <>,
	Maciej <>
Subject: Re: 答复: [Intel-wired-lan] [PATCH 0/2] intel/xdp fixes for fliping rx buffer
Date: Wed, 19 Aug 2020 08:44:32 +0200	[thread overview]
Message-ID: <> (raw)
In-Reply-To: <>

On 2020-08-19 03:37, Li,Rongqing wrote:
 > Hi:
 > Thanks for your explanation.
 > But we can reproduce this bug
 > We use ebpf to redirect only-Vxlan packets to non-zerocopy AF_XDP, 
First we see panic on tcp stack, in tcp_collapse: BUG_ON(offset < 0); it 
is very hard to reproduce.
 > Then we use the scp to do test, and has lots of vxlan packet at the 
same time, scp will be broken frequently.

Ok! Just so that I'm certain of your setup. You receive packets to an
i40e netdev where there's an XDP program. The program does XDP_PASS or
XDP_REDIRECT to e.g. devmap for non-vxlan packets. However, vxlan
packets are redirected to AF_XDP socket(s) in *copy-mode*. Am I
understanding that correct?

I'm assuming this is an x86-64 with 4k page size, right? :-) The page
flipping is a bit different if the PAGE_SIZE is not 4k.

 > With this fixes, scp has not been broken again, and kernel is not 
panic again

Let's dig into your scenario.

Are you saying the following:

Page A:
| "first skb" ----> Rx HW ring entry X
| "second skb"----> Rx HW ring entry X+1 (or X+n)

This is a scenario that shouldn't be allowed, because there are now
two users of the page. If that's the case, the refcounting is
broken. Is that the case?

Check out i40e_can_reuse_rx_page(). The idea with page flipping/reuse
is that the page is only reused if there is only one user.

 > Seem your explanation is unable to solve my analysis:
 >         1. first skb is not for xsk, and forwarded to another device
 >            or socket queue

The data for the "first skb" resides on a page:
| "first skb"
| to be reused
refcount >>1

 >         2. seconds skb is for xsk, copy data to xsk memory, and page
 >            of skb->data is released

Note that page B != page A.

| to be reused/or used by the stack
| "second skb for xsk"
refcount >>1

data is copied to socket, page_frag_free() is called, and the page
count is decreased. The driver will then check if the page can be
reused. If not, it's freed to the page allocator.

 >         3. rx_buff is reusable since only first skb is in it, but
 >            *_rx_buffer_flip will make that page_offset is set to
 >            first skb data

I'm having trouble grasping how this is possible. More than one user
implies that it wont be reused. If this is possible, the
recounting/reuse mechanism is broken, and that is what should be

The AF_XDP redirect should not have semantics different from, say,
devmap redirect. It's just that the page_frag_free() is called earlier
for AF_XDP, instead of from i40e_clean_tx_irq() as the case for

 >         4. then reuse rx buffer, first skb which still is living
 >            will be corrupted.
 > The root cause is difference you said upper, so I only fixes for 
non-zerocopy AF_XDP

I have only addressed non-zerocopy, so we're on the same page (pun
intended) here!


 > -Li

  reply	other threads:[~2020-08-19  7:11 UTC|newest]

Thread overview: 15+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2020-07-17  6:24 Li RongQing
2020-07-17  6:24 ` [PATCH 1/2] xdp: i40e: ixgbe: ixgbevf: not flip rx buffer for copy mode xdp Li RongQing
2020-07-20  7:21   ` [Intel-wired-lan] " Magnus Karlsson
2020-07-21  1:42     ` 答复: " Li,Rongqing
2020-07-21  7:49     ` Li,Rongqing
2020-07-17  6:24 ` [PATCH 2/2] ice/xdp: not adjust " Li RongQing
2020-08-18 14:04 ` [Intel-wired-lan] [PATCH 0/2] intel/xdp fixes for fliping rx buffer Björn Töpel
2020-08-19  1:37   ` 答复: " Li,Rongqing
2020-08-19  6:44     ` Björn Töpel [this message]
2020-08-19  8:17       ` 答复: " Li,Rongqing
2020-08-19  8:31         ` Björn Töpel
2020-08-19  8:52           ` Björn Töpel
2020-08-20 15:13   ` Björn Töpel
2020-08-20 16:51     ` Maciej Fijalkowski
2020-08-20 18:04       ` Björn Töpel

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \ \ \ \ \ \ \ \ \ \ \ \
    --subject='Re: 答复: [Intel-wired-lan] [PATCH 0/2] intel/xdp fixes for fliping rx buffer' \

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).