Netdev Archive on lore.kernel.org
help / color / mirror / Atom feed
* [PATCH bpf-next 0/3] XDP bonding support
@ 2021-06-09 13:55 Jussi Maki
  2021-06-09 13:55 ` [PATCH bpf-next 1/3] net: bonding: Add XDP support to the bonding driver Jussi Maki
                   ` (8 more replies)
  0 siblings, 9 replies; 71+ messages in thread
From: Jussi Maki @ 2021-06-09 13:55 UTC (permalink / raw)
  To: bpf; +Cc: netdev, daniel, j.vosburgh, andy, vfalico, andrii, Jussi Maki

This patchset introduces XDP support to the bonding driver.

Patch 1 contains the implementation, including support for
the recently introduced EXCLUDE_INGRESS. Patch 2 contains a
performance fix to the roundrobin mode which switches rr_tx_counter
to be per-cpu. Patch 3 contains the test suite for the implementation
using a pair of veth devices.

The vmtest.sh is modified to enable the bonding module and install
modules. The config change should probably be done in the libbpf
repository. Andrii: How would you like this done properly?

The motivation for this change is to enable use of bonding (and
802.3ad) in hairpinning L4 load-balancers such as [1] implemented with
XDP and also to transparently support bond devices for projects that
use XDP given most modern NICs have dual port adapters.  An alternative
to this approach would be to implement 802.3ad in user-space and
implement the bonding load-balancing in the XDP program itself, but
is rather a cumbersome endeavor in terms of slave device management
(e.g. by watching netlink) and requires separate programs for native
vs bond cases for the orchestrator. A native in-kernel implementation
overcomes these issues and provides more flexibility.

Below are benchmark results done on two machines with 100Gbit
Intel E810 (ice) NIC and with 32-core 3970X on sending machine, and
16-core 3950X on receiving machine. 64 byte packets were sent with
pktgen-dpdk at full rate. Two issues [2, 3] were identified with the
ice driver, so the tests were performed with iommu=off and patch [2]
applied. Additionally the bonding round robin algorithm was modified
to use per-cpu tx counters as high CPU load (50% vs 10%) and high rate
of cache misses were caused by the shared rr_tx_counter (see patch
2/3). The statistics were collected using "sar -n dev -u 1 10".

 -----------------------|  CPU  |--| rxpck/s |--| txpck/s |----
 without patch (1 dev):
   XDP_DROP:              3.15%      48.6Mpps
   XDP_TX:                3.12%      18.3Mpps     18.3Mpps
   XDP_DROP (RSS):        9.47%      116.5Mpps
   XDP_TX (RSS):          9.67%      25.3Mpps     24.2Mpps
 -----------------------
 with patch, bond (1 dev):
   XDP_DROP:              3.14%      46.7Mpps
   XDP_TX:                3.15%      13.9Mpps     13.9Mpps
   XDP_DROP (RSS):        10.33%     117.2Mpps
   XDP_TX (RSS):          10.64%     25.1Mpps     24.0Mpps
 -----------------------
 with patch, bond (2 devs):
   XDP_DROP:              6.27%      92.7Mpps
   XDP_TX:                6.26%      17.6Mpps     17.5Mpps
   XDP_DROP (RSS):       11.38%      117.2Mpps
   XDP_TX (RSS):         14.30%      28.7Mpps     27.4Mpps
 --------------------------------------------------------------

RSS: Receive Side Scaling, e.g. the packets were sent to a range of
destination IPs.

[1]: https://cilium.io/blog/2021/05/20/cilium-110#standalonelb
[2]: https://lore.kernel.org/bpf/20210601113236.42651-1-maciej.fijalkowski@intel.com/T/#t
[3]: https://lore.kernel.org/bpf/CAHn8xckNXci+X_Eb2WMv4uVYjO2331UWB2JLtXr_58z0Av8+8A@mail.gmail.com/

---

Jussi Maki (3):
  net: bonding: Add XDP support to the bonding driver
  net: bonding: Use per-cpu rr_tx_counter
  selftests/bpf: Add tests for XDP bonding

 drivers/net/bonding/bond_main.c               | 459 +++++++++++++++---
 include/linux/filter.h                        |  13 +-
 include/linux/netdevice.h                     |   5 +
 include/net/bonding.h                         |   3 +-
 kernel/bpf/devmap.c                           |  34 +-
 net/core/filter.c                             |  37 +-
 .../selftests/bpf/prog_tests/xdp_bonding.c    | 342 +++++++++++++
 tools/testing/selftests/bpf/vmtest.sh         |  30 +-
 8 files changed, 843 insertions(+), 80 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/xdp_bonding.c

-- 
2.30.2


^ permalink raw reply	[flat|nested] 71+ messages in thread

end of thread, other threads:[~2021-08-16  9:06 UTC | newest]

Thread overview: 71+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-09 13:55 [PATCH bpf-next 0/3] XDP bonding support Jussi Maki
2021-06-09 13:55 ` [PATCH bpf-next 1/3] net: bonding: Add XDP support to the bonding driver Jussi Maki
2021-06-09 22:29   ` Maciej Fijalkowski
2021-06-09 23:29   ` Jay Vosburgh
2021-06-14  8:02     ` Jussi Maki
2021-06-17  3:40   ` kernel test robot
2021-06-17  6:35   ` kernel test robot
2021-06-22  7:24   ` kernel test robot
2021-06-09 13:55 ` [PATCH bpf-next 2/3] net: bonding: Use per-cpu rr_tx_counter Jussi Maki
2021-06-10  0:04   ` Jay Vosburgh
2021-06-14  7:54     ` Jussi Maki
2021-06-09 13:55 ` [PATCH bpf-next 3/3] selftests/bpf: Add tests for XDP bonding Jussi Maki
2021-06-09 22:07   ` Maciej Fijalkowski
2021-06-14  8:08     ` Jussi Maki
2021-06-14  8:48       ` Magnus Karlsson
2021-06-14 12:20         ` Jussi Maki
2021-06-10 17:24 ` [PATCH bpf-next 0/3] XDP bonding support Andrii Nakryiko
2021-06-14 12:25   ` Jussi Maki
2021-06-14 15:37     ` Jay Vosburgh
2021-06-15  5:34     ` Andrii Nakryiko
2021-06-24  9:18 ` [PATCH bpf-next v2 0/4] " joamaki
2021-06-24  9:18   ` [PATCH bpf-next v2 1/4] net: bonding: Refactor bond_xmit_hash for use with xdp_buff joamaki
2021-06-24  9:18   ` [PATCH bpf-next v2 2/4] net: core: Add support for XDP redirection to slave device joamaki
2021-06-24  9:18   ` [PATCH bpf-next v2 3/4] net: bonding: Add XDP support to the bonding driver joamaki
2021-06-24  9:18   ` [PATCH bpf-next v2 4/4] devmap: Exclude XDP broadcast to master device joamaki
2021-07-01 18:12     ` Jay Vosburgh
2021-07-05 11:44       ` Jussi Maki
2021-07-01 18:20   ` [PATCH bpf-next v2 0/4] XDP bonding support Jay Vosburgh
2021-07-05 10:32     ` Jussi Maki
2021-07-07 11:25 ` [PATCH bpf-next v3 0/5] " Jussi Maki
2021-07-07 11:25   ` [PATCH bpf-next v3 1/5] net: bonding: Refactor bond_xmit_hash for use with xdp_buff Jussi Maki
2021-07-07 11:25   ` [PATCH bpf-next v3 2/5] net: core: Add support for XDP redirection to slave device Jussi Maki
2021-07-07 11:25   ` [PATCH bpf-next v3 3/5] net: bonding: Add XDP support to the bonding driver Jussi Maki
2021-07-13  7:14     ` kernel test robot
2021-07-07 11:25   ` [PATCH bpf-next v3 4/5] devmap: Exclude XDP broadcast to master device Jussi Maki
2021-07-07 11:25   ` [PATCH bpf-next v3 5/5] net: core: Allow netdev_lower_get_next_private_rcu in bh context Jussi Maki
2021-07-28 23:43 ` [PATCH bpf-next v4 0/6] XDP bonding support joamaki
2021-07-28 23:43   ` [PATCH bpf-next v4 1/6] net: bonding: Refactor bond_xmit_hash for use with xdp_buff joamaki
2021-07-28 23:43   ` [PATCH bpf-next v4 2/6] net: core: Add support for XDP redirection to slave device joamaki
2021-07-28 23:43   ` [PATCH bpf-next v4 3/6] net: bonding: Add XDP support to the bonding driver joamaki
2021-07-28 23:43   ` [PATCH bpf-next v4 4/6] devmap: Exclude XDP broadcast to master device joamaki
2021-07-28 23:43   ` [PATCH bpf-next v4 5/6] net: core: Allow netdev_lower_get_next_private_rcu in bh context joamaki
2021-07-28 23:43   ` [PATCH bpf-next v4 6/6] selftests/bpf: Add tests for XDP bonding joamaki
2021-08-03  0:19     ` Andrii Nakryiko
2021-08-03  9:40       ` Jussi Maki
2021-07-30  6:18 ` [PATCH bpf-next v5 0/7] XDP bonding support Jussi Maki
2021-07-30  6:18   ` [PATCH bpf-next v5 1/7] net: bonding: Refactor bond_xmit_hash for use with xdp_buff Jussi Maki
2021-07-30  6:18   ` [PATCH bpf-next v5 2/7] net: core: Add support for XDP redirection to slave device Jussi Maki
2021-07-30  6:18   ` [PATCH bpf-next v5 3/7] net: bonding: Add XDP support to the bonding driver Jussi Maki
2021-07-30  6:18   ` [PATCH bpf-next v5 4/7] devmap: Exclude XDP broadcast to master device Jussi Maki
2021-07-30  6:18   ` [PATCH bpf-next v5 5/7] net: core: Allow netdev_lower_get_next_private_rcu in bh context Jussi Maki
2021-07-30  6:18   ` [PATCH bpf-next v5 6/7] selftests/bpf: Fix xdp_tx.c prog section name Jussi Maki
2021-08-04 23:35     ` Andrii Nakryiko
2021-07-30  6:18   ` [PATCH bpf-next v5 7/7] selftests/bpf: Add tests for XDP bonding Jussi Maki
2021-08-04 23:33     ` Andrii Nakryiko
2021-07-31  5:57 ` [PATCH bpf-next v6 0/7]: XDP bonding support Jussi Maki
2021-07-31  5:57   ` [PATCH bpf-next v6 1/7] net: bonding: Refactor bond_xmit_hash for use with xdp_buff Jussi Maki
2021-08-11  1:52     ` Jonathan Toppins
2021-08-11  8:22       ` Jussi Maki
2021-08-11 14:05         ` Jonathan Toppins
2021-08-16  9:05           ` Jussi Maki
2021-07-31  5:57   ` [PATCH bpf-next v6 2/7] net: core: Add support for XDP redirection to slave device Jussi Maki
2021-07-31  5:57   ` [PATCH bpf-next v6 3/7] net: bonding: Add XDP support to the bonding driver Jussi Maki
2021-07-31  5:57   ` [PATCH bpf-next v6 4/7] devmap: Exclude XDP broadcast to master device Jussi Maki
2021-07-31  5:57   ` [PATCH bpf-next v6 5/7] net: core: Allow netdev_lower_get_next_private_rcu in bh context Jussi Maki
2021-07-31  5:57   ` [PATCH bpf-next v6 6/7] selftests/bpf: Fix xdp_tx.c prog section name Jussi Maki
2021-08-06 22:53     ` Andrii Nakryiko
2021-07-31  5:57   ` [PATCH bpf-next v6 7/7] selftests/bpf: Add tests for XDP bonding Jussi Maki
2021-08-06 22:50     ` Andrii Nakryiko
2021-08-09 14:24       ` Jussi Maki
2021-08-09 21:41         ` Daniel Borkmann

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).