Linux-Fsdevel Archive on lore.kernel.org
help / color / mirror / Atom feed
* aio poll, io_pgetevents and a new in-kernel poll API V3
@ 2018-01-17 19:27 Christoph Hellwig
  2018-01-17 19:27 ` [PATCH 01/36] aio: don't print the page size at boot time Christoph Hellwig
                   ` (36 more replies)
  0 siblings, 37 replies; 48+ messages in thread
From: Christoph Hellwig @ 2018-01-17 19:27 UTC (permalink / raw)
  To: viro
  Cc: Avi Kivity, linux-aio, linux-fsdevel, netdev, linux-api, linux-kernel

Hi all,

this series adds support for the IOCB_CMD_POLL operation to poll for the
readyness of file descriptors using the aio subsystem.  The API is based
on patches that existed in RHAS2.1 and RHEL3, which means it already is
supported by libaio.  To implement the poll support efficiently new
methods to poll are introduced in struct file_operations:  get_poll_head
and poll_mask.  The first one returns a wait_queue_head to wait on
(lifetime is bound by the file), and the second does a non-blocking
check for the POLL* events.  This allows aio poll to work without
any additional context switches, unlike epoll.

To make the interface fully useful a new io_pgetevents system call is
added, which atomically saves and restores the signal mask over the
io_pgetevents system call.  It it the logical equivalent to pselect and
ppoll for io_pgetevents.

The corresponding libaio changes for io_pgetevents support and
documentation, as well as a test case will be posted in a separate
series.

The changes were sponsored by Scylladb, and improve performance
of the seastar framework up to 10%, while also removing the need
for a privileged SCHED_FIFO epoll listener thread.

The patches are on top of Als __poll_t annoations, so I've also
prepared a git branch on top of those here:

    git://git.infradead.org/users/hch/vfs.git aio-poll.3

Gitweb:

    http://git.infradead.org/users/hch/vfs.git/shortlog/refs/heads/aio-poll.3

Libaio changes:

    https://pagure.io/libaio.git io-poll

Seastar changes (not updated for the new io_pgetevens ABI yet):

    https://github.com/avikivity/seastar/commits/aio

Changes since V2:
 - removed a double initialization
 - new vfs_get_poll_head helper
 - document that ->get_poll_head can return NULL
 - call ->poll_mask before sleeping
 - various ACKs
 - add conversion of random to ->poll_mask
 - add conversion of af_alg to ->poll_mask
 - lacking ->poll_mask support now returns -EINVAL for IOCB_CMD_POLL
 - reshuffled the series so that prep patches and everything not
   requiring the new in-kernel poll API is in the beginning

Changes since V1:
 - handle the NULL ->poll case in vfs_poll
 - dropped the file argument to the ->poll_mask socket operation
 - replace the ->pre_poll socket operation with ->get_poll_head as
   in the file operations

^ permalink raw reply	[flat|nested] 48+ messages in thread
* aio poll, io_pgetevents and a new in-kernel poll API V4
@ 2018-01-22 20:12 Christoph Hellwig
  2018-01-22 20:12 ` [PATCH 35/36] timerfd: convert to ->poll_mask Christoph Hellwig
  0 siblings, 1 reply; 48+ messages in thread
From: Christoph Hellwig @ 2018-01-22 20:12 UTC (permalink / raw)
  To: viro
  Cc: Avi Kivity, linux-aio, linux-fsdevel, netdev, linux-api, linux-kernel

Hi all,

this series adds support for the IOCB_CMD_POLL operation to poll for the
readyness of file descriptors using the aio subsystem.  The API is based
on patches that existed in RHAS2.1 and RHEL3, which means it already is
supported by libaio.  To implement the poll support efficiently new
methods to poll are introduced in struct file_operations:  get_poll_head
and poll_mask.  The first one returns a wait_queue_head to wait on
(lifetime is bound by the file), and the second does a non-blocking
check for the POLL* events.  This allows aio poll to work without
any additional context switches, unlike epoll.

To make the interface fully useful a new io_pgetevents system call is
added, which atomically saves and restores the signal mask over the
io_pgetevents system call.  It it the logical equivalent to pselect and
ppoll for io_pgetevents.

The corresponding libaio changes for io_pgetevents support and
documentation, as well as a test case will be posted in a separate
series.

The changes were sponsored by Scylladb, and improve performance
of the seastar framework up to 10%, while also removing the need
for a privileged SCHED_FIFO epoll listener thread.

The patches are on top of Als __poll_t annoations, so I've also
prepared a git branch on top of those here:

    git://git.infradead.org/users/hch/vfs.git aio-poll.4

Gitweb:

    http://git.infradead.org/users/hch/vfs.git/shortlog/refs/heads/aio-poll.4

Libaio changes:

    https://pagure.io/libaio.git io-poll

Seastar changes (not updated for the new io_pgetevens ABI yet):

    https://github.com/avikivity/seastar/commits/aio

Changes since V3:
 - remove the pre-sleep ->poll_mask call in vfs_poll,
   allow ->get_poll_head to return POLL* values.

Changes since V2:
 - removed a double initialization
 - new vfs_get_poll_head helper
 - document that ->get_poll_head can return NULL
 - call ->poll_mask before sleeping
 - various ACKs
 - add conversion of random to ->poll_mask
 - add conversion of af_alg to ->poll_mask
 - lacking ->poll_mask support now returns -EINVAL for IOCB_CMD_POLL
 - reshuffled the series so that prep patches and everything not
   requiring the new in-kernel poll API is in the beginning

Changes since V1:
 - handle the NULL ->poll case in vfs_poll
 - dropped the file argument to the ->poll_mask socket operation
 - replace the ->pre_poll socket operation with ->get_poll_head as
   in the file operations

^ permalink raw reply	[flat|nested] 48+ messages in thread
* aio poll, io_pgetevents and a new in-kernel poll API V5
@ 2018-03-05 21:27 Christoph Hellwig
  2018-03-05 21:27 ` [PATCH 35/36] timerfd: convert to ->poll_mask Christoph Hellwig
  0 siblings, 1 reply; 48+ messages in thread
From: Christoph Hellwig @ 2018-03-05 21:27 UTC (permalink / raw)
  To: viro
  Cc: Avi Kivity, linux-aio, linux-fsdevel, netdev, linux-api, linux-kernel

Hi all,

this series adds support for the IOCB_CMD_POLL operation to poll for the
readyness of file descriptors using the aio subsystem.  The API is based
on patches that existed in RHAS2.1 and RHEL3, which means it already is
supported by libaio.  To implement the poll support efficiently new
methods to poll are introduced in struct file_operations:  get_poll_head
and poll_mask.  The first one returns a wait_queue_head to wait on
(lifetime is bound by the file), and the second does a non-blocking
check for the POLL* events.  This allows aio poll to work without
any additional context switches, unlike epoll.

To make the interface fully useful a new io_pgetevents system call is
added, which atomically saves and restores the signal mask over the
io_pgetevents system call.  It it the logical equivalent to pselect and
ppoll for io_pgetevents.

The corresponding libaio changes for io_pgetevents support and
documentation, as well as a test case will be posted in a separate
series.

The changes were sponsored by Scylladb, and improve performance
of the seastar framework up to 10%, while also removing the need
for a privileged SCHED_FIFO epoll listener thread.

    git://git.infradead.org/users/hch/vfs.git aio-poll.5

Gitweb:

    http://git.infradead.org/users/hch/vfs.git/shortlog/refs/heads/aio-poll.5

Libaio changes:

    https://pagure.io/libaio.git io-poll

Seastar changes (not updated for the new io_pgetevens ABI yet):

    https://github.com/avikivity/seastar/commits/aio

Changes since V4:
 - rebased ontop of Linux 4.16-rc4

Changes since V3:
 - remove the pre-sleep ->poll_mask call in vfs_poll,
   allow ->get_poll_head to return POLL* values.

Changes since V2:
 - removed a double initialization
 - new vfs_get_poll_head helper
 - document that ->get_poll_head can return NULL
 - call ->poll_mask before sleeping
 - various ACKs
 - add conversion of random to ->poll_mask
 - add conversion of af_alg to ->poll_mask
 - lacking ->poll_mask support now returns -EINVAL for IOCB_CMD_POLL
 - reshuffled the series so that prep patches and everything not
   requiring the new in-kernel poll API is in the beginning

Changes since V1:
 - handle the NULL ->poll case in vfs_poll
 - dropped the file argument to the ->poll_mask socket operation
 - replace the ->pre_poll socket operation with ->get_poll_head as
   in the file operations

^ permalink raw reply	[flat|nested] 48+ messages in thread

end of thread, other threads:[~2018-03-05 21:28 UTC | newest]

Thread overview: 48+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-01-17 19:27 aio poll, io_pgetevents and a new in-kernel poll API V3 Christoph Hellwig
2018-01-17 19:27 ` [PATCH 01/36] aio: don't print the page size at boot time Christoph Hellwig
2018-01-17 19:27 ` [PATCH 02/36] aio: remove an outdated comment in aio_complete Christoph Hellwig
2018-01-17 19:27 ` [PATCH 03/36] aio: refactor read/write iocb setup Christoph Hellwig
2018-01-17 19:27 ` [PATCH 04/36] aio: sanitize ki_list handling Christoph Hellwig
2018-01-17 19:27 ` [PATCH 05/36] aio: simplify cancellation Christoph Hellwig
2018-01-17 19:27 ` [PATCH 06/36] aio: delete iocbs from the active_reqs list in kiocb_cancel Christoph Hellwig
2018-01-17 19:27 ` [PATCH 07/36] aio: add delayed cancel support Christoph Hellwig
2018-01-17 19:27 ` [PATCH 08/36] aio: implement io_pgetevents Christoph Hellwig
2018-01-17 19:27 ` [PATCH 09/36] fs: unexport poll_schedule_timeout Christoph Hellwig
2018-01-17 19:27 ` [PATCH 10/36] fs: cleanup do_pollfd Christoph Hellwig
2018-01-17 19:27 ` [PATCH 11/36] fs: update documentation for __poll_t Christoph Hellwig
2018-01-17 19:27 ` [PATCH 12/36] fs: add new vfs_poll and file_can_poll helpers Christoph Hellwig
2018-01-17 19:27 ` [PATCH 13/36] fs: introduce new ->get_poll_head and ->poll_mask methods Christoph Hellwig
2018-01-17 19:27 ` [PATCH 14/36] aio: implement IOCB_CMD_POLL Christoph Hellwig
2018-01-17 19:27 ` [PATCH 15/36] net: refactor socket_poll Christoph Hellwig
2018-01-17 19:27 ` [PATCH 16/36] net: add support for ->poll_mask in proto_ops Christoph Hellwig
2018-01-17 19:27 ` [PATCH 17/36] net: remove sock_no_poll Christoph Hellwig
2018-01-17 19:27 ` [PATCH 18/36] net/tcp: convert to ->poll_mask Christoph Hellwig
2018-01-17 19:27 ` [PATCH 19/36] net/unix: " Christoph Hellwig
2018-01-17 19:27 ` [PATCH 20/36] net: convert datagram_poll users tp ->poll_mask Christoph Hellwig
2018-01-17 19:27 ` [PATCH 21/36] net/dccp: convert to ->poll_mask Christoph Hellwig
2018-01-17 19:27 ` [PATCH 22/36] net/atm: " Christoph Hellwig
2018-01-17 19:27 ` [PATCH 23/36] net/vmw_vsock: " Christoph Hellwig
2018-01-17 19:27 ` [PATCH 24/36] net/tipc: " Christoph Hellwig
2018-01-17 19:27 ` [PATCH 25/36] net/sctp: " Christoph Hellwig
2018-01-17 19:27 ` [PATCH 26/36] net/bluetooth: " Christoph Hellwig
2018-01-17 19:27 ` [PATCH 27/36] net/caif: " Christoph Hellwig
2018-01-17 19:27 ` [PATCH 28/36] net/nfc: " Christoph Hellwig
2018-01-17 19:27 ` [PATCH 29/36] net/phonet: " Christoph Hellwig
2018-01-17 19:27 ` [PATCH 30/36] net/iucv: " Christoph Hellwig
2018-01-17 19:27 ` [PATCH 31/36] net/rxrpc: " Christoph Hellwig
2018-01-17 19:27 ` [PATCH 32/36] crypto: af_alg: " Christoph Hellwig
2018-01-17 19:27 ` [PATCH 33/36] pipe: " Christoph Hellwig
2018-01-17 19:27 ` [PATCH 34/36] eventfd: switch " Christoph Hellwig
2018-01-17 19:27 ` [PATCH 35/36] timerfd: convert " Christoph Hellwig
2018-01-17 19:27 ` [PATCH 36/36] random: " Christoph Hellwig
2018-01-18 15:46 ` aio poll, io_pgetevents and a new in-kernel poll API V3 Jeff Moyer
2018-01-18 16:44   ` Jeff Moyer
2018-01-18 17:42     ` Christoph Hellwig
2018-01-18 17:59       ` Jeff Moyer
2018-01-18 17:55     ` Colin Walters
2018-01-18 18:53       ` Christoph Hellwig
2018-01-18 17:51   ` Avi Kivity
2018-01-18 17:52     ` Avi Kivity
2018-01-18 17:54     ` Jeff Moyer
2018-01-22 20:12 aio poll, io_pgetevents and a new in-kernel poll API V4 Christoph Hellwig
2018-01-22 20:12 ` [PATCH 35/36] timerfd: convert to ->poll_mask Christoph Hellwig
2018-03-05 21:27 aio poll, io_pgetevents and a new in-kernel poll API V5 Christoph Hellwig
2018-03-05 21:27 ` [PATCH 35/36] timerfd: convert to ->poll_mask Christoph Hellwig

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).