Linux-Fsdevel Archive on lore.kernel.org help / color / mirror / Atom feed
From: Omar Sandoval <osandov@osandov.com> To: linux-btrfs@vger.kernel.org Cc: linux-fsdevel@vger.kernel.org Subject: [PATCH 10/11] btrfs-progs: send: stream v2 ioctl flags Date: Fri, 21 Aug 2020 00:40:09 -0700 [thread overview] Message-ID: <fc4038fec127ca2f68ee448b1f5b7fdf3a4372f6.1597994354.git.osandov@osandov.com> (raw) In-Reply-To: <cover.1597994106.git.osandov@osandov.com> From: Boris Burkov <boris@bur.io> To make the btrfs send ioctl use the stream v2 format requires passing BTRFS_SEND_FLAG_STREAM_V2 in flags. Further, to cause the ioctl to emit encoded_write commands for encoded extents, we must set that flag as well as BTRFS_SEND_FLAG_COMPRESSED. Finally, we bump up the version in send.h as well, since we are now fully compatible with v2. Add two command line arguments to btrfs send: --stream-version and --compressed. --stream-version requires an argument which it parses as an integer and sets STREAM_V2 if the argument is 2. --compressed does not require an argument and automatically implies STREAM_V2 as well (COMPRESSED alone causes the ioctl to error out). Some examples to illustrate edge cases: // v1, old format and no encoded_writes btrfs send subvol btrfs send --stream-version 1 subvol // v2 and compressed, we will see encoded_writes btrfs send --compressed subvol btrfs send --compressed --stream-version 2 subvol // v2 only, new format but no encoded_writes btrfs send --stream-version 2 subvol // error: compressed needs version >= 2 btrfs send --compressed --stream-version 1 subvol // error: invalid version (not 1 or 2) btrfs send --stream-version 3 subvol btrfs send --compressed --stream-version 0 subvol btrfs send --compressed --stream-version 10 subvol Signed-off-by: Boris Burkov <boris@bur.io> --- cmds/send.c | 39 +++++++++++++++++++++++++++++++++++++-- ioctl.h | 17 ++++++++++++++++- libbtrfsutil/btrfs.h | 17 ++++++++++++++++- send.h | 2 +- 4 files changed, 70 insertions(+), 5 deletions(-) diff --git a/cmds/send.c b/cmds/send.c index b8e3ba12..4c4eaa84 100644 --- a/cmds/send.c +++ b/cmds/send.c @@ -474,6 +474,7 @@ static int cmd_send(const struct cmd_struct *cmd, int argc, char **argv) int full_send = 1; int new_end_cmd_semantic = 0; u64 send_flags = 0; + long stream_version = 0; memset(&send, 0, sizeof(send)); send.dump_fd = fileno(stdout); @@ -492,11 +493,17 @@ static int cmd_send(const struct cmd_struct *cmd, int argc, char **argv) optind = 0; while (1) { - enum { GETOPT_VAL_SEND_NO_DATA = 256 }; + enum { + GETOPT_VAL_SEND_NO_DATA = 256, + GETOPT_VAL_SEND_STREAM_V2, + GETOPT_VAL_SEND_COMPRESSED + }; static const struct option long_options[] = { { "verbose", no_argument, NULL, 'v' }, { "quiet", no_argument, NULL, 'q' }, - { "no-data", no_argument, NULL, GETOPT_VAL_SEND_NO_DATA } + { "no-data", no_argument, NULL, GETOPT_VAL_SEND_NO_DATA }, + { "stream-version", required_argument, NULL, GETOPT_VAL_SEND_STREAM_V2 }, + { "compressed", no_argument, NULL, GETOPT_VAL_SEND_COMPRESSED } }; int c = getopt_long(argc, argv, "vqec:f:i:p:", long_options, NULL); @@ -584,10 +591,38 @@ static int cmd_send(const struct cmd_struct *cmd, int argc, char **argv) case GETOPT_VAL_SEND_NO_DATA: send_flags |= BTRFS_SEND_FLAG_NO_FILE_DATA; break; + case GETOPT_VAL_SEND_STREAM_V2: + stream_version = strtol(optarg, NULL, 10); + if (stream_version < 1 || stream_version > 2) { + ret = 1; + error("invalid --stream-version. valid values: {1, 2}"); + goto out; + } + if (stream_version == 2) + send_flags |= BTRFS_SEND_FLAG_STREAM_V2; + break; + case GETOPT_VAL_SEND_COMPRESSED: + send_flags |= BTRFS_SEND_FLAG_COMPRESSED; + /* + * We want to default to stream v2 if only compressed is + * set. If stream_version is explicitly set to 0, that + * will trigger its own error condition for being an + * invalid version. + */ + if (stream_version == 0) { + stream_version = 2; + send_flags |= BTRFS_SEND_FLAG_STREAM_V2; + } + break; default: usage_unknown_option(cmd, argv); } } + if (stream_version < 2 && (send_flags & BTRFS_SEND_FLAG_COMPRESSED)) { + ret = 1; + error("--compressed requires --stream-version >= 2"); + goto out; + } if (check_argc_min(argc - optind, 1)) return 1; diff --git a/ioctl.h b/ioctl.h index ade6dcb9..46de8ac8 100644 --- a/ioctl.h +++ b/ioctl.h @@ -653,10 +653,25 @@ BUILD_ASSERT(sizeof(struct btrfs_ioctl_received_subvol_args_32) == 192); */ #define BTRFS_SEND_FLAG_OMIT_END_CMD 0x4 +/* + * Use version 2 of the send stream, which adds new commands and supports larger + * writes. + */ +#define BTRFS_SEND_FLAG_STREAM_V2 0x8 + +/* + * Send compressed data using the ENCODED_WRITE command instead of decompressing + * the data and sending it with the WRITE command. This requires + * BTRFS_SEND_FLAG_STREAM_V2. + */ +#define BTRFS_SEND_FLAG_COMPRESSED 0x10 + #define BTRFS_SEND_FLAG_MASK \ (BTRFS_SEND_FLAG_NO_FILE_DATA | \ BTRFS_SEND_FLAG_OMIT_STREAM_HEADER | \ - BTRFS_SEND_FLAG_OMIT_END_CMD) + BTRFS_SEND_FLAG_OMIT_END_CMD | \ + BTRFS_SEND_FLAG_STREAM_V2 | \ + BTRFS_SEND_FLAG_COMPRESSED) struct btrfs_ioctl_send_args { __s64 send_fd; /* in */ diff --git a/libbtrfsutil/btrfs.h b/libbtrfsutil/btrfs.h index 60d51ff6..8430a40d 100644 --- a/libbtrfsutil/btrfs.h +++ b/libbtrfsutil/btrfs.h @@ -731,10 +731,25 @@ struct btrfs_ioctl_received_subvol_args { */ #define BTRFS_SEND_FLAG_OMIT_END_CMD 0x4 +/* + * Use version 2 of the send stream, which adds new commands and supports larger + * writes. + */ +#define BTRFS_SEND_FLAG_STREAM_V2 0x8 + +/* + * Send compressed data using the ENCODED_WRITE command instead of decompressing + * the data and sending it with the WRITE command. This requires + * BTRFS_SEND_FLAG_STREAM_V2. + */ +#define BTRFS_SEND_FLAG_COMPRESSED 0x10 + #define BTRFS_SEND_FLAG_MASK \ (BTRFS_SEND_FLAG_NO_FILE_DATA | \ BTRFS_SEND_FLAG_OMIT_STREAM_HEADER | \ - BTRFS_SEND_FLAG_OMIT_END_CMD) + BTRFS_SEND_FLAG_OMIT_END_CMD | \ + BTRFS_SEND_FLAG_STREAM_V2 | \ + BTRFS_SEND_FLAG_COMPRESSED) struct btrfs_ioctl_send_args { __s64 send_fd; /* in */ diff --git a/send.h b/send.h index 3c47e0c7..fac90588 100644 --- a/send.h +++ b/send.h @@ -31,7 +31,7 @@ extern "C" { #endif #define BTRFS_SEND_STREAM_MAGIC "btrfs-stream" -#define BTRFS_SEND_STREAM_VERSION 1 +#define BTRFS_SEND_STREAM_VERSION 2 #define BTRFS_SEND_BUF_SIZE_V1 SZ_64K #define BTRFS_SEND_READ_SIZE (1024 * 48) -- 2.28.0
next prev parent reply other threads:[~2020-08-21 7:41 UTC|newest] Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-08-21 7:39 [PATCH 0/9] btrfs: implement send/receive of compressed extents without decompressing Omar Sandoval 2020-08-21 7:39 ` [PATCH 1/9] btrfs: send: get rid of i_size logic in send_write() Omar Sandoval 2020-08-21 17:26 ` Filipe Manana 2020-08-24 17:39 ` Josef Bacik 2020-08-21 7:39 ` [PATCH 2/9] btrfs: send: avoid copying file data Omar Sandoval 2020-08-21 17:29 ` Filipe Manana 2020-08-24 21:34 ` Omar Sandoval 2020-08-24 17:47 ` Josef Bacik 2020-09-11 14:13 ` David Sterba 2020-09-14 22:04 ` Omar Sandoval 2020-09-15 8:14 ` David Sterba 2020-08-21 7:39 ` [PATCH 3/9] btrfs: send: use btrfs_file_extent_end() in send_write_or_clone() Omar Sandoval 2020-08-21 17:30 ` Filipe Manana 2020-08-21 7:39 ` [PATCH 4/9] btrfs: add send_stream_version attribute to sysfs Omar Sandoval 2020-08-21 7:39 ` [PATCH 5/9] btrfs: add send stream v2 definitions Omar Sandoval 2020-08-24 17:49 ` Josef Bacik 2020-08-21 7:39 ` [PATCH 6/9] btrfs: send: write larger chunks when using stream v2 Omar Sandoval 2020-08-24 17:57 ` Josef Bacik 2020-08-21 7:39 ` [PATCH 7/9] btrfs: send: allocate send buffer with alloc_page() and vmap() for v2 Omar Sandoval 2020-08-21 7:39 ` [PATCH 8/9] btrfs: send: send compressed extents with encoded writes Omar Sandoval 2020-08-24 17:32 ` Josef Bacik 2020-08-24 17:52 ` Omar Sandoval 2020-08-21 7:39 ` [PATCH 9/9] btrfs: send: enable support for stream v2 and compressed writes Omar Sandoval 2020-08-21 7:40 ` [PATCH 01/11] btrfs-progs: receive: support v2 send stream larger tlv_len Omar Sandoval 2020-08-21 7:40 ` [PATCH 02/11] btrfs-progs: receive: dynamically allocate sctx->read_buf Omar Sandoval 2020-08-21 7:40 ` [PATCH 03/11] btrfs-progs: receive: support v2 send stream DATA tlv format Omar Sandoval 2020-08-21 7:40 ` [PATCH 04/11] btrfs-progs: receive: add send stream v2 cmds and attrs to send.h Omar Sandoval 2020-08-21 7:40 ` [PATCH 05/11] btrfs-progs: receive: add stub implementation for pwritev2 Omar Sandoval 2020-08-21 7:40 ` [PATCH 06/11] btrfs-progs: receive: process encoded_write commands Omar Sandoval 2020-08-21 7:40 ` [PATCH 07/11] btrfs-progs: receive: encoded_write fallback to explicit decode and write Omar Sandoval 2020-08-21 7:40 ` [PATCH 08/11] btrfs-progs: receive: process fallocate commands Omar Sandoval 2020-08-21 7:40 ` [PATCH 09/11] btrfs-progs: receive: process setflags ioctl commands Omar Sandoval 2020-08-21 7:40 ` Omar Sandoval [this message] 2020-08-21 7:40 ` [PATCH 11/11] btrfs-progs: receive: add tests for basic encoded_write send/receive Omar Sandoval 2020-08-24 19:57 ` [PATCH 0/9] btrfs: implement send/receive of compressed extents without decompressing David Sterba 2020-08-24 22:16 ` Omar Sandoval 2020-09-10 11:28 ` David Sterba
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=fc4038fec127ca2f68ee448b1f5b7fdf3a4372f6.1597994354.git.osandov@osandov.com \ --to=osandov@osandov.com \ --cc=linux-btrfs@vger.kernel.org \ --cc=linux-fsdevel@vger.kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).