Linux-Fsdevel Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Omar Sandoval <osandov@osandov.com>
To: linux-btrfs@vger.kernel.org
Cc: linux-fsdevel@vger.kernel.org
Subject: [PATCH 10/11] btrfs-progs: send: stream v2 ioctl flags
Date: Fri, 21 Aug 2020 00:40:09 -0700 [thread overview]
Message-ID: <fc4038fec127ca2f68ee448b1f5b7fdf3a4372f6.1597994354.git.osandov@osandov.com> (raw)
In-Reply-To: <cover.1597994106.git.osandov@osandov.com>
From: Boris Burkov <boris@bur.io>
To make the btrfs send ioctl use the stream v2 format requires passing
BTRFS_SEND_FLAG_STREAM_V2 in flags. Further, to cause the ioctl to emit
encoded_write commands for encoded extents, we must set that flag as
well as BTRFS_SEND_FLAG_COMPRESSED. Finally, we bump up the version in
send.h as well, since we are now fully compatible with v2.
Add two command line arguments to btrfs send: --stream-version and
--compressed. --stream-version requires an argument which it parses as
an integer and sets STREAM_V2 if the argument is 2. --compressed does
not require an argument and automatically implies STREAM_V2 as well
(COMPRESSED alone causes the ioctl to error out).
Some examples to illustrate edge cases:
// v1, old format and no encoded_writes
btrfs send subvol
btrfs send --stream-version 1 subvol
// v2 and compressed, we will see encoded_writes
btrfs send --compressed subvol
btrfs send --compressed --stream-version 2 subvol
// v2 only, new format but no encoded_writes
btrfs send --stream-version 2 subvol
// error: compressed needs version >= 2
btrfs send --compressed --stream-version 1 subvol
// error: invalid version (not 1 or 2)
btrfs send --stream-version 3 subvol
btrfs send --compressed --stream-version 0 subvol
btrfs send --compressed --stream-version 10 subvol
Signed-off-by: Boris Burkov <boris@bur.io>
---
cmds/send.c | 39 +++++++++++++++++++++++++++++++++++++--
ioctl.h | 17 ++++++++++++++++-
libbtrfsutil/btrfs.h | 17 ++++++++++++++++-
send.h | 2 +-
4 files changed, 70 insertions(+), 5 deletions(-)
diff --git a/cmds/send.c b/cmds/send.c
index b8e3ba12..4c4eaa84 100644
--- a/cmds/send.c
+++ b/cmds/send.c
@@ -474,6 +474,7 @@ static int cmd_send(const struct cmd_struct *cmd, int argc, char **argv)
int full_send = 1;
int new_end_cmd_semantic = 0;
u64 send_flags = 0;
+ long stream_version = 0;
memset(&send, 0, sizeof(send));
send.dump_fd = fileno(stdout);
@@ -492,11 +493,17 @@ static int cmd_send(const struct cmd_struct *cmd, int argc, char **argv)
optind = 0;
while (1) {
- enum { GETOPT_VAL_SEND_NO_DATA = 256 };
+ enum {
+ GETOPT_VAL_SEND_NO_DATA = 256,
+ GETOPT_VAL_SEND_STREAM_V2,
+ GETOPT_VAL_SEND_COMPRESSED
+ };
static const struct option long_options[] = {
{ "verbose", no_argument, NULL, 'v' },
{ "quiet", no_argument, NULL, 'q' },
- { "no-data", no_argument, NULL, GETOPT_VAL_SEND_NO_DATA }
+ { "no-data", no_argument, NULL, GETOPT_VAL_SEND_NO_DATA },
+ { "stream-version", required_argument, NULL, GETOPT_VAL_SEND_STREAM_V2 },
+ { "compressed", no_argument, NULL, GETOPT_VAL_SEND_COMPRESSED }
};
int c = getopt_long(argc, argv, "vqec:f:i:p:", long_options, NULL);
@@ -584,10 +591,38 @@ static int cmd_send(const struct cmd_struct *cmd, int argc, char **argv)
case GETOPT_VAL_SEND_NO_DATA:
send_flags |= BTRFS_SEND_FLAG_NO_FILE_DATA;
break;
+ case GETOPT_VAL_SEND_STREAM_V2:
+ stream_version = strtol(optarg, NULL, 10);
+ if (stream_version < 1 || stream_version > 2) {
+ ret = 1;
+ error("invalid --stream-version. valid values: {1, 2}");
+ goto out;
+ }
+ if (stream_version == 2)
+ send_flags |= BTRFS_SEND_FLAG_STREAM_V2;
+ break;
+ case GETOPT_VAL_SEND_COMPRESSED:
+ send_flags |= BTRFS_SEND_FLAG_COMPRESSED;
+ /*
+ * We want to default to stream v2 if only compressed is
+ * set. If stream_version is explicitly set to 0, that
+ * will trigger its own error condition for being an
+ * invalid version.
+ */
+ if (stream_version == 0) {
+ stream_version = 2;
+ send_flags |= BTRFS_SEND_FLAG_STREAM_V2;
+ }
+ break;
default:
usage_unknown_option(cmd, argv);
}
}
+ if (stream_version < 2 && (send_flags & BTRFS_SEND_FLAG_COMPRESSED)) {
+ ret = 1;
+ error("--compressed requires --stream-version >= 2");
+ goto out;
+ }
if (check_argc_min(argc - optind, 1))
return 1;
diff --git a/ioctl.h b/ioctl.h
index ade6dcb9..46de8ac8 100644
--- a/ioctl.h
+++ b/ioctl.h
@@ -653,10 +653,25 @@ BUILD_ASSERT(sizeof(struct btrfs_ioctl_received_subvol_args_32) == 192);
*/
#define BTRFS_SEND_FLAG_OMIT_END_CMD 0x4
+/*
+ * Use version 2 of the send stream, which adds new commands and supports larger
+ * writes.
+ */
+#define BTRFS_SEND_FLAG_STREAM_V2 0x8
+
+/*
+ * Send compressed data using the ENCODED_WRITE command instead of decompressing
+ * the data and sending it with the WRITE command. This requires
+ * BTRFS_SEND_FLAG_STREAM_V2.
+ */
+#define BTRFS_SEND_FLAG_COMPRESSED 0x10
+
#define BTRFS_SEND_FLAG_MASK \
(BTRFS_SEND_FLAG_NO_FILE_DATA | \
BTRFS_SEND_FLAG_OMIT_STREAM_HEADER | \
- BTRFS_SEND_FLAG_OMIT_END_CMD)
+ BTRFS_SEND_FLAG_OMIT_END_CMD | \
+ BTRFS_SEND_FLAG_STREAM_V2 | \
+ BTRFS_SEND_FLAG_COMPRESSED)
struct btrfs_ioctl_send_args {
__s64 send_fd; /* in */
diff --git a/libbtrfsutil/btrfs.h b/libbtrfsutil/btrfs.h
index 60d51ff6..8430a40d 100644
--- a/libbtrfsutil/btrfs.h
+++ b/libbtrfsutil/btrfs.h
@@ -731,10 +731,25 @@ struct btrfs_ioctl_received_subvol_args {
*/
#define BTRFS_SEND_FLAG_OMIT_END_CMD 0x4
+/*
+ * Use version 2 of the send stream, which adds new commands and supports larger
+ * writes.
+ */
+#define BTRFS_SEND_FLAG_STREAM_V2 0x8
+
+/*
+ * Send compressed data using the ENCODED_WRITE command instead of decompressing
+ * the data and sending it with the WRITE command. This requires
+ * BTRFS_SEND_FLAG_STREAM_V2.
+ */
+#define BTRFS_SEND_FLAG_COMPRESSED 0x10
+
#define BTRFS_SEND_FLAG_MASK \
(BTRFS_SEND_FLAG_NO_FILE_DATA | \
BTRFS_SEND_FLAG_OMIT_STREAM_HEADER | \
- BTRFS_SEND_FLAG_OMIT_END_CMD)
+ BTRFS_SEND_FLAG_OMIT_END_CMD | \
+ BTRFS_SEND_FLAG_STREAM_V2 | \
+ BTRFS_SEND_FLAG_COMPRESSED)
struct btrfs_ioctl_send_args {
__s64 send_fd; /* in */
diff --git a/send.h b/send.h
index 3c47e0c7..fac90588 100644
--- a/send.h
+++ b/send.h
@@ -31,7 +31,7 @@ extern "C" {
#endif
#define BTRFS_SEND_STREAM_MAGIC "btrfs-stream"
-#define BTRFS_SEND_STREAM_VERSION 1
+#define BTRFS_SEND_STREAM_VERSION 2
#define BTRFS_SEND_BUF_SIZE_V1 SZ_64K
#define BTRFS_SEND_READ_SIZE (1024 * 48)
--
2.28.0
next prev parent reply other threads:[~2020-08-21 7:41 UTC|newest]
Thread overview: 37+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-08-21 7:39 [PATCH 0/9] btrfs: implement send/receive of compressed extents without decompressing Omar Sandoval
2020-08-21 7:39 ` [PATCH 1/9] btrfs: send: get rid of i_size logic in send_write() Omar Sandoval
2020-08-21 17:26 ` Filipe Manana
2020-08-24 17:39 ` Josef Bacik
2020-08-21 7:39 ` [PATCH 2/9] btrfs: send: avoid copying file data Omar Sandoval
2020-08-21 17:29 ` Filipe Manana
2020-08-24 21:34 ` Omar Sandoval
2020-08-24 17:47 ` Josef Bacik
2020-09-11 14:13 ` David Sterba
2020-09-14 22:04 ` Omar Sandoval
2020-09-15 8:14 ` David Sterba
2020-08-21 7:39 ` [PATCH 3/9] btrfs: send: use btrfs_file_extent_end() in send_write_or_clone() Omar Sandoval
2020-08-21 17:30 ` Filipe Manana
2020-08-21 7:39 ` [PATCH 4/9] btrfs: add send_stream_version attribute to sysfs Omar Sandoval
2020-08-21 7:39 ` [PATCH 5/9] btrfs: add send stream v2 definitions Omar Sandoval
2020-08-24 17:49 ` Josef Bacik
2020-08-21 7:39 ` [PATCH 6/9] btrfs: send: write larger chunks when using stream v2 Omar Sandoval
2020-08-24 17:57 ` Josef Bacik
2020-08-21 7:39 ` [PATCH 7/9] btrfs: send: allocate send buffer with alloc_page() and vmap() for v2 Omar Sandoval
2020-08-21 7:39 ` [PATCH 8/9] btrfs: send: send compressed extents with encoded writes Omar Sandoval
2020-08-24 17:32 ` Josef Bacik
2020-08-24 17:52 ` Omar Sandoval
2020-08-21 7:39 ` [PATCH 9/9] btrfs: send: enable support for stream v2 and compressed writes Omar Sandoval
2020-08-21 7:40 ` [PATCH 01/11] btrfs-progs: receive: support v2 send stream larger tlv_len Omar Sandoval
2020-08-21 7:40 ` [PATCH 02/11] btrfs-progs: receive: dynamically allocate sctx->read_buf Omar Sandoval
2020-08-21 7:40 ` [PATCH 03/11] btrfs-progs: receive: support v2 send stream DATA tlv format Omar Sandoval
2020-08-21 7:40 ` [PATCH 04/11] btrfs-progs: receive: add send stream v2 cmds and attrs to send.h Omar Sandoval
2020-08-21 7:40 ` [PATCH 05/11] btrfs-progs: receive: add stub implementation for pwritev2 Omar Sandoval
2020-08-21 7:40 ` [PATCH 06/11] btrfs-progs: receive: process encoded_write commands Omar Sandoval
2020-08-21 7:40 ` [PATCH 07/11] btrfs-progs: receive: encoded_write fallback to explicit decode and write Omar Sandoval
2020-08-21 7:40 ` [PATCH 08/11] btrfs-progs: receive: process fallocate commands Omar Sandoval
2020-08-21 7:40 ` [PATCH 09/11] btrfs-progs: receive: process setflags ioctl commands Omar Sandoval
2020-08-21 7:40 ` Omar Sandoval [this message]
2020-08-21 7:40 ` [PATCH 11/11] btrfs-progs: receive: add tests for basic encoded_write send/receive Omar Sandoval
2020-08-24 19:57 ` [PATCH 0/9] btrfs: implement send/receive of compressed extents without decompressing David Sterba
2020-08-24 22:16 ` Omar Sandoval
2020-09-10 11:28 ` David Sterba
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=fc4038fec127ca2f68ee448b1f5b7fdf3a4372f6.1597994354.git.osandov@osandov.com \
--to=osandov@osandov.com \
--cc=linux-btrfs@vger.kernel.org \
--cc=linux-fsdevel@vger.kernel.org \
--subject='Re: [PATCH 10/11] btrfs-progs: send: stream v2 ioctl flags' \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).