LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Long Li <longli@microsoft.com>
To: Greg Kroah-Hartman <gregkh@linuxfoundation.org>
Cc: Bart Van Assche <bvanassche@acm.org>,
	"longli@linuxonhyperv.com" <longli@linuxonhyperv.com>,
	"linux-block@vger.kernel.org" <linux-block@vger.kernel.org>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"linux-hyperv@vger.kernel.org" <linux-hyperv@vger.kernel.org>,
	Jonathan Corbet <corbet@lwn.net>,
	KY Srinivasan <kys@microsoft.com>,
	Haiyang Zhang <haiyangz@microsoft.com>,
	Stephen Hemminger <sthemmin@microsoft.com>,
	Wei Liu <wei.liu@kernel.org>, Dexuan Cui <decui@microsoft.com>,
	Bjorn Andersson <bjorn.andersson@linaro.org>,
	Hans de Goede <hdegoede@redhat.com>,
	"Williams, Dan J" <dan.j.williams@intel.com>,
	Maximilian Luz <luzmaximilian@gmail.com>,
	Mike Rapoport <rppt@kernel.org>,
	Ben Widawsky <ben.widawsky@intel.com>,
	Jiri Slaby <jirislaby@kernel.org>,
	Andra Paraschiv <andraprs@amazon.com>,
	Siddharth Gupta <sidgup@codeaurora.org>,
	Hannes Reinecke <hare@suse.de>
Subject: RE: [Patch v5 0/3] Introduce a driver to support host accelerated access to Microsoft Azure Blob for Azure VM
Date: Sat, 7 Aug 2021 18:29:06 +0000	[thread overview]
Message-ID: <BY5PR21MB1506A93E865A8D6972DD0AAECEF49@BY5PR21MB1506.namprd21.prod.outlook.com> (raw)
In-Reply-To: <YQwvL2N6JpzI+hc8@kroah.com>

> Subject: Re: [Patch v5 0/3] Introduce a driver to support host accelerated
> access to Microsoft Azure Blob for Azure VM
> 
> On Thu, Aug 05, 2021 at 06:24:57PM +0000, Long Li wrote:
> > > Subject: Re: [Patch v5 0/3] Introduce a driver to support host
> > > accelerated access to Microsoft Azure Blob for Azure VM
> > >
> > > On 8/5/21 12:00 AM, longli@linuxonhyperv.com wrote:
> > > > From: Long Li <longli@microsoft.com>
> > > >
> > > > Azure Blob storage [1] is Microsoft's object storage solution for
> > > > the cloud. Users or client applications can access objects in Blob
> > > > storage via HTTP, from anywhere in the world. Objects in Blob
> > > > storage are accessible via the Azure Storage REST API, Azure
> > > > PowerShell, Azure CLI, or an Azure Storage client library. The
> > > > Blob storage interface is not designed to be a POSIX compliant
> interface.
> > > >
> > > > Problem: When a client accesses Blob storage via HTTP, it must go
> > > > through the Blob storage boundary of Azure and get to the storage
> > > > server through multiple servers. This is also true for an Azure VM.
> > > >
> > > > Solution: For an Azure VM, the Blob storage access can be
> > > > accelerated by having Azure host execute the Blob storage requests
> > > > to the backend storage server directly.
> > > >
> > > > This driver implements a VSC (Virtual Service Client) for
> > > > accelerating Blob storage access for an Azure VM by communicating
> > > > with a VSP (Virtual Service
> > > > Provider) on the Azure host. Instead of using HTTP to access the
> > > > Blob storage, an Azure VM passes the Blob storage request to the
> > > > VSP on the Azure host. The Azure host uses its native network to
> > > > perform Blob storage requests to the backend server directly.
> > > >
> > > > This driver doesn't implement Blob storage APIs. It acts as a fast
> > > > channel to pass user-mode Blob storage requests to the Azure host.
> > > > The user-mode program using this driver implements Blob storage
> > > > APIs and packages the Blob storage request as structured data to
> > > > VSC. The request data is modeled as three user provided buffers
> > > > (request, response and data buffers), that are patterned on the
> > > > HTTP model used by existing Azure Blob clients. The VSC passes
> > > > those buffers to VSP for Blob
> > > storage requests.
> > > >
> > > > The driver optimizes Blob storage access for an Azure VM in two ways:
> > > >
> > > > 1. The Blob storage requests are performed by the Azure host to
> > > > the Azure Blob backend storage server directly.
> > > >
> > > > 2. It allows the Azure host to use transport technologies (e.g.
> > > > RDMA) available to the Azure host but not available to the VM, to
> > > > reach to Azure Blob backend servers.
> > > >
> > > > Test results using this driver for an Azure VM:
> > > > 100 Blob clients running on an Azure VM, each reading 100GB Block
> Blobs.
> > > > (10 TB total read data)
> > > > With REST API over HTTP: 94.4 mins Using this driver: 72.5 mins
> > > > Performance (measured in throughput) gain: 30%.
> > > >
> > > > [1]
> > > >
> > >
> https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fdo
> > > cs
> > > > .microsoft.com%2Fen-us%2Fazure%2Fstorage%2Fblobs%2Fstorage-
> blobs-
> > > intro
> > > >
> > >
> duction&amp;data=04%7C01%7Clongli%40microsoft.com%7C6ba60a78f4e74
> > > aeb0b
> > > >
> > >
> b108d95833bf53%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C6376
> > > 378015
> > > >
> > >
> 92577579%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoi
> > > V2luMzIiL
> > > >
> > >
> CJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&amp;sdata=ab5Zl2cQdmUhdT3l
> > > SotDwMl
> > > > DQuE0JaY%2B1REPQ0%2FjXa4%3D&amp;reserved=0
> > >
> > > Is the ioctl interface the only user space interface provided by
> > > this kernel driver? If so, why has this code been implemented as a
> > > kernel driver instead of e.g. a user space library that uses vfio to
> > > interact with a PCIe device? As an example, Qemu supports many
> different virtio device types.
> >
> > The Hyper-V presents one such device for the whole VM. This device is
> > used by all processes on the VM. (The test benchmark used 100
> > processes)
> >
> > Hyper-V doesn't support creating one device for each process. We cannot
> use VFIO in this model.
> 
> I still think this "model" is totally broken and wrong overall.  Again, you are
> creating a custom "block" layer with a character device, forcing all userspace
> programs to use a custom library (where is it at?) just to get their data.

The Azure Blob library (with source code) is available in the following languages:
Java: https://github.com/Azure/azure-sdk-for-java/tree/main/sdk/storage/azure-storage-blob
JavaScript: https://github.com/Azure/azure-sdk-for-js/tree/main/sdk/storage/storage-blob
Python: https://github.com/Azure/azure-sdk-for-python/tree/main/sdk/storage/azure-storage-blob
Go: https://github.com/Azure/azure-storage-blob-go
.NET: https://github.com/Azure/azure-sdk-for-net/tree/main/sdk/storage/Azure.Storage.Blobs
PHP: https://github.com/Azure/azure-storage-php/tree/master/azure-storage-blob
Ruby: https://github.com/azure/azure-storage-ruby/tree/master/blob
C++: https://github.com/Azure/azure-sdk-for-cpp/tree/main/sdk/storage#azure-storage-client-library-for-c

> 
> There's a reason the POSIX model is there, why are you all ignoring it?

The Azure Blob APIs are not designed to be POSIX compatible. This driver is used
to accelerate Blob access for a Blob client running in an Azure VM. It doesn't attempt
to modify the Blob APIs. Changing the Blob APIs will break the existing Blob clients.

Thanks,
Long

  reply	other threads:[~2021-08-07 18:29 UTC|newest]

Thread overview: 35+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-08-05  7:00 longli
2021-08-05  7:00 ` [Patch v5 1/3] Drivers: hv: vmbus: add support to ignore certain PCIE devices longli
2021-08-05  7:00 ` [Patch v5 2/3] Drivers: hv: add Azure Blob driver longli
2021-08-05  7:11   ` Greg Kroah-Hartman
2021-08-05 18:07     ` Long Li
2021-08-05 18:16       ` Greg Kroah-Hartman
2021-08-05 17:06   ` Bart Van Assche
2021-08-05 18:10     ` Long Li
2021-08-05 18:17     ` Greg Kroah-Hartman
2021-09-07 21:42   ` Michael Kelley
2021-08-05  7:00 ` [Patch v5 3/3] Drivers: hv: Add to maintainer for Hyper-V/Azure drivers longli
2021-08-05  7:08 ` [Patch v5 0/3] Introduce a driver to support host accelerated access to Microsoft Azure Blob for Azure VM Greg Kroah-Hartman
2021-08-05 18:27   ` Long Li
2021-08-05 18:33     ` Greg Kroah-Hartman
2021-08-05 17:09 ` Bart Van Assche
2021-08-05 18:24   ` Long Li
2021-08-05 18:34     ` Greg Kroah-Hartman
2021-08-07 18:29       ` Long Li [this message]
2021-08-08  5:14         ` Greg Kroah-Hartman
2021-08-10  3:01           ` Long Li
2021-09-22 23:55             ` Long Li
2021-09-30 22:25               ` Long Li
2021-10-01  7:36                 ` Greg Kroah-Hartman
2021-10-07 18:15                   ` Long Li
2021-10-08  5:54                     ` Greg Kroah-Hartman
2021-10-08 11:11                       ` Vitaly Kuznetsov
2021-10-08 11:19                         ` Greg Kroah-Hartman
2021-10-08 13:28                           ` Vitaly Kuznetsov
2021-10-11 17:57                             ` Long Li
2021-10-13  0:58                               ` Long Li
2021-10-13  7:03                                 ` Greg Kroah-Hartman
2021-10-11 17:55                           ` Long Li
2021-10-11 17:46                         ` Long Li
2021-10-11 17:58                           ` Greg Kroah-Hartman
2021-10-11 19:38                             ` Long Li

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=BY5PR21MB1506A93E865A8D6972DD0AAECEF49@BY5PR21MB1506.namprd21.prod.outlook.com \
    --to=longli@microsoft.com \
    --cc=andraprs@amazon.com \
    --cc=ben.widawsky@intel.com \
    --cc=bjorn.andersson@linaro.org \
    --cc=bvanassche@acm.org \
    --cc=corbet@lwn.net \
    --cc=dan.j.williams@intel.com \
    --cc=decui@microsoft.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=haiyangz@microsoft.com \
    --cc=hare@suse.de \
    --cc=hdegoede@redhat.com \
    --cc=jirislaby@kernel.org \
    --cc=kys@microsoft.com \
    --cc=linux-block@vger.kernel.org \
    --cc=linux-hyperv@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=longli@linuxonhyperv.com \
    --cc=luzmaximilian@gmail.com \
    --cc=rppt@kernel.org \
    --cc=sidgup@codeaurora.org \
    --cc=sthemmin@microsoft.com \
    --cc=wei.liu@kernel.org \
    --subject='RE: [Patch v5 0/3] Introduce a driver to support host accelerated access to Microsoft Azure Blob for Azure VM' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).