LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: "Paraschiv, Andra-Irina" <andraprs@amazon.com>
To: Stefano Garzarella <sgarzare@redhat.com>,
	"Wang, Wei W" <wei.w.wang@intel.com>,
	Stefan Hajnoczi <stefanha@redhat.com>
Cc: "davem@davemloft.net" <davem@davemloft.net>,
	"kuba@kernel.org" <kuba@kernel.org>,
	"mst@redhat.com" <mst@redhat.com>,
	Paolo Bonzini <pbonzini@redhat.com>,
	"kys@microsoft.com" <kys@microsoft.com>,
	"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
	"virtualization@lists.linux-foundation.org" 
	<virtualization@lists.linux-foundation.org>,
	"Yamahata, Isaku" <isaku.yamahata@intel.com>,
	"Nakajima, Jun" <jun.nakajima@intel.com>,
	"Kleen, Andi" <andi.kleen@intel.com>,
	Sergio Lopez Pascual <slp@redhat.com>
Subject: Re: [RFC] hypercall-vsock: add a new vsock transport
Date: Wed, 10 Nov 2021 23:45:54 +0200	[thread overview]
Message-ID: <4617b5db-cca4-fe00-4480-d72cf107ca76@amazon.com> (raw)
In-Reply-To: <20211110111718.5cvt6vgory3fzqld@steredhat>



On 10/11/2021 13:17, Stefano Garzarella wrote:
> 
> On Wed, Nov 10, 2021 at 07:12:36AM +0000, Wang, Wei W wrote:
>> Hi,
>>
>> We plan to add a new vsock transport based on hypercall (e.g. vmcall 
>> on Intel CPUs).
>> It transports AF_VSOCK packets between the guest and host, which is 
>> similar to
>> virtio-vsock, vmci-vsock and hyperv-vsock.
>>
>> Compared to the above listed vsock transports which are designed for 
>> high performance,
>> the main advantages of hypercall-vsock are:
>>
>> 1)       It is VMM agnostic. For example, one guest working on 
>> hypercall-vsock can run on
>>
>> either KVM, Hyperv, or VMware.
>>
>> 2)       It is simpler. It doesn't rely on any complex bus enumeration
>>
>> (e.g. virtio-pci based vsock device may need the whole implementation 
>> of PCI).
>>
>> An example usage is the communication between MigTD and host (Page 8 at
>> https://static.sched.com/hosted_files/kvmforum2021/ef/TDX%20Live%20Migration_Wei%20Wang.pdf). 
>>
>> MigTD communicates to host to assist the migration of the target (user)
>> TD.
>> MigTD is part of the TCB, so its implementation is expected to be as 
>> simple as possible
>> (e.g. bare mental implementation without OS, no PCI driver support).

Thanks for CC. Mixing both threads.

 From Stefan:

"
AF_VSOCK is designed to allow multiple transports, so why not. There is
a cost to developing and maintaining a vsock transport though.

I think Amazon Nitro enclaves use virtio-vsock and I've CCed Andra in
case she has thoughts on the pros/cons and how to minimize the trusted
computing base.

If simplicity is the top priority then VIRTIO's MMIO transport without
indirect descriptors and using the packed virtqueue layout reduces the
size of the implementation:
https://docs.oasis-open.org/virtio/virtio/v1.1/cs01/virtio-v1.1-cs01.html#x1-1440002

Stefan
"


On the Nitro Enclaves project side, virtio-mmio is used for the vsock 
device setup for the enclave. That has worked fine, it has helped to 
have an already available implementation (e.g. virtio-mmio / virtio-pci) 
for adoption and ease of use in different types of setups (e.g. distros, 
kernel versions).

 From Stefano:

> 
> Adding Andra and Sergio, because IIRC Firecracker and libkrun emulates
> virtio-vsock with virtio-mmio so the implementation should be simple and
> also not directly tied to a specific VMM.
> 
> Maybe this fit for your use case too, in this way we don't have to
> maintain another driver.
> 
> Thanks,
> Stefano
> 

Indeed, on the Firecracker side, the vsock device is setup using 
virtio-mmio [1][2][3]. One specific thing is that on the host, instead 
of using vhost, AF_UNIX sockets are used [4].

Thanks,
Andra

[1] 
https://github.com/firecracker-microvm/firecracker/blob/main/src/devices/src/virtio/vsock/mod.rs#L30
[2] 
https://github.com/firecracker-microvm/firecracker/blob/main/src/vmm/src/builder.rs#L936
[3] 
https://github.com/firecracker-microvm/firecracker/blob/main/src/vmm/src/builder.rs#L859
[4] 
https://github.com/firecracker-microvm/firecracker/blob/main/docs/vsock.md



Amazon Development Center (Romania) S.R.L. registered office: 27A Sf. Lazar Street, UBC5, floor 2, Iasi, Iasi County, 700045, Romania. Registered in Romania. Registration number J22/2621/2005.

  reply	other threads:[~2021-11-10 21:46 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <71d7b0463629471e9d4887d7fcef1d8d@intel.com>
2021-11-10  9:34 ` Stefan Hajnoczi
2021-11-11  8:02   ` Wang, Wei W
2021-11-10 10:50 ` Michael S. Tsirkin
2021-11-11  7:58   ` Wang, Wei W
2021-11-11 15:19     ` Michael S. Tsirkin
2021-11-25  6:37     ` Jason Wang
2021-11-25  8:43       ` Wang, Wei W
2021-11-25 12:04         ` Gerd Hoffmann
2021-11-10 11:17 ` Stefano Garzarella
2021-11-10 21:45   ` Paraschiv, Andra-Irina [this message]
2021-11-11  8:14   ` Wang, Wei W
2021-11-11  8:24     ` Paolo Bonzini

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=4617b5db-cca4-fe00-4480-d72cf107ca76@amazon.com \
    --to=andraprs@amazon.com \
    --cc=andi.kleen@intel.com \
    --cc=davem@davemloft.net \
    --cc=isaku.yamahata@intel.com \
    --cc=jun.nakajima@intel.com \
    --cc=kuba@kernel.org \
    --cc=kys@microsoft.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mst@redhat.com \
    --cc=pbonzini@redhat.com \
    --cc=sgarzare@redhat.com \
    --cc=slp@redhat.com \
    --cc=stefanha@redhat.com \
    --cc=virtualization@lists.linux-foundation.org \
    --cc=wei.w.wang@intel.com \
    --subject='Re: [RFC] hypercall-vsock: add a new vsock transport' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).