LKML Archive on lore.kernel.org
 help / color / Atom feed
From: Jean-Philippe Brucker <jean-philippe.brucker@arm.com>
To: Jacob Pan <jacob.jun.pan@linux.intel.com>
Cc: "iommu@lists.linux-foundation.org"
	<iommu@lists.linux-foundation.org>,
	LKML <linux-kernel@vger.kernel.org>,
	Joerg Roedel <joro@8bytes.org>,
	David Woodhouse <dwmw2@infradead.org>,
	Eric Auger <eric.auger@redhat.com>,
	Alex Williamson <alex.williamson@redhat.com>,
	"Tian, Kevin" <kevin.tian@intel.com>,
	Raj Ashok <ashok.raj@intel.com>,
	Andriy Shevchenko <andriy.shevchenko@linux.intel.com>
Subject: Re: [PATCH v3 09/16] iommu: Introduce guest PASID bind function
Date: Wed, 22 May 2019 16:05:53 +0100
Message-ID: <37d1eee7-92c1-7e07-b73d-7af82fdb1652@arm.com> (raw)
In-Reply-To: <20190521155029.0ab0a462@jacob-builder>

On 21/05/2019 23:50, Jacob Pan wrote:
>>> /**
>>>  * struct gpasid_bind_data - Information about device and guest
>>> PASID binding
>>>  * @version:	Version of this data structure
>>>  * @format:	PASID table entry format
>>>  * @flags:	Additional information on guest bind request
>>>  * @gpgd:	Guest page directory base of the guest mm to bind
>>>  * @hpasid:	Process address space ID used for the guest mm
>>> in host IOMMU
>>>  * @gpasid:	Process address space ID used for the guest mm
>>> in guest IOMMU  
>>
>> Trying to understand the full flow:
>> * @gpasid is the one allocated by the guest using a virtual command.
>> The guest writes @gpgd into the virtual PASID table at index @gpasid,
>> then sends an invalidate command to QEMU.
> yes
>> * QEMU issues a gpasid_bind ioctl (on the mdev or its container?).
>> VFIO forwards. The IOMMU driver installs @gpgd into the PASID table
>> using @hpasid, which is associated with the auxiliary domain.
>>
>> But why do we need the @hpasid field here? Does userspace know about
>> it at all, and does VFIO need to pass it to the IOMMU driver?
>>
> We need to support two guest-host PASID mappings through this API. Idea
> comes from Kevin & Yi.
> 1. identity mapping between host and guest PASID
> 2. guest owns its own pasid space
> 
> For option 1, which will plan to support first in this series. There is
> no need for gpasid field since gpasid=hpasid. Guest allocates PASID
> using virtual command interface which gets a host PASID. Then PASID
> cache invalidation in the guest will result in bind_gpasid(), @gpasid is
> not valid in the bind data (indicated by the IOMMU_SVA_GPASID_VAL flag).
> 
> For option 2, guest still uses virtual command to allocate guest pasid,
> but this time QEMU does the allocation for gpasid, at the same time
> QEMU will allocate a host pasid then maintain a G->H PASID lookup.
> When guest invalidate its PASID cache with GPASID, QEMU will find the
> match host PASID then pass both gpasid and hpasid down to the host IOMMU
> driver.
> Host IOMMU driver will store the gpgd at the hpasid entry but keep
> track of the gpasid->hpasid mapping. Host will never program gpasid in
> the IOMMU HW. Host IOMMU driver provides G->H PASID translation for PF
> device drivers that emulates mdev config space, i.e. virtual device
> composition module
> (https://events.linuxfoundation.org/wp-content/uploads/2017/12/Hardware-Assisted-Mediated-Pass-Through-with-VFIO-Kevin-Tian-Intel.pdf).
> 
> These two options is a per VM choice. Hopefully the two diagrams below
> can help to explain. I will put them in the next patch headers.

Thanks for the explanation, makes sense to me now. So the host kernel
needs to know G->H because the guest may write GPASID into the config
space emulated by the host device driver, and device driver then
retrieves the HPASID via an iommu_ops callback? But the device driver
keeps track of aux domains so isn't HPASID retrievable with
aux_get_pasid() already?

> 
> Option 1. Identity G-H PASID mapping diagram.
> 
>     .-------------.  .---------------------------.
>     |   vIOMMU    |  | Guest process mm, FL only |
>     |             |  '---------------------------'
>     .----------------/
>     | PASID Entry |--- PASID cache flush -
>     '-------------'\                      |
>     |             | \                     |
>     |             |  \                    |
>     '-------------'   \________________   |
>                         GPASID = HPASID   |
> Guest                  ^      ^           |
> ------| Shadow |-------| VCMD |-----------|------------
>       v        v       |      |           |
> QEMU                   v      v           |
> ------------------------------------------|------------
> Host             HPASID = ioasid_alloc()  |
>                     |                     v
>                     |       sva_bind_gpasid(HPASID)
>                     |
>     .-------------. |  .----------------------.
>     |   pIOMMU    | |  | Bind FL for GVA-GPA  |
>     |             | | /'----------------------'
>     .----------------'  |
>     | PASID Entry |     V (Nested xlate)
>     '----------------..---------------------.
>     |             |   |Set SL to GPA-HPA    |
>     |             |   '---------------------'
>     '-------------'
> 
> 
> 
> Option 2. Non-identity G-H PASID mapping diagram.
> 
>     .-------------.  .---------------------------.
>     |   vIOMMU    |  | Guest process mm, FL only |
>     |             |  '---------------------------'
>     .----------------/
>     | PASID Entry |--- PASID cache flush -
>     '-------------'\                      | .-------------.
>     |             | \                     | |Guest driver |
>     |             |  \                    | |writes GPASID|
>     '-------------'   \________________   | '-------------'
>                         GPASID            |             |
> Guest                  ^      ^           |             |
> ------| Shadow |-------| VCMD |-----------|------------ |
>       v        v       |      |           |             |
> QEMU                   v      v           |             |
> 	    GPASID = qemu_gpasid_alloc()  |             |
>             keep G->H PASID lookup        |             |
>                    ^                      v             |
> 		   |                 lookup G->H PASID  |
> -------------------|----------------------|------------ |
> Host             HPASID = ioasid_alloc()  |             |
>                     |                     v             |
>                     |     sva_bind_gpasid(HPASID,GPASID)|
>                     |     keep H-G PASID lookup         |
>                     |                          \  -------------------.
>     .-------------. |  .----------------------. \|    VDCM           |
>     |   pIOMMU    | |  | Bind FL for GVA-GPA  |  | H = lookup(GPASID)|
>     |             | | /'----------------------'  | write H to dev    |
>     .----------------'  |                         '------------------'
>     | PASID Entry |     V (Nested xlate)
>     '----------------..---------------------.
>     |             |   |Set SL to GPA-HPA    |
>     |             |   '---------------------'
>     '-------------'
> There is also implications in G-H pasid lookup for PRQ, that would be
> in the later series.
> 
>>>  * @addr_width:	Guest address width. Paging mode can also be
>>> derived.  
>>
>> What does the last sentence mean? @addr_width should probably be in
>> @vtd if it provides implicit information.
>>
> Derive 4 or 5 level paging mode from the address width. It can be in
> @vtd but i thought this can be generic.

Yes I think it's generic enough. It may be worth stating that this is
the *virtual* address width, and removing or clarifying what the paging
mode is (the sentence could be confusing on Arm, as we have different
page granules which cannot be derived from the address width)

Thanks,
Jean

  reply index

Thread overview: 52+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-05-03 22:32 [PATCH v3 00/16] Shared virtual address IOMMU and VT-d support Jacob Pan
2019-05-03 22:32 ` [PATCH v3 01/16] iommu: Introduce attach/detach_pasid_table API Jacob Pan
2019-05-03 22:32 ` [PATCH v3 02/16] iommu: Introduce cache_invalidate API Jacob Pan
2019-05-13  9:14   ` Auger Eric
2019-05-13 11:20     ` Jean-Philippe Brucker
2019-05-13 16:50       ` Auger Eric
2019-05-13 17:09         ` Jean-Philippe Brucker
2019-05-13 22:16           ` Jacob Pan
2019-05-14  7:36             ` Auger Eric
2019-05-14 10:41               ` Jean-Philippe Brucker
2019-05-14 17:44                 ` Jacob Pan
2019-05-14 17:57                   ` Jacob Pan
2019-05-15 11:03                   ` Jean-Philippe Brucker
2019-05-15 14:47                     ` Tian, Kevin
2019-05-15 15:25                       ` Jean-Philippe Brucker
2019-05-14  7:46           ` Auger Eric
2019-05-14 10:42             ` Jean-Philippe Brucker
2019-05-14 11:02               ` Auger Eric
2019-05-14 17:55                 ` Jacob Pan
2019-05-15 15:52                   ` Jean-Philippe Brucker
2019-05-15 16:25                     ` Jacob Pan
2019-05-03 22:32 ` [PATCH v3 03/16] iommu: Add I/O ASID allocator Jacob Pan
2019-05-21  8:21   ` Auger Eric
2019-05-21 17:03     ` Jacob Pan
2019-05-22 12:19       ` Jean-Philippe Brucker
2019-05-21  9:41   ` Auger Eric
2019-05-21 17:05     ` Jacob Pan
2019-05-03 22:32 ` [PATCH v3 04/16] ioasid: Add custom IOASID allocator Jacob Pan
2019-05-21  9:55   ` Auger Eric
2019-05-22 19:42     ` Jacob Pan
2019-05-23  7:14       ` Auger Eric
2019-05-23 15:40         ` Jacob Pan
2019-05-03 22:32 ` [PATCH v3 05/16] iommu/vt-d: Enlightened PASID allocation Jacob Pan
2019-05-03 22:32 ` [PATCH v3 06/16] iommu/vt-d: Add custom allocator for IOASID Jacob Pan
2019-05-03 22:32 ` [PATCH v3 07/16] iommu/vtd: Optimize tlb invalidation for vIOMMU Jacob Pan
2019-05-03 22:32 ` [PATCH v3 08/16] iommu/vt-d: Replace Intel specific PASID allocator with IOASID Jacob Pan
2019-05-03 22:32 ` [PATCH v3 09/16] iommu: Introduce guest PASID bind function Jacob Pan
2019-05-16 14:14   ` Jean-Philippe Brucker
2019-05-16 16:14     ` Jacob Pan
2019-05-20 19:22       ` Jacob Pan
2019-05-21 16:09         ` Jean-Philippe Brucker
2019-05-21 22:50           ` Jacob Pan
2019-05-22 15:05             ` Jean-Philippe Brucker [this message]
2019-05-22 17:15               ` Jacob Pan
2019-05-03 22:32 ` [PATCH v3 10/16] iommu/vt-d: Move domain helper to header Jacob Pan
2019-05-03 22:32 ` [PATCH v3 11/16] iommu/vt-d: Avoid duplicated code for PASID setup Jacob Pan
2019-05-03 22:32 ` [PATCH v3 12/16] iommu/vt-d: Add nested translation helper function Jacob Pan
2019-05-03 22:32 ` [PATCH v3 13/16] iommu/vt-d: Clean up for SVM device list Jacob Pan
2019-05-03 22:32 ` [PATCH v3 14/16] iommu/vt-d: Add bind guest PASID support Jacob Pan
2019-05-03 22:32 ` [PATCH v3 15/16] iommu/vt-d: Support flushing more translation cache types Jacob Pan
2019-05-03 22:32 ` [PATCH v3 16/16] iommu/vt-d: Add svm/sva invalidate function Jacob Pan
2019-05-15 16:31 ` [PATCH v3 00/16] Shared virtual address IOMMU and VT-d support Jacob Pan

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=37d1eee7-92c1-7e07-b73d-7af82fdb1652@arm.com \
    --to=jean-philippe.brucker@arm.com \
    --cc=alex.williamson@redhat.com \
    --cc=andriy.shevchenko@linux.intel.com \
    --cc=ashok.raj@intel.com \
    --cc=dwmw2@infradead.org \
    --cc=eric.auger@redhat.com \
    --cc=iommu@lists.linux-foundation.org \
    --cc=jacob.jun.pan@linux.intel.com \
    --cc=joro@8bytes.org \
    --cc=kevin.tian@intel.com \
    --cc=linux-kernel@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

LKML Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lkml.kernel.org/lkml/0 lkml/git/0.git
	git clone --mirror https://lkml.kernel.org/lkml/1 lkml/git/1.git
	git clone --mirror https://lkml.kernel.org/lkml/2 lkml/git/2.git
	git clone --mirror https://lkml.kernel.org/lkml/3 lkml/git/3.git
	git clone --mirror https://lkml.kernel.org/lkml/4 lkml/git/4.git
	git clone --mirror https://lkml.kernel.org/lkml/5 lkml/git/5.git
	git clone --mirror https://lkml.kernel.org/lkml/6 lkml/git/6.git
	git clone --mirror https://lkml.kernel.org/lkml/7 lkml/git/7.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 lkml lkml/ https://lkml.kernel.org/lkml \
		linux-kernel@vger.kernel.org
	public-inbox-index lkml

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-kernel


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git