LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: John Garry <john.garry@huawei.com>
To: <joro@8bytes.org>, <will@kernel.org>, <robin.murphy@arm.com>,
	<baolu.lu@linux.intel.com>
Cc: <iommu@lists.linux-foundation.org>, <linuxarm@huawei.com>,
	<thierry.reding@gmail.com>, <airlied@linux.ie>, <daniel@ffwll.ch>,
	<jonathanh@nvidia.com>, <sakari.ailus@linux.intel.com>,
	<bingbu.cao@intel.com>, <tian.shu.qiu@intel.com>,
	<mchehab@kernel.org>, <gregkh@linuxfoundation.org>,
	<digetx@gmail.com>, <mst@redhat.com>, <jasowang@redhat.com>,
	<linux-kernel@vger.kernel.org>, <chenxiang66@hisilicon.com>,
	John Garry <john.garry@huawei.com>
Subject: [PATCH v4 0/6] iommu: Allow IOVA rcache range be configured
Date: Wed, 14 Jul 2021 18:36:37 +0800	[thread overview]
Message-ID: <1626259003-201303-1-git-send-email-john.garry@huawei.com> (raw)

For streaming DMA mappings involving an IOMMU and whose IOVA len regularly
exceeds the IOVA rcache upper limit (meaning that they are not cached),
performance can be reduced. 

This may be much more pronounced from commit 4e89dce72521 ("iommu/iova:
Retry from last rb tree node if iova search fails"), as discussed at [0].

IOVAs which cannot be cached are highly involved in the IOVA ageing issue,
as discussed at [1].

This series allows the IOVA rcache range be configured, so that we may
cache all IOVAs per domain, thus improving performance.

A new IOMMU group sysfs file is added - max_opt_dma_size - which is used
indirectly to configure the IOVA rcache range:
/sys/kernel/iommu_groups/X/max_opt_dma_size

This file is updated same as how the IOMMU group default domain type is
updated, i.e. must unbind the only device in the group first.

The inspiration here comes from block layer request queue sysfs
"optimal_io_size" file, in /sys/block/sdX/queue/optimal_io_size

Some figures for storage scenario (when increasing IOVA rcache range to
cover all DMA mapping sizes from the LLD):
v5.13-rc1 baseline:			1200K IOPS
With series:				1800K IOPS

All above are for IOMMU strict mode. Non-strict mode gives ~1800K IOPS in
all scenarios.

[0] https://lore.kernel.org/linux-iommu/20210129092120.1482-1-thunder.leizhen@huawei.com/
[1] https://lore.kernel.org/linux-iommu/1607538189-237944-1-git-send-email-john.garry@huawei.com/

Note that I cc'ed maintainers/reviewers only for the changes associated
with patch #5 since it just touches their code in only a minor way.

John Garry (6):
  iommu: Refactor iommu_group_store_type()
  iova: Allow rcache range upper limit to be flexible
  iommu: Allow iommu_change_dev_def_domain() realloc default domain for
    same type
  iommu: Allow max opt DMA len be set for a group via sysfs
  iova: Add iova_len argument to init_iova_domain()
  dma-iommu: Pass iova len for IOVA domain init

 .../ABI/testing/sysfs-kernel-iommu_groups     |  16 ++
 drivers/gpu/drm/tegra/drm.c                   |   2 +-
 drivers/gpu/host1x/dev.c                      |   2 +-
 drivers/iommu/dma-iommu.c                     |  15 +-
 drivers/iommu/iommu.c                         | 172 ++++++++++++------
 drivers/iommu/iova.c                          |  39 +++-
 drivers/staging/media/ipu3/ipu3-dmamap.c      |   2 +-
 drivers/staging/media/tegra-vde/iommu.c       |   2 +-
 drivers/vdpa/vdpa_sim/vdpa_sim.c              |   2 +-
 include/linux/iommu.h                         |   6 +
 include/linux/iova.h                          |   9 +-
 11 files changed, 194 insertions(+), 73 deletions(-)

-- 
2.26.2


             reply	other threads:[~2021-07-14 10:41 UTC|newest]

Thread overview: 17+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-07-14 10:36 John Garry [this message]
2021-07-14 10:36 ` [PATCH v4 1/6] iommu: Refactor iommu_group_store_type() John Garry
2021-08-02 14:46   ` Will Deacon
2021-07-14 10:36 ` [PATCH v4 2/6] iova: Allow rcache range upper limit to be flexible John Garry
2021-08-02 15:01   ` Will Deacon
2021-08-02 15:23     ` John Garry
2021-08-02 16:09       ` Robin Murphy
2021-07-14 10:36 ` [PATCH v4 3/6] iommu: Allow iommu_change_dev_def_domain() realloc default domain for same type John Garry
2021-07-14 10:36 ` [PATCH v4 4/6] iommu: Allow max opt DMA len be set for a group via sysfs John Garry
2021-07-14 10:36 ` [PATCH v4 5/6] iova: Add iova_len argument to init_iova_domain() John Garry
2021-08-02 15:06   ` Will Deacon
2021-08-02 16:06     ` John Garry
2021-08-02 16:16       ` Robin Murphy
2021-08-02 16:40         ` John Garry
2021-08-02 17:18           ` John Garry
2021-09-21  8:48         ` John Garry
2021-07-14 10:36 ` [PATCH v4 6/6] dma-iommu: Pass iova len for IOVA domain init John Garry

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1626259003-201303-1-git-send-email-john.garry@huawei.com \
    --to=john.garry@huawei.com \
    --cc=airlied@linux.ie \
    --cc=baolu.lu@linux.intel.com \
    --cc=bingbu.cao@intel.com \
    --cc=chenxiang66@hisilicon.com \
    --cc=daniel@ffwll.ch \
    --cc=digetx@gmail.com \
    --cc=gregkh@linuxfoundation.org \
    --cc=iommu@lists.linux-foundation.org \
    --cc=jasowang@redhat.com \
    --cc=jonathanh@nvidia.com \
    --cc=joro@8bytes.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linuxarm@huawei.com \
    --cc=mchehab@kernel.org \
    --cc=mst@redhat.com \
    --cc=robin.murphy@arm.com \
    --cc=sakari.ailus@linux.intel.com \
    --cc=thierry.reding@gmail.com \
    --cc=tian.shu.qiu@intel.com \
    --cc=will@kernel.org \
    --subject='Re: [PATCH v4 0/6] iommu: Allow IOVA rcache range be configured' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).