LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Atish Patra <atish.patra@wdc.com>
To: linux-kernel@vger.kernel.org
Cc: Atish Patra <atish.patra@wdc.com>,
	Alexander Shishkin <alexander.shishkin@linux.intel.com>,
	Anup Patel <anup.patel@wdc.com>, Ard Biesheuvel <ardb@kernel.org>,
	"Darrick J. Wong" <djwong@kernel.org>,
	devicetree@vger.kernel.org, Guo Ren <guoren@linux.alibaba.com>,
	Heinrich Schuchardt <xypron.glpk@gmx.de>,
	Jiri Olsa <jolsa@redhat.com>, John Garry <john.garry@huawei.com>,
	Jonathan Corbet <corbet@lwn.net>,
	linux-doc@vger.kernel.org, linux-perf-users@vger.kernel.org,
	linux-riscv@lists.infradead.org,
	Nick Kossifidis <mick@ics.forth.gr>,
	Palmer Dabbelt <palmer@dabbelt.com>,
	Paul Walmsley <paul.walmsley@sifive.com>,
	Rob Herring <robh+dt@kernel.org>,
	Vincent Chen <vincent.chen@sifive.com>
Subject: [v3 00/10] Improve RISC-V Perf support using SBI PMU and sscofpmf extension
Date: Fri, 10 Sep 2021 12:27:47 -0700	[thread overview]
Message-ID: <20210910192757.2309100-1-atish.patra@wdc.com> (raw)

This series adds improved perf support for RISC-V based system using
SBI PMU extension[1] and Sscofpmf extension[2]. The SBI PMU extension allows
the kernel to program the counters for different events and start/stop counters
while the sscofpmf extension allows the counter overflow interrupt and privilege
mode filtering. An hardware platform can leverage SBI PMU extension without
the sscofpmf extension if it supports mcountinhibit and mcounteren. However,
the reverse is not true. With both of these extension enabled, a platform can
take advantage of all both event counting and sampling using perf tool. 

This series introduces a platform perf driver instead of a existing arch
specific implementation. The new perf implementation has adopted a modular
approach where most of the generic event handling is done in the core library
while individual PMUs need to only implement necessary features specific to
the PMU. This is easily extensible and any future RISC-V PMU implementation
can leverage this. Currently, SBI PMU driver & legacy PMU driver are implemented
as a part of this series.

The legacy driver tries to reimplement the existing minimal perf under a new
config to maintain backward compatibility. This implementation only allows
monitoring of always running cycle/instruction counters. Moreover, they can
not be started or stopped. In general, this is very limited and not very useful.
That's why, I am not very keen to carry the support into the new driver.
However, I don't want to break perf for any existing hardware platforms.
If nobody really uses perf currently, I will be happy to drop PATCH 4.

This series has been tested in Qemu on both RV64 & RV32. Qemu[5] & OpenSBI [3]
patches are required to test it. Qemu changes are not backward compatible.
That means, you can not use perf anymore on older Qemu versions with latest
OpenSBI and/or Kernel. However, newer kernel will just use legacy pmu driver if
old OpenSBI is detected or hardware doesn't implement mcountinhibit.

Here is an output of perf stat/report while running hackbench with OpenSBI & Linux
kernel patches applied [3].

Perf stat:
=========

[root@fedora-riscv riscv]# perf stat -e r8000000000000005 -e r8000000000000007
-e r8000000000000006 -e r0000000000020002 -e r0000000000020004 -e branch-misses
-e cache-misses -e dTLB-load-misses -e dTLB-store-misses -e iTLB-load-misses
-e cycles -e instructions ./hackbench -pipe 15 process
Running with 15*40 (== 600) tasks.
Time: 6.578

 Performance counter stats for './hackbench -pipe 15 process':

             6,491      r8000000000000005      (52.59%) --> SBI_PMU_FW_SET_TIMER
            20,433      r8000000000000007      (60.74%) --> SBI_PMU_FW_IPI_RECVD
            21,271      r8000000000000006      (68.71%) --> SBI_PMU_FW_IPI_SENT
                 0      r0000000000020002      (76.55%)
     <not counted>      r0000000000020004      (0.00%)
     <not counted>      branch-misses          (0.00%)
     <not counted>      cache-misses           (0.00%)
        57,537,853      dTLB-load-misses       (9.49%)
         2,821,147      dTLB-store-misses      (18.64%)
        52,928,130      iTLB-load-misses       (27.53%)
    89,521,791,110      cycles                 (36.08%)
    90,678,132,464      instructions #    1.01  insn per cycle (44.44%)

       6.975908032 seconds time elapsed

       3.130950000 seconds user
      24.353310000 seconds sys

The patches can also be found in the github[4].

Perf record:
============
[root@fedora-riscv riscv]# perf record -e cycles -e instructions -e \
dTLB-load-misses -e dTLB-store-misses -c 1000 ./hackbench -pipe 15 process 15
Running with 15*40 (== 600) tasks.
Time: 1.238
[ perf record: Woken up 1 times to write data ]
[ perf record: Captured and wrote 0.106 MB perf.data (1020 samples) ]

[root@fedora-riscv riscv]# perf report  
Available samples                                                               
372 cycles                                                                     ◆
372 instructions                                                               ▒
262 dTLB-load-misses                                                           ▒
14 dTLB-store-misses      

The patches can also be found in the github[4].

[1] https://github.com/riscv-non-isa/riscv-sbi-doc/blob/master/riscv-sbi.adoc
[2] https://drive.google.com/file/d/171j4jFjIkKdj5LWcExphq4xG_2sihbfd/edit
[3] https://github.com/atishp04/opensbi/tree/pmu_sscofpmf 
[4] https://github.com/atishp04/linux/tree/riscv_pmu_v3
[5] https://github.com/atishp04/qemu/tree/riscv_pmu_v2

Changes from v3->v4:
1. Added interrupt overflow support.
2. Cleaned up legacy driver initialization.
3. Supports perf record now.
4. Added the DT binding and maintainers file.
5. Changed cpu hotplug notifier to be multi-state.
6. OpenSBI doesn't disable cycle/instret counter during boot. Update the
   perf code to disable all the counter during the boot.

Changes from v1->v2
1. Implemented the latest SBI PMU extension specification.
2. The core platform driver was changed to operate as a library while only
   sbi based PMU is built as a driver. The legacy one is just a fallback if
   SBI PMU extension is not available.

Atish Patra (10):
RISC-V: Remove the current perf implementation
RISC-V: Add CSR encodings for all HPMCOUNTERS
RISC-V: Add a perf core library for pmu drivers
RISC-V: Add a simple platform driver for RISC-V legacy perf
RISC-V: Add RISC-V SBI PMU extension definitions
dt-binding: pmu: Add RISC-V PMU DT bindings
RISC-V: Add perf platform driver based on SBI PMU extension
RISC-V: Add interrupt support for perf
Documentation: riscv: Remove the old documentation
MAINTAINERS: Add entry for RISC-V PMU drivers

.../devicetree/bindings/perf/riscv,pmu.yaml   |  51 ++
Documentation/riscv/pmu.rst                   | 255 ------
MAINTAINERS                                   |  10 +
arch/riscv/Kconfig                            |  13 -
arch/riscv/include/asm/csr.h                  |  66 +-
arch/riscv/include/asm/perf_event.h           |  72 --
arch/riscv/include/asm/sbi.h                  |  97 +++
arch/riscv/kernel/Makefile                    |   1 -
arch/riscv/kernel/perf_event.c                | 485 ------------
drivers/perf/Kconfig                          |  25 +
drivers/perf/Makefile                         |   5 +
drivers/perf/riscv_pmu.c                      | 331 ++++++++
drivers/perf/riscv_pmu_legacy.c               | 143 ++++
drivers/perf/riscv_pmu_sbi.c                  | 731 ++++++++++++++++++
include/linux/cpuhotplug.h                    |   1 +
include/linux/perf/riscv_pmu.h                |  69 ++
16 files changed, 1528 insertions(+), 827 deletions(-)
create mode 100644 Documentation/devicetree/bindings/perf/riscv,pmu.yaml
delete mode 100644 Documentation/riscv/pmu.rst
delete mode 100644 arch/riscv/kernel/perf_event.c
create mode 100644 drivers/perf/riscv_pmu.c
create mode 100644 drivers/perf/riscv_pmu_legacy.c
create mode 100644 drivers/perf/riscv_pmu_sbi.c
create mode 100644 include/linux/perf/riscv_pmu.h

--
2.31.1


             reply	other threads:[~2021-09-10 19:28 UTC|newest]

Thread overview: 18+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-10 19:27 Atish Patra [this message]
2021-09-10 19:27 ` [v3 01/10] RISC-V: Remove the current perf implementation Atish Patra
     [not found]   ` <238571632287002@mail.yandex.ru>
2021-09-22  5:25     ` Nikita Shubin
2021-09-10 19:27 ` [v3 02/10] RISC-V: Add CSR encodings for all HPMCOUNTERS Atish Patra
2021-09-10 19:27 ` [v3 03/10] RISC-V: Add a perf core library for pmu drivers Atish Patra
2021-09-10 19:27 ` [v3 04/10] RISC-V: Add a simple platform driver for RISC-V legacy perf Atish Patra
2021-09-10 19:27 ` [v3 05/10] RISC-V: Add RISC-V SBI PMU extension definitions Atish Patra
2021-09-10 19:27 ` [v3 06/10] dt-binding: pmu: Add RISC-V PMU DT bindings Atish Patra
2021-09-13 12:18   ` Rob Herring
2021-09-14 15:50   ` Rob Herring
2021-09-10 19:27 ` [v3 07/10] RISC-V: Add perf platform driver based on SBI PMU extension Atish Patra
2021-09-10 19:27 ` [v3 08/10] RISC-V: Add interrupt support for perf Atish Patra
2021-09-10 19:27 ` [v3 09/10] Documentation: riscv: Remove the old documentation Atish Patra
2021-09-10 19:27 ` [v3 10/10] MAINTAINERS: Add entry for RISC-V PMU drivers Atish Patra
2021-10-04 18:20 ` [v3 00/10] Improve RISC-V Perf support using SBI PMU and sscofpmf extension Palmer Dabbelt
2021-10-06 18:27   ` Atish Patra
2021-10-05  9:17 ` Nikita Shubin
2021-10-06 18:28   ` Atish Patra

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210910192757.2309100-1-atish.patra@wdc.com \
    --to=atish.patra@wdc.com \
    --cc=alexander.shishkin@linux.intel.com \
    --cc=anup.patel@wdc.com \
    --cc=ardb@kernel.org \
    --cc=corbet@lwn.net \
    --cc=devicetree@vger.kernel.org \
    --cc=djwong@kernel.org \
    --cc=guoren@linux.alibaba.com \
    --cc=john.garry@huawei.com \
    --cc=jolsa@redhat.com \
    --cc=linux-doc@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-perf-users@vger.kernel.org \
    --cc=linux-riscv@lists.infradead.org \
    --cc=mick@ics.forth.gr \
    --cc=palmer@dabbelt.com \
    --cc=paul.walmsley@sifive.com \
    --cc=robh+dt@kernel.org \
    --cc=vincent.chen@sifive.com \
    --cc=xypron.glpk@gmx.de \
    --subject='Re: [v3 00/10] Improve RISC-V Perf support using SBI PMU and sscofpmf extension' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).