LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Pekka Paalanen <pq@iki.fi>
To: mingo@redhat.com
Cc: Pekka Paalanen <pq@iki.fi>, linux-kernel@vger.kernel.org
Subject: [RFC PATCH] x86: mmiotrace - trace memory mapped IO
Date: Sun, 27 Jan 2008 19:55:36 +0200	[thread overview]
Message-ID: <20080127195536.50809974@daedalus.pq.iki.fi> (raw)
In-Reply-To: <20080127185238.4bcac54b@daedalus.pq.iki.fi>

Hello,

the patch itself is slightly bigger than recommended for LKML, so
it is available at:
http://jumi.lut.fi/~paalanen/scratch/mmio24/0002-x86-mmiotrace-trace-memory-mapped-IO.patch.txt

This seems to work fine on UP machine, I have not tested yet with
with an SMP machine. Originally mmio-trace was UP-only, so there
probably still are SMP issues. Mmio-trace has existed with its current
purpose for about a year now.

The following is a copy of the patch description:

Mmiotrace is a tool for trapping memory mapped IO (MMIO) accesses within
the kernel. It is used for debugging and especially for reverse
engineering evil binary drivers.

Mmiotrace works by wrapping the ioremap family of kernel functions and
marking the returned pages as not present. Access to the IO memory
triggers a page fault, which will be handled by mmiotrace's custom page
fault handler. This will single-step the faulted instruction with the
MMIO page marked as present. Access logs are directed to user space via
relay and debug_fs.

This page fault approach is necessary, because binary drivers have
readl/writel etc. calls inlined and therefore extremely difficult to
trap with with e.g. kprobes.

This patch depends on the custom page fault handlers patch.

---

More information on mmio-trace can be found at
http://nouveau.freedesktop.org/wiki/MmioTrace
and the out-of-three module with user space parts is at
http://gitweb.freedesktop.org/?p=users/pq/mmio-trace.git;a=summary

The out-of-tree git does not include all changes I made for the
in-tree version.

Mmiotrace does not appear to procude excruciating slowness when used.
Starting X with traced nvidia binary drivers takes a couple of seconds
longer than normally. Starting X, running glxgears for a moment and
shutting down X results in an mmio-trace log of roughly 1.5 million
events. Replace glxgears by running Quake3-demo's first demo play
twice and end up with 3.5 million events. Quake3-demo framerate with
mmio-traced driver goes down to 59 fps, compared to the normal 83 fps.

I know of one bad failure case with mmio-trace. It may trigger a
complete system freeze when tracing a certain nvidia driver version
on a machine with an 8000-series (nv50) graphics chip. Netconsole,
serial console and NMI watchdog have so far been useless, and I
am starting to believe that it is the nvidia driver that does
something bad to make mmio-trace fall over. I have heard that more
recent nvidia drivers do not make it crash anymore.

I'm hoping mmiotrace could be included in 2.6.25. It would help
FOSS driver projects (Nouveau, maybe madwifi, too) to gather the
information they need. Very often the information comes from
ordinary users, for whom going through kernel patching would be
a major struggle.


Thanks,
Pekka

  reply	other threads:[~2008-01-27 17:55 UTC|newest]

Thread overview: 43+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-01-27 16:52 [PATCH] x86: Add a list for custom page fault handlers Pekka Paalanen
2008-01-27 17:55 ` Pekka Paalanen [this message]
2008-01-30 22:39   ` [RFC PATCH] x86: mmiotrace - trace memory mapped IO Pekka Paalanen
2008-01-27 19:29 ` [PATCH] x86: Add a list for custom page fault handlers Ingo Molnar
2008-01-27 21:03 ` Peter Zijlstra
2008-01-30  2:28 ` Harvey Harrison
2008-01-30  2:34   ` Harvey Harrison
2008-01-30 18:08     ` Pekka Paalanen
2008-01-31 15:07       ` Ingo Molnar
2008-01-31 16:02         ` [PATCH v2] " Pekka Paalanen
2008-01-31 16:15           ` Arjan van de Ven
2008-02-03  6:55             ` Pekka Paalanen
2008-02-03  7:03               ` Ingo Molnar
2008-02-03 21:40                 ` Pekka Paalanen
2008-02-05 20:28                 ` [PATCH 1/4] x86 mmiotrace: use lookup_address() Pekka Paalanen
2008-02-05 20:30                   ` [PATCH 2/4] x86 mmiotrace: fix relay-buffer-full flag for SMP Pekka Paalanen
2008-02-05 20:44                     ` Eric Dumazet
2008-02-05 21:14                       ` Pekka Paalanen
2008-02-05 21:35                         ` Eric Dumazet
2008-02-09 17:53                           ` [PATCH] x86 mmiotrace: Use percpu instead of arrays Pekka Paalanen
2008-02-05 20:31                   ` [PATCH 3/4] x86 mmiotrace: comment about user space ABI Pekka Paalanen
2008-02-05 20:39                   ` [PATCH 4/4] x86 mmiotrace: move files into arch/x86/mm/ Pekka Paalanen
2008-02-06  3:02                     ` Randy Dunlap
2008-02-09 11:21                       ` Pekka Paalanen
2008-02-07 12:53                     ` Ingo Molnar
2008-02-07 12:56                       ` Christoph Hellwig
2008-02-09 17:52                         ` [RFC PATCH] x86: explicit call to mmiotrace in do_page_fault() Pekka Paalanen
2008-02-09 18:01                           ` Arjan van de Ven
2008-02-09 18:23                             ` Pekka Paalanen
2008-02-09 18:56                               ` Pekka Enberg
2008-02-09 19:11                                 ` Pekka Paalanen
2008-02-09 19:19                                   ` Pekka Enberg
2008-02-09 18:39                             ` Peter Zijlstra
2008-02-09 18:39                           ` Peter Zijlstra
2008-02-10 18:05                             ` [RFC PATCH v2] " Pekka Paalanen
2008-02-11  2:12                               ` Pavel Roskin
2008-02-11 18:04                                 ` Pekka Paalanen
2008-02-06  5:00                   ` [PATCH 1/4] x86 mmiotrace: use lookup_address() Christoph Hellwig
2008-02-07 12:52                     ` Ingo Molnar
2008-01-31 16:16           ` [RFC PATCH v2] x86: mmiotrace - trace memory mapped IO Pekka Paalanen
2008-01-31 16:29             ` Arjan van de Ven
2008-02-03  7:21               ` Pekka Paalanen
2008-01-30 18:20 ` [PATCH] x86: Add a list for custom page fault handlers Arjan van de Ven

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20080127195536.50809974@daedalus.pq.iki.fi \
    --to=pq@iki.fi \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@redhat.com \
    --subject='Re: [RFC PATCH] x86: mmiotrace - trace memory mapped IO' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).