LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* [PATCHv4 00/11] perf tools: Add perf data CTF conversion
@ 2015-01-30 10:42 Jiri Olsa
  2015-01-30 10:42 ` [PATCH 01/11] perf tools: Add feature check for libbabeltrace Jiri Olsa
                   ` (10 more replies)
  0 siblings, 11 replies; 14+ messages in thread
From: Jiri Olsa @ 2015-01-30 10:42 UTC (permalink / raw)
  To: linux-kernel
  Cc: Arnaldo Carvalho de Melo, David Ahern, Dominique Toupin,
	Frederic Weisbecker, Jeremie Galarneau, Jiri Olsa,
	Mathieu Desnoyers, Namhyung Kim, Paul Mackerras, Peter Zijlstra,
	Sebastian Andrzej Siewior, Tom Zanussi, Wang Nan

hi,
this is follow up on original RFC patchset:
  http://marc.info/?t=140732735600004&r=1&w=2

Basically we are adding 'perf data convert' command to
allow conversion of perf data file into CTF [1] data.

v4 changes:
  - sorted out babeltrace new code regressions
  - rebased to latest acme's perf/core
  - includes fixes from Wang Nan, patches 9/10/11
  - patch 1/11 changelog updated with babeltrace
    compilation instructions [Arnaldo]

v3 changes:
  - rebased to latest acme's perf/core

v2 changes:
  - addressed comments from Namhyung
  - rebased to latest acme's perf/core

Changes from RFC:
  - able to generate CTF data, that are possible to be displayed under
    tracecompas GUI [3], please check several screenshots in here [4]
  - storing CTF data streams per cpu
  - several cleanups

Examples:
- Catch default perf data (cycles event):
  $ perf record ls
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.012 MB perf.data (~546 samples) ]

- To display converted CTF data run [2]:
  $ babeltrace ./ctf-data/
  [03:19:13.962125533] (+?.?????????) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 }
  [03:19:13.962130001] (+0.000004468) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 }
  [03:19:13.962131936] (+0.000001935) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 8 }
  [03:19:13.962133732] (+0.000001796) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 114 }
  [03:19:13.962135557] (+0.000001825) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 2087 }
  [03:19:13.962137627] (+0.000002070) cycles: { }, { ip = 0xFFFFFFFF81361938, tid = 20714, pid = 20714, period = 37582 }
  [03:19:13.962161091] (+0.000023464) cycles: { }, { ip = 0xFFFFFFFF8124218F, tid = 20714, pid = 20714, period = 600246 }
  [03:19:13.962517569] (+0.000356478) cycles: { }, { ip = 0xFFFFFFFF811A75DB, tid = 20714, pid = 20714, period = 1325731 }
  [03:19:13.969518008] (+0.007000439) cycles: { }, { ip = 0x34080917B2, tid = 20714, pid = 20714, period = 1144298 }

- To get some nice output in tracecompas GUI [3], please capture sched:*
  and syscall tracepoints like:
  # perf record -e 'sched:*,raw_syscalls:*' -a
  ^C[ perf record: Woken up 0 times to write data ]
  [ perf record: Captured and wrote 412.347 MB perf.data (~18015721 samples) ]

- To convert perf data file run:
  # perf data convert --to-ctf=./ctf 
  [ perf data convert: Converted 'perf.data' into CTF data './ctf' ]
  [ perf data convert: Converted and wrote 408.421 MB (3964792 samples) ]

- To display converted CTF data run [2]:
  # babeltrace ./ctf-data/
  [23:32:20.165354855] (+0.000000507) sched:sched_wakeup: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810BCA72, perf_tid = 0, perf_pid = 0, perf_id = 462554, perf_period = 1, common_type = 265, ...
  [23:32:20.165359078] (+0.000001181) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 0, perf_pid = 0, perf_id = 462562, perf_period = 1, common_type = 263, ...
  [23:32:20.165364686] (+0.000000328) sched:sched_stat_runtime: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 5326, perf_pid = 5326, perf_id = 462610, perf_period = 11380, ...
  [23:32:20.165366067] (+0.000001205) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 5326, perf_pid = 5326, perf_id = 462562, perf_period = 1, common_type  ...
  [23:32:20.165723312] (+0.000001479) sched:sched_stat_runtime: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 11821, perf_pid = 11821, perf_id = 462612, perf_period = 1000265, ...
  [23:32:20.065282391] (+?.?????????) raw_syscalls:sys_enter: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810230AF, perf_tid = 26155, perf_pid = 26155, perf_id = 462635, perf_period = 1, ...
  [23:32:20.065286422] (+0.000004031) raw_syscalls:sys_exit: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810231D8, perf_tid = 26155, perf_pid = 26155, perf_id = 462639, perf_period = 1, ...

- Or run tracecompass and open the CTF data ;-)

Changes are also reachable in here:
  git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
  perf/core_ctf_convert

thanks,
jirka

[1] Common Trace Format - http://www.efficios.com/ctf
[2] babeltrace - http://www.efficios.com/babeltrace
[3] Trace compass - http://projects.eclipse.org/projects/tools.tracecompass
[4] screenshots - http://people.redhat.com/~jolsa/tracecompass-perf/

Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Dominique Toupin <dominique.toupin@ericsson.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jeremie Galarneau <jgalar@efficios.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Namhyung Kim <namhyung@gmail.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Tom Zanussi <tzanussi@gmail.com>
Cc: Wang Nan <wangnan0@huawei.com>
---
Jiri Olsa (5):
      perf tools: Add feature check for libbabeltrace
      perf tools: Add new perf data command
      perf data: Add perf data to CTF conversion support
      perf data: Enable stream flush within processing
      perf data: Add support for setting ordered_events queue size

Sebastian Andrzej Siewior (3):
      perf data: Add a 'perf' prefix to the generic fields
      perf data: Add tracepoint events fields CTF conversion support
      perf data: Switch to multiple cpu stream files

Wang Nan (3):
      tools lib traceevent: Add priv field to struct format_field
      perf data: Fix duplicate field names and avoid reserved keywords
      perf data: Fix signess of value

 tools/lib/traceevent/event-parse.c                    |    2 +
 tools/lib/traceevent/event-parse.h                    |    2 +
 tools/perf/Documentation/perf-data.txt                |   40 +++
 tools/perf/Documentation/perf.txt                     |    7 +-
 tools/perf/Makefile.perf                              |    9 +-
 tools/perf/builtin-data.c                             |  119 ++++++++
 tools/perf/builtin.h                                  |    1 +
 tools/perf/command-list.txt                           |    1 +
 tools/perf/config/Makefile                            |   24 ++
 tools/perf/config/feature-checks/Makefile             |    8 +-
 tools/perf/config/feature-checks/test-all.c           |    5 +
 tools/perf/config/feature-checks/test-libbabeltrace.c |    8 +
 tools/perf/perf.c                                     |    1 +
 tools/perf/util/data-convert-bt.c                     | 1186 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 tools/perf/util/data-convert-bt.h                     |    8 +
 tools/perf/util/debug.c                               |    2 +
 tools/perf/util/debug.h                               |    1 +
 17 files changed, 1420 insertions(+), 4 deletions(-)
 create mode 100644 tools/perf/Documentation/perf-data.txt
 create mode 100644 tools/perf/builtin-data.c
 create mode 100644 tools/perf/config/feature-checks/test-libbabeltrace.c
 create mode 100644 tools/perf/util/data-convert-bt.c
 create mode 100644 tools/perf/util/data-convert-bt.h

^ permalink raw reply	[flat|nested] 14+ messages in thread
* [PATCHv5 00/11] perf tools: Add perf data CTF conversion
@ 2015-02-20 22:16 Jiri Olsa
  2015-02-20 22:16 ` [PATCH 01/11] perf tools: Add feature check for libbabeltrace Jiri Olsa
  0 siblings, 1 reply; 14+ messages in thread
From: Jiri Olsa @ 2015-02-20 22:16 UTC (permalink / raw)
  To: linux-kernel
  Cc: Arnaldo Carvalho de Melo, David Ahern, Dominique Toupin,
	Frederic Weisbecker, Jeremie Galarneau, Jiri Olsa,
	Mathieu Desnoyers, Namhyung Kim, Paul Mackerras, Peter Zijlstra,
	Sebastian Andrzej Siewior, Tom Zanussi, Wang Nan

hi,
this is follow up on original RFC patchset:
  http://marc.info/?t=140732735600004&r=1&w=2

Basically we are adding 'perf data convert' command to
allow conversion of perf data file into CTF [1] data.

v5 changes:
  - added struct field_format::alias instead of priv void*
  - rebased to latest acme's perf/core

v4 changes:
  - sorted out babeltrace new code regressions
  - rebased to latest acme's perf/core
  - includes fixes from Wang Nan, patches 9/10/11
  - patch 1/11 changelog updated with babeltrace
    compilation instructions [Arnaldo]

v3 changes:
  - rebased to latest acme's perf/core

v2 changes:
  - addressed comments from Namhyung
  - rebased to latest acme's perf/core

Changes from RFC:
  - able to generate CTF data, that are possible to be displayed under
    tracecompas GUI [3], please check several screenshots in here [4]
  - storing CTF data streams per cpu
  - several cleanups

Examples:
- Catch default perf data (cycles event):
  $ perf record ls
  [ perf record: Woken up 1 times to write data ]
  [ perf record: Captured and wrote 0.012 MB perf.data (~546 samples) ]

- To display converted CTF data run [2]:
  $ babeltrace ./ctf-data/
  [03:19:13.962125533] (+?.?????????) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 }
  [03:19:13.962130001] (+0.000004468) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 1 }
  [03:19:13.962131936] (+0.000001935) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 8 }
  [03:19:13.962133732] (+0.000001796) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 114 }
  [03:19:13.962135557] (+0.000001825) cycles: { }, { ip = 0xFFFFFFFF8105443A, tid = 20714, pid = 20714, period = 2087 }
  [03:19:13.962137627] (+0.000002070) cycles: { }, { ip = 0xFFFFFFFF81361938, tid = 20714, pid = 20714, period = 37582 }
  [03:19:13.962161091] (+0.000023464) cycles: { }, { ip = 0xFFFFFFFF8124218F, tid = 20714, pid = 20714, period = 600246 }
  [03:19:13.962517569] (+0.000356478) cycles: { }, { ip = 0xFFFFFFFF811A75DB, tid = 20714, pid = 20714, period = 1325731 }
  [03:19:13.969518008] (+0.007000439) cycles: { }, { ip = 0x34080917B2, tid = 20714, pid = 20714, period = 1144298 }

- To get some nice output in tracecompas GUI [3], please capture sched:*
  and syscall tracepoints like:
  # perf record -e 'sched:*,raw_syscalls:*' -a
  ^C[ perf record: Woken up 0 times to write data ]
  [ perf record: Captured and wrote 412.347 MB perf.data (~18015721 samples) ]

- To convert perf data file run:
  # perf data convert --to-ctf=./ctf 
  [ perf data convert: Converted 'perf.data' into CTF data './ctf' ]
  [ perf data convert: Converted and wrote 408.421 MB (3964792 samples) ]

- To display converted CTF data run [2]:
  # babeltrace ./ctf-data/
  [23:32:20.165354855] (+0.000000507) sched:sched_wakeup: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810BCA72, perf_tid = 0, perf_pid = 0, perf_id = 462554, perf_period = 1, common_type = 265, ...
  [23:32:20.165359078] (+0.000001181) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 0, perf_pid = 0, perf_id = 462562, perf_period = 1, common_type = 263, ...
  [23:32:20.165364686] (+0.000000328) sched:sched_stat_runtime: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 5326, perf_pid = 5326, perf_id = 462610, perf_period = 11380, ...
  [23:32:20.165366067] (+0.000001205) sched:sched_switch: { cpu_id = 0 }, { perf_ip = 0xFFFFFFFF8172A110, perf_tid = 5326, perf_pid = 5326, perf_id = 462562, perf_period = 1, common_type  ...
  [23:32:20.165723312] (+0.000001479) sched:sched_stat_runtime: { cpu_id = 2 }, { perf_ip = 0xFFFFFFFF810C8AE5, perf_tid = 11821, perf_pid = 11821, perf_id = 462612, perf_period = 1000265, ...
  [23:32:20.065282391] (+?.?????????) raw_syscalls:sys_enter: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810230AF, perf_tid = 26155, perf_pid = 26155, perf_id = 462635, perf_period = 1, ...
  [23:32:20.065286422] (+0.000004031) raw_syscalls:sys_exit: { cpu_id = 1 }, { perf_ip = 0xFFFFFFFF810231D8, perf_tid = 26155, perf_pid = 26155, perf_id = 462639, perf_period = 1, ...

- Or run tracecompass and open the CTF data ;-)

Changes are also reachable in here:
  git://git.kernel.org/pub/scm/linux/kernel/git/jolsa/perf.git
  perf/core_ctf_convert

thanks,
jirka

[1] Common Trace Format - http://www.efficios.com/ctf
[2] babeltrace - http://www.efficios.com/babeltrace
[3] Trace compass - http://projects.eclipse.org/projects/tools.tracecompass
[4] screenshots - http://people.redhat.com/~jolsa/tracecompass-perf/

Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Dominique Toupin <dominique.toupin@ericsson.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jeremie Galarneau <jgalar@efficios.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Cc: Namhyung Kim <namhyung@gmail.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Tom Zanussi <tzanussi@gmail.com>
Cc: Wang Nan <wangnan0@huawei.com>
---
Jiri Olsa (6):
      perf tools: Add feature check for libbabeltrace
      perf tools: Add new perf data command
      perf data: Add perf data to CTF conversion support
      perf data: Enable stream flush within processing
      perf data: Add support for setting ordered_events queue size
      tools lib traceevent: Add alias field to struct format_field

Sebastian Andrzej Siewior (3):
      perf data: Add a 'perf' prefix to the generic fields
      perf data: Add tracepoint events fields CTF conversion support
      perf data: Switch to multiple cpu stream files

Wang Nan (2):
      perf data: Fix duplicate field names and avoid reserved keywords
      perf data: Fix signess of value

 tools/lib/traceevent/event-parse.c                 |    1 +
 tools/lib/traceevent/event-parse.h                 |    1 +
 tools/perf/Build                                   |    1 +
 tools/perf/Documentation/perf-data.txt             |   40 +
 tools/perf/Documentation/perf.txt                  |    7 +-
 tools/perf/Makefile.perf                           |    4 +-
 tools/perf/builtin-data.c                          |  119 ++
 tools/perf/builtin.h                               |    1 +
 tools/perf/command-list.txt                        |    1 +
 tools/perf/config/Makefile                         |   25 +
 tools/perf/config/feature-checks/Makefile          |    8 +-
 tools/perf/config/feature-checks/test-all.c        |    5 +
 .../config/feature-checks/test-libbabeltrace.c     |    8 +
 tools/perf/perf.c                                  |    1 +
 tools/perf/util/Build                              |    2 +
 tools/perf/util/data-convert-bt.c                  | 1174 ++++++++++++++++++++
 tools/perf/util/data-convert-bt.h                  |    8 +
 tools/perf/util/debug.c                            |    2 +
 tools/perf/util/debug.h                            |    1 +
 19 files changed, 1405 insertions(+), 4 deletions(-)
 create mode 100644 tools/perf/Documentation/perf-data.txt
 create mode 100644 tools/perf/builtin-data.c
 create mode 100644 tools/perf/config/feature-checks/test-libbabeltrace.c
 create mode 100644 tools/perf/util/data-convert-bt.c
 create mode 100644 tools/perf/util/data-convert-bt.h

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2015-02-20 22:18 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-01-30 10:42 [PATCHv4 00/11] perf tools: Add perf data CTF conversion Jiri Olsa
2015-01-30 10:42 ` [PATCH 01/11] perf tools: Add feature check for libbabeltrace Jiri Olsa
2015-01-30 10:42 ` [PATCH 02/11] perf tools: Add new perf data command Jiri Olsa
2015-01-30 10:42 ` [PATCH 03/11] perf data: Add perf data to CTF conversion support Jiri Olsa
2015-01-30 10:42 ` [PATCH 04/11] perf data: Add a 'perf' prefix to the generic fields Jiri Olsa
2015-01-30 10:42 ` [PATCH 05/11] perf data: Add tracepoint events fields CTF conversion support Jiri Olsa
2015-01-30 10:43 ` [PATCH 06/11] perf data: Switch to multiple cpu stream files Jiri Olsa
2015-01-30 10:43 ` [PATCH 07/11] perf data: Enable stream flush within processing Jiri Olsa
2015-01-30 10:43 ` [PATCH 08/11] perf data: Add support for setting ordered_events queue size Jiri Olsa
2015-01-30 10:43 ` [PATCH 09/11] tools lib traceevent: Add priv field to struct format_field Jiri Olsa
2015-01-30 10:47   ` Jiri Olsa
2015-01-30 10:43 ` [PATCH 10/11] perf data: Fix duplicate field names and avoid reserved keywords Jiri Olsa
2015-01-30 10:43 ` [PATCH 11/11] perf data: Fix signess of value Jiri Olsa
2015-02-20 22:16 [PATCHv5 00/11] perf tools: Add perf data CTF conversion Jiri Olsa
2015-02-20 22:16 ` [PATCH 01/11] perf tools: Add feature check for libbabeltrace Jiri Olsa

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).