LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Jiri Olsa <jolsa@kernel.org>
To: linux-kernel@vger.kernel.org
Cc: Jiri Olsa <jolsa@kernel.org>,
	Arnaldo Carvalho de Melo <acme@redhat.com>,
	David Ahern <dsahern@gmail.com>,
	Frederic Weisbecker <fweisbec@gmail.com>,
	Jeremie Galarneau <jgalar@efficios.com>,
	Namhyung Kim <namhyung@gmail.com>,
	Paul Mackerras <paulus@samba.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Sebastian Andrzej Siewior <bigeasy@linutronix.de>,
	Tom Zanussi <tzanussi@gmail.com>, Wang Nan <wangnan0@huawei.com>
Subject: [PATCH 07/11] perf data: Enable stream flush within processing
Date: Fri, 30 Jan 2015 11:43:01 +0100	[thread overview]
Message-ID: <1422614585-23411-8-git-send-email-jolsa@kernel.org> (raw)
In-Reply-To: <1422614585-23411-1-git-send-email-jolsa@kernel.org>

For big data files the size of data allocated for stream
instance could get really high. It's needed to flush the
data out of the stream once in a while.

Unfortunately there's no size indication in the stream
object, so we govern the flush based on the number of
stored events. Current flush limit is set ot 100000
events.

Cc: Arnaldo Carvalho de Melo <acme@redhat.com>
Cc: David Ahern <dsahern@gmail.com>
Cc: Frederic Weisbecker <fweisbec@gmail.com>
Cc: Jeremie Galarneau <jgalar@efficios.com>
Cc: Jiri Olsa <jolsa@kernel.org>
Cc: Namhyung Kim <namhyung@gmail.com>
Cc: Paul Mackerras <paulus@samba.org>
Cc: Peter Zijlstra <peterz@infradead.org>
Cc: Sebastian Andrzej Siewior <bigeasy@linutronix.de>
Cc: Tom Zanussi <tzanussi@gmail.com>
Cc: Wang Nan <wangnan0@huawei.com>
Signed-off-by: Jiri Olsa <jolsa@kernel.org>
---
 tools/perf/util/data-convert-bt.c | 26 ++++++++++++++++++++++++--
 1 file changed, 24 insertions(+), 2 deletions(-)

diff --git a/tools/perf/util/data-convert-bt.c b/tools/perf/util/data-convert-bt.c
index 4bb769e081a8..3d73e51ee10d 100644
--- a/tools/perf/util/data-convert-bt.c
+++ b/tools/perf/util/data-convert-bt.c
@@ -43,6 +43,7 @@ struct evsel_priv {
 struct ctf_stream {
 	struct bt_ctf_stream *stream;
 	int cpu;
+	u32 count;
 };
 
 struct ctf_writer {
@@ -392,7 +393,10 @@ static int ctf_stream__flush(struct ctf_stream *cs)
 		if (err)
 			pr_err("CTF stream %d flush failed\n", cs->cpu);
 
-		pr("Flush stream for cpu %d\n", cs->cpu);
+		pr("Flush stream for cpu %d (%u samples)\n",
+		   cs->cpu, cs->count);
+
+		cs->count = 0;
 	}
 
 	return err;
@@ -490,6 +494,19 @@ static int get_sample_cpu(struct ctf_writer *cw, struct perf_sample *sample,
 	return cpu;
 }
 
+#define STREAM_FLUSH_COUNT 100000
+
+/*
+ * Currently we have no other way to determine the
+ * time for the stream flush other than keep track
+ * of the number of events and check it against
+ * threshold.
+ */
+static bool is_flush_needed(struct ctf_stream *cs)
+{
+	return cs->count >= STREAM_FLUSH_COUNT;
+}
+
 static int process_sample_event(struct perf_tool *tool,
 				union perf_event *_event __maybe_unused,
 				struct perf_sample *sample,
@@ -535,8 +552,13 @@ static int process_sample_event(struct perf_tool *tool,
 	}
 
 	cs = ctf_stream(cw, get_sample_cpu(cw, sample, evsel));
-	if (cs)
+	if (cs) {
+		if (is_flush_needed(cs))
+			ctf_stream__flush(cs);
+
+		cs->count++;
 		bt_ctf_stream_append_event(cs->stream, event);
+	}
 
 	bt_ctf_event_put(event);
 	return cs ? 0 : -1;
-- 
1.9.3


  parent reply	other threads:[~2015-01-30 10:44 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-01-30 10:42 [PATCHv4 00/11] perf tools: Add perf data CTF conversion Jiri Olsa
2015-01-30 10:42 ` [PATCH 01/11] perf tools: Add feature check for libbabeltrace Jiri Olsa
2015-01-30 10:42 ` [PATCH 02/11] perf tools: Add new perf data command Jiri Olsa
2015-01-30 10:42 ` [PATCH 03/11] perf data: Add perf data to CTF conversion support Jiri Olsa
2015-01-30 10:42 ` [PATCH 04/11] perf data: Add a 'perf' prefix to the generic fields Jiri Olsa
2015-01-30 10:42 ` [PATCH 05/11] perf data: Add tracepoint events fields CTF conversion support Jiri Olsa
2015-01-30 10:43 ` [PATCH 06/11] perf data: Switch to multiple cpu stream files Jiri Olsa
2015-01-30 10:43 ` Jiri Olsa [this message]
2015-01-30 10:43 ` [PATCH 08/11] perf data: Add support for setting ordered_events queue size Jiri Olsa
2015-01-30 10:43 ` [PATCH 09/11] tools lib traceevent: Add priv field to struct format_field Jiri Olsa
2015-01-30 10:47   ` Jiri Olsa
2015-01-30 10:43 ` [PATCH 10/11] perf data: Fix duplicate field names and avoid reserved keywords Jiri Olsa
2015-01-30 10:43 ` [PATCH 11/11] perf data: Fix signess of value Jiri Olsa
2015-02-20 22:16 [PATCHv5 00/11] perf tools: Add perf data CTF conversion Jiri Olsa
2015-02-20 22:17 ` [PATCH 07/11] perf data: Enable stream flush within processing Jiri Olsa

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=1422614585-23411-8-git-send-email-jolsa@kernel.org \
    --to=jolsa@kernel.org \
    --cc=acme@redhat.com \
    --cc=bigeasy@linutronix.de \
    --cc=dsahern@gmail.com \
    --cc=fweisbec@gmail.com \
    --cc=jgalar@efficios.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=namhyung@gmail.com \
    --cc=paulus@samba.org \
    --cc=peterz@infradead.org \
    --cc=tzanussi@gmail.com \
    --cc=wangnan0@huawei.com \
    --subject='Re: [PATCH 07/11] perf data: Enable stream flush within processing' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).