LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: "Paul E. McKenney" <paulmck@linux.vnet.ibm.com>
To: Joel Fernandes <joelaf@google.com>
Cc: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>,
	rostedt <rostedt@goodmis.org>, Namhyung Kim <namhyung@kernel.org>,
	Masami Hiramatsu <mhiramat@kernel.org>,
	linux-kernel <linux-kernel@vger.kernel.org>,
	linux-rt-users <linux-rt-users@vger.kernel.org>,
	Peter Zijlstra <peterz@infradead.org>,
	Ingo Molnar <mingo@redhat.com>,
	Tom Zanussi <tom.zanussi@linux.intel.com>,
	Thomas Gleixner <tglx@linutronix.de>,
	Boqun Feng <boqun.feng@gmail.com>, fweisbec <fweisbec@gmail.com>,
	Randy Dunlap <rdunlap@infradead.org>,
	kbuild test robot <fengguang.wu@intel.com>,
	baohong liu <baohong.liu@intel.com>,
	vedang patel <vedang.patel@intel.com>,
	kernel-team <kernel-team@lge.com>
Subject: Re: [RFC v4 3/4] irqflags: Avoid unnecessary calls to trace_ if you can
Date: Tue, 24 Apr 2018 17:10:49 -0700	[thread overview]
Message-ID: <20180425001049.GX26088@linux.vnet.ibm.com> (raw)
In-Reply-To: <68e4c123-a223-5e26-e57a-da2515041bf3@google.com>

On Tue, Apr 24, 2018 at 04:46:54PM -0700, Joel Fernandes wrote:
> 
> 
> On 04/24/2018 04:21 PM, Mathieu Desnoyers wrote:
> >----- On Apr 24, 2018, at 2:59 PM, Joel Fernandes joelaf@google.com wrote:
> >>On Tue, Apr 24, 2018 at 11:26 AM, Paul E. McKenney
> >><paulmck@linux.vnet.ibm.com> wrote:
> >>>On Tue, Apr 24, 2018 at 11:23:02AM -0700, Paul E. McKenney wrote:
> >>>>On Tue, Apr 24, 2018 at 10:26:58AM -0700, Paul E. McKenney wrote:
> >>>>>On Tue, Apr 24, 2018 at 09:01:34AM -0700, Joel Fernandes wrote:
> >>>>>>On Tue, Apr 24, 2018 at 8:56 AM, Paul E. McKenney
> >>>>>><paulmck@linux.vnet.ibm.com> wrote:
> >>>>>>>On Mon, Apr 23, 2018 at 05:22:44PM -0400, Steven Rostedt wrote:
> >>>>>>>>On Mon, 23 Apr 2018 13:12:21 -0400 (EDT)
> >>>>>>>>Mathieu Desnoyers <mathieu.desnoyers@efficios.com> wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>>I'm inclined to explicitly declare the tracepoints with their given
> >>>>>>>>>synchronization method. Tracepoint probe callback functions for currently
> >>>>>>>>>existing tracepoints expect to have preemption disabled when invoked.
> >>>>>>>>>This assumption will not be true anymore for srcu-tracepoints.
> >>>>>>>>
> >>>>>>>>Actually, why not have a flag attached to the tracepoint_func that
> >>>>>>>>states if it expects preemption to be enabled or not? If a
> >>>>>>>>trace_##event##_srcu() is called, then simply disable preemption before
> >>>>>>>>calling the callbacks for it. That way if a callback is fine for use
> >>>>>>>>with srcu, then it would require calling
> >>>>>>>>
> >>>>>>>>       register_trace_##event##_may_sleep();
> >>>>>>>>
> >>>>>>>>Then if someone uses this on a tracepoint where preemption is disabled,
> >>>>>>>>we simply do not call it.
> >>>>>>>
> >>>>>>>One more stupid question...  If we are having to trace so much stuff
> >>>>>>>in the idle loop, are we perhaps grossly overstating the extent of that
> >>>>>>>"idle" loop?  For being called "idle", this code seems quite busy!
> >>>>>>
> >>>>>>;-)
> >>>>>>The performance hit I am observing is when running a heavy workload,
> >>>>>>like hackbench or something like that. That's what I am trying to
> >>>>>>correct.
> >>>>>>By the way is there any limitation on using SRCU too early during
> >>>>>>boot? I backported Mathieu's srcu tracepoint patches but the kernel
> >>>>>>hangs pretty early in the boot. I register lockdep probes in
> >>>>>>start_kernel. I am hoping that's not why.
> >>>>>>
> >>>>>>I could also have just screwed up the backporting... may be for my
> >>>>>>testing, I will just replace the rcu API with the srcu instead of all
> >>>>>>of Mathieu's new TRACE_EVENT macros for SRCU, since all I am trying to
> >>>>>>do right now is measure the performance of my patches with SRCU.
> >>>>>
> >>>>>Gah, yes, there is an entry on my capacious todo list on making SRCU
> >>>>>grace periods work during early boot and mid-boot.  Let me see what
> >>>>>I can do...
> >>>>
> >>>>OK, just need to verify that you are OK with call_srcu()'s callbacks
> >>>>not being invoked until sometime during core_initcall() time.  (If you
> >>>>really do need them to be invoked before that, in theory it is possible,
> >>>>but in practice it is weird, even for RCU.)
> >>>
> >>>Oh, and that early at boot, you will need to use DEFINE_SRCU() or
> >>>DEFINE_STATIC_SRCU() rather than dynamic allocation and initialization.
> >>>
> >>>                                                         Thanx, Paul
> >>>
> >>
> >>Oh ok.
> >>
> >>About call_rcu, calling it later may be an issue since we register the
> >>probes in start_kernel, for the first probe call_rcu will be sched,
> >>but for the second one I think it'll try to call_rcu to get rid of the
> >>first one.
> >>
> >>This is the relevant code that gets called when probes are added:
> >>
> >>static inline void release_probes(struct tracepoint_func *old)
> >>{
> >>        if (old) {
> >>                struct tp_probes *tp_probes = container_of(old,
> >>                        struct tp_probes, probes[0]);
> >>                call_rcu_sched(&tp_probes->rcu, rcu_free_old_probes);
> >>        }
> >>}
> >>
> >>Maybe we can somehow defer the call_srcu until later? Would that be possible?
> >>
> >>also Mathieu, you didn't modify the call_rcu_sched in your prototype
> >>to be changed to use call_srcu, should you be doing that?
> >
> >You're right, I think I should have introduced a call_srcu in there.
> >It's missing in my prototype.
> >
> >However, in the prototype I did, we need to wait for *both* sched-rcu
> >and SRCU grace periods, because we don't track which site is using which
> >rcu flavor.
> >
> >So you could achieve this relatively easily by means of two chained
> >RCU callbacks, e.g.:
> >
> >release_probes() calls call_rcu_sched(... , rcu_free_old_probes)
> >
> >and then in rcu_free_old_probes() do:
> >
> >call_srcu(... , srcu_free_old_probes)
> >
> >and perform kfree(container_of(head, struct tp_probes, rcu));
> >within srcu_free_old_probes.
> >
> >It is somewhat a hack, but should work.
> 
> Sounds good, thanks.
> 
> Also I found the reason for my boot issue. It was because the
> init_srcu_struct in the prototype was being done in an initcall.
> Instead if I do it in start_kernel before the tracepoint is used, it
> fixes it (although I don't know if this is dangerous to do like this
> but I can get it to boot atleast.. Let me know if this isn't the
> right way to do it, or if something else could go wrong)
> 
> diff --git a/init/main.c b/init/main.c
> index 34823072ef9e..ecc88319c6da 100644
> --- a/init/main.c
> +++ b/init/main.c
> @@ -631,6 +631,7 @@ asmlinkage __visible void __init start_kernel(void)
>         WARN(!irqs_disabled(), "Interrupts were enabled early\n");
>         early_boot_irqs_disabled = false;
> 
> +       init_srcu_struct(&tracepoint_srcu);
>         lockdep_init_early();
> 
>         local_irq_enable();
> --
> 
> I benchmarked it and the performance also looks quite good compared
> to the rcu tracepoint version.
> 
> If you, Paul and other think doing the init_srcu_struct like this
> should be Ok, then I can try to work more on your srcu prototype and
> roll into my series and post them in the next RFC series (or let me
> know if you wanted to work your srcu stuff in a separate series..).

That is definitely not what I was expecting, but let's see if it works
anyway...  ;-)

But first, I was instead expecting something like this:

DEFINE_SRCU(tracepoint_srcu);

With this approach, some of the initialization happens at compile time
and the rest happens at the first call_srcu().

This will work -only- if the first call_srcu() doesn't happen until after
workqueue_init_early() has been invoked.  Which I believe must have been
the case in your testing, because otherwise it looks like __call_srcu()
would have complained bitterly.

On the other hand, if you need to invoke call_srcu() before the call
to workqueue_init_early(), then you need the patch that I am beating
into shape.  Plus you would need to use DEFINE_SRCU() and to avoid
invoking init_srcu_struct().

							Thanx, Paul

  reply	other threads:[~2018-04-25  0:09 UTC|newest]

Thread overview: 44+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-04-17  4:07 [RFC v4 0/4] Centralize and unify usage of preempt/irq tracepoints Joel Fernandes
2018-04-17  4:07 ` [RFC v4 1/4] tracepoint: Add API to not do lockdep checks during RCU ops Joel Fernandes
2018-04-17  4:07 ` [RFC v4 2/4] softirq: reorder trace_softirqs_on to prevent lockdep splat Joel Fernandes
2018-04-17  4:07 ` [RFC v4 3/4] irqflags: Avoid unnecessary calls to trace_ if you can Joel Fernandes
2018-04-18  9:02   ` Masami Hiramatsu
2018-04-19  5:43     ` Namhyung Kim
2018-04-20  7:07       ` Joel Fernandes
2018-04-23  1:14         ` Joel Fernandes
2018-04-23  3:19           ` Paul E. McKenney
2018-04-23 14:31             ` Mathieu Desnoyers
2018-04-23 14:53               ` Steven Rostedt
2018-04-23 14:59                 ` Mathieu Desnoyers
2018-04-23 15:12                   ` Paul E. McKenney
2018-04-23 16:18                   ` Steven Rostedt
2018-04-23 17:12                     ` Mathieu Desnoyers
2018-04-23 17:24                       ` Joel Fernandes
2018-04-23 21:22                       ` Steven Rostedt
2018-04-24 15:56                         ` Paul E. McKenney
2018-04-24 16:01                           ` Joel Fernandes
2018-04-24 17:26                             ` Paul E. McKenney
2018-04-24 18:23                               ` Paul E. McKenney
2018-04-24 18:26                                 ` Paul E. McKenney
2018-04-24 18:59                                   ` Joel Fernandes
2018-04-24 19:01                                     ` Joel Fernandes
2018-04-24 19:09                                     ` Paul E. McKenney
2018-04-24 19:16                                       ` Joel Fernandes
2018-04-24 23:21                                     ` Mathieu Desnoyers
2018-04-24 23:46                                       ` Joel Fernandes
2018-04-25  0:10                                         ` Paul E. McKenney [this message]
2018-04-25  4:20                                           ` Paul E. McKenney
2018-04-25 21:27                                             ` Joel Fernandes
2018-04-25 21:35                                               ` Paul E. McKenney
2018-04-25 21:40                                               ` Mathieu Desnoyers
2018-04-25 22:51                                                 ` Steven Rostedt
2018-04-26 15:03                                                   ` Mathieu Desnoyers
2018-04-26 16:08                                                     ` Mathieu Desnoyers
2018-04-25 23:13                                                 ` Joel Fernandes
2018-04-26 15:13                                                   ` Mathieu Desnoyers
2018-04-26 15:20                                                     ` Joel Fernandes
2018-04-26 15:49                                                     ` Paul E. McKenney
2018-04-23 15:49                 ` Joel Fernandes
2018-04-26  2:18             ` Joel Fernandes
2018-05-01  1:18     ` Joel Fernandes
2018-04-17  4:07 ` [RFC v4 4/4] tracing: Centralize preemptirq tracepoints and unify their usage Joel Fernandes

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180425001049.GX26088@linux.vnet.ibm.com \
    --to=paulmck@linux.vnet.ibm.com \
    --cc=baohong.liu@intel.com \
    --cc=boqun.feng@gmail.com \
    --cc=fengguang.wu@intel.com \
    --cc=fweisbec@gmail.com \
    --cc=joelaf@google.com \
    --cc=kernel-team@lge.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-rt-users@vger.kernel.org \
    --cc=mathieu.desnoyers@efficios.com \
    --cc=mhiramat@kernel.org \
    --cc=mingo@redhat.com \
    --cc=namhyung@kernel.org \
    --cc=peterz@infradead.org \
    --cc=rdunlap@infradead.org \
    --cc=rostedt@goodmis.org \
    --cc=tglx@linutronix.de \
    --cc=tom.zanussi@linux.intel.com \
    --cc=vedang.patel@intel.com \
    --subject='Re: [RFC v4 3/4] irqflags: Avoid unnecessary calls to trace_ if you can' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).