LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Steven Rostedt <rostedt@goodmis.org>
To: Heiko Carstens <heiko.carstens@de.ibm.com>
Cc: Ingo Molnar <mingo@elte.hu>,
	Peter Zijlstra <a.p.zijlstra@chello.nl>,
	linux-kernel@vger.kernel.org
Subject: Re: bug: ftrace & lockdep badness
Date: Wed, 5 Nov 2008 11:04:37 -0500 (EST)	[thread overview]
Message-ID: <alpine.DEB.1.10.0811051054430.11900@gandalf.stny.rr.com> (raw)
In-Reply-To: <20081105123714.GA23504@osiris.boeblingen.de.ibm.com>



On Wed, 5 Nov 2008, Heiko Carstens wrote:

> Hi Steven,
> 
> enabling preemptirqsoff tracing gives me this on a lockdep enabled kernel:
> 
> Badness at kernel/lockdep.c:2894

[...]

> Call Trace:
> ([<0000000033abb188>] 0x33abb188)
>  [<000000000007e76a>] lock_acquire+0x52/0xbc
>  [<000000000037618c>] _spin_lock_irqsave+0x6c/0xb0
>  [<000000000009d52e>] ring_buffer_reset_cpu+0x6e/0x114
>  [<00000000000a46c0>] tracing_reset+0x84/0xe8
>  [<00000000000a5632>] trace_preempt_off+0x112/0x178
>  [<0000000000378888>] add_preempt_count+0xa4/0x130
>  [<00000000000539e0>] __local_bh_disable+0x54/0xcc
>  [<0000000000053a82>] local_bh_disable+0x2a/0x38

[...]

> Seems to be that the following happens:
> 
> __local_bh_disable() calls add_preempt_count() which adds something to
> preempt_count().
> After this addition softirq_count() will return something != 0.
> However current->softirqs_enabled is still 1 since we haven't called
> trace_softirqs_off() yet.
> But we call trace_preempt_off() instead which will grab a lock in the
> ringbuffer code and that will trigger this check in lockdep.c:
> 
>         if (!hardirq_count()) {
>                 if (softirq_count())
>                         DEBUG_LOCKS_WARN_ON(current->softirqs_enabled);
> 
> I assume this is a known bug. Is there any fix available?

Yes, this does seem to be the case.

Perhaps Peter or Ingo can come up with some ideas on how to solve this. 
Let me try to rephrase the problem.

In __local_bh_disable() we call add_preempt_count(SOFTIRQ_OFFSET)

in add_preempt_count (in sched.c)

  preempt_count() += val;
  if (preempt_count() == val)
	trace_preempt_off(...)

in trace_preempt_off we record into the ring buffer. But the ring buffer 
will call spin_lock_irqsave (it use to be raw_spin_lock, where this was 
not an issue). Note, there is development to make the ring buffer 
lockless, but until that time, we need to come up with a real solution.

Now we enter the lockdep code, but we are not in a full state transition 
and we get a WARN_ON triggered by the lockdep code (have SOFTIRQ_OFFSET 
set but we did not yet tell lockdep we are in a softirq).

This is the type of problems we deal with when we have the tracer tracing 
lockdep code at the same time the lockdep code is checking the tracer.

-- Steve

  parent reply	other threads:[~2008-11-05 16:04 UTC|newest]

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2008-11-05 12:37 Heiko Carstens
2008-11-05 13:07 ` Steven Rostedt
2008-11-05 16:04 ` Steven Rostedt [this message]
2008-11-05 16:30   ` Ingo Molnar
2008-11-05 16:47     ` Steven Rostedt
2008-11-05 17:01       ` Frédéric Weisbecker
2008-11-05 17:08       ` Ingo Molnar
2008-11-05 17:13         ` Steven Rostedt

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=alpine.DEB.1.10.0811051054430.11900@gandalf.stny.rr.com \
    --to=rostedt@goodmis.org \
    --cc=a.p.zijlstra@chello.nl \
    --cc=heiko.carstens@de.ibm.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@elte.hu \
    --subject='Re: bug: ftrace & lockdep badness' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).