LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Steven Rostedt <rostedt@goodmis.org>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Jonathan Corbet <corbet@lwn.net>, Yinghai Lu <yinghai@kernel.org>,
	Ingo Molnar <mingo@elte.hu>, Robert Hancock <hancockr@shaw.ca>,
	e1000-devel@lists.sourceforge.net,
	LKML <linux-kernel@vger.kernel.org>
Subject: Re: 2.6.28-rc2 hates my e1000e
Date: Sat, 1 Nov 2008 13:35:36 -0400 (EDT)	[thread overview]
Message-ID: <alpine.DEB.1.10.0811011326390.9409@gandalf.stny.rr.com> (raw)
In-Reply-To: <alpine.LFD.2.00.0811010958230.6226@nehalem.linux-foundation.org>


[ Note, lots of family activities this weekend, so my response may be 
  slow ]

On Sat, 1 Nov 2008, Linus Torvalds wrote:
> On Sat, 1 Nov 2008, Jonathan Corbet wrote:
> 
> > Networking is fine in the absence of NFS.  I retried things and
> > stress-tested it in a few ways with no trouble.  I think your last patch
> > fixes the network card just fine.
> > 
> > Then I tried NFS again, watching more closely this time around.
> > Everything locks up.  In fact, the soft lockup watchdog starts to
> > scream:
> 
> Interesting. I wonder why it happens for NFS, but not apparently for all 
> your other modules.
> 
> It does look very much like a ftrace issue, though, not NFS or 
> network-related. Steven? Is this something that you are aware of already, 
> with what looks like a lockup in ftrace_record_ip()?
> 

No, I have not seen this before. The code is now pretty straight forward.

Jon, could you do a gdb vmlinux and a li *ftrace_record_ip+0xcb to find 
the exact line that is?

Showing the call path of this we have:

in module.c:

  	/* sechdrs[0].sh_size is always zero */
	mseg = section_objs(hdr, sechdrs, secstrings, "__mcount_loc",
			    sizeof(*mseg), &num_mcount);
	ftrace_init_module(mseg, mseg + num_mcount);

Where we pass a table of mcount callers to the ftrace_init_module.

void ftrace_init_module(unsigned long *start, unsigned long *end)
{
	if (ftrace_disabled || start == end)
		return;
	ftrace_convert_nops(start, end);
}

I wonder if I should test to make sure start is < end :-/

ftrace_convert_nops does the following under a mutex.

	while (p < end) {
		addr = ftrace_call_adjust(*p++);
		ftrace_record_ip(addr);
	}


And ftrace_record_ip does:

ftrace_record_ip(unsigned long ip)
{
	struct dyn_ftrace *rec;
	if (!ftrace_enabled || ftrace_disabled)
		return NULL;
	rec = ftrace_alloc_dyn_node(ip);
	if (!rec)
		return NULL;
	rec->ip = ip;
	list_add(&rec->list, &ftrace_new_addrs);
	return rec;
}


The ftrace_alloc_dyn_node does allocate a page if we are running low, but 
there's no other loops or locks that I can see us deadlocking on.

-- Steve


  reply	other threads:[~2008-11-01 17:35 UTC|newest]

Thread overview: 21+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
     [not found] <fa.P5jmIAOhSJ192sfWD+qlec07fe8@ifi.uio.no>
2008-10-31  0:45 ` Robert Hancock
2008-10-31  2:58   ` Jonathan Corbet
2008-10-31  4:08     ` Yinghai Lu
2008-10-31 15:07       ` Linus Torvalds
2008-10-31 15:44         ` Linus Torvalds
2008-10-31 16:00           ` Jonathan Corbet
2008-10-31 16:24             ` Linus Torvalds
2008-10-31 16:35               ` Yinghai Lu
2008-10-31 16:46                 ` Linus Torvalds
2008-10-31 16:51               ` Jonathan Corbet
2008-10-31 16:58                 ` Linus Torvalds
2008-10-31 20:20                   ` Yinghai Lu
2008-11-01 15:01                   ` Jonathan Corbet
2008-11-01 17:16                     ` Linus Torvalds
2008-11-01 17:35                       ` Steven Rostedt [this message]
2008-11-01 19:50                       ` Yinghai Lu
2008-11-01 22:45                         ` Jonathan Corbet
2008-11-01 22:47                           ` Yinghai Lu
2008-11-01 23:18                           ` Linus Torvalds
2008-11-02  1:26                             ` Robert Hancock
2008-10-30 23:44 Jonathan Corbet

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=alpine.DEB.1.10.0811011326390.9409@gandalf.stny.rr.com \
    --to=rostedt@goodmis.org \
    --cc=corbet@lwn.net \
    --cc=e1000-devel@lists.sourceforge.net \
    --cc=hancockr@shaw.ca \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@elte.hu \
    --cc=torvalds@linux-foundation.org \
    --cc=yinghai@kernel.org \
    --subject='Re: 2.6.28-rc2 hates my e1000e' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).