LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Petr Mladek <pmladek@suse.com>
To: Steven Rostedt <rostedt@goodmis.org>
Cc: Sergey Senozhatsky <sergey.senozhatsky.work@gmail.com>,
	akpm@linux-foundation.org, linux-mm@kvack.org,
	Peter Zijlstra <peterz@infradead.org>, Jan Kara <jack@suse.cz>,
	Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>,
	Tejun Heo <tj@kernel.org>,
	linux-kernel@vger.kernel.org,
	Sergey Senozhatsky <sergey.senozhatsky@gmail.com>
Subject: Re: [PATCH] printk: Ratelimit messages printed by console drivers
Date: Fri, 20 Apr 2018 16:01:57 +0200	[thread overview]
Message-ID: <20180420140157.2nx5nkojj7l2y7if@pathway.suse.cz> (raw)
In-Reply-To: <20180420080428.622a8e7f@gandalf.local.home>

On Fri 2018-04-20 08:04:28, Steven Rostedt wrote:
> On Fri, 20 Apr 2018 11:12:24 +0200
> Petr Mladek <pmladek@suse.com> wrote:
> 
> > Yes, my number was arbitrary. The important thing is that it was long
> > enough. Or do you know about an console that will not be able to write
> > 100 lines within one hour?
> 
> The problem is the way rate limit works. If you print 100 lines (or
> 1000) in 5 seconds, then you just stopped printing from that context
> for 59 minutes and 55 seconds. That's a long time to block printing.

Are we talking about the same context?

I am talking about console drivers called from console_unlock(). It is
very special context because it is more or less recursive:

     + could cause infinite loop
     + the errors are usually the same again and again

As a result, if you get too many messages from this context:

     + you are lost (recursion)
     + more messages != new information

And you need to fix the problem anyway. Otherwise, the system
logging is a mess.


> What happens if you had a couple of NMIs go off that takes up that
> time, and then you hit a bug 10 minutes later from that context. You
> just lost it.

I do not understand how this is related to the NMI context.
The messages in NMI context are not throttled!

OK, the original patch throttled also NMI messages when NMI
interrupted console drivers. But it is easy to fix.


> This is a magnitude larger than any other user of rate limit in the
> kernel. The most common time is 5 seconds. The longest I can find is 1
> minute. You are saying you want to block printing from this context for
> 60 minutes!

I see 1 day long limits in dio_warn_stale_pagecache() and
xfs_scrub_experimental_warning().

Note that most ratelimiting is related to a single message. Also it
is in situation where the system should recover within seconds.


> That is HUGE! I don't understand your rational for such a huge number.
> What data do you have to back that up with?

We want to allow seeing the entire lockdep splat (Sergey wants more
than 100 lines). Also it is not that unusual that slow console is busy
several minutes when too many things are happening.

I proposed that long delay because I want to be on the safe side.
Also I do not see a huge benefit in repeating the same messages
too often.

Alternative solution would be to allow first, lets say 250, lines
and then nothing. I mean to change the approach from rate-limiting
to print-once.

Best Regards,
Petr

  reply	other threads:[~2018-04-20 14:02 UTC|newest]

Thread overview: 27+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-04-13 12:47 Petr Mladek
2018-04-13 14:12 ` Steven Rostedt
2018-04-14  2:35   ` Sergey Senozhatsky
2018-04-16  1:47     ` Sergey Senozhatsky
2018-04-16  4:25       ` Sergey Senozhatsky
2018-04-19 12:53         ` Petr Mladek
2018-04-20  2:15           ` Sergey Senozhatsky
2018-04-20  9:12             ` Petr Mladek
2018-04-20 12:04               ` Steven Rostedt
2018-04-20 14:01                 ` Petr Mladek [this message]
2018-04-20 14:17                   ` Steven Rostedt
2018-04-20 14:19                     ` Steven Rostedt
2018-04-20 14:57                     ` Petr Mladek
2018-04-20 15:13                       ` Steven Rostedt
2018-04-23 10:32                         ` Petr Mladek
2018-04-23 11:36                           ` Steven Rostedt
2018-04-23 12:45                             ` Petr Mladek
2018-04-25  5:31                               ` Sergey Senozhatsky
2018-04-26  9:42                                 ` Petr Mladek
2018-04-27 10:22                                   ` Sergey Senozhatsky
2018-05-09 12:00                                     ` Petr Mladek
2018-05-09 12:59                                       ` Steven Rostedt
2019-02-26 10:24                                       ` Tetsuo Handa
2019-02-28  4:45                                         ` Sergey Senozhatsky
2018-04-23  5:21               ` Sergey Senozhatsky
2018-04-23 12:26                 ` Petr Mladek
2018-04-23 13:00                   ` Sergey Senozhatsky

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20180420140157.2nx5nkojj7l2y7if@pathway.suse.cz \
    --to=pmladek@suse.com \
    --cc=akpm@linux-foundation.org \
    --cc=jack@suse.cz \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=penguin-kernel@I-love.SAKURA.ne.jp \
    --cc=peterz@infradead.org \
    --cc=rostedt@goodmis.org \
    --cc=sergey.senozhatsky.work@gmail.com \
    --cc=sergey.senozhatsky@gmail.com \
    --cc=tj@kernel.org \
    --subject='Re: [PATCH] printk: Ratelimit messages printed by console drivers' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).