LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Sergey Senozhatsky <email@example.com>
To: Petr Mladek <firstname.lastname@example.org>
Cc: Steven Rostedt <email@example.com>,
Sergey Senozhatsky <firstname.lastname@example.org>,
Peter Zijlstra <email@example.com>, Jan Kara <firstname.lastname@example.org>,
Tetsuo Handa <penguin-kernel@I-love.SAKURA.ne.jp>,
Tejun Heo <email@example.com>,
Sergey Senozhatsky <firstname.lastname@example.org>
Subject: Re: [PATCH] printk: Ratelimit messages printed by console drivers
Date: Wed, 25 Apr 2018 14:31:46 +0900 [thread overview]
Message-ID: <20180425053146.GA25288@jagdpanzerIV> (raw)
On (04/23/18 14:45), Petr Mladek wrote:
> I am not sure how slow are the slowest consoles. If I take that
> everything should be faster than 1200 bauds. Then 10 minutes
> should be enough for 1000 lines and 80 characters per-line:
Well, the problem with the numbers is that they are too... simple...
let me put it this way.
What if I don't have a slow serial console? Or what if I have NMI
watchdog set to 40 seconds? Or what if I don't have NMIs at all?
Why am I all of a sudden limited by "1200 bauds"?
Another problem is that we limit the *wrong* thing.
Not only because we can [and probably need to] rate-limit the misbehaving
code that calls printk()-s, instead of printk(). But because we claim
that we limit the "number of lines" added recursively. This is wrong.
We limit the number of times vprintk_func() was called, which is != the
number of added lines. Because vprintk_func() is also called for pr_cont()
or printk(KERN_CONT) or printk("missing new line"). Backtraces contain
tons and tons of pr_cont()-s - registers print out, list of modules
print out, stack print out, code print out. Even this thing at the
bottom of a trace:
Code: 01 ca 49 89 d1 48 89 d1 48 c1 ea 23 48 8b 14 d5 80 23 63 82 49 c1 e9 0c 48 c1 e9 1b 48 85 d2 74 0a 0f b6 c9 48 c1 e1 04 48 01 ca <48> 8b 12 49 c1 e1 06 b9 00 00 00 80 89 7d 80 89 75 84 48 8b 3d
is nothing but a bunch of pr_cont()-s, each of which will individually
end up in vprintk_func(). Error reports in general can contain even more
pr_cont() calls. E.g. core kernel code can hex dump slab memory, while
being called from one of console drivers.
Another problem is that nothing tells us that we *actually* have an
infinite loop. Nothing tells us that every call_console_drivers()
adds more messages to the logbuf. We see just one thing - the current
call_console_drivers() is about to add some lines to the logbuf later
on. OK, why is this a problem? This can be a one time thing. Or
console_unlock() may be in a schedulable context, getting rescheduled
after every line it prints [either implicitly after
printk_safe_exit_irqrestore(), or explicitly by calling into the
scheduler - cond_resched()].
Most likely, we don't even realize how many things we are about to
> Alternatively, it seems that we are going to call console drivers
> outside printk_safe context => the messages will appear in the main
> log buffer immediately => only small risk of a ping-pong with printk
> safe buffers. We might reset the counter when all messages are handled
> in console_unlock(). It will be more complex patch than when using
> ratelimiting but it still should be sane.
We may have some sort of vprintk_func()-based solution, may be.
But we first need a real reason. Right now it looks to me like
we have "a solution" to a problem which we have never witnessed.
That vprintk_func()-based solution, if there will be no other
options on the table, must be much smarter than anything that
we have seen so far.
next prev parent reply other threads:[~2018-04-25 5:31 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2018-04-13 12:47 Petr Mladek
2018-04-13 14:12 ` Steven Rostedt
2018-04-14 2:35 ` Sergey Senozhatsky
2018-04-16 1:47 ` Sergey Senozhatsky
2018-04-16 4:25 ` Sergey Senozhatsky
2018-04-19 12:53 ` Petr Mladek
2018-04-20 2:15 ` Sergey Senozhatsky
2018-04-20 9:12 ` Petr Mladek
2018-04-20 12:04 ` Steven Rostedt
2018-04-20 14:01 ` Petr Mladek
2018-04-20 14:17 ` Steven Rostedt
2018-04-20 14:19 ` Steven Rostedt
2018-04-20 14:57 ` Petr Mladek
2018-04-20 15:13 ` Steven Rostedt
2018-04-23 10:32 ` Petr Mladek
2018-04-23 11:36 ` Steven Rostedt
2018-04-23 12:45 ` Petr Mladek
2018-04-25 5:31 ` Sergey Senozhatsky [this message]
2018-04-26 9:42 ` Petr Mladek
2018-04-27 10:22 ` Sergey Senozhatsky
2018-05-09 12:00 ` Petr Mladek
2018-05-09 12:59 ` Steven Rostedt
2019-02-26 10:24 ` Tetsuo Handa
2019-02-28 4:45 ` Sergey Senozhatsky
2018-04-23 5:21 ` Sergey Senozhatsky
2018-04-23 12:26 ` Petr Mladek
2018-04-23 13:00 ` Sergey Senozhatsky
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--subject='Re: [PATCH] printk: Ratelimit messages printed by console drivers' \
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).