LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* 5.14-rc breaks iotop swap io monitoring.
@ 2021-08-24  3:16 Dave Jones
  2021-08-24  3:18 ` Dave Jones
  2021-08-24  8:31 ` Peter Zijlstra
  0 siblings, 2 replies; 10+ messages in thread
From: Dave Jones @ 2021-08-24  3:16 UTC (permalink / raw)
  To: Linux Kernel; +Cc: Peter Zijlstra, Johannes Weiner, Linus Torvalds

I just noticed that in 5.14-rc running iotop, it complains on startup
that "CONFIG_TASK_DELAY_ACCT not enabled in kernel, cannot determine
SWAPIN and IO %" and sure enough a bunch of columns show up as
'unavailable'.

My suspicion is this commit, which I've not confirmed yet as it doesn't
cleanly revert.  I'll poke at it some more in the morning, but figured
I'd bring it up sans-evidence in the hope that I'll wake up to someone
having an Ah-ha moment.

	Dave


commit c5895d3f06cbb80ccb311f1dcb37074651030cb6
Author: Peter Zijlstra <peterz@infradead.org>
Date:   Tue May 4 22:43:42 2021 +0200

    sched: Simplify sched_info_on()
    
    The situation around sched_info is somewhat complicated, it is used by
    sched_stats and delayacct and, indirectly, kvm.
    
    If SCHEDSTATS=Y (but disabled by default) sched_info_on() is
    unconditionally true -- this is the case for all distro kernel configs
    I checked.
    
    If for some reason SCHEDSTATS=N, but TASK_DELAY_ACCT=Y, then
    sched_info_on() can return false when delayacct is disabled,
    presumably because there would be no other users left; except kvm is.
    
    Instead of complicating matters further by accurately accounting
    sched_stat and kvm state, simply unconditionally enable when
    SCHED_INFO=Y, matching the common distro case.
    
    Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
    Reviewed-by: Thomas Gleixner <tglx@linutronix.de>
    Reviewed-by: Ingo Molnar <mingo@kernel.org>
    Acked-by: Johannes Weiner <hannes@cmpxchg.org>
    Link: https://lkml.kernel.org/r/20210505111525.121458839@infradead.org

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: 5.14-rc breaks iotop swap io monitoring.
  2021-08-24  3:16 5.14-rc breaks iotop swap io monitoring Dave Jones
@ 2021-08-24  3:18 ` Dave Jones
  2021-08-24  8:31 ` Peter Zijlstra
  1 sibling, 0 replies; 10+ messages in thread
From: Dave Jones @ 2021-08-24  3:18 UTC (permalink / raw)
  To: Linux Kernel, Peter Zijlstra, Johannes Weiner, Linus Torvalds

On Mon, Aug 23, 2021 at 11:16:48PM -0400, Dave Jones wrote:
 > I just noticed that in 5.14-rc running iotop, it complains on startup
 > that "CONFIG_TASK_DELAY_ACCT not enabled in kernel, cannot determine
 > SWAPIN and IO %" and sure enough a bunch of columns show up as
 > 'unavailable'.
 > 
 > My suspicion is this commit, which I've not confirmed yet as it doesn't
 > cleanly revert.  I'll poke at it some more in the morning, but figured
 > I'd bring it up sans-evidence in the hope that I'll wake up to someone
 > having an Ah-ha moment.

Forgot to mention, my config has both SCHEDSTATS, SCHED_INFO and TASK_DELAY_ACCT set
to =y

 	Dave
 

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: 5.14-rc breaks iotop swap io monitoring.
  2021-08-24  3:16 5.14-rc breaks iotop swap io monitoring Dave Jones
  2021-08-24  3:18 ` Dave Jones
@ 2021-08-24  8:31 ` Peter Zijlstra
  2021-08-24 15:19   ` Dave Jones
  1 sibling, 1 reply; 10+ messages in thread
From: Peter Zijlstra @ 2021-08-24  8:31 UTC (permalink / raw)
  To: Dave Jones, Linux Kernel, Johannes Weiner, Linus Torvalds

On Mon, Aug 23, 2021 at 11:16:48PM -0400, Dave Jones wrote:
> I just noticed that in 5.14-rc running iotop, it complains on startup
> that "CONFIG_TASK_DELAY_ACCT not enabled in kernel, cannot determine
> SWAPIN and IO %" and sure enough a bunch of columns show up as
> 'unavailable'.
> 
> My suspicion is this commit, which I've not confirmed yet as it doesn't
> cleanly revert.  I'll poke at it some more in the morning, but figured
> I'd bring it up sans-evidence in the hope that I'll wake up to someone
> having an Ah-ha moment.

If you need DELAYACCT I'm thinking:

  e4042ad49235 ("delayacct: Default disabled")

and

  0cd7c741f01d ("delayacct: Add sysctl to enable at runtime")

might be more relevant..

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: 5.14-rc breaks iotop swap io monitoring.
  2021-08-24  8:31 ` Peter Zijlstra
@ 2021-08-24 15:19   ` Dave Jones
  2021-08-24 16:34     ` Linus Torvalds
  0 siblings, 1 reply; 10+ messages in thread
From: Dave Jones @ 2021-08-24 15:19 UTC (permalink / raw)
  To: Peter Zijlstra; +Cc: Linux Kernel, Johannes Weiner, Linus Torvalds

On Tue, Aug 24, 2021 at 10:31:16AM +0200, Peter Zijlstra wrote:
 > On Mon, Aug 23, 2021 at 11:16:48PM -0400, Dave Jones wrote:
 > > I just noticed that in 5.14-rc running iotop, it complains on startup
 > > that "CONFIG_TASK_DELAY_ACCT not enabled in kernel, cannot determine
 > > SWAPIN and IO %" and sure enough a bunch of columns show up as
 > > 'unavailable'.
 > > 
 > > My suspicion is this commit, which I've not confirmed yet as it doesn't
 > > cleanly revert.  I'll poke at it some more in the morning, but figured
 > > I'd bring it up sans-evidence in the hope that I'll wake up to someone
 > > having an Ah-ha moment.
 > 
 > If you need DELAYACCT I'm thinking:
 > 
 >   e4042ad49235 ("delayacct: Default disabled")
 > 
 > and
 > 
 >   0cd7c741f01d ("delayacct: Add sysctl to enable at runtime")

That does sound more relevant. however, even after echo 1  > /proc/sys/kernel/task_delayacct,
it still fails in the same way.

	Dave


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: 5.14-rc breaks iotop swap io monitoring.
  2021-08-24 15:19   ` Dave Jones
@ 2021-08-24 16:34     ` Linus Torvalds
  2021-08-24 17:35       ` Peter Zijlstra
  2021-08-24 21:33       ` Dave Jones
  0 siblings, 2 replies; 10+ messages in thread
From: Linus Torvalds @ 2021-08-24 16:34 UTC (permalink / raw)
  To: Dave Jones, Peter Zijlstra, Linux Kernel, Johannes Weiner,
	Linus Torvalds

On Tue, Aug 24, 2021 at 8:19 AM Dave Jones <davej@codemonkey.org.uk> wrote:
>
> On Tue, Aug 24, 2021 at 10:31:16AM +0200, Peter Zijlstra wrote:
>  >
>  > If you need DELAYACCT I'm thinking:
>  >
>  >   e4042ad49235 ("delayacct: Default disabled")
>  >
>  > and
>  >
>  >   0cd7c741f01d ("delayacct: Add sysctl to enable at runtime")
>
> That does sound more relevant. however, even after echo 1  > /proc/sys/kernel/task_delayacct,
> it still fails in the same way.

Hmm.

What happens if you boot with the 'delayacct' kernel parameter.

Even if you enable it at run-time, processes that have been started
before it was enabled won't actually have the 'tsk->delays'
allocation. So I'm not sure how effective the run-time thing is.

            Linus

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: 5.14-rc breaks iotop swap io monitoring.
  2021-08-24 16:34     ` Linus Torvalds
@ 2021-08-24 17:35       ` Peter Zijlstra
  2021-08-24 21:33       ` Dave Jones
  1 sibling, 0 replies; 10+ messages in thread
From: Peter Zijlstra @ 2021-08-24 17:35 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: Dave Jones, Linux Kernel, Johannes Weiner

On Tue, Aug 24, 2021 at 09:34:02AM -0700, Linus Torvalds wrote:

> Even if you enable it at run-time, processes that have been started
> before it was enabled won't actually have the 'tsk->delays'
> allocation. So I'm not sure how effective the run-time thing is.

Right; we *could* fix that by looping through the tasklist; grabbing a
task ref, doing the alloc and restarting until we've iterated the whole
tasklist and not found a task without a delays thing.

Didn't think it would be worth it, but can be done..

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: 5.14-rc breaks iotop swap io monitoring.
  2021-08-24 16:34     ` Linus Torvalds
  2021-08-24 17:35       ` Peter Zijlstra
@ 2021-08-24 21:33       ` Dave Jones
  2021-08-24 23:26         ` Linus Torvalds
  1 sibling, 1 reply; 10+ messages in thread
From: Dave Jones @ 2021-08-24 21:33 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: Peter Zijlstra, Linux Kernel, Johannes Weiner

On Tue, Aug 24, 2021 at 09:34:02AM -0700, Linus Torvalds wrote:
 > On Tue, Aug 24, 2021 at 8:19 AM Dave Jones <davej@codemonkey.org.uk> wrote:
 > >
 > > On Tue, Aug 24, 2021 at 10:31:16AM +0200, Peter Zijlstra wrote:
 > >  >
 > >  > If you need DELAYACCT I'm thinking:
 > >  >
 > >  >   e4042ad49235 ("delayacct: Default disabled")
 > >  >
 > >  > and
 > >  >
 > >  >   0cd7c741f01d ("delayacct: Add sysctl to enable at runtime")
 > >
 > > That does sound more relevant. however, even after echo 1  > /proc/sys/kernel/task_delayacct,
 > > it still fails in the same way.
 > 
 > Hmm.
 > 
 > What happens if you boot with the 'delayacct' kernel parameter.
 > 
 > Even if you enable it at run-time, processes that have been started
 > before it was enabled won't actually have the 'tsk->delays'
 > allocation. So I'm not sure how effective the run-time thing is.

Yeah that boot option does make iotop work again.

	Dave


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: 5.14-rc breaks iotop swap io monitoring.
  2021-08-24 21:33       ` Dave Jones
@ 2021-08-24 23:26         ` Linus Torvalds
  2021-08-25  0:29           ` Dave Jones
  2021-08-25  7:36           ` Peter Zijlstra
  0 siblings, 2 replies; 10+ messages in thread
From: Linus Torvalds @ 2021-08-24 23:26 UTC (permalink / raw)
  To: Dave Jones, Linus Torvalds, Peter Zijlstra, Linux Kernel,
	Johannes Weiner

On Tue, Aug 24, 2021 at 2:41 PM Dave Jones <davej@codemonkey.org.uk> wrote:
>
> On Tue, Aug 24, 2021 at 09:34:02AM -0700, Linus Torvalds wrote:
>  >
>  > What happens if you boot with the 'delayacct' kernel parameter.
>  >
>  > Even if you enable it at run-time, processes that have been started
>  > before it was enabled won't actually have the 'tsk->delays'
>  > allocation. So I'm not sure how effective the run-time thing is.
>
> Yeah that boot option does make iotop work again.

Ok, so it's that delay allocation thing is it.

I'm inclined to let it be and see if somebody else notices, and how
big of a deal it is.

The 'delayacct' kernel command line is an acceptable workaround if
this is something only a few people will even notice or care about.

I wonder how much people care about some statistics and iotop these days.

I also assume the swap stats still show up in "vmstat" etc, and that
it's just that iotop ended up using fancier interfaces?

Or do I assume wrongly?

           Linus

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: 5.14-rc breaks iotop swap io monitoring.
  2021-08-24 23:26         ` Linus Torvalds
@ 2021-08-25  0:29           ` Dave Jones
  2021-08-25  7:36           ` Peter Zijlstra
  1 sibling, 0 replies; 10+ messages in thread
From: Dave Jones @ 2021-08-25  0:29 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: Peter Zijlstra, Linux Kernel, Johannes Weiner

On Tue, Aug 24, 2021 at 04:26:59PM -0700, Linus Torvalds wrote:
 > >  > What happens if you boot with the 'delayacct' kernel parameter.
 > >  >
 > > Yeah that boot option does make iotop work again.
 > 
 > Ok, so it's that delay allocation thing is it.
 > 
 > I'm inclined to let it be and see if somebody else notices, and how
 > big of a deal it is.
 >
 > The 'delayacct' kernel command line is an acceptable workaround if
 > this is something only a few people will even notice or care about.
 > 
 > I wonder how much people care about some statistics and iotop these days.
 > 
 > I also assume the swap stats still show up in "vmstat" etc, and that
 > it's just that iotop ended up using fancier interfaces?

afaict, vmstat is fine (though none of my machines are using swap, so
they're just reporting 0)

	Dave


^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: 5.14-rc breaks iotop swap io monitoring.
  2021-08-24 23:26         ` Linus Torvalds
  2021-08-25  0:29           ` Dave Jones
@ 2021-08-25  7:36           ` Peter Zijlstra
  1 sibling, 0 replies; 10+ messages in thread
From: Peter Zijlstra @ 2021-08-25  7:36 UTC (permalink / raw)
  To: Linus Torvalds; +Cc: Dave Jones, Linux Kernel, Johannes Weiner

On Tue, Aug 24, 2021 at 04:26:59PM -0700, Linus Torvalds wrote:
> I also assume the swap stats still show up in "vmstat" etc, and that
> it's just that iotop ended up using fancier interfaces?

vmstat still works; I just started a fair number of stupid large
programs and I see swpd si and so changing.

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2021-08-25  7:37 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-24  3:16 5.14-rc breaks iotop swap io monitoring Dave Jones
2021-08-24  3:18 ` Dave Jones
2021-08-24  8:31 ` Peter Zijlstra
2021-08-24 15:19   ` Dave Jones
2021-08-24 16:34     ` Linus Torvalds
2021-08-24 17:35       ` Peter Zijlstra
2021-08-24 21:33       ` Dave Jones
2021-08-24 23:26         ` Linus Torvalds
2021-08-25  0:29           ` Dave Jones
2021-08-25  7:36           ` Peter Zijlstra

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).