LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* 2.6.24.2-rt2
@ 2008-02-22  0:24 Steven Rostedt
  2008-02-26 10:29 ` 2.6.24.2-rt2 Jan Kiszka
  0 siblings, 1 reply; 7+ messages in thread
From: Steven Rostedt @ 2008-02-22  0:24 UTC (permalink / raw)
  To: LKML, RT; +Cc: Ingo Molnar, Thomas Gleixner

We are pleased to announce the 2.6.24.2-rt2 tree, which can be
downloaded from the location:

  http://rt.et.redhat.com/download/

Information on the RT patch can be found at:

  http://rt.wiki.kernel.org/index.php/Main_Page

Changes since 2.6.24-rt1

  - ported to 2.6.24.2

  - *** New ftrace utility ***
      The old latency_tracer has now been replaced with the cleaned up
      version that is being prepared for mainline.

  - compiler warning fix (Shi Weihua)


to build a 2.6.24.2-rt2 tree, the following patches should be applied:

  http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.24.tar.bz2
  http://kernel.org/pub/linux/kernel/v2.6/patch-2.6.24.2.bz2
  http://rt.et.redhat.com/download/patch-2.6.24.2-rt2.bz2


And like always, my RT version of Matt Mackall's ketchup will get this
for you nicely:

  http://people.redhat.com/srostedt/rt/tools/ketchup-0.9.8-rt3


The broken out patches are also available.



-- Steve




^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: 2.6.24.2-rt2
  2008-02-22  0:24 2.6.24.2-rt2 Steven Rostedt
@ 2008-02-26 10:29 ` Jan Kiszka
  2008-02-26 12:01   ` 2.6.24.2-rt2 Jan Kiszka
  0 siblings, 1 reply; 7+ messages in thread
From: Jan Kiszka @ 2008-02-26 10:29 UTC (permalink / raw)
  To: Steven Rostedt; +Cc: LKML, RT, Ingo Molnar, Thomas Gleixner

Steven Rostedt wrote:
> We are pleased to announce the 2.6.24.2-rt2 tree, which can be
> downloaded from the location:
> 
>   http://rt.et.redhat.com/download/
> 
> Information on the RT patch can be found at:
> 
>   http://rt.wiki.kernel.org/index.php/Main_Page
> 
> Changes since 2.6.24-rt1
> 
>   - ported to 2.6.24.2
> 
>   - *** New ftrace utility ***
>       The old latency_tracer has now been replaced with the cleaned up
>       version that is being prepared for mainline.
> 
>   - compiler warning fix (Shi Weihua)

This important fix is still missing:

http://lkml.org/lkml/2008/2/5/295


At this chance: We still see the same unbalanced sched-other load on our
NUMA box as Gernot once reported [1]:

top - 11:19:20 up 4 min,  1 user,  load average: 29.52, 9.54, 3.37
Tasks: 502 total,  41 running, 461 sleeping,   0 stopped,   0 zombie
Cpu0  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu2  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu4  :  0.0%us,  0.3%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu5  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu6  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu7  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu8  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu9  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu10 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu11 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu12 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu13 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu14 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu15 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  65513284k total,  1032032k used, 64481252k free,     6444k buffers
Swap:  3204896k total,        0k used,  3204896k free,    37312k cached

 PR   PID  NI  VIRT  SHR S %CPU %MEM    TIME+  COMMAND
 20  5603   0  705m  464 R  100  0.3   1:18.19 load-balance-te
 20  5604   0  705m  464 R  100  0.3   1:18.16 load-balance-te
 20  5605   0  705m  464 R  100  0.3   1:18.18 load-balance-te
 20  5608   0  705m  464 R  100  0.3   1:18.18 load-balance-te
 20  5611   0  705m  464 R   25  0.3   0:19.58 load-balance-te
 20  5620   0  705m  464 R   25  0.3   0:19.54 load-balance-te
 20  5606   0  705m  464 R   25  0.3   0:19.56 load-balance-te
 20  5616   0  705m  464 R   25  0.3   0:19.54 load-balance-te
 20  5607   0  705m  464 R   20  0.3   0:15.64 load-balance-te
 20  5609   0  705m  464 R   20  0.3   0:15.66 load-balance-te
 20  5614   0  705m  464 R   20  0.3   0:15.68 load-balance-te
 20  5617   0  705m  464 R   20  0.3   0:15.64 load-balance-te
 20  5619   0  705m  464 R   20  0.3   0:15.64 load-balance-te
 20  5610   0  705m  464 R   17  0.3   0:13.10 load-balance-te
 20  5618   0  705m  464 R   17  0.3   0:13.04 load-balance-te
 20  5621   0  705m  464 R   17  0.3   0:13.02 load-balance-te
 20  5622   0  705m  464 R   17  0.3   0:13.02 load-balance-te
 20  5623   0  705m  464 R   17  0.3   0:13.06 load-balance-te
 20  5624   0  705m  464 R   17  0.3   0:13.02 load-balance-te
 20  5615   0  705m  464 R    5  0.3   0:03.76 load-balance-te
 20  5633   0  705m  464 R    5  0.3   0:03.72 load-balance-te
 20  5634   0  705m  464 R    5  0.3   0:03.82 load-balance-te
 20  5635   0  705m  464 R    5  0.3   0:03.74 load-balance-te
 20  5636   0  705m  464 R    5  0.3   0:03.72 load-balance-te
 20  5638   0  705m  464 R    5  0.3   0:03.72 load-balance-te
 20  5640   0  705m  464 R    5  0.3   0:03.72 load-balance-te
 20  5612   0  705m  464 R    5  0.3   0:03.78 load-balance-te
 20  5613   0  705m  464 R    5  0.3   0:03.78 load-balance-te
 20  5625   0  705m  464 R    5  0.3   0:03.74 load-balance-te
 20  5626   0  705m  464 R    5  0.3   0:03.74 load-balance-te
 20  5627   0  705m  464 R    5  0.3   0:03.82 load-balance-te
 20  5628   0  705m  464 R    5  0.3   0:03.72 load-balance-te
 20  5629   0  705m  464 R    5  0.3   0:03.72 load-balance-te
 20  5630   0  705m  464 R    5  0.3   0:03.72 load-balance-te
 20  5631   0  705m  464 R    5  0.3   0:03.72 load-balance-te
 20  5632   0  705m  464 R    5  0.3   0:03.72 load-balance-te
 20  5637   0  705m  464 R    5  0.3   0:03.72 load-balance-te
 20  5639   0  705m  464 R    5  0.3   0:03.74 load-balance-te
 20  5641   0  705m  464 R    5  0.3   0:03.70 load-balance-te
 20  5642   0  705m  464 R    5  0.3   0:03.80 load-balance-te

I just accidentally compiled the kernel with NR_CPUS=8 first, and then
the load was balanced. Weird.

Some time ago I tried to dig into this but didn't get far due to the
limited bisectability of -rt (was at least true for .23-rtX) and limited
time. But based on this search, I suspect now that the issue is
introduced via some patch somewhere at the end of the series. If anyone
has a good pointer what to check, I would try to look into this again.

Jan

[1] http://thread.gmane.org/gmane.linux.rt.user/1640/focus=1746

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: 2.6.24.2-rt2
  2008-02-26 10:29 ` 2.6.24.2-rt2 Jan Kiszka
@ 2008-02-26 12:01   ` Jan Kiszka
  2008-02-26 15:15     ` 2.6.24.2-rt2 Steven Rostedt
  0 siblings, 1 reply; 7+ messages in thread
From: Jan Kiszka @ 2008-02-26 12:01 UTC (permalink / raw)
  To: Steven Rostedt; +Cc: LKML, RT, Ingo Molnar, Thomas Gleixner

Jan Kiszka wrote:
> At this chance: We still see the same unbalanced sched-other load on our
> NUMA box as Gernot once reported [1]:
> 
> top - 11:19:20 up 4 min,  1 user,  load average: 29.52, 9.54, 3.37
> Tasks: 502 total,  41 running, 461 sleeping,   0 stopped,   0 zombie
> Cpu0  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu1  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu2  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu3  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu4  :  0.0%us,  0.3%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu5  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu6  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu7  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu8  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu9  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu10 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu11 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu12 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu13 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu14 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu15 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Mem:  65513284k total,  1032032k used, 64481252k free,     6444k buffers
> Swap:  3204896k total,        0k used,  3204896k free,    37312k cached
> 

ETOOMANYKERNELS, this was from 2.6.23.12-rt14. 2.6.24.2-rt2 shows a
different patter under identical load:

top - 12:55:27 up 2 min,  1 user,  load average: 9.97, 2.42, 0.81
Tasks: 491 total,  42 running, 449 sleeping,   0 stopped,   0 zombie
Cpu0  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu1  : 99.7%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.3%si,  0.0%st
Cpu2  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu3  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu4  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu5  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu6  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu7  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu8  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu9  :  0.3%us,  0.3%sy,  0.0%ni, 99.3%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu10 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu11 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu12 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu13 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu14 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Cpu15 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
Mem:  65512480k total,   580704k used, 64931776k free,     8964k buffers
Swap:  3204896k total,        0k used,  3204896k free,   129720k cached

Jan

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: 2.6.24.2-rt2
  2008-02-26 12:01   ` 2.6.24.2-rt2 Jan Kiszka
@ 2008-02-26 15:15     ` Steven Rostedt
  2008-02-26 15:38       ` 2.6.24.2-rt2 Jan Kiszka
  0 siblings, 1 reply; 7+ messages in thread
From: Steven Rostedt @ 2008-02-26 15:15 UTC (permalink / raw)
  To: Jan Kiszka; +Cc: LKML, RT, Ingo Molnar, Thomas Gleixner



On Tue, 26 Feb 2008, Jan Kiszka wrote:

> Jan Kiszka wrote:
> > At this chance: We still see the same unbalanced sched-other load on our
> > NUMA box as Gernot once reported [1]:
> >
> > top - 11:19:20 up 4 min,  1 user,  load average: 29.52, 9.54, 3.37
> > Tasks: 502 total,  41 running, 461 sleeping,   0 stopped,   0 zombie
> > Cpu0  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> > Cpu1  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> > Cpu2  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> > Cpu3  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> > Cpu4  :  0.0%us,  0.3%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> > Cpu5  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> > Cpu6  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> > Cpu7  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> > Cpu8  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> > Cpu9  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> > Cpu10 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> > Cpu11 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> > Cpu12 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> > Cpu13 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> > Cpu14 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> > Cpu15 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> > Mem:  65513284k total,  1032032k used, 64481252k free,     6444k buffers
> > Swap:  3204896k total,        0k used,  3204896k free,    37312k cached
> >
>
> ETOOMANYKERNELS, this was from 2.6.23.12-rt14. 2.6.24.2-rt2 shows a
> different patter under identical load:

There has been CFS updates, which may account for the differences. Seems
better though.

>
> top - 12:55:27 up 2 min,  1 user,  load average: 9.97, 2.42, 0.81
> Tasks: 491 total,  42 running, 449 sleeping,   0 stopped,   0 zombie
> Cpu0  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu1  : 99.7%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.3%si,  0.0%st
> Cpu2  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu3  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu4  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu5  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu6  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu7  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu8  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu9  :  0.3%us,  0.3%sy,  0.0%ni, 99.3%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu10 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu11 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu12 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu13 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu14 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Cpu15 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
> Mem:  65512480k total,   580704k used, 64931776k free,     8964k buffers
> Swap:  3204896k total,        0k used,  3204896k free,   129720k cached
>

What's the NUMA topology? What tasks are running, and at what priorities?

Those three idle CPUS, should they have tasks running on them?

-- Steve


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: 2.6.24.2-rt2
  2008-02-26 15:15     ` 2.6.24.2-rt2 Steven Rostedt
@ 2008-02-26 15:38       ` Jan Kiszka
  2008-02-26 16:55         ` 2.6.24.2-rt2 Steven Rostedt
  0 siblings, 1 reply; 7+ messages in thread
From: Jan Kiszka @ 2008-02-26 15:38 UTC (permalink / raw)
  To: Steven Rostedt; +Cc: LKML, RT, Ingo Molnar, Thomas Gleixner

Steven Rostedt wrote:
> 
> On Tue, 26 Feb 2008, Jan Kiszka wrote:
> 
>> Jan Kiszka wrote:
>>> At this chance: We still see the same unbalanced sched-other load on our
>>> NUMA box as Gernot once reported [1]:
>>>
>>> top - 11:19:20 up 4 min,  1 user,  load average: 29.52, 9.54, 3.37
>>> Tasks: 502 total,  41 running, 461 sleeping,   0 stopped,   0 zombie
>>> Cpu0  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>>> Cpu1  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>>> Cpu2  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>>> Cpu3  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>>> Cpu4  :  0.0%us,  0.3%sy,  0.0%ni, 99.7%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>>> Cpu5  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>>> Cpu6  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>>> Cpu7  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>>> Cpu8  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>>> Cpu9  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>>> Cpu10 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>>> Cpu11 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>>> Cpu12 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>>> Cpu13 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>>> Cpu14 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>>> Cpu15 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>>> Mem:  65513284k total,  1032032k used, 64481252k free,     6444k buffers
>>> Swap:  3204896k total,        0k used,  3204896k free,    37312k cached
>>>
>> ETOOMANYKERNELS, this was from 2.6.23.12-rt14. 2.6.24.2-rt2 shows a
>> different patter under identical load:
> 
> There has been CFS updates, which may account for the differences. Seems
> better though.
> 
>> top - 12:55:27 up 2 min,  1 user,  load average: 9.97, 2.42, 0.81
>> Tasks: 491 total,  42 running, 449 sleeping,   0 stopped,   0 zombie
>> Cpu0  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>> Cpu1  : 99.7%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.3%si,  0.0%st
>> Cpu2  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>> Cpu3  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>> Cpu4  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>> Cpu5  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>> Cpu6  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>> Cpu7  :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>> Cpu8  :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>> Cpu9  :  0.3%us,  0.3%sy,  0.0%ni, 99.3%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>> Cpu10 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>> Cpu11 :  0.0%us,  0.0%sy,  0.0%ni,100.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>> Cpu12 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>> Cpu13 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>> Cpu14 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>> Cpu15 :100.0%us,  0.0%sy,  0.0%ni,  0.0%id,  0.0%wa,  0.0%hi,  0.0%si,  0.0%st
>> Mem:  65512480k total,   580704k used, 64931776k free,     8964k buffers
>> Swap:  3204896k total,        0k used,  3204896k free,   129720k cached
>>
> 
> What's the NUMA topology?

4 nodes. I'm not sure if it is really NUMA related, but the same kernel
runs that test as expected on a non-NUMA 2x2 box.

> What tasks are running, and at what priorities?

40 pthreads, created with default parameters from a main thread which
runs with default parameters as well. The threads simply run endless loops.

> 
> Those three idle CPUS, should they have tasks running on them?

For sure, given the overload situation of the system (40x full load vs.
16 cores). Neither did we fiddle with any parameter of the system
(knowingly, its a standard openSUSE 10.3 underneath) nor did we set
thread affinities.

Jan

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: 2.6.24.2-rt2
  2008-02-26 15:38       ` 2.6.24.2-rt2 Jan Kiszka
@ 2008-02-26 16:55         ` Steven Rostedt
  2008-02-26 17:15           ` 2.6.24.2-rt2 Jan Kiszka
  0 siblings, 1 reply; 7+ messages in thread
From: Steven Rostedt @ 2008-02-26 16:55 UTC (permalink / raw)
  To: Jan Kiszka; +Cc: LKML, RT, Ingo Molnar, Thomas Gleixner



On Tue, 26 Feb 2008, Jan Kiszka wrote:
> >>
> >
> > What's the NUMA topology?
>
> 4 nodes. I'm not sure if it is really NUMA related, but the same kernel
> runs that test as expected on a non-NUMA 2x2 box.
>
> > What tasks are running, and at what priorities?
>
> 40 pthreads, created with default parameters from a main thread which
> runs with default parameters as well. The threads simply run endless loops.
>
> >
> > Those three idle CPUS, should they have tasks running on them?
>
> For sure, given the overload situation of the system (40x full load vs.
> 16 cores). Neither did we fiddle with any parameter of the system
> (knowingly, its a standard openSUSE 10.3 underneath) nor did we set
> thread affinities.
>

Do you get different behaviour with 2.6.24.2?

-- Steve


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: 2.6.24.2-rt2
  2008-02-26 16:55         ` 2.6.24.2-rt2 Steven Rostedt
@ 2008-02-26 17:15           ` Jan Kiszka
  0 siblings, 0 replies; 7+ messages in thread
From: Jan Kiszka @ 2008-02-26 17:15 UTC (permalink / raw)
  To: Steven Rostedt; +Cc: LKML, RT, Ingo Molnar, Thomas Gleixner

Steven Rostedt wrote:
> 
> On Tue, 26 Feb 2008, Jan Kiszka wrote:
>>> What's the NUMA topology?
>> 4 nodes. I'm not sure if it is really NUMA related, but the same kernel
>> runs that test as expected on a non-NUMA 2x2 box.
>>
>>> What tasks are running, and at what priorities?
>> 40 pthreads, created with default parameters from a main thread which
>> runs with default parameters as well. The threads simply run endless loops.
>>
>>> Those three idle CPUS, should they have tasks running on them?
>> For sure, given the overload situation of the system (40x full load vs.
>> 16 cores). Neither did we fiddle with any parameter of the system
>> (knowingly, its a standard openSUSE 10.3 underneath) nor did we set
>> thread affinities.
>>
> 
> Do you get different behaviour with 2.6.24.2?

Last time I checked mainline (I think 2.6.24), it was fine. It was
definitely fine for 2.6.23. But I'm going to revalidate this once the
machine is free again (tomorrow).

Jan

-- 
Siemens AG, Corporate Technology, CT SE 2
Corporate Competence Center Embedded Linux

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2008-02-26 17:16 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2008-02-22  0:24 2.6.24.2-rt2 Steven Rostedt
2008-02-26 10:29 ` 2.6.24.2-rt2 Jan Kiszka
2008-02-26 12:01   ` 2.6.24.2-rt2 Jan Kiszka
2008-02-26 15:15     ` 2.6.24.2-rt2 Steven Rostedt
2008-02-26 15:38       ` 2.6.24.2-rt2 Jan Kiszka
2008-02-26 16:55         ` 2.6.24.2-rt2 Steven Rostedt
2008-02-26 17:15           ` 2.6.24.2-rt2 Jan Kiszka

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).