LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* SLUB 0:1 SLAB (OOM during massive parallel kernel builds)
@ 2007-10-23 18:16 Alexey Dobriyan
  2007-10-23 19:35 ` Christoph Lameter
  0 siblings, 1 reply; 14+ messages in thread
From: Alexey Dobriyan @ 2007-10-23 18:16 UTC (permalink / raw)
  To: clameter, penberg; +Cc: linux-kernel, linux-mm

Abstract:

Nasty OOM killings appear during massive parallel kernel builds with
SLUB, but not with SLAB. By nasty I mean, cc1 processes are killed --
object files in .ccache and tree are corrupted which makes me to
blow up them entirely.


Setup:

92G ext2 partition mounted with noatime. .ccache, master tree, object
trees, build logs residing on it. 2G of RAM, 1G of swap partition.
CFQ scheduler. Core 2 Duo E6400. x86_64 arch.


Workload:

build ~250 configs some fat, some not, in parallel from one source
using O= and save logs to separate dir. This is what I do regularly here.


Observations:

With SLAB this workload never went to OOM killer.
With SLUB and pretty much all debugging enabled, it finishes to the end
(albeit slowly).
With SLUB and no debugging, OOM killer kicks in.

After I notice it at evening, e.g ssh to box doesn't work -- new OOM killings
appear.


Below is dmesg from fresh boot,  OOM messages and .config. Please, give me a
clue on how to proceed from there. If needed I can prepare cut-down scripts
for reproducing and more precise description of setup.

================================================================================
[this is the first OOM]
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 142   Cold: hi:   62, btch:  15 usd:  53
CPU    1: Hot: hi:  186, btch:  31 usd: 159   Cold: hi:   62, btch:  15 usd:  54
Active:386969 inactive:66065 dirty:0 writeback:0 unstable:0
 free:3431 slab:11685 mapped:243 pagetables:14621 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:2008kB inactive:1172kB present:12744kB pages_scanned:6852 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5688kB min:5708kB low:7132kB high:8560kB active:1548100kB inactive:261040kB present:2051880kB pages_scanned:12798112 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 13*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 518*4kB 6*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5688kB
Swap cache: add 4908248, delete 4908248, find 699858/1120395, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
17925 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 188662 or a child
Killed process 25288 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025e7bc>] do_mmap_pgoff+0x2fc/0x360
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 146   Cold: hi:   62, btch:  15 usd:  61
CPU    1: Hot: hi:  186, btch:  31 usd: 140   Cold: hi:   62, btch:  15 usd:  58
Active:306488 inactive:146569 dirty:0 writeback:0 unstable:0
 free:3415 slab:11631 mapped:2 pagetables:14611 bounce:0
DMA free:8044kB min:32kB low:40kB high:48kB active:1596kB inactive:1576kB present:12744kB pages_scanned:14865 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5616kB min:5708kB low:7132kB high:8560kB active:1226588kB inactive:582396kB present:2051880kB pages_scanned:13650148 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 15*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8044kB
DMA32: 498*4kB 5*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5616kB
Swap cache: add 4908570, delete 4908570, find 699873/1120438, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9622 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 187901 or a child
Killed process 25289 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff802de57e>] _atomic_dec_and_lock+0x4e/0x70
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 147   Cold: hi:   62, btch:  15 usd:  56
CPU    1: Hot: hi:  186, btch:  31 usd: 179   Cold: hi:   62, btch:  15 usd:  27
Active:423015 inactive:30323 dirty:0 writeback:0 unstable:0
 free:3429 slab:11630 mapped:2 pagetables:14605 bounce:0
DMA free:8044kB min:32kB low:40kB high:48kB active:1600kB inactive:1572kB present:12744kB pages_scanned:14993 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5672kB min:5708kB low:7132kB high:8560kB active:1690460kB inactive:119720kB present:2051880kB pages_scanned:27640275 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 15*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8044kB
DMA32: 500*4kB 11*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5672kB
Swap cache: add 4908633, delete 4908633, find 699873/1120439, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9326 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 187140 or a child
Killed process 25291 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 159   Cold: hi:   62, btch:  15 usd:  39
CPU    1: Hot: hi:  186, btch:  31 usd: 179   Cold: hi:   62, btch:  15 usd:  57
Active:325742 inactive:127034 dirty:0 writeback:0 unstable:0
 free:3444 slab:11630 mapped:2 pagetables:14599 bounce:0
DMA free:8044kB min:32kB low:40kB high:48kB active:1600kB inactive:1572kB present:12744kB pages_scanned:14993 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5732kB min:5708kB low:7132kB high:8560kB active:1301244kB inactive:506676kB present:2051880kB pages_scanned:3441512 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 15*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8044kB
DMA32: 509*4kB 14*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5732kB
Swap cache: add 4908693, delete 4908693, find 699873/1120439, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9818 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 186379 or a child
Killed process 25292 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025e7bc>] do_mmap_pgoff+0x2fc/0x360
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 155   Cold: hi:   62, btch:  15 usd:  29
CPU    1: Hot: hi:  186, btch:  31 usd: 161   Cold: hi:   62, btch:  15 usd:  61
Active:11064 inactive:441279 dirty:0 writeback:0 unstable:0
 free:3445 slab:11629 mapped:2 pagetables:14593 bounce:0
DMA free:8044kB min:32kB low:40kB high:48kB active:1600kB inactive:1572kB present:12744kB pages_scanned:15057 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5736kB min:5708kB low:7132kB high:8560kB active:42656kB inactive:1763544kB present:2051880kB pages_scanned:4028526 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 15*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8044kB
DMA32: 520*4kB 9*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5736kB
Swap cache: add 4908753, delete 4908753, find 699873/1120439, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10207 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 185618 or a child
Killed process 25293 (akmk)
as invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025c43c>] remove_vma+0x4c/0x60
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 151   Cold: hi:   62, btch:  15 usd:  57
CPU    1: Hot: hi:  186, btch:  31 usd: 165   Cold: hi:   62, btch:  15 usd:  47
Active:363161 inactive:89780 dirty:0 writeback:0 unstable:0
 free:3430 slab:11629 mapped:2 pagetables:14587 bounce:0
DMA free:8044kB min:32kB low:40kB high:48kB active:1604kB inactive:1568kB present:12744kB pages_scanned:15185 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5676kB min:5708kB low:7132kB high:8560kB active:1453272kB inactive:355248kB present:2051880kB pages_scanned:37350819 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 15*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8044kB
DMA32: 511*4kB 6*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5676kB
Swap cache: add 4908822, delete 4908822, find 699873/1120439, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9525 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 184857 or a child
Killed process 25297 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 148   Cold: hi:   62, btch:  15 usd:  21
CPU    1: Hot: hi:  186, btch:  31 usd: 180   Cold: hi:   62, btch:  15 usd:  57
Active:256272 inactive:196720 dirty:0 writeback:0 unstable:0
 free:3430 slab:11627 mapped:2 pagetables:14581 bounce:0
DMA free:8044kB min:32kB low:40kB high:48kB active:1604kB inactive:1568kB present:12744kB pages_scanned:15249 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5676kB min:5708kB low:7132kB high:8560kB active:1023484kB inactive:785312kB present:2051880kB pages_scanned:4581896 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 15*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8044kB
DMA32: 513*4kB 5*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5676kB
Swap cache: add 4908882, delete 4908882, find 699873/1120439, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9399 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 184096 or a child
Killed process 25302 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 159   Cold: hi:   62, btch:  15 usd:  44
CPU    1: Hot: hi:  186, btch:  31 usd: 188   Cold: hi:   62, btch:  15 usd:  57
Active:304839 inactive:148090 dirty:0 writeback:0 unstable:0
 free:3429 slab:11627 mapped:2 pagetables:14575 bounce:0
DMA free:8044kB min:32kB low:40kB high:48kB active:1588kB inactive:1568kB present:12744kB pages_scanned:15441 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5672kB min:5708kB low:7132kB high:8560kB active:1215336kB inactive:593352kB present:2051880kB pages_scanned:13164262 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 15*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8044kB
DMA32: 512*4kB 5*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5672kB
Swap cache: add 4908968, delete 4908968, find 699873/1120441, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9331 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 183335 or a child
Killed process 25304 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff802de57e>] _atomic_dec_and_lock+0x4e/0x70
 [<ffffffff80282712>] dput+0xb2/0x140
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 155   Cold: hi:   62, btch:  15 usd:  30
CPU    1: Hot: hi:  186, btch:  31 usd: 165   Cold: hi:   62, btch:  15 usd:  53
Active:169953 inactive:282282 dirty:0 writeback:0 unstable:0
 free:3418 slab:11628 mapped:2 pagetables:14569 bounce:0
DMA free:8044kB min:32kB low:40kB high:48kB active:1604kB inactive:1568kB present:12744kB pages_scanned:15505 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5628kB min:5708kB low:7132kB high:8560kB active:678208kB inactive:1127560kB present:2051880kB pages_scanned:4001527 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 15*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8044kB
DMA32: 498*4kB 5*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5616kB
Swap cache: add 4909029, delete 4909029, find 699873/1120441, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10125 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 182574 or a child
Killed process 25305 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 180   Cold: hi:   62, btch:  15 usd:  53
CPU    1: Hot: hi:  186, btch:  31 usd: 164   Cold: hi:   62, btch:  15 usd:  59
Active:11285 inactive:440793 dirty:0 writeback:0 unstable:0
 free:3429 slab:11626 mapped:1 pagetables:14563 bounce:0
DMA free:8040kB min:32kB low:40kB high:48kB active:1724kB inactive:1428kB present:12744kB pages_scanned:7123 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5676kB min:5708kB low:7132kB high:8560kB active:45772kB inactive:1759440kB present:2051880kB pages_scanned:4827940 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 14*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8040kB
DMA32: 513*4kB 5*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5676kB
Swap cache: add 4909091, delete 4909091, find 699873/1120441, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10098 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 181813 or a child
Killed process 25307 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 163   Cold: hi:   62, btch:  15 usd:  49
CPU    1: Hot: hi:  186, btch:  31 usd: 160   Cold: hi:   62, btch:  15 usd:  61
Active:140594 inactive:311525 dirty:0 writeback:0 unstable:0
 free:3430 slab:11625 mapped:1 pagetables:14558 bounce:0
DMA free:8040kB min:32kB low:40kB high:48kB active:1724kB inactive:1428kB present:12744kB pages_scanned:7315 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5680kB min:5708kB low:7132kB high:8560kB active:560652kB inactive:1244672kB present:2051880kB pages_scanned:21209367 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 14*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8040kB
DMA32: 514*4kB 5*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5680kB
Swap cache: add 4909153, delete 4909153, find 699873/1120441, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10009 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 181052 or a child
Killed process 25308 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025e7bc>] do_mmap_pgoff+0x2fc/0x360
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 155   Cold: hi:   62, btch:  15 usd:  60
CPU    1: Hot: hi:  186, btch:  31 usd: 172   Cold: hi:   62, btch:  15 usd:  59
Active:163601 inactive:288519 dirty:0 writeback:0 unstable:0
 free:3445 slab:11625 mapped:2 pagetables:14553 bounce:0
DMA free:8040kB min:32kB low:40kB high:48kB active:1724kB inactive:1428kB present:12744kB pages_scanned:7379 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5740kB min:5708kB low:7132kB high:8560kB active:652680kB inactive:1152648kB present:2051880kB pages_scanned:3581883 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 14*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8040kB
DMA32: 529*4kB 5*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5740kB
Swap cache: add 4909215, delete 4909215, find 699873/1120441, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9926 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 180291 or a child
Killed process 25309 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff802de57e>] _atomic_dec_and_lock+0x4e/0x70
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 115   Cold: hi:   62, btch:  15 usd:  42
CPU    1: Hot: hi:  186, btch:  31 usd:  58   Cold: hi:   62, btch:  15 usd:  59
Active:8992 inactive:444596 dirty:0 writeback:0 unstable:0
 free:3422 slab:11588 mapped:1 pagetables:14545 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1556kB inactive:1248kB present:12744kB pages_scanned:14668 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5656kB min:5708kB low:7132kB high:8560kB active:34412kB inactive:1777136kB present:2051880kB pages_scanned:8923551 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 514*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5656kB
Swap cache: add 4931119, delete 4931119, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9791 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 179530 or a child
Killed process 25313 (akmk)
fixdep invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802e0c03>] prio_tree_insert+0x23/0x270
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8026b0f5>] __slab_free+0x1e5/0x330
 [<ffffffff8025c43c>] remove_vma+0x4c/0x60
 [<ffffffff802de57e>] _atomic_dec_and_lock+0x4e/0x70
 [<ffffffff80282712>] dput+0xb2/0x140
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 114   Cold: hi:   62, btch:  15 usd:  14
CPU    1: Hot: hi:  186, btch:  31 usd:  72   Cold: hi:   62, btch:  15 usd:  30
Active:29737 inactive:423544 dirty:0 writeback:0 unstable:0
 free:3433 slab:11586 mapped:1 pagetables:14539 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1696kB inactive:1396kB present:12744kB pages_scanned:14830 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5700kB min:5708kB low:7132kB high:8560kB active:117252kB inactive:1692780kB present:2051880kB pages_scanned:3553979 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 516*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5664kB
Swap cache: add 4931180, delete 4931180, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10157 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 178769 or a child
Killed process 25318 (akmk)
bash invoked oom-killer: gfp_mask=0x1280d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802593a5>] handle_mm_fault+0x485/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8026b0f5>] __slab_free+0x1e5/0x330
 [<ffffffff8025c43c>] remove_vma+0x4c/0x60
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 114   Cold: hi:   62, btch:  15 usd:  52
CPU    1: Hot: hi:  186, btch:  31 usd:  84   Cold: hi:   62, btch:  15 usd:  48
Active:190765 inactive:262200 dirty:0 writeback:0 unstable:0
 free:3424 slab:11586 mapped:2 pagetables:14533 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1696kB inactive:1396kB present:12744kB pages_scanned:15022 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5664kB min:5708kB low:7132kB high:8560kB active:759060kB inactive:1049708kB present:2051880kB pages_scanned:7773323 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 516*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5664kB
Swap cache: add 4931241, delete 4931241, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10263 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 178008 or a child
Killed process 25321 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 114   Cold: hi:   62, btch:  15 usd:  35
CPU    1: Hot: hi:  186, btch:  31 usd:  94   Cold: hi:   62, btch:  15 usd:  50
Active:161540 inactive:291160 dirty:0 writeback:0 unstable:0
 free:3424 slab:11587 mapped:1 pagetables:14527 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1696kB inactive:1396kB present:12744kB pages_scanned:15086 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5664kB min:5708kB low:7132kB high:8560kB active:644464kB inactive:1163132kB present:2051880kB pages_scanned:35575277 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 514*4kB 3*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5664kB
Swap cache: add 4931301, delete 4931301, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10530 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 177247 or a child
Killed process 25326 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 114   Cold: hi:   62, btch:  15 usd:  55
CPU    1: Hot: hi:  186, btch:  31 usd: 107   Cold: hi:   62, btch:  15 usd:  14
Active:40327 inactive:412498 dirty:0 writeback:0 unstable:0
 free:3433 slab:11587 mapped:1 pagetables:14521 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1684kB inactive:1392kB present:12744kB pages_scanned:15214 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5700kB min:5708kB low:7132kB high:8560kB active:157320kB inactive:1650776kB present:2051880kB pages_scanned:3954392 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 516*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5664kB
Swap cache: add 4931361, delete 4931361, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10400 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 176486 or a child
Killed process 25331 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025e7bc>] do_mmap_pgoff+0x2fc/0x360
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 113   Cold: hi:   62, btch:  15 usd:  47
CPU    1: Hot: hi:  186, btch:  31 usd: 119   Cold: hi:   62, btch:  15 usd:  49
Active:363535 inactive:88985 dirty:0 writeback:0 unstable:0
 free:3484 slab:11587 mapped:1 pagetables:14516 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1700kB inactive:1392kB present:12744kB pages_scanned:15214 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5904kB min:5708kB low:7132kB high:8560kB active:1452440kB inactive:354548kB present:2051880kB pages_scanned:4327701 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 566*4kB 7*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5904kB
Swap cache: add 4931422, delete 4931422, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10564 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 175725 or a child
Killed process 25339 (akmk)
fixdep invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802e0c03>] prio_tree_insert+0x23/0x270
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025e7bc>] do_mmap_pgoff+0x2fc/0x360
 [<ffffffff802735be>] sys_newfstat+0x2e/0x50
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 114   Cold: hi:   62, btch:  15 usd:  50
CPU    1: Hot: hi:  186, btch:  31 usd: 129   Cold: hi:   62, btch:  15 usd:  43
Active:15013 inactive:437762 dirty:0 writeback:0 unstable:0
 free:3424 slab:11586 mapped:1 pagetables:14510 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1700kB inactive:1392kB present:12744kB pages_scanned:15278 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5664kB min:5708kB low:7132kB high:8560kB active:58352kB inactive:1749656kB present:2051880kB pages_scanned:21495882 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 516*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5664kB
Swap cache: add 4931482, delete 4931482, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10239 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 174964 or a child
Killed process 25347 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff80282705>] dput+0xa5/0x140
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 112   Cold: hi:   62, btch:  15 usd:  60
CPU    1: Hot: hi:  186, btch:  31 usd: 142   Cold: hi:   62, btch:  15 usd:  58
Active:156024 inactive:296851 dirty:0 writeback:0 unstable:0
 free:3424 slab:11585 mapped:1 pagetables:14504 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1704kB inactive:1388kB present:12744kB pages_scanned:15470 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5664kB min:5708kB low:7132kB high:8560kB active:624500kB inactive:1183968kB present:2051880kB pages_scanned:32711791 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 516*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5664kB
Swap cache: add 4931542, delete 4931542, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10070 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 174203 or a child
Killed process 25354 (akmk)
rm invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8026b0f5>] __slab_free+0x1e5/0x330
 [<ffffffff8025c43c>] remove_vma+0x4c/0x60
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 108   Cold: hi:   62, btch:  15 usd:  48
CPU    1: Hot: hi:  186, btch:  31 usd: 155   Cold: hi:   62, btch:  15 usd:  58
Active:382351 inactive:70856 dirty:0 writeback:0 unstable:0
 free:3424 slab:11584 mapped:1 pagetables:14499 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1704kB inactive:1388kB present:12744kB pages_scanned:15534 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5664kB min:5708kB low:7132kB high:8560kB active:1527700kB inactive:282036kB present:2051880kB pages_scanned:6581042 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 504*4kB 8*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5664kB
Swap cache: add 4931603, delete 4931603, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9703 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 173442 or a child
Killed process 25369 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025c43c>] remove_vma+0x4c/0x60
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 118   Cold: hi:   62, btch:  15 usd:  60
CPU    1: Hot: hi:  186, btch:  31 usd: 162   Cold: hi:   62, btch:  15 usd:  20
Active:256144 inactive:196747 dirty:0 writeback:0 unstable:0
 free:3424 slab:11585 mapped:1 pagetables:14493 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1704kB inactive:1408kB present:12744kB pages_scanned:15598 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5664kB min:5708kB low:7132kB high:8560kB active:1020568kB inactive:788012kB present:2051880kB pages_scanned:5734170 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 506*4kB 7*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5664kB
Swap cache: add 4931664, delete 4931664, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9917 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 172681 or a child
Killed process 25379 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025e7bc>] do_mmap_pgoff+0x2fc/0x360
 [<ffffffff802735be>] sys_newfstat+0x2e/0x50
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 118   Cold: hi:   62, btch:  15 usd:  25
CPU    1: Hot: hi:  186, btch:  31 usd: 172   Cold: hi:   62, btch:  15 usd:  46
Active:423920 inactive:28829 dirty:0 writeback:0 unstable:0
 free:3418 slab:11584 mapped:6 pagetables:14487 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1704kB inactive:1436kB present:12744kB pages_scanned:15726 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5640kB min:5708kB low:7132kB high:8560kB active:1693976kB inactive:113880kB present:2051880kB pages_scanned:4012861 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 501*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5604kB
Swap cache: add 4931725, delete 4931725, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10104 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 171920 or a child
Killed process 25388 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 117   Cold: hi:   62, btch:  15 usd:  56
CPU    1: Hot: hi:  186, btch:  31 usd: 181   Cold: hi:   62, btch:  15 usd:  36
Active:433664 inactive:18732 dirty:0 writeback:0 unstable:0
 free:3439 slab:11583 mapped:3 pagetables:14483 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1708kB inactive:1432kB present:12744kB pages_scanned:15726 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5724kB min:5708kB low:7132kB high:8560kB active:1732948kB inactive:73496kB present:2051880kB pages_scanned:5885606 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 513*4kB 11*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5724kB
Swap cache: add 4931785, delete 4931785, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10317 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 171159 or a child
Killed process 25395 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025e7bc>] do_mmap_pgoff+0x2fc/0x360
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 136   Cold: hi:   62, btch:  15 usd:  58
CPU    1: Hot: hi:  186, btch:  31 usd: 160   Cold: hi:   62, btch:  15 usd:  44
Active:191270 inactive:261290 dirty:0 writeback:0 unstable:0
 free:3425 slab:11584 mapped:3 pagetables:14477 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1708kB inactive:1432kB present:12744kB pages_scanned:15918 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5668kB min:5708kB low:7132kB high:8560kB active:763372kB inactive:1043728kB present:2051880kB pages_scanned:31767483 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 517*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5668kB
Swap cache: add 4931845, delete 4931845, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10106 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 170398 or a child
Killed process 25397 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 135   Cold: hi:   62, btch:  15 usd:  15
CPU    1: Hot: hi:  186, btch:  31 usd: 171   Cold: hi:   62, btch:  15 usd:  44
Active:162477 inactive:289899 dirty:0 writeback:0 unstable:0
 free:3419 slab:11584 mapped:4 pagetables:14471 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1708kB inactive:1432kB present:12744kB pages_scanned:15982 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5644kB min:5708kB low:7132kB high:8560kB active:648200kB inactive:1158164kB present:2051880kB pages_scanned:3195902 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 502*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5608kB
Swap cache: add 4931906, delete 4931906, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10326 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 169637 or a child
Killed process 25401 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff802de57e>] _atomic_dec_and_lock+0x4e/0x70
 [<ffffffff80282712>] dput+0xb2/0x140
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 140   Cold: hi:   62, btch:  15 usd:  57
CPU    1: Hot: hi:  186, btch:  31 usd: 183   Cold: hi:   62, btch:  15 usd:  57
Active:166888 inactive:285231 dirty:0 writeback:0 unstable:0
 free:3440 slab:11584 mapped:4 pagetables:14465 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1708kB inactive:1432kB present:12744kB pages_scanned:16046 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5728kB min:5708kB low:7132kB high:8560kB active:665844kB inactive:1139492kB present:2051880kB pages_scanned:3455232 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 532*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5728kB
Swap cache: add 4931966, delete 4931966, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10416 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 168876 or a child
Killed process 25407 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 158   Cold: hi:   62, btch:  15 usd:  17
CPU    1: Hot: hi:  186, btch:  31 usd: 182   Cold: hi:   62, btch:  15 usd:  48
Active:8518 inactive:443658 dirty:0 writeback:0 unstable:0
 free:3425 slab:11584 mapped:4 pagetables:14459 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1708kB inactive:1432kB present:12744kB pages_scanned:16110 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5668kB min:5708kB low:7132kB high:8560kB active:34348kB inactive:1771152kB present:2051880kB pages_scanned:18797855 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 511*4kB 5*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5668kB
Swap cache: add 4932027, delete 4932027, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10343 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 168115 or a child
Killed process 25409 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025e7bc>] do_mmap_pgoff+0x2fc/0x360
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 156   Cold: hi:   62, btch:  15 usd:  57
CPU    1: Hot: hi:  186, btch:  31 usd: 184   Cold: hi:   62, btch:  15 usd:  47
Active:168463 inactive:283970 dirty:0 writeback:0 unstable:0
 free:3426 slab:11581 mapped:4 pagetables:14453 bounce:0
DMA free:8032kB min:32kB low:40kB high:48kB active:1712kB inactive:1428kB present:12744kB pages_scanned:16238 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5672kB min:5708kB low:7132kB high:8560kB active:672140kB inactive:1134452kB present:2051880kB pages_scanned:10945764 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 6*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
DMA32: 518*4kB 2*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5672kB
Swap cache: add 4932088, delete 4932088, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9997 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 167354 or a child
Killed process 25418 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025d4d1>] vma_merge+0x161/0x1f0
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 154   Cold: hi:   62, btch:  15 usd:  47
CPU    1: Hot: hi:  186, btch:  31 usd: 163   Cold: hi:   62, btch:  15 usd:  47
Active:336342 inactive:115515 dirty:0 writeback:0 unstable:0
 free:3445 slab:11581 mapped:1 pagetables:14447 bounce:0
DMA free:8044kB min:32kB low:40kB high:48kB active:2096kB inactive:996kB present:12744kB pages_scanned:9433 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5736kB min:5708kB low:7132kB high:8560kB active:1343272kB inactive:461064kB present:2051880kB pages_scanned:3723650 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 9*4kB 89*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8044kB
DMA32: 524*4kB 7*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5736kB
Swap cache: add 4932148, delete 4932148, find 700368/1123222, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10461 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 166593 or a child
Killed process 25428 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 167   Cold: hi:   62, btch:  15 usd:  54
CPU    1: Hot: hi:  186, btch:  31 usd: 123   Cold: hi:   62, btch:  15 usd:  52
Active:2906 inactive:450980 dirty:0 writeback:0 unstable:0
 free:3419 slab:11579 mapped:16 pagetables:14474 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1752kB inactive:1396kB present:12744kB pages_scanned:11565 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5640kB min:5708kB low:7132kB high:8560kB active:9872kB inactive:1802524kB present:2051880kB pages_scanned:17405264 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 514*4kB 2*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5640kB
Swap cache: add 4936171, delete 4936171, find 700451/1123735, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9649 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 165832 or a child
Killed process 25438 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 164   Cold: hi:   62, btch:  15 usd:  42
CPU    1: Hot: hi:  186, btch:  31 usd: 136   Cold: hi:   62, btch:  15 usd:  54
Active:448616 inactive:4562 dirty:0 writeback:0 unstable:0
 free:3421 slab:11578 mapped:16 pagetables:14468 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1752kB inactive:1396kB present:12744kB pages_scanned:11565 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5648kB min:5708kB low:7132kB high:8560kB active:1792712kB inactive:16852kB present:2051880kB pages_scanned:10911644 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 512*4kB 4*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5648kB
Swap cache: add 4936231, delete 4936231, find 700451/1123735, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10298 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 165071 or a child
Killed process 25448 (akmk)
make invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8024bda6>] read_cache_page+0x6/0x40
 [<ffffffff8027cb6e>] filldir+0xae/0xf0
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8021e8ca>] __wake_up_sync+0x4a/0x80
 [<ffffffff802762a8>] pipe_release+0x58/0xc0
 [<ffffffff80282705>] dput+0xa5/0x140
 [<ffffffff8022ed6e>] recalc_sigpending+0xe/0x30
 [<ffffffff8022ee37>] sigprocmask+0x67/0xf0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 164   Cold: hi:   62, btch:  15 usd:  54
CPU    1: Hot: hi:  186, btch:  31 usd: 147   Cold: hi:   62, btch:  15 usd:  15
Active:4623 inactive:448756 dirty:0 writeback:0 unstable:0
 free:3451 slab:11580 mapped:16 pagetables:14462 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1752kB inactive:1396kB present:12744kB pages_scanned:11629 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5768kB min:5708kB low:7132kB high:8560kB active:18848kB inactive:1791452kB present:2051880kB pages_scanned:9077611 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 550*4kB 0*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5768kB
Swap cache: add 4936291, delete 4936291, find 700451/1123735, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10102 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 164310 or a child
Killed process 25465 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff802de57e>] _atomic_dec_and_lock+0x4e/0x70
 [<ffffffff80282712>] dput+0xb2/0x140
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 174   Cold: hi:   62, btch:  15 usd:  48
CPU    1: Hot: hi:  186, btch:  31 usd: 145   Cold: hi:   62, btch:  15 usd:  38
Active:124740 inactive:328306 dirty:0 writeback:0 unstable:0
 free:3434 slab:11581 mapped:16 pagetables:14456 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1736kB inactive:1396kB present:12744kB pages_scanned:11757 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5700kB min:5708kB low:7132kB high:8560kB active:494664kB inactive:1314388kB present:2051880kB pages_scanned:20846696 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 531*4kB 1*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5700kB
Swap cache: add 4936353, delete 4936353, find 700451/1123735, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10305 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 163549 or a child
Killed process 25474 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025d4d1>] vma_merge+0x161/0x1f0
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 176   Cold: hi:   62, btch:  15 usd:  51
CPU    1: Hot: hi:  186, btch:  31 usd: 158   Cold: hi:   62, btch:  15 usd:  18
Active:227125 inactive:225745 dirty:0 writeback:0 unstable:0
 free:3434 slab:11579 mapped:16 pagetables:14450 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1752kB inactive:1396kB present:12744kB pages_scanned:11821 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5700kB min:5708kB low:7132kB high:8560kB active:906748kB inactive:901584kB present:2051880kB pages_scanned:9477646 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 525*4kB 4*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5700kB
Swap cache: add 4936413, delete 4936413, find 700451/1123735, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10422 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 162788 or a child
Killed process 25520 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 176   Cold: hi:   62, btch:  15 usd:  46
CPU    1: Hot: hi:  186, btch:  31 usd: 169   Cold: hi:   62, btch:  15 usd:  53
Active:192981 inactive:260376 dirty:0 writeback:0 unstable:0
 free:3434 slab:11580 mapped:16 pagetables:14444 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1752kB inactive:1396kB present:12744kB pages_scanned:11885 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5700kB min:5708kB low:7132kB high:8560kB active:770296kB inactive:1039996kB present:2051880kB pages_scanned:14468241 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 527*4kB 3*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5700kB
Swap cache: add 4936473, delete 4936473, find 700451/1123735, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9903 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 162027 or a child
Killed process 25521 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025d4d1>] vma_merge+0x161/0x1f0
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 175   Cold: hi:   62, btch:  15 usd:  45
CPU    1: Hot: hi:  186, btch:  31 usd: 185   Cold: hi:   62, btch:  15 usd:  55
Active:144219 inactive:308764 dirty:0 writeback:0 unstable:0
 free:3434 slab:11575 mapped:1 pagetables:14438 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1764kB inactive:1196kB present:12744kB pages_scanned:76102 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5700kB min:5708kB low:7132kB high:8560kB active:575112kB inactive:1233748kB present:2051880kB pages_scanned:35804643 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 525*4kB 4*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5700kB
Swap cache: add 4936534, delete 4936534, find 700451/1123735, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10106 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 161266 or a child
Killed process 25526 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025d4d1>] vma_merge+0x161/0x1f0
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 150   Cold: hi:   62, btch:  15 usd:  20
CPU    1: Hot: hi:  186, btch:  31 usd: 184   Cold: hi:   62, btch:  15 usd:  54
Active:332097 inactive:121133 dirty:0 writeback:0 unstable:0
 free:3450 slab:11575 mapped:62 pagetables:14432 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1912kB inactive:1160kB present:12744kB pages_scanned:76189 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5764kB min:5708kB low:7132kB high:8560kB active:1326476kB inactive:483372kB present:2051880kB pages_scanned:2112 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 549*4kB 0*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5764kB
Swap cache: add 4936603, delete 4936603, find 700451/1123736, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9944 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 160505 or a child
Killed process 25542 (akmk)
cron invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff802de57e>] _atomic_dec_and_lock+0x4e/0x70
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8022f180>] do_sigaction+0x90/0x1b0
 [<ffffffff8022f379>] sys_rt_sigaction+0x99/0xd0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 132   Cold: hi:   62, btch:  15 usd:  44
CPU    1: Hot: hi:  186, btch:  31 usd: 162   Cold: hi:   62, btch:  15 usd:  55
Active:58995 inactive:393811 dirty:0 writeback:0 unstable:0
 free:3436 slab:11574 mapped:1 pagetables:14426 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1928kB inactive:1160kB present:12744kB pages_scanned:76317 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5708kB min:5708kB low:7132kB high:8560kB active:236160kB inactive:1571908kB present:2051880kB pages_scanned:14107701 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 535*4kB 0*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5708kB
Swap cache: add 4936672, delete 4936672, find 700451/1123737, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10160 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 159744 or a child
Killed process 25719 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 130   Cold: hi:   62, btch:  15 usd:  59
CPU    1: Hot: hi:  186, btch:  31 usd: 174   Cold: hi:   62, btch:  15 usd:  61
Active:135048 inactive:317433 dirty:0 writeback:0 unstable:0
 free:3436 slab:11574 mapped:1 pagetables:14420 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1928kB inactive:1160kB present:12744kB pages_scanned:76381 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5708kB min:5708kB low:7132kB high:8560kB active:535832kB inactive:1271004kB present:2051880kB pages_scanned:21688955 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 523*4kB 6*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5708kB
Swap cache: add 4936733, delete 4936733, find 700451/1123737, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10384 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 158983 or a child
Killed process 25764 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 129   Cold: hi:   62, btch:  15 usd:  60
CPU    1: Hot: hi:  186, btch:  31 usd: 156   Cold: hi:   62, btch:  15 usd:  45
Active:142210 inactive:310656 dirty:0 writeback:0 unstable:0
 free:3416 slab:11575 mapped:1 pagetables:14414 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1928kB inactive:1160kB present:12744kB pages_scanned:76381 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5628kB min:5708kB low:7132kB high:8560kB active:568896kB inactive:1239288kB present:2051880kB pages_scanned:4144069 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 506*4kB 0*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5592kB
Swap cache: add 4936795, delete 4936795, find 700451/1123737, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10131 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 158222 or a child
Killed process 25768 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff802de57e>] _atomic_dec_and_lock+0x4e/0x70
 [<ffffffff80282712>] dput+0xb2/0x140
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 111   Cold: hi:   62, btch:  15 usd:  52
CPU    1: Hot: hi:  186, btch:  31 usd: 140   Cold: hi:   62, btch:  15 usd:  57
Active:406593 inactive:46196 dirty:0 writeback:0 unstable:0
 free:3422 slab:11572 mapped:1 pagetables:14410 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1928kB inactive:1160kB present:12744kB pages_scanned:76637 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5652kB min:5708kB low:7132kB high:8560kB active:1626552kB inactive:181576kB present:2051880kB pages_scanned:29550878 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 521*4kB 0*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5652kB
Swap cache: add 4936865, delete 4936865, find 700451/1123738, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9963 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 157461 or a child
Killed process 25774 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff802de57e>] _atomic_dec_and_lock+0x4e/0x70
 [<ffffffff80282712>] dput+0xb2/0x140
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 103   Cold: hi:   62, btch:  15 usd:  50
CPU    1: Hot: hi:  186, btch:  31 usd: 152   Cold: hi:   62, btch:  15 usd:  60
Active:245331 inactive:207700 dirty:0 writeback:0 unstable:0
 free:3422 slab:11572 mapped:1 pagetables:14404 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1928kB inactive:1160kB present:12744kB pages_scanned:76893 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5652kB min:5708kB low:7132kB high:8560kB active:979396kB inactive:829640kB present:2051880kB pages_scanned:31298335 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 519*4kB 1*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5652kB
Swap cache: add 4936925, delete 4936925, find 700451/1123738, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9683 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 156700 or a child
Killed process 25777 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd:  95   Cold: hi:   62, btch:  15 usd:  50
CPU    1: Hot: hi:  186, btch:  31 usd: 156   Cold: hi:   62, btch:  15 usd:  49
Active:244490 inactive:207880 dirty:0 writeback:0 unstable:0
 free:3453 slab:11572 mapped:2 pagetables:14398 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1928kB inactive:1160kB present:12744kB pages_scanned:77021 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5776kB min:5708kB low:7132kB high:8560kB active:976032kB inactive:830360kB present:2051880kB pages_scanned:3029907 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 538*4kB 7*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5776kB
Swap cache: add 4936985, delete 4936985, find 700451/1123738, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10296 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 155939 or a child
Killed process 25816 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff80239930>] autoremove_wake_function+0x0/0x30
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd:  95   Cold: hi:   62, btch:  15 usd:  61
CPU    1: Hot: hi:  186, btch:  31 usd: 162   Cold: hi:   62, btch:  15 usd:  54
Active:229501 inactive:223055 dirty:0 writeback:0 unstable:0
 free:3423 slab:11570 mapped:1 pagetables:14392 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1928kB inactive:1160kB present:12744kB pages_scanned:77149 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5656kB min:5708kB low:7132kB high:8560kB active:916076kB inactive:891172kB present:2051880kB pages_scanned:4123304 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 522*4kB 0*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5656kB
Swap cache: add 4937046, delete 4937046, find 700451/1123738, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10055 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 155178 or a child
Killed process 25822 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd:  85   Cold: hi:   62, btch:  15 usd:  47
CPU    1: Hot: hi:  186, btch:  31 usd: 171   Cold: hi:   62, btch:  15 usd:  16
Active:222092 inactive:230428 dirty:0 writeback:0 unstable:0
 free:3423 slab:11572 mapped:1 pagetables:14386 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1932kB inactive:1196kB present:12744kB pages_scanned:77405 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5656kB min:5708kB low:7132kB high:8560kB active:886436kB inactive:920516kB present:2051880kB pages_scanned:37952251 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 522*4kB 0*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5656kB
Swap cache: add 4937123, delete 4937123, find 700451/1123740, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10025 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 154417 or a child
Killed process 25825 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd:  79   Cold: hi:   62, btch:  15 usd:  49
CPU    1: Hot: hi:  186, btch:  31 usd: 184   Cold: hi:   62, btch:  15 usd:  50
Active:43813 inactive:408398 dirty:0 writeback:0 unstable:0
 free:3423 slab:11571 mapped:1 pagetables:14380 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1932kB inactive:1196kB present:12744kB pages_scanned:77597 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5656kB min:5708kB low:7132kB high:8560kB active:171016kB inactive:1634700kB present:2051880kB pages_scanned:21288516 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 522*4kB 0*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5656kB
Swap cache: add 4937184, delete 4937184, find 700451/1123740, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10247 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 153656 or a child
Killed process 25830 (akmk)
mips-unknown-li invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8026b0f5>] __slab_free+0x1e5/0x330
 [<ffffffff8025c43c>] remove_vma+0x4c/0x60
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd:  77   Cold: hi:   62, btch:  15 usd:  60
CPU    1: Hot: hi:  186, btch:  31 usd: 165   Cold: hi:   62, btch:  15 usd:  15
Active:205842 inactive:246412 dirty:0 writeback:0 unstable:0
 free:3421 slab:11572 mapped:2 pagetables:14375 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1932kB inactive:1196kB present:12744kB pages_scanned:77597 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5648kB min:5708kB low:7132kB high:8560kB active:821656kB inactive:984212kB present:2051880kB pages_scanned:2989881 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 508*4kB 0*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5600kB
Swap cache: add 4937245, delete 4937245, find 700451/1123740, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10215 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 152895 or a child
Killed process 25868 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd:  83   Cold: hi:   62, btch:  15 usd:  57
CPU    1: Hot: hi:  186, btch:  31 usd: 176   Cold: hi:   62, btch:  15 usd:  60
Active:261918 inactive:190031 dirty:0 writeback:0 unstable:0
 free:3424 slab:11573 mapped:3 pagetables:14369 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1932kB inactive:1196kB present:12744kB pages_scanned:77661 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5660kB min:5708kB low:7132kB high:8560kB active:1045740kB inactive:758928kB present:2051880kB pages_scanned:9313873 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 519*4kB 2*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5660kB
Swap cache: add 4937306, delete 4937306, find 700451/1123740, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10408 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 152134 or a child
Killed process 25877 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff802de57e>] _atomic_dec_and_lock+0x4e/0x70
 [<ffffffff80282712>] dput+0xb2/0x140
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd:  77   Cold: hi:   62, btch:  15 usd:  54
CPU    1: Hot: hi:  186, btch:  31 usd: 163   Cold: hi:   62, btch:  15 usd:  17
Active:216397 inactive:236001 dirty:0 writeback:0 unstable:0
 free:3424 slab:11573 mapped:1 pagetables:14365 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1932kB inactive:1196kB present:12744kB pages_scanned:77725 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5660kB min:5708kB low:7132kB high:8560kB active:863656kB inactive:942808kB present:2051880kB pages_scanned:2789619 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 517*4kB 3*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5660kB
Swap cache: add 4937366, delete 4937366, find 700451/1123740, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9910 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 151373 or a child
Killed process 25883 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd:  77   Cold: hi:   62, btch:  15 usd:  58
CPU    1: Hot: hi:  186, btch:  31 usd: 172   Cold: hi:   62, btch:  15 usd:  14
Active:292998 inactive:159173 dirty:0 writeback:0 unstable:0
 free:3418 slab:11572 mapped:5 pagetables:14359 bounce:0
DMA free:8036kB min:32kB low:40kB high:48kB active:1936kB inactive:1192kB present:12744kB pages_scanned:77789 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5636kB min:5708kB low:7132kB high:8560kB active:1170056kB inactive:635612kB present:2051880kB pages_scanned:2765158 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 5*4kB 90*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
DMA32: 504*4kB 2*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5600kB
Swap cache: add 4937427, delete 4937427, find 700451/1123740, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10153 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 150612 or a child
Killed process 25889 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025c43c>] remove_vma+0x4c/0x60
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff802e29d1>] __up_write+0x21/0x130
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 183   Cold: hi:   62, btch:  15 usd:  58
CPU    1: Hot: hi:  186, btch:  31 usd: 159   Cold: hi:   62, btch:  15 usd:  38
Active:257823 inactive:196433 dirty:0 writeback:0 unstable:0
 free:3427 slab:11583 mapped:2 pagetables:14345 bounce:0
DMA free:8028kB min:32kB low:40kB high:48kB active:1732kB inactive:1272kB present:12744kB pages_scanned:7256 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5680kB min:5708kB low:7132kB high:8560kB active:1029560kB inactive:784460kB present:2051880kB pages_scanned:15873083 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 1*4kB 91*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8028kB
DMA32: 530*4kB 1*8kB 18*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5680kB
Swap cache: add 4941921, delete 4941921, find 700476/1124255, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9988 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 149851 or a child
Killed process 25905 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff8025d4d1>] vma_merge+0x161/0x1f0
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 183   Cold: hi:   62, btch:  15 usd:  10
CPU    1: Hot: hi:  186, btch:  31 usd: 172   Cold: hi:   62, btch:  15 usd:  17
Active:6576 inactive:448414 dirty:0 writeback:0 unstable:0
 free:3421 slab:11582 mapped:4 pagetables:14339 bounce:0
DMA free:8028kB min:32kB low:40kB high:48kB active:1732kB inactive:1272kB present:12744kB pages_scanned:7256 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5656kB min:5708kB low:7132kB high:8560kB active:24572kB inactive:1792384kB present:2051880kB pages_scanned:6507504 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 1*4kB 91*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8028kB
DMA32: 515*4kB 1*8kB 18*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5620kB
Swap cache: add 4941981, delete 4941981, find 700476/1124255, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
9337 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 149090 or a child
Killed process 25924 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 180   Cold: hi:   62, btch:  15 usd:  52
CPU    1: Hot: hi:  186, btch:  31 usd: 183   Cold: hi:   62, btch:  15 usd:  51
Active:2045 inactive:451969 dirty:0 writeback:0 unstable:0
 free:3427 slab:11582 mapped:2 pagetables:14335 bounce:0
DMA free:8028kB min:32kB low:40kB high:48kB active:1732kB inactive:1272kB present:12744kB pages_scanned:7320 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5680kB min:5708kB low:7132kB high:8560kB active:8556kB inactive:1804556kB present:2051880kB pages_scanned:10503383 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 1*4kB 91*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8028kB
DMA32: 530*4kB 1*8kB 18*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5680kB
Swap cache: add 4942041, delete 4942041, find 700476/1124255, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10080 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 148329 or a child
Killed process 25927 (akmk)
cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Call Trace:
 [<ffffffff8024cfe6>] oom_kill_process+0xf6/0x110
 [<ffffffff8024d49d>] out_of_memory+0x1ad/0x1f0
 [<ffffffff8024f79e>] __alloc_pages+0x28e/0x3a0
 [<ffffffff802518d1>] __do_page_cache_readahead+0x101/0x270
 [<ffffffff8024c43a>] filemap_fault+0x28a/0x3c0
 [<ffffffff8025742c>] __do_fault+0x6c/0x430
 [<ffffffff802592db>] handle_mm_fault+0x3bb/0x700
 [<ffffffff8021c09f>] do_page_fault+0x1bf/0x830
 [<ffffffff803d1583>] thread_return+0x3d/0x53a
 [<ffffffff8025d296>] do_munmap+0x296/0x2f0
 [<ffffffff803d3679>] error_exit+0x0/0x51

Mem-info:
DMA per-cpu:
CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
DMA32 per-cpu:
CPU    0: Hot: hi:  186, btch:  31 usd: 167   Cold: hi:   62, btch:  15 usd:  60
CPU    1: Hot: hi:  186, btch:  31 usd: 164   Cold: hi:   62, btch:  15 usd:  31
Active:115924 inactive:338027 dirty:0 writeback:0 unstable:0
 free:3428 slab:11580 mapped:4 pagetables:14330 bounce:0
DMA free:8028kB min:32kB low:40kB high:48kB active:1732kB inactive:1292kB present:12744kB pages_scanned:7384 all_unreclaimable? yes
lowmem_reserve[]: 0 2003 2003 2003
DMA32 free:5684kB min:5708kB low:7132kB high:8560kB active:461840kB inactive:1350816kB present:2051880kB pages_scanned:5104608 all_unreclaimable? yes
lowmem_reserve[]: 0 0 0 0
DMA: 1*4kB 91*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8028kB
DMA32: 529*4kB 2*8kB 18*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5684kB
Swap cache: add 4942111, delete 4942111, find 700476/1124255, race 92+89
Free swap  = 0kB
Total swap = 987956kB
Free swap:            0kB
524176 pages of RAM
8772 reserved pages
10143 pages shared
0 pages swap cached
Out of memory: kill process 25286 (build-all) score 147568 or a child
Killed process 25934 (akmk)


================================================================================
Linux version 2.6.23-0b776eb5426752d4e53354ac89e3710d857e09a7 (ad@core2) (gcc version 4.1.1 (Gentoo 4.1.1-r3)) #2 SMP Tue Oct 23 21:41:03 MSD 2007
Command line: root=/dev/sda2 netconsole=@10.10.0.42/eth0,9353@10.10.0.1/00:80:48:45:EC:73 ignore_loglevel
BIOS-provided physical RAM map:
 BIOS-e820: 0000000000000000 - 000000000009fc00 (usable)
 BIOS-e820: 000000000009fc00 - 00000000000a0000 (reserved)
 BIOS-e820: 00000000000e4000 - 0000000000100000 (reserved)
 BIOS-e820: 0000000000100000 - 000000007ff90000 (usable)
 BIOS-e820: 000000007ff90000 - 000000007ff9e000 (ACPI data)
 BIOS-e820: 000000007ff9e000 - 000000007ffe0000 (ACPI NVS)
 BIOS-e820: 000000007ffe0000 - 0000000080000000 (reserved)
 BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
 BIOS-e820: 00000000ffb00000 - 0000000100000000 (reserved)
Entering add_active_range(0, 0, 159) 0 entries of 256 used
Entering add_active_range(0, 256, 524176) 1 entries of 256 used
end_pfn_map = 1048576
DMI 2.4 present.
ACPI: RSDP 000FA980, 0024 (r2 ACPIAM)
ACPI: XSDT 7FF90100, 0054 (r1 KOZIRO FRONTIER  2000707 MSFT       97)
ACPI: FACP 7FF90290, 00F4 (r3 MSTEST OEMFACP   2000707 MSFT       97)
ACPI: DSDT 7FF905C0, 8FA9 (r1  A0637 A0637000        0 INTL 20060113)
ACPI: FACS 7FF9E000, 0040
ACPI: APIC 7FF90390, 006C (r1 MSTEST OEMAPIC   2000707 MSFT       97)
ACPI: MCFG 7FF90400, 003C (r1 MSTEST OEMMCFG   2000707 MSFT       97)
ACPI: SLIC 7FF90440, 0176 (r1 KOZIRO FRONTIER  2000707 MSFT       97)
ACPI: OEMB 7FF9E040, 007B (r1 MSTEST AMI_OEM   2000707 MSFT       97)
ACPI: HPET 7FF99570, 0038 (r1 MSTEST OEMHPET   2000707 MSFT       97)
Entering add_active_range(0, 0, 159) 0 entries of 256 used
Entering add_active_range(0, 256, 524176) 1 entries of 256 used
Zone PFN ranges:
  DMA             0 ->     4096
  DMA32        4096 ->  1048576
  Normal    1048576 ->  1048576
Movable zone start PFN for each node
early_node_map[2] active PFN ranges
    0:        0 ->      159
    0:      256 ->   524176
On node 0 totalpages: 524079
  DMA zone: 56 pages used for memmap
  DMA zone: 757 pages reserved
  DMA zone: 3186 pages, LIFO batch:0
  DMA32 zone: 7110 pages used for memmap
  DMA32 zone: 512970 pages, LIFO batch:31
  Normal zone: 0 pages used for memmap
  Movable zone: 0 pages used for memmap
ACPI: Local APIC address 0xfee00000
ACPI: LAPIC (acpi_id[0x01] lapic_id[0x00] enabled)
Processor #0 (Bootup-CPU)
ACPI: LAPIC (acpi_id[0x02] lapic_id[0x01] enabled)
Processor #1
ACPI: LAPIC (acpi_id[0x03] lapic_id[0x82] disabled)
ACPI: LAPIC (acpi_id[0x04] lapic_id[0x83] disabled)
ACPI: IOAPIC (id[0x02] address[0xfec00000] gsi_base[0])
IOAPIC[0]: apic_id 2, address 0xfec00000, GSI 0-23
ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 dfl dfl)
ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
ACPI: IRQ0 used by override.
ACPI: IRQ2 used by override.
ACPI: IRQ9 used by override.
Setting APIC routing to flat
ACPI: HPET id: 0x8086a202 base: 0xfed00000
Using ACPI (MADT) for SMP configuration information
Allocating PCI resources starting at 88000000 (gap: 80000000:7ee00000)
PERCPU: Allocating 22896 bytes of per cpu data
Built 1 zonelists in Zone order, mobility grouping on.  Total pages: 516156
Kernel command line: root=/dev/sda2 netconsole=@10.10.0.42/eth0,9353@10.10.0.1/00:80:48:45:EC:73 ignore_loglevel
debug: ignoring loglevel setting.
Initializing CPU#0
PID hash table entries: 4096 (order: 12, 32768 bytes)
Extended CMOS year: 2000
hpet clockevent registered
TSC calibrated against HPET
time.c: Detected 2135.039 MHz processor.
Console: colour VGA+ 80x25
console [tty0] enabled
Dentry cache hash table entries: 262144 (order: 9, 2097152 bytes)
Inode-cache hash table entries: 131072 (order: 8, 1048576 bytes)
Memory: 2061224k/2096704k available (1873k kernel code, 34912k reserved, 651k data, 192k init)
SLUB: Genslabs=11, HWalign=64, Order=0-1, MinObjects=4, CPUs=2, Nodes=1
Calibrating delay using timer specific routine.. 4272.48 BogoMIPS (lpj=21362443)
Mount-cache hash table entries: 256
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 2048K
using mwait in idle threads.
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 0
CPU0: Thermal monitoring enabled (TM2)
Freeing SMP alternatives: 18k freed
ACPI: Core revision 20070126
Using local APIC timer interrupts.
APIC timer calibration result 16679990
Detected 16.679 MHz APIC timer.
Booting processor 1/2 APIC 0x1
Initializing CPU#1
Calibrating delay using timer specific routine.. 4270.10 BogoMIPS (lpj=21350544)
CPU: L1 I cache: 32K, L1 D cache: 32K
CPU: L2 cache: 2048K
CPU: Physical Processor ID: 0
CPU: Processor Core ID: 1
CPU1: Thermal monitoring enabled (TM2)
Intel(R) Core(TM)2 CPU          6400  @ 2.13GHz stepping 02
checking TSC synchronization [CPU#0 -> CPU#1]: passed.
Brought up 2 CPUs
net_namespace: 120 bytes
NET: Registered protocol family 16
ACPI: bus type pci registered
PCI: Using configuration type 1
ACPI: EC: Look up EC in DSDT
ACPI: Interpreter enabled
ACPI: (supports S0 S5)
ACPI: Using IOAPIC for interrupt routing
ACPI: PCI Root Bridge [PCI0] (0000:00)
PCI quirk: region 0800-087f claimed by ICH6 ACPI/GPIO/TCO
PCI quirk: region 0480-04bf claimed by ICH6 GPIO
PCI: Transparent bridge - 0000:00:1e.0
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P2._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P1._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P4._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P7._PRT]
ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P8._PRT]
ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 10 *11 12 14 15)
ACPI: PCI Interrupt Link [LNKB] (IRQs 3 4 *5 6 7 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 10 11 12 14 *15)
ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 *10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKF] (IRQs 3 4 5 6 7 10 11 12 14 15) *0, disabled.
ACPI: PCI Interrupt Link [LNKG] (IRQs *3 4 5 6 7 10 11 12 14 15)
ACPI: PCI Interrupt Link [LNKH] (IRQs 3 4 5 6 7 10 11 12 *14 15)
ACPI Warning (tbutils-0217): Incorrect checksum in table [OEMB] -  08, should be FB [20070126]
Linux Plug and Play Support v0.97 (c) Adam Belay
pnp: PnP ACPI init
ACPI: bus type pnp registered
pnp: PnP ACPI: found 16 devices
ACPI: ACPI bus type pnp unregistered
SCSI subsystem initialized
libata version 3.00 loaded.
PCI: Using ACPI for IRQ routing
PCI: If a device doesn't work, try "pci=routeirq".  If it helps, post a report
Time: tsc clocksource has been installed.
Switched to high resolution mode on CPU 0
Switched to high resolution mode on CPU 1
system 00:01: iomem range 0xfed14000-0xfed19fff has been reserved
system 00:08: ioport range 0x290-0x297 has been reserved
system 00:09: iomem range 0xffafe000-0xffb0cbff could not be reserved
system 00:09: iomem range 0xffb00000-0xffbfffff could not be reserved
system 00:09: iomem range 0xfed1c000-0xfed1ffff has been reserved
system 00:09: iomem range 0xfed20000-0xfed8ffff has been reserved
system 00:0c: iomem range 0xfec00000-0xfec00fff has been reserved
system 00:0c: iomem range 0xfee00000-0xfee00fff could not be reserved
system 00:0e: iomem range 0xe0000000-0xefffffff has been reserved
system 00:0f: iomem range 0x0-0x9ffff could not be reserved
system 00:0f: iomem range 0xc0000-0xcffff has been reserved
system 00:0f: iomem range 0xe0000-0xfffff could not be reserved
system 00:0f: iomem range 0x100000-0x7fffffff could not be reserved
PCI: Bridge: 0000:00:01.0
  IO window: a000-afff
  MEM window: f8800000-fe8fffff
  PREFETCH window: bfe00000-dfdfffff
PCI: Bridge: 0000:00:1c.0
  IO window: disabled.
  MEM window: disabled.
  PREFETCH window: dfe00000-dfefffff
PCI: Bridge: 0000:00:1c.3
  IO window: disabled.
  MEM window: fea00000-feafffff
  PREFETCH window: disabled.
PCI: Bridge: 0000:00:1c.4
  IO window: b000-bfff
  MEM window: fe900000-fe9fffff
  PREFETCH window: disabled.
PCI: Bridge: 0000:00:1e.0
  IO window: disabled.
  MEM window: disabled.
  PREFETCH window: disabled.
ACPI: PCI Interrupt 0000:00:01.0[A] -> GSI 16 (level, low) -> IRQ 16
PCI: Setting latency timer of device 0000:00:01.0 to 64
ACPI: PCI Interrupt 0000:00:1c.0[A] -> GSI 16 (level, low) -> IRQ 16
PCI: Setting latency timer of device 0000:00:1c.0 to 64
ACPI: PCI Interrupt 0000:00:1c.3[D] -> GSI 19 (level, low) -> IRQ 19
PCI: Setting latency timer of device 0000:00:1c.3 to 64
ACPI: PCI Interrupt 0000:00:1c.4[A] -> GSI 16 (level, low) -> IRQ 16
PCI: Setting latency timer of device 0000:00:1c.4 to 64
PCI: Setting latency timer of device 0000:00:1e.0 to 64
NET: Registered protocol family 2
IP route cache hash table entries: 65536 (order: 7, 524288 bytes)
TCP established hash table entries: 131072 (order: 9, 3145728 bytes)
TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
TCP: Hash tables configured (established 131072 bind 65536)
TCP reno registered
io scheduler noop registered
io scheduler cfq registered (default)
Boot video device is 0000:01:00.0
Real Time Clock Driver v1.12ac
ACPI: PCI Interrupt 0000:03:00.0[A] -> GSI 19 (level, low) -> IRQ 19
PCI: Setting latency timer of device 0000:03:00.0 to 64
atl1 0000:03:00.0: version 2.0.7
netconsole: local port 6665
netconsole: local IP 10.10.0.42
netconsole: interface eth0
netconsole: remote port 9353
netconsole: remote IP 10.10.0.1
netconsole: remote ethernet address 00:80:48:45:ec:73
netconsole: device eth0 not up yet, forcing it
atl1 0000:03:00.0: eth0 link is up 100 Mbps full duplex
console [netcon0] enabled
netconsole: network logging started
ahci 0000:02:00.0: version 3.0
ACPI: PCI Interrupt 0000:02:00.0[A] -> GSI 16 (level, low) -> IRQ 16
ahci 0000:02:00.0: AHCI 0001.0000 32 slots 2 ports 3 Gbps 0x3 impl SATA mode
ahci 0000:02:00.0: flags: 64bit ncq pm led clo pmp pio slum part 
PCI: Setting latency timer of device 0000:02:00.0 to 64
scsi0 : ahci
scsi1 : ahci
ata1: SATA max UDMA/133 abar m8192@0xfe9fe000 port 0xfe9fe100 irq 16
ata2: SATA max UDMA/133 abar m8192@0xfe9fe000 port 0xfe9fe180 irq 16
ata1: SATA link down (SStatus 0 SControl 300)
ata2: SATA link down (SStatus 0 SControl 300)
ata_piix 0000:00:1f.2: version 2.12
ata_piix 0000:00:1f.2: MAP [ P0 P2 P1 P3 ]
ACPI: PCI Interrupt 0000:00:1f.2[B] -> GSI 19 (level, low) -> IRQ 19
PCI: Setting latency timer of device 0000:00:1f.2 to 64
scsi2 : ata_piix
scsi3 : ata_piix
ata3: SATA max UDMA/133 cmd 0xec00 ctl 0xe880 bmdma 0xe400 irq 19
ata4: SATA max UDMA/133 cmd 0xe800 ctl 0xe480 bmdma 0xe408 irq 19
ata3.00: ATA-7: ST3250620AS, 3.AAE, max UDMA/133
ata3.00: 488397168 sectors, multi 16: LBA48 NCQ (depth 0/32)
ata3.00: configured for UDMA/133
ata4.00: ATA-7: ST3320620AS, 3.AAK, max UDMA/133
ata4.00: 625142448 sectors, multi 16: LBA48 NCQ (depth 0/32)
ata4.00: configured for UDMA/133
scsi 2:0:0:0: Direct-Access     ATA      ST3250620AS      3.AA PQ: 0 ANSI: 5
sd 2:0:0:0: [sda] 488397168 512-byte hardware sectors (250059 MB)
sd 2:0:0:0: [sda] Write Protect is off
sd 2:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sd 2:0:0:0: [sda] 488397168 512-byte hardware sectors (250059 MB)
sd 2:0:0:0: [sda] Write Protect is off
sd 2:0:0:0: [sda] Mode Sense: 00 3a 00 00
sd 2:0:0:0: [sda] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sda: sda1 sda2 sda3
sd 2:0:0:0: [sda] Attached SCSI disk
scsi 3:0:0:0: Direct-Access     ATA      ST3320620AS      3.AA PQ: 0 ANSI: 5
sd 3:0:0:0: [sdb] 625142448 512-byte hardware sectors (320073 MB)
sd 3:0:0:0: [sdb] Write Protect is off
sd 3:0:0:0: [sdb] Mode Sense: 00 3a 00 00
sd 3:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
sd 3:0:0:0: [sdb] 625142448 512-byte hardware sectors (320073 MB)
sd 3:0:0:0: [sdb] Write Protect is off
sd 3:0:0:0: [sdb] Mode Sense: 00 3a 00 00
sd 3:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
 sdb: sdb1
sd 3:0:0:0: [sdb] Attached SCSI disk
ata_piix 0000:00:1f.5: MAP [ P0 -- P1 -- ]
ACPI: PCI Interrupt 0000:00:1f.5[B] -> GSI 19 (level, low) -> IRQ 19
PCI: Setting latency timer of device 0000:00:1f.5 to 64
scsi4 : ata_piix
scsi5 : ata_piix
ata5: SATA max UDMA/133 cmd 0xd400 ctl 0xd080 bmdma 0xc880 irq 19
ata6: SATA max UDMA/133 cmd 0xd000 ctl 0xcc00 bmdma 0xc888 irq 19
ACPI: PCI Interrupt 0000:02:00.1[B] -> GSI 17 (level, low) -> IRQ 17
PCI: Setting latency timer of device 0000:02:00.1 to 64
scsi6 : pata_jmicron
scsi7 : pata_jmicron
ata7: PATA max UDMA/100 cmd 0xbc00 ctl 0xb880 bmdma 0xb400 irq 17
ata8: PATA max UDMA/100 cmd 0xb800 ctl 0xb480 bmdma 0xb408 irq 17
PNP: PS/2 Controller [PNP0303:PS2K] at 0x60,0x64 irq 1
PNP: PS/2 appears to have AUX port disabled, if this is incorrect please boot with i8042.nopnp
serio: i8042 KBD port at 0x60,0x64 irq 1
coretemp coretemp.0: Using undocumented features, absolute temperature might be wrong!
coretemp coretemp.1: Using undocumented features, absolute temperature might be wrong!
w83627ehf: Found W83627DHG chip at 0x290
TCP cubic registered
NET: Registered protocol family 1
input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input0
kjournald starting.  Commit interval 5 seconds
EXT3-fs: mounted filesystem with ordered data mode.
VFS: Mounted root (ext3 filesystem) readonly.
Freeing unused kernel memory: 192k freed
EXT3 FS on sda2, internal journal
Adding 987956k swap on /dev/sda1.  Priority:-1 extents:1 across:987956k
================================================================================
#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.23
# Tue Oct 23 21:39:10 2007
#
CONFIG_X86_64=y
CONFIG_64BIT=y
CONFIG_X86=y
CONFIG_GENERIC_TIME=y
CONFIG_GENERIC_TIME_VSYSCALL=y
CONFIG_GENERIC_CMOS_UPDATE=y
CONFIG_CLOCKSOURCE_WATCHDOG=y
CONFIG_GENERIC_CLOCKEVENTS=y
CONFIG_GENERIC_CLOCKEVENTS_BROADCAST=y
CONFIG_ZONE_DMA32=y
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_SEMAPHORE_SLEEPERS=y
CONFIG_MMU=y
CONFIG_ZONE_DMA=y
CONFIG_RWSEM_GENERIC_SPINLOCK=y
CONFIG_GENERIC_HWEIGHT=y
CONFIG_GENERIC_CALIBRATE_DELAY=y
CONFIG_X86_CMPXCHG=y
CONFIG_EARLY_PRINTK=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
CONFIG_ARCH_POPULATES_NODE_MAP=y
CONFIG_DMI=y
CONFIG_AUDIT_ARCH=y
CONFIG_GENERIC_BUG=y
# CONFIG_ARCH_HAS_ILOG2_U32 is not set
# CONFIG_ARCH_HAS_ILOG2_U64 is not set
CONFIG_DEFCONFIG_LIST="/lib/modules/$UNAME_RELEASE/.config"

#
# General setup
#
CONFIG_EXPERIMENTAL=y
CONFIG_LOCK_KERNEL=y
CONFIG_INIT_ENV_ARG_LIMIT=32
CONFIG_LOCALVERSION=""
# CONFIG_LOCALVERSION_AUTO is not set
CONFIG_SWAP=y
CONFIG_SYSVIPC=y
CONFIG_SYSVIPC_SYSCTL=y
# CONFIG_POSIX_MQUEUE is not set
# CONFIG_BSD_PROCESS_ACCT is not set
# CONFIG_TASKSTATS is not set
# CONFIG_USER_NS is not set
# CONFIG_AUDIT is not set
CONFIG_IKCONFIG=y
CONFIG_IKCONFIG_PROC=y
CONFIG_LOG_BUF_SHIFT=15
# CONFIG_CGROUPS is not set
# CONFIG_FAIR_GROUP_SCHED is not set
# CONFIG_SYSFS_DEPRECATED is not set
# CONFIG_RELAY is not set
# CONFIG_BLK_DEV_INITRD is not set
# CONFIG_CC_OPTIMIZE_FOR_SIZE is not set
CONFIG_SYSCTL=y
CONFIG_EMBEDDED=y
# CONFIG_SYSCTL_SYSCALL is not set
CONFIG_KALLSYMS=y
# CONFIG_KALLSYMS_ALL is not set
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_HOTPLUG=y
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
CONFIG_BASE_FULL=y
CONFIG_FUTEX=y
# CONFIG_EPOLL is not set
# CONFIG_SIGNALFD is not set
# CONFIG_EVENTFD is not set
CONFIG_SHMEM=y
# CONFIG_VM_EVENT_COUNTERS is not set
CONFIG_SLUB_DEBUG=y
# CONFIG_SLAB is not set
CONFIG_SLUB=y
# CONFIG_SLOB is not set
CONFIG_RT_MUTEXES=y
# CONFIG_TINY_SHMEM is not set
CONFIG_BASE_SMALL=0
# CONFIG_MODULES is not set
CONFIG_BLOCK=y
# CONFIG_BLK_DEV_IO_TRACE is not set
# CONFIG_BLK_DEV_BSG is not set

#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
# CONFIG_IOSCHED_AS is not set
# CONFIG_IOSCHED_DEADLINE is not set
CONFIG_IOSCHED_CFQ=y
# CONFIG_DEFAULT_AS is not set
# CONFIG_DEFAULT_DEADLINE is not set
CONFIG_DEFAULT_CFQ=y
# CONFIG_DEFAULT_NOOP is not set
CONFIG_DEFAULT_IOSCHED="cfq"

#
# Processor type and features
#
CONFIG_TICK_ONESHOT=y
CONFIG_NO_HZ=y
CONFIG_HIGH_RES_TIMERS=y
CONFIG_GENERIC_CLOCKEVENTS_BUILD=y
CONFIG_X86_PC=y
# CONFIG_X86_VSMP is not set
# CONFIG_MK8 is not set
# CONFIG_MPSC is not set
CONFIG_MCORE2=y
# CONFIG_GENERIC_CPU is not set
CONFIG_X86_L1_CACHE_BYTES=64
CONFIG_X86_L1_CACHE_SHIFT=6
CONFIG_X86_INTERNODE_CACHE_BYTES=64
CONFIG_X86_TSC=y
CONFIG_X86_GOOD_APIC=y
# CONFIG_MICROCODE is not set
# CONFIG_X86_MSR is not set
# CONFIG_X86_CPUID is not set
CONFIG_X86_HT=y
CONFIG_X86_IO_APIC=y
CONFIG_X86_LOCAL_APIC=y
CONFIG_MTRR=y
CONFIG_SMP=y
# CONFIG_SCHED_SMT is not set
# CONFIG_SCHED_MC is not set
# CONFIG_PREEMPT_NONE is not set
CONFIG_PREEMPT_VOLUNTARY=y
# CONFIG_PREEMPT is not set
# CONFIG_PREEMPT_BKL is not set
# CONFIG_NUMA is not set
CONFIG_ARCH_SPARSEMEM_ENABLE=y
CONFIG_ARCH_FLATMEM_ENABLE=y
CONFIG_SELECT_MEMORY_MODEL=y
CONFIG_FLATMEM_MANUAL=y
# CONFIG_DISCONTIGMEM_MANUAL is not set
# CONFIG_SPARSEMEM_MANUAL is not set
CONFIG_FLATMEM=y
CONFIG_FLAT_NODE_MEM_MAP=y
# CONFIG_SPARSEMEM_STATIC is not set
CONFIG_SPARSEMEM_VMEMMAP_ENABLE=y
CONFIG_SPLIT_PTLOCK_CPUS=4
CONFIG_RESOURCES_64BIT=y
CONFIG_ZONE_DMA_FLAG=1
CONFIG_BOUNCE=y
CONFIG_VIRT_TO_BUS=y
CONFIG_NR_CPUS=2
CONFIG_PHYSICAL_ALIGN=0x200000
# CONFIG_HOTPLUG_CPU is not set
CONFIG_ARCH_ENABLE_MEMORY_HOTPLUG=y
CONFIG_HPET_TIMER=y
CONFIG_HPET_EMULATE_RTC=y
# CONFIG_IOMMU is not set
# CONFIG_CALGARY_IOMMU is not set
CONFIG_X86_MCE=y
CONFIG_X86_MCE_INTEL=y
# CONFIG_X86_MCE_AMD is not set
# CONFIG_KEXEC is not set
# CONFIG_CRASH_DUMP is not set
# CONFIG_RELOCATABLE is not set
CONFIG_PHYSICAL_START=0x200000
# CONFIG_SECCOMP is not set
# CONFIG_CC_STACKPROTECTOR is not set
CONFIG_HZ_100=y
# CONFIG_HZ_250 is not set
# CONFIG_HZ_300 is not set
# CONFIG_HZ_1000 is not set
CONFIG_HZ=100
CONFIG_GENERIC_HARDIRQS=y
CONFIG_GENERIC_IRQ_PROBE=y
CONFIG_ISA_DMA_API=y
CONFIG_GENERIC_PENDING_IRQ=y

#
# Power management options
#
CONFIG_PM=y
# CONFIG_PM_LEGACY is not set
# CONFIG_PM_DEBUG is not set
CONFIG_SUSPEND_SMP_POSSIBLE=y
# CONFIG_SUSPEND is not set
CONFIG_HIBERNATION_SMP_POSSIBLE=y
# CONFIG_HIBERNATION is not set
CONFIG_ACPI=y
# CONFIG_ACPI_PROCFS is not set
# CONFIG_ACPI_PROC_EVENT is not set
# CONFIG_ACPI_BUTTON is not set
# CONFIG_ACPI_FAN is not set
# CONFIG_ACPI_DOCK is not set
# CONFIG_ACPI_PROCESSOR is not set
# CONFIG_ACPI_ASUS is not set
# CONFIG_ACPI_TOSHIBA is not set
CONFIG_ACPI_BLACKLIST_YEAR=0
# CONFIG_ACPI_DEBUG is not set
CONFIG_ACPI_EC=y
CONFIG_ACPI_POWER=y
CONFIG_ACPI_SYSTEM=y
# CONFIG_X86_PM_TIMER is not set
# CONFIG_ACPI_CONTAINER is not set

#
# CPU Frequency scaling
#
# CONFIG_CPU_FREQ is not set
# CONFIG_CPU_IDLE is not set

#
# Bus options (PCI etc.)
#
CONFIG_PCI=y
CONFIG_PCI_DIRECT=y
# CONFIG_PCI_MMCONFIG is not set
CONFIG_PCI_DOMAINS=y
# CONFIG_DMAR is not set
# CONFIG_PCIEPORTBUS is not set
CONFIG_ARCH_SUPPORTS_MSI=y
CONFIG_PCI_MSI=y
# CONFIG_PCI_DEBUG is not set
# CONFIG_HT_IRQ is not set
# CONFIG_PCCARD is not set
# CONFIG_HOTPLUG_PCI is not set

#
# Executable file formats / Emulations
#
CONFIG_BINFMT_ELF=y
# CONFIG_BINFMT_MISC is not set
# CONFIG_IA32_EMULATION is not set
# CONFIG_COMPAT_FOR_U64_ALIGNMENT is not set

#
# Networking
#
CONFIG_NET=y

#
# Networking options
#
# CONFIG_PACKET is not set
CONFIG_UNIX=y
# CONFIG_NET_KEY is not set
CONFIG_INET=y
# CONFIG_IP_MULTICAST is not set
# CONFIG_IP_ADVANCED_ROUTER is not set
CONFIG_IP_FIB_HASH=y
# CONFIG_IP_PNP is not set
# CONFIG_NET_IPIP is not set
# CONFIG_NET_IPGRE is not set
# CONFIG_ARPD is not set
# CONFIG_SYN_COOKIES is not set
# CONFIG_INET_AH is not set
# CONFIG_INET_ESP is not set
# CONFIG_INET_IPCOMP is not set
# CONFIG_INET_XFRM_TUNNEL is not set
# CONFIG_INET_TUNNEL is not set
# CONFIG_INET_XFRM_MODE_TRANSPORT is not set
# CONFIG_INET_XFRM_MODE_TUNNEL is not set
# CONFIG_INET_XFRM_MODE_BEET is not set
# CONFIG_INET_LRO is not set
# CONFIG_INET_DIAG is not set
# CONFIG_TCP_CONG_ADVANCED is not set
CONFIG_TCP_CONG_CUBIC=y
CONFIG_DEFAULT_TCP_CONG="cubic"
# CONFIG_TCP_MD5SIG is not set
# CONFIG_IPV6 is not set
# CONFIG_INET6_XFRM_TUNNEL is not set
# CONFIG_INET6_TUNNEL is not set
# CONFIG_NETWORK_SECMARK is not set
# CONFIG_NETFILTER is not set
# CONFIG_IP_DCCP is not set
# CONFIG_IP_SCTP is not set
# CONFIG_TIPC is not set
# CONFIG_ATM is not set
# CONFIG_BRIDGE is not set
# CONFIG_VLAN_8021Q is not set
# CONFIG_DECNET is not set
# CONFIG_LLC2 is not set
# CONFIG_IPX is not set
# CONFIG_ATALK is not set
# CONFIG_X25 is not set
# CONFIG_LAPB is not set
# CONFIG_ECONET is not set
# CONFIG_WAN_ROUTER is not set
# CONFIG_NET_SCHED is not set

#
# Network testing
#
# CONFIG_NET_PKTGEN is not set
# CONFIG_HAMRADIO is not set
# CONFIG_IRDA is not set
# CONFIG_BT is not set
# CONFIG_AF_RXRPC is not set

#
# Wireless
#
# CONFIG_CFG80211 is not set
# CONFIG_WIRELESS_EXT is not set
# CONFIG_MAC80211 is not set
# CONFIG_IEEE80211 is not set
# CONFIG_RFKILL is not set
# CONFIG_NET_9P is not set

#
# Device Drivers
#

#
# Generic Driver Options
#
CONFIG_UEVENT_HELPER_PATH="/sbin/hotplug"
CONFIG_STANDALONE=y
CONFIG_PREVENT_FIRMWARE_BUILD=y
# CONFIG_FW_LOADER is not set
# CONFIG_DEBUG_DRIVER is not set
# CONFIG_DEBUG_DEVRES is not set
# CONFIG_SYS_HYPERVISOR is not set
# CONFIG_CONNECTOR is not set
# CONFIG_MTD is not set
# CONFIG_PARPORT is not set
CONFIG_PNP=y
# CONFIG_PNP_DEBUG is not set

#
# Protocols
#
CONFIG_PNPACPI=y
# CONFIG_BLK_DEV is not set
# CONFIG_MISC_DEVICES is not set
# CONFIG_IDE is not set

#
# SCSI device support
#
# CONFIG_RAID_ATTRS is not set
CONFIG_SCSI=y
CONFIG_SCSI_DMA=y
# CONFIG_SCSI_TGT is not set
# CONFIG_SCSI_NETLINK is not set
# CONFIG_SCSI_PROC_FS is not set

#
# SCSI support type (disk, tape, CD-ROM)
#
CONFIG_BLK_DEV_SD=y
# CONFIG_CHR_DEV_ST is not set
# CONFIG_CHR_DEV_OSST is not set
CONFIG_BLK_DEV_SR=y
# CONFIG_BLK_DEV_SR_VENDOR is not set
# CONFIG_CHR_DEV_SG is not set
# CONFIG_CHR_DEV_SCH is not set

#
# Some SCSI devices (e.g. CD jukebox) support multiple LUNs
#
# CONFIG_SCSI_MULTI_LUN is not set
# CONFIG_SCSI_CONSTANTS is not set
# CONFIG_SCSI_LOGGING is not set
# CONFIG_SCSI_SCAN_ASYNC is not set

#
# SCSI Transports
#
# CONFIG_SCSI_SPI_ATTRS is not set
# CONFIG_SCSI_FC_ATTRS is not set
# CONFIG_SCSI_ISCSI_ATTRS is not set
# CONFIG_SCSI_SAS_LIBSAS is not set
# CONFIG_SCSI_SRP_ATTRS is not set
# CONFIG_SCSI_LOWLEVEL is not set
CONFIG_ATA=y
# CONFIG_ATA_NONSTANDARD is not set
CONFIG_ATA_ACPI=y
CONFIG_SATA_AHCI=y
# CONFIG_SATA_SVW is not set
CONFIG_ATA_PIIX=y
# CONFIG_SATA_MV is not set
# CONFIG_SATA_NV is not set
# CONFIG_PDC_ADMA is not set
# CONFIG_SATA_QSTOR is not set
# CONFIG_SATA_PROMISE is not set
# CONFIG_SATA_SX4 is not set
# CONFIG_SATA_SIL is not set
# CONFIG_SATA_SIL24 is not set
# CONFIG_SATA_SIS is not set
# CONFIG_SATA_ULI is not set
# CONFIG_SATA_VIA is not set
# CONFIG_SATA_VITESSE is not set
# CONFIG_SATA_INIC162X is not set
# CONFIG_PATA_ACPI is not set
# CONFIG_PATA_ALI is not set
# CONFIG_PATA_AMD is not set
# CONFIG_PATA_ARTOP is not set
# CONFIG_PATA_ATIIXP is not set
# CONFIG_PATA_CMD640_PCI is not set
# CONFIG_PATA_CMD64X is not set
# CONFIG_PATA_CS5520 is not set
# CONFIG_PATA_CS5530 is not set
# CONFIG_PATA_CYPRESS is not set
# CONFIG_PATA_EFAR is not set
# CONFIG_ATA_GENERIC is not set
# CONFIG_PATA_HPT366 is not set
# CONFIG_PATA_HPT37X is not set
# CONFIG_PATA_HPT3X2N is not set
# CONFIG_PATA_HPT3X3 is not set
# CONFIG_PATA_IT821X is not set
# CONFIG_PATA_IT8213 is not set
CONFIG_PATA_JMICRON=y
# CONFIG_PATA_TRIFLEX is not set
# CONFIG_PATA_MARVELL is not set
# CONFIG_PATA_MPIIX is not set
# CONFIG_PATA_OLDPIIX is not set
# CONFIG_PATA_NETCELL is not set
# CONFIG_PATA_NS87410 is not set
# CONFIG_PATA_NS87415 is not set
# CONFIG_PATA_OPTI is not set
# CONFIG_PATA_OPTIDMA is not set
# CONFIG_PATA_PDC_OLD is not set
# CONFIG_PATA_RADISYS is not set
# CONFIG_PATA_RZ1000 is not set
# CONFIG_PATA_SC1200 is not set
# CONFIG_PATA_SERVERWORKS is not set
# CONFIG_PATA_PDC2027X is not set
# CONFIG_PATA_SIL680 is not set
# CONFIG_PATA_SIS is not set
# CONFIG_PATA_VIA is not set
# CONFIG_PATA_WINBOND is not set
# CONFIG_PATA_PLATFORM is not set
# CONFIG_MD is not set
# CONFIG_FUSION is not set

#
# IEEE 1394 (FireWire) support
#
# CONFIG_FIREWIRE is not set
# CONFIG_IEEE1394 is not set
# CONFIG_I2O is not set
# CONFIG_MACINTOSH_DRIVERS is not set
CONFIG_NETDEVICES=y
# CONFIG_NETDEVICES_MULTIQUEUE is not set
# CONFIG_DUMMY is not set
# CONFIG_BONDING is not set
# CONFIG_MACVLAN is not set
# CONFIG_EQUALIZER is not set
# CONFIG_TUN is not set
# CONFIG_VETH is not set
# CONFIG_NET_SB1000 is not set
# CONFIG_IP1000 is not set
# CONFIG_ARCNET is not set
# CONFIG_NET_ETHERNET is not set
CONFIG_MII=y
CONFIG_NETDEV_1000=y
# CONFIG_ACENIC is not set
# CONFIG_DL2K is not set
# CONFIG_E1000 is not set
# CONFIG_E1000E is not set
# CONFIG_NS83820 is not set
# CONFIG_HAMACHI is not set
# CONFIG_YELLOWFIN is not set
# CONFIG_R8169 is not set
# CONFIG_SIS190 is not set
# CONFIG_SKGE is not set
# CONFIG_SKY2 is not set
# CONFIG_SK98LIN is not set
# CONFIG_VIA_VELOCITY is not set
# CONFIG_TIGON3 is not set
# CONFIG_BNX2 is not set
# CONFIG_QLA3XXX is not set
CONFIG_ATL1=y
# CONFIG_NETDEV_10000 is not set
# CONFIG_TR is not set

#
# Wireless LAN
#
# CONFIG_WLAN_PRE80211 is not set
# CONFIG_WLAN_80211 is not set
# CONFIG_WAN is not set
# CONFIG_FDDI is not set
# CONFIG_HIPPI is not set
# CONFIG_PPP is not set
# CONFIG_SLIP is not set
# CONFIG_NET_FC is not set
# CONFIG_SHAPER is not set
CONFIG_NETCONSOLE=y
# CONFIG_NETCONSOLE_DYNAMIC is not set
CONFIG_NETPOLL=y
# CONFIG_NETPOLL_TRAP is not set
CONFIG_NET_POLL_CONTROLLER=y
# CONFIG_ISDN is not set
# CONFIG_PHONE is not set

#
# Input device support
#
CONFIG_INPUT=y
# CONFIG_INPUT_FF_MEMLESS is not set
# CONFIG_INPUT_POLLDEV is not set

#
# Userland interfaces
#
# CONFIG_INPUT_MOUSEDEV is not set
# CONFIG_INPUT_JOYDEV is not set
# CONFIG_INPUT_EVDEV is not set
# CONFIG_INPUT_EVBUG is not set

#
# Input Device Drivers
#
CONFIG_INPUT_KEYBOARD=y
CONFIG_KEYBOARD_ATKBD=y
# CONFIG_KEYBOARD_SUNKBD is not set
# CONFIG_KEYBOARD_LKKBD is not set
# CONFIG_KEYBOARD_XTKBD is not set
# CONFIG_KEYBOARD_NEWTON is not set
# CONFIG_KEYBOARD_STOWAWAY is not set
# CONFIG_INPUT_MOUSE is not set
# CONFIG_INPUT_JOYSTICK is not set
# CONFIG_INPUT_TABLET is not set
# CONFIG_INPUT_TOUCHSCREEN is not set
# CONFIG_INPUT_MISC is not set

#
# Hardware I/O ports
#
CONFIG_SERIO=y
CONFIG_SERIO_I8042=y
# CONFIG_SERIO_SERPORT is not set
# CONFIG_SERIO_CT82C710 is not set
# CONFIG_SERIO_PCIPS2 is not set
CONFIG_SERIO_LIBPS2=y
# CONFIG_SERIO_RAW is not set
# CONFIG_GAMEPORT is not set

#
# Character devices
#
CONFIG_VT=y
CONFIG_VT_CONSOLE=y
CONFIG_HW_CONSOLE=y
# CONFIG_VT_HW_CONSOLE_BINDING is not set
# CONFIG_SERIAL_NONSTANDARD is not set

#
# Serial drivers
#
# CONFIG_SERIAL_8250 is not set
CONFIG_FIX_EARLYCON_MEM=y

#
# Non-8250 serial port support
#
# CONFIG_SERIAL_JSM is not set
CONFIG_UNIX98_PTYS=y
# CONFIG_LEGACY_PTYS is not set
# CONFIG_IPMI_HANDLER is not set
# CONFIG_HW_RANDOM is not set
# CONFIG_NVRAM is not set
CONFIG_RTC=y
# CONFIG_R3964 is not set
# CONFIG_APPLICOM is not set
# CONFIG_MWAVE is not set
# CONFIG_PC8736x_GPIO is not set
# CONFIG_RAW_DRIVER is not set
# CONFIG_HPET is not set
# CONFIG_HANGCHECK_TIMER is not set
# CONFIG_TCG_TPM is not set
# CONFIG_TELCLOCK is not set
CONFIG_DEVPORT=y
# CONFIG_I2C is not set

#
# SPI support
#
# CONFIG_SPI is not set
# CONFIG_SPI_MASTER is not set
# CONFIG_W1 is not set
# CONFIG_POWER_SUPPLY is not set
CONFIG_HWMON=y
CONFIG_HWMON_VID=y
# CONFIG_SENSORS_ABITUGURU is not set
# CONFIG_SENSORS_ABITUGURU3 is not set
# CONFIG_SENSORS_K8TEMP is not set
# CONFIG_SENSORS_F71805F is not set
# CONFIG_SENSORS_F71882FG is not set
CONFIG_SENSORS_CORETEMP=y
# CONFIG_SENSORS_IT87 is not set
# CONFIG_SENSORS_PC87360 is not set
# CONFIG_SENSORS_PC87427 is not set
# CONFIG_SENSORS_SIS5595 is not set
# CONFIG_SENSORS_SMSC47M1 is not set
# CONFIG_SENSORS_SMSC47B397 is not set
# CONFIG_SENSORS_VIA686A is not set
# CONFIG_SENSORS_VT1211 is not set
# CONFIG_SENSORS_VT8231 is not set
# CONFIG_SENSORS_W83627HF is not set
CONFIG_SENSORS_W83627EHF=y
# CONFIG_SENSORS_HDAPS is not set
# CONFIG_SENSORS_APPLESMC is not set
# CONFIG_HWMON_DEBUG_CHIP is not set
# CONFIG_WATCHDOG is not set

#
# Sonics Silicon Backplane
#
CONFIG_SSB_POSSIBLE=y
# CONFIG_SSB is not set

#
# Multifunction device drivers
#
# CONFIG_MFD_SM501 is not set

#
# Multimedia devices
#
# CONFIG_VIDEO_DEV is not set
# CONFIG_DVB_CORE is not set
# CONFIG_DAB is not set

#
# Graphics support
#
# CONFIG_AGP is not set
# CONFIG_DRM is not set
# CONFIG_VGASTATE is not set
# CONFIG_VIDEO_OUTPUT_CONTROL is not set
# CONFIG_FB is not set
# CONFIG_BACKLIGHT_LCD_SUPPORT is not set

#
# Display device support
#
# CONFIG_DISPLAY_SUPPORT is not set

#
# Console display driver support
#
CONFIG_VGA_CONSOLE=y
# CONFIG_VGACON_SOFT_SCROLLBACK is not set
# CONFIG_VIDEO_SELECT is not set
CONFIG_DUMMY_CONSOLE=y

#
# Sound
#
# CONFIG_SOUND is not set
# CONFIG_HID_SUPPORT is not set
# CONFIG_USB_SUPPORT is not set
# CONFIG_MMC is not set
# CONFIG_NEW_LEDS is not set
# CONFIG_INFINIBAND is not set
# CONFIG_EDAC is not set
# CONFIG_RTC_CLASS is not set
# CONFIG_DMADEVICES is not set
# CONFIG_VIRTUALIZATION is not set

#
# Userspace I/O
#
# CONFIG_UIO is not set

#
# Firmware Drivers
#
# CONFIG_EDD is not set
# CONFIG_DELL_RBU is not set
# CONFIG_DCDBAS is not set
# CONFIG_DMIID is not set

#
# File systems
#
CONFIG_EXT2_FS=y
# CONFIG_EXT2_FS_XATTR is not set
# CONFIG_EXT2_FS_XIP is not set
CONFIG_EXT3_FS=y
# CONFIG_EXT3_FS_XATTR is not set
# CONFIG_EXT4DEV_FS is not set
CONFIG_JBD=y
# CONFIG_REISERFS_FS is not set
# CONFIG_JFS_FS is not set
# CONFIG_FS_POSIX_ACL is not set
# CONFIG_XFS_FS is not set
# CONFIG_GFS2_FS is not set
# CONFIG_OCFS2_FS is not set
# CONFIG_MINIX_FS is not set
# CONFIG_ROMFS_FS is not set
# CONFIG_INOTIFY is not set
# CONFIG_QUOTA is not set
# CONFIG_DNOTIFY is not set
# CONFIG_AUTOFS_FS is not set
# CONFIG_AUTOFS4_FS is not set
# CONFIG_FUSE_FS is not set

#
# CD-ROM/DVD Filesystems
#
# CONFIG_ISO9660_FS is not set
# CONFIG_UDF_FS is not set

#
# DOS/FAT/NT Filesystems
#
# CONFIG_MSDOS_FS is not set
# CONFIG_VFAT_FS is not set
# CONFIG_NTFS_FS is not set

#
# Pseudo filesystems
#
CONFIG_PROC_FS=y
# CONFIG_PROC_KCORE is not set
CONFIG_PROC_SYSCTL=y
CONFIG_SYSFS=y
CONFIG_TMPFS=y
# CONFIG_TMPFS_POSIX_ACL is not set
# CONFIG_HUGETLBFS is not set
# CONFIG_HUGETLB_PAGE is not set
# CONFIG_CONFIGFS_FS is not set

#
# Miscellaneous filesystems
#
# CONFIG_ADFS_FS is not set
# CONFIG_AFFS_FS is not set
# CONFIG_HFS_FS is not set
# CONFIG_HFSPLUS_FS is not set
# CONFIG_BEFS_FS is not set
# CONFIG_BFS_FS is not set
# CONFIG_EFS_FS is not set
# CONFIG_CRAMFS is not set
# CONFIG_VXFS_FS is not set
# CONFIG_HPFS_FS is not set
# CONFIG_QNX4FS_FS is not set
# CONFIG_SYSV_FS is not set
# CONFIG_UFS_FS is not set
# CONFIG_NETWORK_FILESYSTEMS is not set

#
# Partition Types
#
# CONFIG_PARTITION_ADVANCED is not set
CONFIG_MSDOS_PARTITION=y
# CONFIG_NLS is not set
# CONFIG_DLM is not set

#
# Kernel hacking
#
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
# CONFIG_PRINTK_TIME is not set
# CONFIG_ENABLE_MUST_CHECK is not set
CONFIG_MAGIC_SYSRQ=y
# CONFIG_UNUSED_SYMBOLS is not set
# CONFIG_DEBUG_FS is not set
# CONFIG_HEADERS_CHECK is not set
CONFIG_DEBUG_KERNEL=y
# CONFIG_DEBUG_SHIRQ is not set
# CONFIG_DETECT_SOFTLOCKUP is not set
# CONFIG_SCHED_DEBUG is not set
# CONFIG_SCHEDSTATS is not set
# CONFIG_TIMER_STATS is not set
# CONFIG_SLUB_DEBUG_ON is not set
# CONFIG_DEBUG_RT_MUTEXES is not set
# CONFIG_RT_MUTEX_TESTER is not set
# CONFIG_DEBUG_SPINLOCK is not set
# CONFIG_DEBUG_MUTEXES is not set
# CONFIG_DEBUG_LOCK_ALLOC is not set
# CONFIG_PROVE_LOCKING is not set
# CONFIG_LOCK_STAT is not set
# CONFIG_DEBUG_SPINLOCK_SLEEP is not set
# CONFIG_DEBUG_LOCKING_API_SELFTESTS is not set
# CONFIG_DEBUG_KOBJECT is not set
CONFIG_DEBUG_BUGVERBOSE=y
# CONFIG_DEBUG_INFO is not set
# CONFIG_DEBUG_VM is not set
# CONFIG_DEBUG_LIST is not set
# CONFIG_DEBUG_SG is not set
# CONFIG_FRAME_POINTER is not set
# CONFIG_FORCED_INLINING is not set
# CONFIG_BOOT_PRINTK_DELAY is not set
# CONFIG_FAULT_INJECTION is not set
# CONFIG_SAMPLES is not set
# CONFIG_DEBUG_RODATA is not set
# CONFIG_DEBUG_STACKOVERFLOW is not set
# CONFIG_DEBUG_STACK_USAGE is not set

#
# Security options
#
# CONFIG_KEYS is not set
# CONFIG_SECURITY is not set
# CONFIG_SECURITY_FILE_CAPABILITIES is not set
# CONFIG_CRYPTO is not set

#
# Library routines
#
CONFIG_BITREVERSE=y
# CONFIG_CRC_CCITT is not set
# CONFIG_CRC16 is not set
# CONFIG_CRC_ITU_T is not set
CONFIG_CRC32=y
# CONFIG_CRC7 is not set
# CONFIG_LIBCRC32C is not set
CONFIG_PLIST=y
CONFIG_HAS_IOMEM=y
CONFIG_HAS_IOPORT=y
CONFIG_HAS_DMA=y

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: SLUB 0:1 SLAB (OOM during massive parallel kernel builds)
  2007-10-23 18:16 SLUB 0:1 SLAB (OOM during massive parallel kernel builds) Alexey Dobriyan
@ 2007-10-23 19:35 ` Christoph Lameter
  2007-10-23 19:48   ` Pekka Enberg
  2007-10-23 20:04   ` Pekka Enberg
  0 siblings, 2 replies; 14+ messages in thread
From: Christoph Lameter @ 2007-10-23 19:35 UTC (permalink / raw)
  To: Alexey Dobriyan; +Cc: penberg, linux-kernel, linux-mm

On Tue, 23 Oct 2007, Alexey Dobriyan wrote:

> Nasty OOM killings appear during massive parallel kernel builds with
> SLUB, but not with SLAB. By nasty I mean, cc1 processes are killed --
> object files in .ccache and tree are corrupted which makes me to
> blow up them entirely.

Hmmmm... Memory is corrupted? I guess a race destroys memory? 

> 
> With SLAB this workload never went to OOM killer.
> With SLUB and pretty much all debugging enabled, it finishes to the end
> (albeit slowly).
> With SLUB and no debugging, OOM killer kicks in.

Not sure what this is. Maybe the slowing SLUB solves the race.
Could you try and boot with

slub_debug=F

to enable limited debugging?

> cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Regular Order 0 alloc .... but why is there no memory available , 
reclaimed?


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: SLUB 0:1 SLAB (OOM during massive parallel kernel builds)
  2007-10-23 19:35 ` Christoph Lameter
@ 2007-10-23 19:48   ` Pekka Enberg
  2007-10-23 19:57     ` Christoph Lameter
  2007-10-23 20:04   ` Pekka Enberg
  1 sibling, 1 reply; 14+ messages in thread
From: Pekka Enberg @ 2007-10-23 19:48 UTC (permalink / raw)
  To: Christoph Lameter; +Cc: Alexey Dobriyan, linux-kernel, linux-mm

Hi,

On Tue, 23 Oct 2007, Alexey Dobriyan wrote:
> > cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0

Christoph Lameter wrote:
> Regular Order 0 alloc .... but why is there no memory available , 
> reclaimed?

I am too wondering where all that memory is going. Logging /proc/meminfo 
and slabinfo -S (see Documentation/vm/slabinfo.c) every few minutes or 
so probably might help answering that question.

				Pekka


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: SLUB 0:1 SLAB (OOM during massive parallel kernel builds)
  2007-10-23 19:48   ` Pekka Enberg
@ 2007-10-23 19:57     ` Christoph Lameter
  2007-10-24  8:09       ` Mel Gorman
  0 siblings, 1 reply; 14+ messages in thread
From: Christoph Lameter @ 2007-10-23 19:57 UTC (permalink / raw)
  To: Pekka Enberg; +Cc: Alexey Dobriyan, linux-kernel, linux-mm, mel

On Tue, 23 Oct 2007, Pekka Enberg wrote:

> > > cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0
> 
> Christoph Lameter wrote:
> > Regular Order 0 alloc .... but why is there no memory available , reclaimed?
> 
> I am too wondering where all that memory is going. Logging /proc/meminfo and
> slabinfo -S (see Documentation/vm/slabinfo.c) every few minutes or so probably
> might help answering that question

__GFP_HARDWALL and __GFP_MOVABLE is set. I guess we have no cpusets on the 
box? Wonder what the antifrag status is? Can we get /proc/pagetypeinfo?



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: SLUB 0:1 SLAB (OOM during massive parallel kernel builds)
  2007-10-23 19:35 ` Christoph Lameter
  2007-10-23 19:48   ` Pekka Enberg
@ 2007-10-23 20:04   ` Pekka Enberg
  2007-10-23 20:09     ` Christoph Lameter
  1 sibling, 1 reply; 14+ messages in thread
From: Pekka Enberg @ 2007-10-23 20:04 UTC (permalink / raw)
  To: Christoph Lameter; +Cc: Alexey Dobriyan, linux-kernel, linux-mm

Hi,

On Tue, 23 Oct 2007, Alexey Dobriyan wrote:
> > With SLAB this workload never went to OOM killer.
> > With SLUB and pretty much all debugging enabled, it finishes to the end
> > (albeit slowly).
> > With SLUB and no debugging, OOM killer kicks in.

On 10/23/07, Christoph Lameter <clameter@sgi.com> wrote:
> Not sure what this is. Maybe the slowing SLUB solves the race.

What kind of race are you thinking of? What I initially thought was
that the problem is that SLUB messes up some other VM heuristics due
to different object sizes, not holding on to empty slabs, and/or page
allocator pass-through. I guess only object size is affected by
debugging, though?

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: SLUB 0:1 SLAB (OOM during massive parallel kernel builds)
  2007-10-23 20:04   ` Pekka Enberg
@ 2007-10-23 20:09     ` Christoph Lameter
  2007-10-23 20:41       ` Pekka Enberg
  0 siblings, 1 reply; 14+ messages in thread
From: Christoph Lameter @ 2007-10-23 20:09 UTC (permalink / raw)
  To: Pekka Enberg; +Cc: Alexey Dobriyan, linux-kernel, linux-mm

On Tue, 23 Oct 2007, Pekka Enberg wrote:

> On 10/23/07, Christoph Lameter <clameter@sgi.com> wrote:
> > Not sure what this is. Maybe the slowing SLUB solves the race.
> 
> What kind of race are you thinking of? What I initially thought was
> that the problem is that SLUB messes up some other VM heuristics due
> to different object sizes, not holding on to empty slabs, and/or page
> allocator pass-through. I guess only object size is affected by
> debugging, though?

The number of objects per page is reduced by enabling full debugging. That 
triggers a potential of more order 1 allocations but we are failing at 
order 0 allocs. slub_debug=F does not increase object size with debugging 
information but keeps things as they are while doing as much verification 
of slab integrity as possible.

One--potentially far fetched theory--is that the difference in alloc 
performance may trigger a race. But I guess it is more likely that we will 
find something wrong with the gfp flags (that I recategorized for 2.6.24).



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: SLUB 0:1 SLAB (OOM during massive parallel kernel builds)
  2007-10-23 20:09     ` Christoph Lameter
@ 2007-10-23 20:41       ` Pekka Enberg
  2007-10-24 16:56         ` Christoph Lameter
  0 siblings, 1 reply; 14+ messages in thread
From: Pekka Enberg @ 2007-10-23 20:41 UTC (permalink / raw)
  To: Christoph Lameter; +Cc: Alexey Dobriyan, linux-kernel, Linux MM

Hi Christoph,

(I fixed linux-mm cc to kvack.org.)

On 10/23/07, Christoph Lameter <clameter@sgi.com> wrote:
> The number of objects per page is reduced by enabling full debugging. That
> triggers a potential of more order 1 allocations but we are failing at
> order 0 allocs.

Yeah, but we're _not failing_ when debugging is enabled. Thus, it's
likely, that the _failing_ (non-debug) case has potential for more
order 0 allocs, no? I am just guessing here but maybe it's
slab_order() behaving differently from calculate_slab_order() so that
we see more order 0 pressure in SLUB than SLAB?

                                     Pekka

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: SLUB 0:1 SLAB (OOM during massive parallel kernel builds)
  2007-10-23 19:57     ` Christoph Lameter
@ 2007-10-24  8:09       ` Mel Gorman
  2007-10-24 19:28         ` Alexey Dobriyan
  0 siblings, 1 reply; 14+ messages in thread
From: Mel Gorman @ 2007-10-24  8:09 UTC (permalink / raw)
  To: Christoph Lameter; +Cc: Pekka Enberg, Alexey Dobriyan, linux-kernel, linux-mm

On (23/10/07 12:57), Christoph Lameter didst pronounce:
> On Tue, 23 Oct 2007, Pekka Enberg wrote:
> 
> > > > cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0
> > 
> > Christoph Lameter wrote:
> > > Regular Order 0 alloc .... but why is there no memory available , reclaimed?
> > 
> > I am too wondering where all that memory is going. Logging /proc/meminfo and
> > slabinfo -S (see Documentation/vm/slabinfo.c) every few minutes or so probably
> > might help answering that question
> 
> __GFP_HARDWALL and __GFP_MOVABLE is set. I guess we have no cpusets on the 
> box? Wonder what the antifrag status is? Can we get /proc/pagetypeinfo?
> 

I do not believe the anti-frag status will make a difference as this is
a order-0 failure. Looking at the OOM messages;

Failure 1
> Active:386969 inactive:66065 dirty:0 writeback:0 unstable:0 free:3431 slab:11685 mapped:243 pagetables:14621 bounce:0
> DMA free:8036kB min:32kB low:40kB high:48kB active:2008kB inactive:1172kB present:12744kB pages_scanned:6852 all_unreclaimable? yes
> lowmem_reserve[]: 0 2003 2003 2003
> DMA32 free:5688kB min:5708kB low:7132kB high:8560kB active:1548100kB inactive:261040kB present:2051880kB pages_scanned:12798 112 all_unreclaimable? yes
> lowmem_reserve[]: 0 0 0 0
> DMA: 13*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8036kB
> DMA32: 518*4kB 6*8kB 19*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5688kB
> Swap cache: add 4908248, delete 4908248, find 699858/1120395, race 92+89
> Free swap  = 0kB
> Total swap = 987956kB
> Free swap:            0kB

No reclaimable memory, no free swap and free memory is below the min free
watermark on the DMA32 zone being allocated from. That system is genuinely
out of memory although I am surprised it didn't fallback to ZONE_DMA but
the likely explanation is the lowmem reserves.

Failure 2
> Active:306488 inactive:146569 dirty:0 writeback:0 unstable:0 free:3415 slab:11631 mapped:2 pagetables:14611 bounce:0
> DMA free:8044kB min:32kB low:40kB high:48kB active:1596kB
> inactive:1576kB present:12744kB pages_scanned:14865 all_unreclaimable?  yes
> lowmem_reserve[]: 0 2003 2003 2003
> DMA32 free:5616kB min:5708kB low:7132kB high:8560kB active:1226588kB inactive:582396kB present:2051880kB pages_scanned:13650148 all_unreclaimable? yes
> lowmem_reserve[]: 0 0 0 0
> DMA: 15*4kB 86*8kB 60*16kB 26*32kB 6*64kB 8*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8044kB
> DMA32: 498*4kB 5*8kB 20*16kB 12*32kB 1*64kB 0*128kB 1*256kB 1*512kB 0*1024kB 1*2048kB 0*4096kB = 5616kB
> Swap cache: add 4908570, delete 4908570, find 699873/1120438, race 92+89
> Free swap  = 0kB
> Total swap = 987956kB
> Free swap:            0kB

Same again - really OOM.

It seems to be the same all the way through.

I am somewhat at a loss to explain why SLUB would fail in this case when
SLAB wouldn't. One possibility is that SLAB is artifically holding up
processes in allocation so that fewer are running at the same time allowing
the overall job to complete. For a compile-time benchmark, it might not be
very noticable as the processes are already pretty short-lived. It would be
interesting to add more swap so that SLUB doesn't fail and see if there is
a difference in time-to-completion.

If SLUB is faster, it might imply that it is letting more compile-jobs
running at the same time. Output of vmstat during both runs (preferably with
an indication on the SLUB failure where the OOM occured) may tell us. If the
"running" and "blocked" column for SLUB are significantly higher than the
averages for SLAB, would it support that theory Christoph?

Another possibility is that SLUBs runtime overhead is higher although I find
that one harder to believe.

-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab
-- 
-- 
Mel Gorman
Part-time Phd Student                          Linux Technology Center
University of Limerick                         IBM Dublin Software Lab

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: SLUB 0:1 SLAB (OOM during massive parallel kernel builds)
  2007-10-23 20:41       ` Pekka Enberg
@ 2007-10-24 16:56         ` Christoph Lameter
  0 siblings, 0 replies; 14+ messages in thread
From: Christoph Lameter @ 2007-10-24 16:56 UTC (permalink / raw)
  To: Pekka Enberg; +Cc: Alexey Dobriyan, linux-kernel, Linux MM

On Tue, 23 Oct 2007, Pekka Enberg wrote:

> Yeah, but we're _not failing_ when debugging is enabled. Thus, it's
> likely, that the _failing_ (non-debug) case has potential for more
> order 0 allocs, no? I am just guessing here but maybe it's
> slab_order() behaving differently from calculate_slab_order() so that
> we see more order 0 pressure in SLUB than SLAB?

Seesm that order 0 pressure is better than order 1?



^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: SLUB 0:1 SLAB (OOM during massive parallel kernel builds)
  2007-10-24  8:09       ` Mel Gorman
@ 2007-10-24 19:28         ` Alexey Dobriyan
  2007-10-25  2:15           ` Christoph Lameter
  0 siblings, 1 reply; 14+ messages in thread
From: Alexey Dobriyan @ 2007-10-24 19:28 UTC (permalink / raw)
  To: Mel Gorman; +Cc: Christoph Lameter, Pekka Enberg, linux-kernel, linux-mm

On Wed, Oct 24, 2007 at 09:09:38AM +0100, Mel Gorman wrote:
> On (23/10/07 12:57), Christoph Lameter didst pronounce:
> > On Tue, 23 Oct 2007, Pekka Enberg wrote:
> > 
> > > > > cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0
> > > 
> > > Christoph Lameter wrote:
> > > > Regular Order 0 alloc .... but why is there no memory available , reclaimed?

Thanks, guys for suggestions.

I redid workload with slub_debug=F and collected /proc/meminfo, slabinfo
-S and /proc/pagetyinfo. It OOMed. Here are first few messages from OOM
killer and few last messages from proc files. During run collecting
script managed to wakeup once in 1-3 minutes. There are 12 minutes between
last and second-to-last messages which is I assume when X happened.
If needed, I can send complete writeout.

I'm going to add more swap now and make timings.




[   36.146064] Adding 987956k swap on /dev/sda1.  Priority:-1 extents:1 across:987956k
[12728.699150] cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=-17
[12728.699299] 
[12728.699299] Call Trace:
[12728.699422]  [<ffffffff8024d176>] oom_kill_process+0xf6/0x110
[12728.699505]  [<ffffffff8024d62d>] out_of_memory+0x1ad/0x1f0
[12728.699587]  [<ffffffff8024f92e>] __alloc_pages+0x28e/0x3a0
[12728.699669]  [<ffffffff80251a61>] __do_page_cache_readahead+0x101/0x270
[12728.699757]  [<ffffffff8024c5ca>] filemap_fault+0x28a/0x3c0
[12728.699839]  [<ffffffff802575bc>] __do_fault+0x6c/0x430
[12728.699920]  [<ffffffff8025946b>] handle_mm_fault+0x3bb/0x700
[12728.700004]  [<ffffffff8021c1ff>] do_page_fault+0x1bf/0x830
[12728.700086]  [<ffffffff8025e94c>] do_mmap_pgoff+0x2fc/0x360
[12728.700168]  [<ffffffff8025d426>] do_munmap+0x296/0x2f0
[12728.700249]  [<ffffffff802e2d11>] __up_write+0x21/0x130
[12728.700331]  [<ffffffff803d3c99>] error_exit+0x0/0x51
[12728.700409] 
[12728.700465] Mem-info:
[12728.700526] DMA per-cpu:
[12728.700589] CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
[12728.700732] CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
[12728.700874] DMA32 per-cpu:
[12728.700938] CPU    0: Hot: hi:  186, btch:  31 usd:  94   Cold: hi:   62, btch:  15 usd:  30
[12728.701081] CPU    1: Hot: hi:  186, btch:  31 usd: 115   Cold: hi:   62, btch:  15 usd:  57
[12728.701224] Active:433892 inactive:22010 dirty:0 writeback:0 unstable:0
[12728.701225]  free:3433 slab:10898 mapped:9 pagetables:14315 bounce:0
[12728.701398] DMA free:8032kB min:32kB low:40kB high:48kB active:2716kB inactive:2208kB present:12744kB pages_scanned:9299 all_unreclaimable? yes
[12728.701567] lowmem_reserve[]: 0 2003 2003 2003
[12728.701654] DMA32 free:5700kB min:5708kB low:7132kB high:8560kB active:1732852kB inactive:85720kB present:2051880kB pages_scanned:18914517 all_unreclaimable? yes
[12728.701877] lowmem_reserve[]: 0 0 0 0
[12728.701961] DMA: 8*4kB 2*8kB 11*16kB 12*32kB 12*64kB 12*128kB 4*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
[12728.702139] DMA32: 49*4kB 14*8kB 9*16kB 2*32kB 1*64kB 2*128kB 1*256kB 1*512kB 0*1024kB 0*2048kB 1*4096kB = 5700kB
[12728.702319] Swap cache: add 5538322, delete 5538322, find 910933/1354692, race 80+107
[12728.702458] Free swap  = 0kB
[12728.702522] Total swap = 987956kB
[12728.702590] Free swap:            0kB
[12728.709794] 524176 pages of RAM
[12728.709866] 8772 reserved pages
[12728.709931] 8761 pages shared
[12728.709996] 0 pages swap cached
[12728.710063] Out of memory: kill process 2951 (sshd) score 276 or a child
[12728.710380] Killed process 2951 (sshd)
[12735.176084] cc1 invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=-17
[12735.176230] 
[12735.176230] Call Trace:
[12735.176359]  [<ffffffff8024d176>] oom_kill_process+0xf6/0x110
[12735.176444]  [<ffffffff8024d62d>] out_of_memory+0x1ad/0x1f0
[12735.176526]  [<ffffffff8024f92e>] __alloc_pages+0x28e/0x3a0
[12735.176608]  [<ffffffff80251a61>] __do_page_cache_readahead+0x101/0x270
[12735.176697]  [<ffffffff8024c5ca>] filemap_fault+0x28a/0x3c0
[12735.176779]  [<ffffffff802575bc>] __do_fault+0x6c/0x430
[12735.176862]  [<ffffffff8025946b>] handle_mm_fault+0x3bb/0x700
[12735.176946]  [<ffffffff8021c1ff>] do_page_fault+0x1bf/0x830
[12735.177027]  [<ffffffff8021e6bd>] __dequeue_entity+0x3d/0x50
[12735.177113]  [<ffffffff803d1b90>] thread_return+0x3d/0x54d
[12735.177194]  [<ffffffff8025e94c>] do_mmap_pgoff+0x2fc/0x360
[12735.177278]  [<ffffffff8025d426>] do_munmap+0x296/0x2f0
[12735.177358]  [<ffffffff803d3c99>] error_exit+0x0/0x51
[12735.177436] 
[12735.177493] Mem-info:
[12735.177555] DMA per-cpu:
[12735.177617] CPU    0: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
[12735.177763] CPU    1: Hot: hi:    0, btch:   1 usd:   0   Cold: hi:    0, btch:   1 usd:   0
[12735.177918] DMA32 per-cpu:
[12735.177983] CPU    0: Hot: hi:  186, btch:  31 usd:  94   Cold: hi:   62, btch:  15 usd:  45
[12735.178128] CPU    1: Hot: hi:  186, btch:  31 usd: 133   Cold: hi:   62, btch:  15 usd:  47
[12735.178271] Active:129449 inactive:326076 dirty:0 writeback:0 unstable:0
[12735.178272]  free:3433 slab:10896 mapped:10 pagetables:14304 bounce:0
[12735.178446] DMA free:8032kB min:32kB low:40kB high:48kB active:2588kB inactive:2336kB present:12744kB pages_scanned:9363 all_unreclaimable? yes
[12735.178618] lowmem_reserve[]: 0 2003 2003 2003
[12735.178706] DMA32 free:5700kB min:5708kB low:7132kB high:8560kB active:512264kB inactive:1304912kB present:2051880kB pages_scanned:3251584 all_unreclaimable? yes
[12735.178933] lowmem_reserve[]: 0 0 0 0
[12735.179014] DMA: 8*4kB 2*8kB 11*16kB 12*32kB 12*64kB 12*128kB 4*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 8032kB
[12735.179195] DMA32: 33*4kB 22*8kB 9*16kB 2*32kB 1*64kB 2*128kB 1*256kB 1*512kB 0*1024kB 0*2048kB 1*4096kB = 5700kB
[12735.179382] Swap cache: add 5538428, delete 5538396, find 910933/1354692, race 80+107
[12735.179530] Free swap  = 0kB
[12735.179594] Total swap = 987956kB
[12735.179661] Free swap:            0kB
[12735.186929] 524176 pages of RAM
[12735.186999] 8772 reserved pages
[12735.187069] 8986 pages shared
[12735.187134] 32 pages swap cached
[12735.187201] Out of memory: kill process 3007 (cron) score 272 or a child
[12735.187524] Killed process 5528 (cron)



---------------------------------------------------------
Wed Oct 24 11:54:31 MSD 2007
MemTotal:      2061616 kB
MemFree:        881376 kB
Buffers:          8092 kB
Cached:         260672 kB
SwapCached:      71364 kB
Active:         768928 kB
Inactive:       216452 kB
SwapTotal:      987956 kB
SwapFree:       260064 kB
Dirty:          101652 kB
Writeback:        1740 kB
AnonPages:      648864 kB
Mapped:          26252 kB
Slab:            67316 kB
SReclaimable:    26600 kB
SUnreclaim:      40716 kB
PageTables:      46800 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   2018764 kB
Committed_AS:  1855552 kB
VmallocTotal: 34359738367 kB
VmallocUsed:       896 kB
VmallocChunk: 34359737107 kB
Name                   Objects Objsize    Space Slabs/Part/Cpu  O/S O %Fr %Ef Flg
dentry                   56820     200    11.6M       2841/0/0   20 0   0  97 aF
ext2_inode_cache         13823     624     9.4M       2304/1/0    6 0   0  91 aF
vm_area_struct           38861     168     7.3M     1794/716/0   24 0  39  88 F
sighand_cache             2062    2088     6.2M      764/184/0    3 1  24  68 AF
radix_tree_node           9463     552     5.5M      1357/10/0    7 0   0  93 F
task_struct               2068    1776     4.7M      580/178/0    4 1  30  77 F
buffer_head              43228     104     4.5M       1109/9/0   39 0   0  98 aF
:F-0000768                4082     744     3.7M      917/300/0    5 0  32  80 *AF
:F-0000192               15205     192     3.5M      863/534/0   21 0  61  82 *F
:F-0002048                1578    2048     3.2M        397/9/0    4 1   2  99 *F
:F-0000704                2035     704     1.7M       211/92/0   11 1  43  82 *AF
:F-0000064               15489      64     1.6M      403/191/0   64 0  47  60 *F
:F-0001024                1207    1024     1.3M       322/64/0    4 0  19  93 *F
inode_cache               2122     480     1.1M       272/27/0    8 0   9  91 aF
anon_vma                 18342      24   655.3K       160/98/0  128 0  61  67 F
:F-0000128                2455     128   561.1K      137/112/0   32 0  81  55 *F
ext3_inode_cache           570     640   499.7K       122/48/0    6 0  39  73 aF
:F-0000080                3825      80   307.2K         75/0/0   51 0   0  99 *F
cfq_io_context            1292     152   225.2K        55/20/0   26 0  36  87 F
cfq_queue                 1281     136   180.2K        44/16/0   30 0  36  96 F
shmem_inode_cache          244     672   167.9K         41/2/0    6 0   4  97 F
blkdev_requests            203     288    69.6K        17/11/0   14 0  64  83 F
:F-0000512                  63     512    53.2K         13/9/0    8 0  69  60 *F
:F-0000032                1398      32    49.1K         12/3/0  128 0  25  91 *F
idr_layer_cache             77     528    45.0K         11/0/0    7 0   0  90 F
:F-0004096                   9    4096    40.9K          5/1/0    2 1  20  90 *AF
:F-0000256                 105     256    32.7K          8/3/0   16 0  37  82 *F
kmalloc-96                 196      96    28.6K          7/3/0   42 0  42  65 F
file_lock_cache            130     176    24.5K          6/2/0   22 0  33  93 F
:F-0000016                1458      16    24.5K          6/1/0  256 0  16  94 *F
bdev_cache                  22     680    16.3K          2/0/0   11 1   0  91 AaF
sock_inode_cache            27     560    16.3K          4/1/0    7 0  25  92 AaF
journal_head                91      96    16.3K          4/2/0   42 0  50  53 aF
proc_inode_cache            15     512    12.2K          3/1/0    7 0  33  62 aF
:F-0000320                  24     320     8.1K          2/0/0   12 0   0  93 *AF
:F-0000160                  50     160     8.1K          2/0/0   25 0   0  97 *F
revoke_record              256      32     8.1K          2/0/0  128 0   0 100 AaF
TCP                          5    1464     8.1K          1/0/0    5 1   0  89 AF
journal_handle             340      24     8.1K          2/0/0  170 0   0  99 aF
skbuff_fclone_cache         18     388     8.1K          2/0/0    9 0   0  85 AF
kmalloc-8                 1024       8     8.1K          2/0/0  512 0   0 100 F
blkdev_queue                 5    1552     8.1K          1/0/0    5 1   0  94 F
scsi_cmd_cache              20     368     8.1K          2/0/0   10 0   0  89 AF
Acpi-Parse                 102      40     4.0K          1/0/0  102 0   0  99 F
revoke_table               256      16     4.0K          1/0/0  256 0   0 100 aF
Page block order: 10
Pages per block:  1024

Free pages count per migrate type at order       0      1      2      3      4      5      6      7      8      9     10 
Node    0, zone      DMA, type    Unmovable     12     11      5      9     11     10      3      0      0      0      0 
Node    0, zone      DMA, type  Reclaimable      0      0      0      0      0      0      0      0      0      0      0 
Node    0, zone      DMA, type      Movable    100     69     43     17     16     10      3      0      0      0      0 
Node    0, zone      DMA, type      Reserve      0      0      0      0      0      0      0      0      0      0      1 
Node    0, zone      DMA, type       <NULL>      0      0      0      0      0      0      0      0      0      0      0 
Node    0, zone    DMA32, type    Unmovable   3522    912     37      0      0      0      0      0      0      0      0 
Node    0, zone    DMA32, type  Reclaimable   4185   2320    514     43      1      0      0      0      0      0      0 
Node    0, zone    DMA32, type      Movable  31785  40871  15256   2709    140      0      0      0      0      0      0 
Node    0, zone    DMA32, type      Reserve     16     59     50     17      3      2      1      1      0      0      1 
Node    0, zone    DMA32, type       <NULL>      0      0      0      0      0      0      0      0      0      0      0 

Number of blocks type     Unmovable  Reclaimable      Movable      Reserve       <NULL> 
Node 0, zone      DMA            1            0            2            1            0 
Node 0, zone    DMA32           46           21          439            2            0 
---------------------------------------------------------
Wed Oct 24 11:55:31 MSD 2007
MemTotal:      2061616 kB
MemFree:        241412 kB
Buffers:         10580 kB
Cached:         293072 kB
SwapCached:      94344 kB
Active:        1415064 kB
Inactive:       201048 kB
SwapTotal:      987956 kB
SwapFree:       403872 kB
Dirty:          112412 kB
Writeback:        6144 kB
AnonPages:     1222908 kB
Mapped:          42924 kB
Slab:            72848 kB
SReclaimable:    31252 kB
SUnreclaim:      41596 kB
PageTables:      48832 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   2018764 kB
Committed_AS:  2313396 kB
VmallocTotal: 34359738367 kB
VmallocUsed:       896 kB
VmallocChunk: 34359737107 kB
Name                   Objects Objsize    Space Slabs/Part/Cpu  O/S O %Fr %Ef Flg
dentry                   70457     200    14.4M       3523/1/0   20 0   0  97 aF
ext2_inode_cache         16272     624    11.1M      2730/89/0    6 0   3  90 aF
vm_area_struct           39679     168     7.3M     1790/355/0   24 0  19  90 F
radix_tree_node          11207     552     6.5M       1601/0/0    7 0   0  94 F
sighand_cache             2111    2088     6.0M       744/84/0    3 1  11  72 AF
buffer_head              49069     104     5.1M      1261/29/0   39 0   2  98 aF
task_struct               2118    1776     4.7M      578/113/0    4 1  19  79 F
:F-0000768                4182     744     3.7M      910/158/0    5 0  17  83 *AF
:F-0000192               15377     192     3.5M      865/558/0   21 0  64  83 *F
:F-0002048                1614    2048     3.3M       411/19/0    4 1   4  98 *F
:F-0000704                2084     704     1.7M       211/53/0   11 1  25  84 *AF
:F-0000064               17173      64     1.6M      403/199/0   64 0  49  66 *F
:F-0001024                1232    1024     1.3M       322/44/0    4 0  13  95 *F
inode_cache               2143     480     1.1M       272/11/0    8 0   4  92 aF
anon_vma                 18747      24   655.3K       160/47/0  128 0  29  68 F
:F-0000128                2652     128   540.6K       132/88/0   32 0  66  62 *F
ext3_inode_cache           596     640   499.7K       122/34/0    6 0  27  76 aF
:F-0000080                3825      80   307.2K         75/0/0   51 0   0  99 *F
cfq_io_context            1228     152   225.2K        55/22/0   26 0  40  82 F
cfq_queue                 1211     136   180.2K        44/17/0   30 0  38  91 F
shmem_inode_cache          244     672   167.9K         41/2/0    6 0   4  97 F
:F-0004096                  16    4096    73.7K          9/2/0    2 1  22  88 *AF
blkdev_requests            188     288    61.4K         15/6/0   14 0  40  88 F
:F-0000512                  63     512    53.2K         13/9/0    8 0  69  60 *F
:F-0000032                1398      32    49.1K         12/3/0  128 0  25  91 *F
idr_layer_cache             77     528    45.0K         11/0/0    7 0   0  90 F
file_lock_cache            125     176    32.7K          8/6/0   22 0  75  67 F
:F-0000256                 109     256    32.7K          8/2/0   16 0  25  85 *F
kmalloc-96                 196      96    28.6K          7/3/0   42 0  42  65 F
:F-0000016                1458      16    24.5K          6/1/0  256 0  16  94 *F
bdev_cache                  22     680    16.3K          2/0/0   11 1   0  91 AaF
sock_inode_cache            27     560    16.3K          4/1/0    7 0  25  92 AaF
journal_head               143      96    16.3K          4/1/0   42 0  25  83 aF
proc_inode_cache            15     512    12.2K          3/1/0    7 0  33  62 aF
:F-0000320                  24     320     8.1K          2/0/0   12 0   0  93 *AF
:F-0000160                  50     160     8.1K          2/0/0   25 0   0  97 *F
revoke_record              256      32     8.1K          2/0/0  128 0   0 100 AaF
TCP                          5    1464     8.1K          1/0/0    5 1   0  89 AF
journal_handle             340      24     8.1K          2/0/0  170 0   0  99 aF
skbuff_fclone_cache         18     388     8.1K          2/0/0    9 0   0  85 AF
kmalloc-8                 1024       8     8.1K          2/0/0  512 0   0 100 F
blkdev_queue                 5    1552     8.1K          1/0/0    5 1   0  94 F
scsi_cmd_cache              20     368     8.1K          2/0/0   10 0   0  89 AF
Acpi-Parse                 102      40     4.0K          1/0/0  102 0   0  99 F
revoke_table               256      16     4.0K          1/0/0  256 0   0 100 aF
Page block order: 10
Pages per block:  1024

Free pages count per migrate type at order       0      1      2      3      4      5      6      7      8      9     10 
Node    0, zone      DMA, type    Unmovable     10      9      9     10     12     11      2      0      0      0      0 
Node    0, zone      DMA, type  Reclaimable      0      0      0      0      0      0      0      0      0      0      0 
Node    0, zone      DMA, type      Movable      1      0      2      0      0      1      2      0      0      0      0 
Node    0, zone      DMA, type      Reserve      0      0      0      0      0      0      0      0      0      0      1 
Node    0, zone      DMA, type       <NULL>      0      0      0      0      0      0      0      0      0      0      0 
Node    0, zone    DMA32, type    Unmovable    129     10      0      0      0      0      0      0      0      0      0 
Node    0, zone    DMA32, type  Reclaimable    135      0      0      0      0      0      0      0      0      0      0 
Node    0, zone    DMA32, type      Movable   2850   1272    395     58      0      0      0      0      0      0      0 
Node    0, zone    DMA32, type      Reserve      3      1      2      4      3      2      1      1      0      0      1 
Node    0, zone    DMA32, type       <NULL>      0      0      0      0      0      0      0      0      0      0      0 

Number of blocks type     Unmovable  Reclaimable      Movable      Reserve       <NULL> 
Node 0, zone      DMA            1            0            2            1            0 
Node 0, zone    DMA32           46           21          439            2            0 
---------------------------------------------------------
Wed Oct 24 11:56:57 MSD 2007
MemTotal:      2061616 kB
MemFree:         14188 kB
Buffers:          2836 kB
Cached:         194004 kB
SwapCached:      42244 kB
Active:        1437864 kB
Inactive:       359544 kB
SwapTotal:      987956 kB
SwapFree:       223988 kB
Dirty:          133824 kB
Writeback:        4200 kB
AnonPages:     1590604 kB
Mapped:          39624 kB
Slab:            72556 kB
SReclaimable:    30180 kB
SUnreclaim:      42376 kB
PageTables:      56664 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   2018764 kB
Committed_AS:  3018888 kB
VmallocTotal: 34359738367 kB
VmallocUsed:       896 kB
VmallocChunk: 34359737107 kB
Name                   Objects Objsize    Space Slabs/Part/Cpu  O/S O %Fr %Ef Flg
dentry                   44251     200    12.5M    3059/1512/0   20 0  49  70 aF
ext2_inode_cache         12586     624    11.3M    2771/1644/0    6 0  59  69 aF
vm_area_struct           43961     168     7.5M      1833/17/0   24 0   0  98 F
sighand_cache             2396    2088     6.5M        799/1/0    3 1   0  76 AF
radix_tree_node           6130     552     6.4M    1571/1264/0    7 0  80  52 F
buffer_head              33796     104     5.3M     1296/982/0   39 0  75  66 aF
task_struct               2399    1776     4.9M        600/1/0    4 1   0  86 F
:F-0000768                4750     744     3.8M        951/5/0    5 0   0  90 *AF
:F-0000192               16629     192     3.5M      865/264/0   21 0  30  90 *F
:F-0002048                1513    2048     3.1M       385/17/0    4 1   4  98 *F
:F-0000704                2362     704     1.7M        215/3/0   11 1   1  94 *AF
:F-0000064               13948      64     1.6M      403/309/0   64 0  76  54 *F
:F-0001024                1337    1024     1.3M        337/9/0    4 0   2  99 *F
inode_cache               1533     480     1.0M      263/123/0    8 0  46  68 aF
anon_vma                 20885      24   671.7K       164/14/0  128 0   8  74 F
:F-0000128                3461     128   557.0K       136/70/0   32 0  51  79 *F
ext3_inode_cache           599     640   499.7K       122/56/0    6 0  45  76 aF
:F-0000080                3825      80   307.2K         75/0/0   51 0   0  99 *F
cfq_io_context            1415     152   225.2K         55/3/0   26 0   5  95 F
cfq_queue                 1241     136   180.2K         44/5/0   30 0  11  93 F
shmem_inode_cache          244     672   167.9K         41/2/0    6 0   4  97 F
:F-0004096                  22    4096    90.1K         11/0/0    2 1   0 100 *AF
:F-0000016                2281      16    57.3K         14/8/0  256 0  57  63 *F
blkdev_requests            157     288    57.3K         14/9/0   14 0  64  78 F
:F-0000512                  64     512    53.2K         13/9/0    8 0  69  61 *F
:F-0000032                1398      32    49.1K         12/3/0  128 0  25  91 *F
idr_layer_cache             77     528    45.0K         11/0/0    7 0   0  90 F
:F-0000256                 105     256    32.7K          8/3/0   16 0  37  82 *F
kmalloc-96                 196      96    28.6K          7/3/0   42 0  42  65 F
bdev_cache                  22     680    16.3K          2/0/0   11 1   0  91 AaF
sock_inode_cache            27     560    16.3K          4/1/0    7 0  25  92 AaF
journal_head                91      96    16.3K          4/2/0   42 0  50  53 aF
proc_inode_cache            15     512    12.2K          3/1/0    7 0  33  62 aF
file_lock_cache             45     176    12.2K          3/1/0   22 0  33  64 F
:F-0000320                  24     320     8.1K          2/0/0   12 0   0  93 *AF
:F-0000160                  50     160     8.1K          2/0/0   25 0   0  97 *F
revoke_record              256      32     8.1K          2/0/0  128 0   0 100 AaF
TCP                          5    1464     8.1K          1/0/0    5 1   0  89 AF
journal_handle             340      24     8.1K          2/0/0  170 0   0  99 aF
skbuff_fclone_cache         18     388     8.1K          2/0/0    9 0   0  85 AF
kmalloc-8                 1024       8     8.1K          2/0/0  512 0   0 100 F
blkdev_queue                 5    1552     8.1K          1/0/0    5 1   0  94 F
scsi_cmd_cache              20     368     8.1K          2/0/0   10 0   0  89 AF
Acpi-Parse                 102      40     4.0K          1/0/0  102 0   0  99 F
revoke_table               256      16     4.0K          1/0/0  256 0   0 100 aF
Page block order: 10
Pages per block:  1024

Free pages count per migrate type at order       0      1      2      3      4      5      6      7      8      9     10 
Node    0, zone      DMA, type    Unmovable      1      2      8     10     12     11      2      0      0      0      0 
Node    0, zone      DMA, type  Reclaimable      0      0      0      0      0      0      0      0      0      0      0 
Node    0, zone      DMA, type      Movable      1      4      1      3      0      1      2      0      0      0      0 
Node    0, zone      DMA, type      Reserve      0      0      0      0      0      0      0      0      0      0      1 
Node    0, zone      DMA, type       <NULL>      0      0      0      0      0      0      0      0      0      0      0 
Node    0, zone    DMA32, type    Unmovable     11      0      0      0      0      0      0      0      0      0      0 
Node    0, zone    DMA32, type  Reclaimable    110      0      0      0      0      0      0      0      0      0      0 
Node    0, zone    DMA32, type      Movable     10     31      0      0      0      0      0      0      0      0      0 
Node    0, zone    DMA32, type      Reserve      9      1      0      0      3      2      1      1      0      0      1 
Node    0, zone    DMA32, type       <NULL>      0      0      0      0      0      0      0      0      0      0      0 

Number of blocks type     Unmovable  Reclaimable      Movable      Reserve       <NULL> 
Node 0, zone      DMA            1            0            2            1            0 
Node 0, zone    DMA32           46           21          439            2            0 
---------------------------------------------------------
Wed Oct 24 12:08:52 MSD 2007
MemTotal:      2061616 kB
MemFree:         13808 kB
Buffers:          2952 kB
Cached:          34332 kB
SwapCached:        872 kB
Active:         904352 kB
Inactive:       923296 kB
SwapTotal:      987956 kB
SwapFree:           24 kB
Dirty:           24576 kB
Writeback:        1540 kB
AnonPages:     1805896 kB
Mapped:           1692 kB
Slab:            51532 kB
SReclaimable:    12584 kB
SUnreclaim:      38948 kB
PageTables:      57952 kB
NFS_Unstable:        0 kB
Bounce:              0 kB
CommitLimit:   2018764 kB
Committed_AS:  3473824 kB
VmallocTotal: 34359738367 kB
VmallocUsed:       896 kB
VmallocChunk: 34359737107 kB
Name                   Objects Objsize    Space Slabs/Part/Cpu  O/S O %Fr %Ef Flg
vm_area_struct           44135     168     7.5M     1851/109/0   24 0   5  97 F
sighand_cache             2392    2088     6.5M       804/19/0    3 1   2  75 AF
task_struct               2396    1776     4.9M       604/19/0    4 1   3  86 F
:F-0000768                4745     744     3.9M       957/36/0    5 0   3  90 *AF
dentry                    3364     200     3.8M      936/896/0   20 0  95  17 aF
:F-0000192               16630     192     3.5M      865/263/0   21 0  30  90 *F
:F-0002048                1448    2048     3.1M       386/42/0    4 1  10  93 *F
ext2_inode_cache          1367     624     2.3M      582/483/0    6 0  82  35 aF
:F-0000704                2365     704     1.7M       217/21/0   11 1   9  93 *AF
:F-0000064                8905      64     1.6M      403/337/0   64 0  83  34 *F
:F-0001024                1344    1024     1.4M       343/19/0    4 0   5  97 *F
inode_cache               1518     480     1.0M      263/126/0    8 0  47  67 aF
anon_vma                 20856      24   675.8K       165/46/0  128 0  27  74 F
:F-0000128                2922     128   577.5K      141/106/0   32 0  75  64 *F
radix_tree_node            861     552   544.7K       133/24/0    7 0  18  87 F
ext3_inode_cache           317     640   421.8K       103/87/0    6 0  84  48 aF
:F-0000080                3825      80   307.2K         75/0/0   51 0   0  99 *F
:F-0004096                  51    4096   229.3K         28/5/0    2 1  17  91 *AF
cfq_io_context            1447     152   229.3K         56/6/0   26 0  10  95 F
cfq_queue                 1426     136   196.6K         48/5/0   30 0  10  98 F
buffer_head                232     104   167.9K        41/38/0   39 0  92  14 aF
shmem_inode_cache          244     672   167.9K         41/2/0    6 0   4  97 F
blkdev_requests            168     288    86.0K        21/19/0   14 0  90  56 F
:F-0000512                  64     512    53.2K         13/9/0    8 0  69  61 *F
:F-0000032                1398      32    49.1K         12/3/0  128 0  25  91 *F
idr_layer_cache             77     528    45.0K         11/0/0    7 0   0  90 F
:F-0000016                1657      16    32.7K          8/3/0  256 0  37  80 *F
:F-0000256                 106     256    32.7K          8/3/0   16 0  37  82 *F
kmalloc-96                 196      96    28.6K          7/3/0   42 0  42  65 F
sock_inode_cache            27     560    16.3K          4/1/0    7 0  25  92 AaF
bdev_cache                  22     680    16.3K          2/0/0   11 1   0  91 AaF
proc_inode_cache            15     512    12.2K          3/1/0    7 0  33  62 aF
file_lock_cache             45     176    12.2K          3/1/0   22 0  33  64 F
journal_head                85      96    12.2K          3/1/0   42 0  33  66 aF
:F-0000320                  24     320     8.1K          2/0/0   12 0   0  93 *AF
:F-0000160                  50     160     8.1K          2/0/0   25 0   0  97 *F
revoke_record              256      32     8.1K          2/0/0  128 0   0 100 AaF
TCP                          5    1464     8.1K          1/0/0    5 1   0  89 AF
journal_handle             340      24     8.1K          2/0/0  170 0   0  99 aF
skbuff_fclone_cache         18     388     8.1K          2/0/0    9 0   0  85 AF
kmalloc-8                 1024       8     8.1K          2/0/0  512 0   0 100 F
blkdev_queue                 5    1552     8.1K          1/0/0    5 1   0  94 F
scsi_cmd_cache              20     368     8.1K          2/0/0   10 0   0  89 AF
Acpi-Parse                 102      40     4.0K          1/0/0  102 0   0  99 F
revoke_table               256      16     4.0K          1/0/0  256 0   0 100 aF
Page block order: 10
Pages per block:  1024

Free pages count per migrate type at order       0      1      2      3      4      5      6      7      8      9     10 
Node    0, zone      DMA, type    Unmovable      9      1     10     10     12     11      2      0      0      0      0 
Node    0, zone      DMA, type  Reclaimable      0      0      0      0      0      0      0      0      0      0      0 
Node    0, zone      DMA, type      Movable      2      0      1      2      0      1      2      0      0      0      0 
Node    0, zone      DMA, type      Reserve      0      0      0      0      0      0      0      0      0      0      1 
Node    0, zone      DMA, type       <NULL>      0      0      0      0      0      0      0      0      0      0      0 
Node    0, zone    DMA32, type    Unmovable     73      0      0      0      0      0      0      0      0      0      0 
Node    0, zone    DMA32, type  Reclaimable     57      1      0      0      0      0      0      0      0      0      0 
Node    0, zone    DMA32, type      Movable      1     39      0      0      0      0      0      0      0      0      0 
Node    0, zone    DMA32, type      Reserve     26      9      9      2      1      2      1      1      0      0      1 
Node    0, zone    DMA32, type       <NULL>      0      0      0      0      0      0      0      0      0      0      0 

Number of blocks type     Unmovable  Reclaimable      Movable      Reserve       <NULL> 
Node 0, zone      DMA            1            0            2            1            0 
Node 0, zone    DMA32           46           21          439            2            0 
---------------------------------------------------------

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: SLUB 0:1 SLAB (OOM during massive parallel kernel builds)
  2007-10-24 19:28         ` Alexey Dobriyan
@ 2007-10-25  2:15           ` Christoph Lameter
  2007-10-25  2:34             ` Nick Piggin
  0 siblings, 1 reply; 14+ messages in thread
From: Christoph Lameter @ 2007-10-25  2:15 UTC (permalink / raw)
  To: Alexey Dobriyan; +Cc: Mel Gorman, Pekka Enberg, linux-kernel, linux-mm

On Wed, 24 Oct 2007, Alexey Dobriyan wrote:

> [12728.701398] DMA free:8032kB min:32kB low:40kB high:48kB active:2716kB 
> inactive:2208kB present:12744kB pages_scanned:9299 all_unreclaimable? 
> yes [12728.701567] lowmem_reserve[]: 0 2003 2003 2003 [12728.701654] 

Ummm... all unreclaimable is set! Are you mlocking the pages in memory? Or 
what causes this? All pages under writeback? What is the dirty ratio set 
to?

> pages_scanned:9363 all_unreclaimable? yes [12735.178618] 

And again.

> lowmem_reserve[]: 0 2003 2003 2003 [12735.178706] DMA32 free:5700kB 
> min:5708kB low:7132kB high:8560kB active:512264kB inactive:1304912kB 
> present:2051880kB pages_scanned:3251584 all_unreclaimable? yes 

And again.


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: SLUB 0:1 SLAB (OOM during massive parallel kernel builds)
  2007-10-25  2:15           ` Christoph Lameter
@ 2007-10-25  2:34             ` Nick Piggin
  2007-10-25  2:43               ` Christoph Lameter
  0 siblings, 1 reply; 14+ messages in thread
From: Nick Piggin @ 2007-10-25  2:34 UTC (permalink / raw)
  To: Christoph Lameter
  Cc: Alexey Dobriyan, Mel Gorman, Pekka Enberg, linux-kernel, linux-mm

On Thursday 25 October 2007 12:15, Christoph Lameter wrote:
> On Wed, 24 Oct 2007, Alexey Dobriyan wrote:
> > [12728.701398] DMA free:8032kB min:32kB low:40kB high:48kB active:2716kB
> > inactive:2208kB present:12744kB pages_scanned:9299 all_unreclaimable?
> > yes [12728.701567] lowmem_reserve[]: 0 2003 2003 2003 [12728.701654]
>
> Ummm... all unreclaimable is set! Are you mlocking the pages in memory? Or
> what causes this? All pages under writeback? What is the dirty ratio set
> to?

Why is SLUB behaving differently, though.

Memory efficiency wouldn't be the reason, would it? I mean, SLUB
should be more efficient than SLAB, plus have less data lying around
in queues.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: SLUB 0:1 SLAB (OOM during massive parallel kernel builds)
  2007-10-25  2:34             ` Nick Piggin
@ 2007-10-25  2:43               ` Christoph Lameter
  2007-10-25  3:06                 ` Nick Piggin
  0 siblings, 1 reply; 14+ messages in thread
From: Christoph Lameter @ 2007-10-25  2:43 UTC (permalink / raw)
  To: Nick Piggin
  Cc: Alexey Dobriyan, Mel Gorman, Pekka Enberg, linux-kernel, linux-mm

On Thu, 25 Oct 2007, Nick Piggin wrote:

> > Ummm... all unreclaimable is set! Are you mlocking the pages in memory? Or
> > what causes this? All pages under writeback? What is the dirty ratio set
> > to?
> 
> Why is SLUB behaving differently, though.

Nore sure. Are we really sure that this does not occur using SLAB?
 
> Memory efficiency wouldn't be the reason, would it? I mean, SLUB
> should be more efficient than SLAB, plus have less data lying around
> in queues.

SLAB may have data around in queues which (if the stars align the right 
way) may allow it to go longer without having to get a page from the page 
allocator.

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: SLUB 0:1 SLAB (OOM during massive parallel kernel builds)
  2007-10-25  2:43               ` Christoph Lameter
@ 2007-10-25  3:06                 ` Nick Piggin
  0 siblings, 0 replies; 14+ messages in thread
From: Nick Piggin @ 2007-10-25  3:06 UTC (permalink / raw)
  To: Christoph Lameter
  Cc: Alexey Dobriyan, Mel Gorman, Pekka Enberg, linux-kernel, linux-mm

On Thursday 25 October 2007 12:43, Christoph Lameter wrote:
> On Thu, 25 Oct 2007, Nick Piggin wrote:
> > > Ummm... all unreclaimable is set! Are you mlocking the pages in memory?
> > > Or what causes this? All pages under writeback? What is the dirty ratio
> > > set to?
> >
> > Why is SLUB behaving differently, though.
>
> Nore sure. Are we really sure that this does not occur using SLAB?

>From the reports it seems pretty consistent. I guess it could well
be something that may occur with SLAB *if the conditions are a bit
different*...

> > Memory efficiency wouldn't be the reason, would it? I mean, SLUB
> > should be more efficient than SLAB, plus have less data lying around
> > in queues.
>
> SLAB may have data around in queues which (if the stars align the right
> way) may allow it to go longer without having to get a page from the page
> allocator.

But page allocs from slab isn't where the OOMs are occurring, so this
seems unlikely (also, the all_unreclaimable logic now should be pretty
strict, so you have to really run the machine out of memory (1GB of
swap gets fully used, then his DMA32 zone is scanned 8 times without
reclaiming a single page).

That said, parallel kernel compiling can really change a lot in memory
footprint depending on small variations in timing. So it might not be
anything to worry about.

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2007-10-25  3:13 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-10-23 18:16 SLUB 0:1 SLAB (OOM during massive parallel kernel builds) Alexey Dobriyan
2007-10-23 19:35 ` Christoph Lameter
2007-10-23 19:48   ` Pekka Enberg
2007-10-23 19:57     ` Christoph Lameter
2007-10-24  8:09       ` Mel Gorman
2007-10-24 19:28         ` Alexey Dobriyan
2007-10-25  2:15           ` Christoph Lameter
2007-10-25  2:34             ` Nick Piggin
2007-10-25  2:43               ` Christoph Lameter
2007-10-25  3:06                 ` Nick Piggin
2007-10-23 20:04   ` Pekka Enberg
2007-10-23 20:09     ` Christoph Lameter
2007-10-23 20:41       ` Pekka Enberg
2007-10-24 16:56         ` Christoph Lameter

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).