LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Yasuaki Ishimatsu <yasu.isimatu@gmail.com>
To: Dave Young <dyoung@redhat.com>
Cc: x86@kernel.org, linux-kernel@vger.kernel.org, tglx@linutronix.de,
	bhe@redhat.com, mingo@redhat.com, hpa@zytor.com,
	akpm@linux-foundation.org
Subject: Re: [PATCH] x86/numa: kernel stack corruption fix
Date: Mon, 06 Apr 2015 07:26:56 -0700 (PDT)	[thread overview]
Message-ID: <552297b0.8a858c0a.5345.fffff99a@mx.google.com> (raw)
In-Reply-To: <20150403071513.GC22579@dhcp-128-53.nay.redhat.com>

Hi,

On Fri, 3 Apr 2015 15:15:13 +0800
Dave Young <dyoung@redhat.com> wrote:

> Hi,
> 
> On 04/02/15 at 12:36pm, Yasuaki Ishimatsu wrote:
> > 
> > On Wed, 1 Apr 2015 12:53:46 +0800
> > Dave Young <dyoung@redhat.com> wrote:
> > 
> > > I got below kernel panic during kdump test on Thinkpad T420 laptop:
> > > 
> > > [    0.000000] No NUMA configuration found                                      
> > > [    0.000000] Faking a node at [mem 0x0000000000000000-0x0000000037ba4fff]     
> > > [    0.000000] Kernel panic - not syncing: stack-protector: Kernel stack is cor 
> > > upted in: ffffffff81d21910                                                     r
> > > [    0.000000]                                                                  
> > > [    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.0.0-rc6+ #44           
> > > [    0.000000] Hardware name: LENOVO 4236NUC/4236NUC, BIOS 83ET76WW (1.46 ) 07/ 
> > > 5/2013                                                                         0
> > > [    0.000000]  0000000000000000 c70296ddd809e4f6 ffffffff81b67ce8 ffffffff817c 
> > > a26                                                                            2
> > > [    0.000000]  0000000000000000 ffffffff81a61c90 ffffffff81b67d68 ffffffff817b 
> > > 8d2                                                                            c
> > > [    0.000000]  0000000000000010 ffffffff81b67d78 ffffffff81b67d18 c70296ddd809 
> > > 4f6                                                                            e
> > > [    0.000000] Call Trace:                                                      
> > > [    0.000000]  [<ffffffff817c2a26>] dump_stack+0x45/0x57                       
> > > [    0.000000]  [<ffffffff817bc8d2>] panic+0xd0/0x204                           
> > > [    0.000000]  [<ffffffff81d21910>] ? numa_clear_kernel_node_hotplug+0xe6/0xf2 
> > > [    0.000000]  [<ffffffff8107741b>] __stack_chk_fail+0x1b/0x20                 
> > > [    0.000000]  [<ffffffff81d21910>] numa_clear_kernel_node_hotplug+0xe6/0xf2   
> > > [    0.000000]  [<ffffffff81d21e5d>] numa_init+0x1a5/0x520                      
> > > [    0.000000]  [<ffffffff81d222b1>] x86_numa_init+0x19/0x3d                    
> > > [    0.000000]  [<ffffffff81d22460>] initmem_init+0x9/0xb                       
> > > [    0.000000]  [<ffffffff81d0d00c>] setup_arch+0x94f/0xc82                     
> > > [    0.000000]  [<ffffffff81d05120>] ? early_idt_handlers+0x120/0x120           
> > > [    0.000000]  [<ffffffff817bd0bb>] ? printk+0x55/0x6b                         
> > > [    0.000000]  [<ffffffff81d05120>] ? early_idt_handlers+0x120/0x120           
> > > [    0.000000]  [<ffffffff81d05d9b>] start_kernel+0xe8/0x4d6                    
> > > [    0.000000]  [<ffffffff81d05120>] ? early_idt_handlers+0x120/0x120           
> > > [    0.000000]  [<ffffffff81d05120>] ? early_idt_handlers+0x120/0x120           
> > > [    0.000000]  [<ffffffff81d055ee>] x86_64_start_reservations+0x2a/0x2c        
> > > [    0.000000]  [<ffffffff81d05751>] x86_64_start_kernel+0x161/0x184            
> > > [    0.000000] ---[ end Kernel panic - not syncing: stack-protector: Kernel sta 
> > > k is corrupted in: ffffffff81d21910                                            c
> > > [    0.000000]                                                                  
> > > PANIC: early exception 0d rip 10:ffffffff8105d2a6 error 7eb cr2 ffff8800371dd00 
> > > [    0.000000] CPU: 0 PID: 0 Comm: swapper Not tainted 4.0.0-rc6+ #44          0
> > > [    0.000000] Hardware name: LENOVO 4236NUC/4236NUC, BIOS 83ET76WW (1.46 ) 07/ 
> > > 5/2013                                                                         0
> > > [    0.000000]  0000000000000000 c70296ddd809e4f6 ffffffff81b67c60 ffffffff817c 
> > > a26                                                                            2
> > > [    0.000000]  0000000000000096 ffffffff81a61c90 ffffffff81b67d68 fffffff00000 
> > > 084 0000000000000a0d 0000000000000a00                                          0
> > > [    0.000000] Call Trace:                                                      
> > > [    0.000000]  [<ffffffff817c2a26>] dump_stack+0x45/0x57                       
> > > [    0.000000]  [<ffffffff81d051b0>] early_idt_handler+0x90/0xb7                
> > > [    0.000000]  [<ffffffff8105d2a6>] ? native_irq_enable+0x6/0x10               
> > > [    0.000000]  [<ffffffff817bc9c5>] ? panic+0x1c3/0x204                        
> > > [    0.000000]  [<ffffffff81d21910>] ? numa_clear_kernel_node_hotplug+0xe6/0xf2 
> > > [    0.000000]  [<ffffffff8107741b>] __stack_chk_fail+0x1b/0x20                 
> > > [    0.000000]  [<ffffffff81d21910>] numa_clear_kernel_node_hotplug+0xe6/0xf2   
> > > [    0.000000]  [<ffffffff81d21e5d>] numa_init+0x1a5/0x520                      
> > > [    0.000000]  [<ffffffff81d222b1>] x86_numa_init+0x19/0x3d                    
> > > [    0.000000]  [<ffffffff81d22460>] initmem_init+0x9/0xb                       
> > > [    0.000000]  [<ffffffff81d0d00c>] setup_arch+0x94f/0xc82                     
> > > [    0.000000]  [<ffffffff81d05120>] ? early_idt_handlers+0x120/0x120           
> > > [    0.000000]  [<ffffffff817bd0bb>] ? printk+0x55/0x6b                         
> > > [    0.000000]  [<ffffffff81d05120>] ? early_idt_handlers+0x120/0x120           
> > > [    0.000000]  [<ffffffff81d05d9b>] start_kernel+0xe8/0x4d6                    
> > > [    0.000000]  [<ffffffff81d05120>] ? early_idt_handlers+0x120/0x120           
> > > [    0.000000]  [<ffffffff81d05120>] ? early_idt_handlers+0x120/0x120           
> > > [    0.000000]  [<ffffffff81d055ee>] x86_64_start_reservations+0x2a/0x2c        
> > > [    0.000000]  [<ffffffff81d05751>] x86_64_start_kernel+0x161/0x184            
> > > [    0.000000] RIP 0x46                                                         
> > > 
> > > This is caused by writing over end of numa mask bitmap.
> > > 
> > > numa_clear_kernel_node try to set node id in a mask bitmap, it iterating all
> > > reserved region and assume every regions have valid nid. It is not true because
> > > There's an exception for graphic memory quirks. see function trim_snb_memory
> > > in arch/x86/kernel/setup.c
> > > 
> > > It is easily to reproduce the bug in kdump kernel because kdump kernel use
> > > prereserved memory instead of whole memory, but kexec pass other reserved memory
> > > ranges to 2nd kernel as well. like below in my test:
> > > kdump kernel ram 0x2d000000 - 0x37bfffff
> > > One of the reserved regions: 0x40000000 - 0x40100000
> > > 
> > > The above reserved region includes 0x40004000, a page excluded in
> > > trim_snb_memory. For this memblock reserved region the nid is not set it is
> > > still default value MAX_NUMNODES. later node_set callback will set bit
> > > MAX_NUMNODES in nodemask bitmap thus stack corruption happen. 
> > > 
> > > Fixing this by adding a check, do not call node_set in case nid is MAX_NUMNODES.
> > > 
> > > Signed-off-by: Dave Young <dyoung@redhat.com>
> > > ---
> > 
> > Looks good to me.
> > 
> > Reviewed-by: Yasuaki Ishimatsu <isimatu.yasuaki@jp.fujitsu.com>
> 

> Per suggestion from Xishi Qiu, I would like to add below comment in code and repost
> Could I carry with your Reviewed-by line? 
> 
> /* In case booting with numa=off and only using part of system ram
>  * ie. mem=nn[kMG] or in kdump kernel, numa_meminfo may not include all the
>  * memblock.reserved memory because trim_snb_memory() reserves specific pages 
>  * for Sandy Bridge graphics. */

I don't think the issue is related to numa=off. If we use numa=off,
all memory ranges which are informed by e820 are managed to Node 0.

The issue occurs when reserved memory ranges are not included
in system ram which informed by e820.

Thanks,
Yasuaki Ishimatsu

> 
> Thanks
> Dave

  parent reply	other threads:[~2015-04-06 14:26 UTC|newest]

Thread overview: 19+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-04-01  4:53 Dave Young
2015-04-01  5:11 ` Dave Young
2015-04-01  7:27   ` Xishi Qiu
2015-04-01  7:41     ` Dave Young
2015-04-01  8:21       ` Xishi Qiu
2015-04-01  8:34         ` Xishi Qiu
2015-04-01  9:17           ` Dave Young
2015-04-01  9:33             ` Xishi Qiu
2015-04-02 19:15       ` Yasuaki Ishimatsu
2015-04-03  7:03         ` Dave Young
2015-04-02  1:51   ` Xishi Qiu
2015-04-02  3:24     ` Dave Young
2015-04-02 19:36 ` Yasuaki Ishimatsu
2015-04-03  7:15   ` Dave Young
2015-04-03  7:17     ` Ingo Molnar
2015-04-03  7:23       ` Dave Young
2015-04-06 14:26     ` Yasuaki Ishimatsu [this message]
2015-04-07  3:33       ` Dave Young
2015-04-07 14:15         ` Yasuaki Ishimatsu

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=552297b0.8a858c0a.5345.fffff99a@mx.google.com \
    --to=yasu.isimatu@gmail.com \
    --cc=akpm@linux-foundation.org \
    --cc=bhe@redhat.com \
    --cc=dyoung@redhat.com \
    --cc=hpa@zytor.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=mingo@redhat.com \
    --cc=tglx@linutronix.de \
    --cc=x86@kernel.org \
    --subject='Re: [PATCH] x86/numa: kernel stack corruption fix' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).