LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
From: Yinghai Lu <yinghai@kernel.org>
To: Thomas Gleixner <tglx@linutronix.de>, Ingo Molnar <mingo@elte.hu>,
"H. Peter Anvin" <hpa@zytor.com>
Cc: "linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: [PATCH 3/6] x86, 64bit, numa: Put pgtable to local node memory
Date: Mon, 27 Dec 2010 16:48:17 -0800 [thread overview]
Message-ID: <4D1933D1.9020609@kernel.org> (raw)
In-Reply-To: <4D19320B.7030007@kernel.org>
Introduce init_memory_mapping_high(), and use it with 64bit.
It will go with every memory segment above 4g to create page table to the
memory range itself.
before this patch all page tables was on one node.
with this patch, one RED-PEN is killed
debug out for 8 sockets system after patch
[ 0.000000] initial memory mapped : 0 - 20000000
[ 0.000000] init_memory_mapping: [0x00000000000000-0x0000007f74ffff]
[ 0.000000] 0000000000 - 007f600000 page 2M
[ 0.000000] 007f600000 - 007f750000 page 4k
[ 0.000000] kernel direct mapping tables up to 7f750000 @ [0x7f74c000-0x7f74ffff]
[ 0.000000] RAMDISK: 7bc84000 - 7f745000
....
[ 0.000000] Adding active range (0, 0x10, 0x95) 0 entries of 3200 used
[ 0.000000] Adding active range (0, 0x100, 0x7f750) 1 entries of 3200 used
[ 0.000000] Adding active range (0, 0x100000, 0x1080000) 2 entries of 3200 used
[ 0.000000] Adding active range (1, 0x1080000, 0x2080000) 3 entries of 3200 used
[ 0.000000] Adding active range (2, 0x2080000, 0x3080000) 4 entries of 3200 used
[ 0.000000] Adding active range (3, 0x3080000, 0x4080000) 5 entries of 3200 used
[ 0.000000] Adding active range (4, 0x4080000, 0x5080000) 6 entries of 3200 used
[ 0.000000] Adding active range (5, 0x5080000, 0x6080000) 7 entries of 3200 used
[ 0.000000] Adding active range (6, 0x6080000, 0x7080000) 8 entries of 3200 used
[ 0.000000] Adding active range (7, 0x7080000, 0x8080000) 9 entries of 3200 used
[ 0.000000] init_memory_mapping: [0x00000100000000-0x0000107fffffff]
[ 0.000000] 0100000000 - 1080000000 page 2M
[ 0.000000] kernel direct mapping tables up to 1080000000 @ [0x107ffbd000-0x107fffffff]
[ 0.000000] memblock_x86_reserve_range: [0x107ffc2000-0x107fffffff] PGTABLE
[ 0.000000] init_memory_mapping: [0x00001080000000-0x0000207fffffff]
[ 0.000000] 1080000000 - 2080000000 page 2M
[ 0.000000] kernel direct mapping tables up to 2080000000 @ [0x207ff7d000-0x207fffffff]
[ 0.000000] memblock_x86_reserve_range: [0x207ffc0000-0x207fffffff] PGTABLE
[ 0.000000] init_memory_mapping: [0x00002080000000-0x0000307fffffff]
[ 0.000000] 2080000000 - 3080000000 page 2M
[ 0.000000] kernel direct mapping tables up to 3080000000 @ [0x307ff3d000-0x307fffffff]
[ 0.000000] memblock_x86_reserve_range: [0x307ffc0000-0x307fffffff] PGTABLE
[ 0.000000] init_memory_mapping: [0x00003080000000-0x0000407fffffff]
[ 0.000000] 3080000000 - 4080000000 page 2M
[ 0.000000] kernel direct mapping tables up to 4080000000 @ [0x407fefd000-0x407fffffff]
[ 0.000000] memblock_x86_reserve_range: [0x407ffc0000-0x407fffffff] PGTABLE
[ 0.000000] init_memory_mapping: [0x00004080000000-0x0000507fffffff]
[ 0.000000] 4080000000 - 5080000000 page 2M
[ 0.000000] kernel direct mapping tables up to 5080000000 @ [0x507febd000-0x507fffffff]
[ 0.000000] memblock_x86_reserve_range: [0x507ffc0000-0x507fffffff] PGTABLE
[ 0.000000] init_memory_mapping: [0x00005080000000-0x0000607fffffff]
[ 0.000000] 5080000000 - 6080000000 page 2M
[ 0.000000] kernel direct mapping tables up to 6080000000 @ [0x607fe7d000-0x607fffffff]
[ 0.000000] memblock_x86_reserve_range: [0x607ffc0000-0x607fffffff] PGTABLE
[ 0.000000] init_memory_mapping: [0x00006080000000-0x0000707fffffff]
[ 0.000000] 6080000000 - 7080000000 page 2M
[ 0.000000] kernel direct mapping tables up to 7080000000 @ [0x707fe3d000-0x707fffffff]
[ 0.000000] memblock_x86_reserve_range: [0x707ffc0000-0x707fffffff] PGTABLE
[ 0.000000] init_memory_mapping: [0x00007080000000-0x0000807fffffff]
[ 0.000000] 7080000000 - 8080000000 page 2M
[ 0.000000] kernel direct mapping tables up to 8080000000 @ [0x807fdfc000-0x807fffffff]
[ 0.000000] memblock_x86_reserve_range: [0x807ffbf000-0x807fffffff] PGTABLE
[ 0.000000] Initmem setup node 0 [0000000000000000-000000107fffffff]
[ 0.000000] NODE_DATA [0x0000107ffbd000-0x0000107ffc1fff]
[ 0.000000] Initmem setup node 1 [0000001080000000-000000207fffffff]
[ 0.000000] NODE_DATA [0x0000207ffbb000-0x0000207ffbffff]
[ 0.000000] Initmem setup node 2 [0000002080000000-000000307fffffff]
[ 0.000000] NODE_DATA [0x0000307ffbb000-0x0000307ffbffff]
[ 0.000000] Initmem setup node 3 [0000003080000000-000000407fffffff]
[ 0.000000] NODE_DATA [0x0000407ffbb000-0x0000407ffbffff]
[ 0.000000] Initmem setup node 4 [0000004080000000-000000507fffffff]
[ 0.000000] NODE_DATA [0x0000507ffbb000-0x0000507ffbffff]
[ 0.000000] Initmem setup node 5 [0000005080000000-000000607fffffff]
[ 0.000000] NODE_DATA [0x0000607ffbb000-0x0000607ffbffff]
[ 0.000000] Initmem setup node 6 [0000006080000000-000000707fffffff]
[ 0.000000] NODE_DATA [0x0000707ffbb000-0x0000707ffbffff]
[ 0.000000] Initmem setup node 7 [0000007080000000-000000807fffffff]
[ 0.000000] NODE_DATA [0x0000807ffba000-0x0000807ffbefff]
Signed-off-by: Yinghai Lu <yinghai@kernel.org>
---
arch/x86/include/asm/page_types.h | 2 +
arch/x86/kernel/setup.c | 8 -----
arch/x86/mm/amdtopology_64.c | 8 +++--
arch/x86/mm/init.c | 8 -----
arch/x86/mm/init_64.c | 54 ++++++++++++++++++++++++++++++++++++++
arch/x86/mm/numa_64.c | 6 ++--
arch/x86/mm/srat_64.c | 2 +
7 files changed, 68 insertions(+), 20 deletions(-)
Index: linux-2.6/arch/x86/include/asm/page_types.h
===================================================================
--- linux-2.6.orig/arch/x86/include/asm/page_types.h
+++ linux-2.6/arch/x86/include/asm/page_types.h
@@ -53,6 +53,8 @@ static inline u64 get_max_mapped(void)
extern unsigned long init_memory_mapping(unsigned long start,
unsigned long end);
+void init_memory_mapping_high(void);
+
extern void initmem_init(unsigned long start_pfn, unsigned long end_pfn,
int acpi, int k8);
extern void free_initmem(void);
Index: linux-2.6/arch/x86/kernel/setup.c
===================================================================
--- linux-2.6.orig/arch/x86/kernel/setup.c
+++ linux-2.6/arch/x86/kernel/setup.c
@@ -941,14 +941,6 @@ void __init setup_arch(char **cmdline_p)
max_low_pfn_mapped = init_memory_mapping(0, max_low_pfn<<PAGE_SHIFT);
max_pfn_mapped = max_low_pfn_mapped;
-#ifdef CONFIG_X86_64
- if (max_pfn > max_low_pfn) {
- max_pfn_mapped = init_memory_mapping(1UL<<32,
- max_pfn<<PAGE_SHIFT);
- /* can we preseve max_low_pfn ?*/
- max_low_pfn = max_pfn;
- }
-#endif
memblock.current_limit = get_max_mapped();
/*
Index: linux-2.6/arch/x86/mm/amdtopology_64.c
===================================================================
--- linux-2.6.orig/arch/x86/mm/amdtopology_64.c
+++ linux-2.6/arch/x86/mm/amdtopology_64.c
@@ -220,12 +220,14 @@ int __init amd_scan_nodes(void)
apicid_base = boot_cpu_physical_apicid;
}
- for_each_node_mask(i, node_possible_map) {
- int j;
-
+ for_each_node_mask(i, node_possible_map)
memblock_x86_register_active_regions(i,
nodes[i].start >> PAGE_SHIFT,
nodes[i].end >> PAGE_SHIFT);
+ init_memory_mapping_high();
+ for_each_node_mask(i, node_possible_map) {
+ int j;
+
for (j = apicid_base; j < cores + apicid_base; j++)
apicid_to_node[(i << bits) + j] = i;
setup_node_bootmem(i, nodes[i].start, nodes[i].end);
Index: linux-2.6/arch/x86/mm/init_64.c
===================================================================
--- linux-2.6.orig/arch/x86/mm/init_64.c
+++ linux-2.6/arch/x86/mm/init_64.c
@@ -607,9 +607,63 @@ void __init initmem_init(unsigned long s
int acpi, int k8)
{
memblock_x86_register_active_regions(0, start_pfn, end_pfn);
+ init_memory_mapping_high();
}
#endif
+struct mapping_work_data {
+ unsigned long start;
+ unsigned long end;
+ unsigned long pfn_mapped;
+};
+
+static int __init_refok
+mapping_work_fn(unsigned long start_pfn, unsigned long end_pfn, void *datax)
+{
+ struct mapping_work_data *data = datax;
+ unsigned long pfn_mapped;
+ unsigned long final_start, final_end;
+
+ final_start = max_t(unsigned long, start_pfn<<PAGE_SHIFT, data->start);
+ final_end = min_t(unsigned long, end_pfn<<PAGE_SHIFT, data->end);
+
+ if (final_end <= final_start)
+ return 0;
+
+ pfn_mapped = init_memory_mapping(final_start, final_end);
+
+ if (pfn_mapped > data->pfn_mapped)
+ data->pfn_mapped = pfn_mapped;
+
+ return 0;
+}
+
+static unsigned long __init_refok
+init_memory_mapping_active_regions(unsigned long start, unsigned long end)
+{
+ struct mapping_work_data data;
+
+ data.start = start;
+ data.end = end;
+ data.pfn_mapped = 0;
+
+ work_with_active_regions(MAX_NUMNODES, mapping_work_fn, &data);
+
+ return data.pfn_mapped;
+}
+
+void __init_refok init_memory_mapping_high(void)
+{
+ if (max_pfn > max_low_pfn) {
+ max_pfn_mapped = init_memory_mapping_active_regions(1UL<<32,
+ max_pfn<<PAGE_SHIFT);
+ /* can we preserve max_low_pfn ? */
+ max_low_pfn = max_pfn;
+
+ memblock.current_limit = get_max_mapped();
+ }
+}
+
void __init paging_init(void)
{
unsigned long max_zone_pfns[MAX_NR_ZONES];
Index: linux-2.6/arch/x86/mm/numa_64.c
===================================================================
--- linux-2.6.orig/arch/x86/mm/numa_64.c
+++ linux-2.6/arch/x86/mm/numa_64.c
@@ -590,11 +590,12 @@ static int __init numa_emulation(unsigne
* the e820 memory map.
*/
remove_all_active_ranges();
- for_each_node_mask(i, node_possible_map) {
+ for_each_node_mask(i, node_possible_map)
memblock_x86_register_active_regions(i, nodes[i].start >> PAGE_SHIFT,
nodes[i].end >> PAGE_SHIFT);
+ init_memory_mapping_high();
+ for_each_node_mask(i, node_possible_map)
setup_node_bootmem(i, nodes[i].start, nodes[i].end);
- }
acpi_fake_nodes(nodes, num_nodes);
numa_init_array();
return 0;
@@ -645,6 +646,7 @@ void __init initmem_init(unsigned long s
for (i = 0; i < nr_cpu_ids; i++)
numa_set_node(i, 0);
memblock_x86_register_active_regions(0, start_pfn, last_pfn);
+ init_memory_mapping_high();
setup_node_bootmem(0, start_pfn << PAGE_SHIFT, last_pfn << PAGE_SHIFT);
}
Index: linux-2.6/arch/x86/mm/srat_64.c
===================================================================
--- linux-2.6.orig/arch/x86/mm/srat_64.c
+++ linux-2.6/arch/x86/mm/srat_64.c
@@ -443,6 +443,8 @@ int __init acpi_scan_nodes(unsigned long
return -1;
}
+ init_memory_mapping_high();
+
/* Account for nodes with cpus and no memory */
nodes_or(node_possible_map, nodes_parsed, cpu_nodes_parsed);
Index: linux-2.6/arch/x86/mm/init.c
===================================================================
--- linux-2.6.orig/arch/x86/mm/init.c
+++ linux-2.6/arch/x86/mm/init.c
@@ -65,16 +65,10 @@ static void __init find_early_table_spac
#ifdef CONFIG_X86_32
/* for fixmap */
tables += roundup(__end_of_fixed_addresses * sizeof(pte_t), PAGE_SIZE);
-#endif
- /*
- * RED-PEN putting page tables only on node 0 could
- * cause a hotspot and fill up ZONE_DMA. The page tables
- * need roughly 0.5KB per GB.
- */
-#ifdef CONFIG_X86_32
good_end = max_pfn_mapped << PAGE_SHIFT;
#endif
+
base = memblock_find_in_range(start, good_end, tables, PAGE_SIZE);
if (base == MEMBLOCK_ERROR)
panic("Cannot find space for the kernel page tables");
next prev parent reply other threads:[~2010-12-28 0:49 UTC|newest]
Thread overview: 48+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-12-18 0:58 [PATCH 0/3] memblock related top down Yinghai Lu
2010-12-28 0:47 ` [PATCH 0/6] memblock related further cleanup for x86 Yinghai Lu
[not found] ` <4D19320B.7030007@kernel.org>
2010-12-28 0:47 ` [PATCH 1/6] x86: Change get_max_mapped() to inline Yinghai Lu
2010-12-29 23:05 ` H. Peter Anvin
2010-12-29 23:30 ` Yinghai Lu
2010-12-29 23:37 ` Yinghai Lu
2010-12-29 23:42 ` H. Peter Anvin
2010-12-29 23:45 ` [PATCH -v2 " Yinghai Lu
2011-02-16 8:50 ` [tip:x86/mm] " tip-bot for Yinghai Lu
2010-12-28 0:48 ` [PATCH 2/6] x86, 64bit, numa: Allocate memnodemap under max_pfn_mapped Yinghai Lu
2011-02-16 8:50 ` [tip:x86/mm] x86-64, " tip-bot for Yinghai Lu
2010-12-28 0:48 ` Yinghai Lu [this message]
2010-12-29 23:46 ` [PATCH -v2 3/6] x86, 64bit, numa: Put pgtable to local node memory Yinghai Lu
2010-12-29 23:50 ` H. Peter Anvin
2010-12-30 0:11 ` Yinghai Lu
2010-12-30 0:39 ` David Rientjes
2010-12-30 0:58 ` H. Peter Anvin
2010-12-30 1:07 ` David Rientjes
2010-12-30 1:53 ` H. Peter Anvin
2010-12-30 9:06 ` Ingo Molnar
2010-12-30 10:28 ` [boot crash] " Ingo Molnar
2010-12-30 10:30 ` Ingo Molnar
2010-12-30 12:01 ` [build failure] " Ingo Molnar
2010-12-30 18:53 ` David Rientjes
2010-12-30 18:54 ` [patch] x86, numa: Fix CONFIG_DEBUG_PER_CPU_MAPS without NUMA emulation David Rientjes
2011-01-06 20:39 ` David Rientjes
2011-01-06 21:25 ` Yinghai Lu
2011-01-06 23:58 ` David Rientjes
2011-01-07 15:32 ` [tip:x86/numa] " tip-bot for David Rientjes
2010-12-30 21:18 ` [boot crash] Re: [PATCH -v2 3/6] x86, 64bit, numa: Put pgtable to local node memory Yinghai Lu
[not found] ` <AANLkTimpyRXS9dNFATh4GqtA46uynutAvK7Qb5S7WPFO@mail.gmail.com>
[not found] ` <20110105134434.GA22816@elte.hu>
2011-01-05 21:24 ` Yinghai Lu
2011-01-07 20:41 ` [PATCH] memblock: Don't adjust size in memblock_find_base() Yinghai Lu
2011-01-10 5:56 ` Benjamin Herrenschmidt
2011-01-10 6:59 ` Yinghai Lu
2011-01-12 1:35 ` Yinghai Lu
2011-01-29 0:37 ` [PATCH - resend] " Yinghai Lu
2011-02-03 20:25 ` Andrew Morton
2011-02-03 20:37 ` Yinghai Lu
2011-02-16 8:51 ` [tip:x86/mm] x86-64, numa: Put pgtable to local node memory tip-bot for Yinghai Lu
2010-12-28 0:48 ` [PATCH 4/6] x86: Use early pre-allocated page table buffer top-down Yinghai Lu
2010-12-28 0:48 ` [PATCH 5/6] x86, 64bit: Move out cleanup higmap [_brk_end, _end) out of init_memory_mapping() Yinghai Lu
2011-02-16 8:51 ` [tip:x86/mm] x86-64: " tip-bot for Yinghai Lu
2010-12-28 0:48 ` [PATCH 6/6] x86: Rename e820_table_* to pgt_buf_* Yinghai Lu
2010-12-30 21:54 ` [PATCH -v2 " Yinghai Lu
2010-12-28 20:21 ` [PATCH 0/3] memblock related top down H. Peter Anvin
2010-12-28 21:36 ` Yinghai Lu
2010-12-28 22:09 ` H. Peter Anvin
2010-12-28 22:25 ` Yinghai Lu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=4D1933D1.9020609@kernel.org \
--to=yinghai@kernel.org \
--cc=hpa@zytor.com \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@elte.hu \
--cc=tglx@linutronix.de \
--subject='Re: [PATCH 3/6] x86, 64bit, numa: Put pgtable to local node memory' \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).