Why is "Memory split" Kconfig option only for EMBEDDED?

Why is "Memory split" Kconfig option only for EMBEDDED?

[Norbert Kiesel]

Hi,

I remember reading on LKML some time ago that using VMSPLIT_3G_OPT would
be optimal for a machine with exactly 1GB memory (like my current
desktop). Why is that option only prompted for after selecting EMBEDDED
(which I normally don't select for desktop machines)?

Best,
  Norbert

Re: Why is "Memory split" Kconfig option only for EMBEDDED?

[Arjan van de Ven]

On Wed, 2006-12-06 at 12:42 +0100, Norbert Kiesel wrote:
> Hi,
> 
> I remember reading on LKML some time ago that using VMSPLIT_3G_OPT would
> be optimal for a machine with exactly 1GB memory (like my current
> desktop). Why is that option only prompted for after selecting EMBEDDED
> (which I normally don't select for desktop machines

because it changes the userspace ABI and has some other caveats.... this
is not something you should muck with lightly 


-- 
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org

Re: Why is "Memory split" Kconfig option only for EMBEDDED?

[Norbert Kiesel]

On Wed, 2006-12-06 at 12:58 +0100, Arjan van de Ven wrote:
> On Wed, 2006-12-06 at 12:42 +0100, Norbert Kiesel wrote:
> > Hi,
> > 
> > I remember reading on LKML some time ago that using VMSPLIT_3G_OPT would
> > be optimal for a machine with exactly 1GB memory (like my current
> > desktop). Why is that option only prompted for after selecting EMBEDDED
> > (which I normally don't select for desktop machines
> 
> because it changes the userspace ABI and has some other caveats.... this
> is not something you should muck with lightly 
> 

Hmm, but it's also marked EXPERIMENTAL. Would that not be the
sufficient?  Assuming I don't use any external/binary drivers and a
self-compiled kernel w//o any additional patches: is there really any
downside?

</nk>
 

Re: Why is "Memory split" Kconfig option only for EMBEDDED?

[Arjan van de Ven]

On Wed, 2006-12-06 at 13:19 +0100, Norbert Kiesel wrote:
> On Wed, 2006-12-06 at 12:58 +0100, Arjan van de Ven wrote:
> > On Wed, 2006-12-06 at 12:42 +0100, Norbert Kiesel wrote:
> > > Hi,
> > > 
> > > I remember reading on LKML some time ago that using VMSPLIT_3G_OPT would
> > > be optimal for a machine with exactly 1GB memory (like my current
> > > desktop). Why is that option only prompted for after selecting EMBEDDED
> > > (which I normally don't select for desktop machines
> > 
> > because it changes the userspace ABI and has some other caveats.... this
> > is not something you should muck with lightly 
> > 
> 
> Hmm, but it's also marked EXPERIMENTAL. Would that not be the
> sufficient?  Assuming I don't use any external/binary drivers and a
> self-compiled kernel w//o any additional patches: is there really any
> downside?

I said *userspace ABI*. You're changing something that userspace has
known about and was documented since the start of Linux. So userspace
application binaries can break, and at least you're changing the rules
on them. That's fine if you know what you're doing.. but in a general
system... not a good default, hence the EMBEDDED.


I assumed entirely that you're not using binary drivers, those are just
horrid in the first place :)

-- 
if you want to mail me at work (you don't), use arjan (at) linux.intel.com
Test the interaction between Linux and your BIOS via http://www.linuxfirmwarekit.org

Re: Why is "Memory split" Kconfig option only for EMBEDDED?

[Adrian Bunk]

On Wed, Dec 06, 2006 at 01:19:08PM +0100, Norbert Kiesel wrote:
> On Wed, 2006-12-06 at 12:58 +0100, Arjan van de Ven wrote:
> > On Wed, 2006-12-06 at 12:42 +0100, Norbert Kiesel wrote:
> > > Hi,
> > > 
> > > I remember reading on LKML some time ago that using VMSPLIT_3G_OPT would
> > > be optimal for a machine with exactly 1GB memory (like my current
> > > desktop). Why is that option only prompted for after selecting EMBEDDED
> > > (which I normally don't select for desktop machines
> > 
> > because it changes the userspace ABI and has some other caveats.... this
> > is not something you should muck with lightly 
> > 
> 
> Hmm, but it's also marked EXPERIMENTAL. Would that not be the
> sufficient?  Assuming I don't use any external/binary drivers and a
> self-compiled kernel w//o any additional patches: is there really any
> downside?

- Wine doesn't work (I'm not sure about VMSPLIT_3G_OPT, but
                     VMSPLIT_2G definitely breaks Wine)
- AFAIR some people reported problems with some Java programs
  after fiddling with the vmsplit options

EMBEDDED isn't exactly the right way to hide it, but the vmsplit options 
aren't something you can safely change.

> </nk>

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Why is "Memory split" Kconfig option only for EMBEDDED?

[Norbert Kiesel]

On Wed, 2006-12-06 at 13:45 +0100, Arjan van de Ven wrote:
> On Wed, 2006-12-06 at 13:19 +0100, Norbert Kiesel wrote:
> > On Wed, 2006-12-06 at 12:58 +0100, Arjan van de Ven wrote:
> > > On Wed, 2006-12-06 at 12:42 +0100, Norbert Kiesel wrote:
> > > > Hi,
> > > > 
> > > > I remember reading on LKML some time ago that using VMSPLIT_3G_OPT would
> > > > be optimal for a machine with exactly 1GB memory (like my current
> > > > desktop). Why is that option only prompted for after selecting EMBEDDED
> > > > (which I normally don't select for desktop machines
> > > 
> > > because it changes the userspace ABI and has some other caveats.... this
> > > is not something you should muck with lightly 
> > > 
> > 
> > Hmm, but it's also marked EXPERIMENTAL. Would that not be the
> > sufficient?  Assuming I don't use any external/binary drivers and a
> > self-compiled kernel w//o any additional patches: is there really any
> > downside?
> 
> I said *userspace ABI*. You're changing something that userspace has
> known about and was documented since the start of Linux. So userspace
> application binaries can break, and at least you're changing the rules
> on them. That's fine if you know what you're doing.. but in a general
> system... not a good default, hence the EMBEDDED.

Thanks for the reply. I was not asking to change the default, I just
want to see the option in e.g. menuconfig. And the help text already has
a very strong advise to leave it at VMSPLIT_3G.

Anyway, I don't want to stress this further: I'm happy enough with my
Kconfig that has "if EMBEDDED" removed for the prompt. 

Best,
  Norbert

Re: Why is "Memory split" Kconfig option only for EMBEDDED?

[Jan Engelhardt]

On Dec 6 2006 14:36, Norbert Kiesel wrote:
>On Wed, 2006-12-06 at 13:45 +0100, Arjan van de Ven wrote:
>> On Wed, 2006-12-06 at 13:19 +0100, Norbert Kiesel wrote:
>> > On Wed, 2006-12-06 at 12:58 +0100, Arjan van de Ven wrote:
>> > > On Wed, 2006-12-06 at 12:42 +0100, Norbert Kiesel wrote:
>> > > > Hi,
>> > > > 
>> > > > I remember reading on LKML some time ago that using
>> > > > VMSPLIT_3G_OPT would be optimal for a machine with
>> > > > exactly 1GB memory (like my current desktop). Why is
>> > > > that option only prompted for after selecting EMBEDDED
>> > > > (which I normally don't select for desktop machines
>> > > 
>> > > because it changes the userspace ABI and has some other
>> > > caveats.... this is not something you should muck with
>> > > lightly
>> > 
>> > Hmm, but it's also marked EXPERIMENTAL. Would that not be
>> > the sufficient?  Assuming I don't use any external/binary
>> > drivers and a self-compiled kernel w//o any additional
>> > patches: is there really any downside?
>> 
>> I said *userspace ABI*. You're changing something that
>> userspace has known about and was documented since the start
>> of Linux. So userspace application binaries can break, and at
>> least you're changing the rules on them. That's fine if you
>> know what you're doing.. but in a general system... not a
>> good default, hence the EMBEDDED.
>
>Thanks for the reply. I was not asking to change the default, I
>just want to see the option in e.g. menuconfig. And the help
>text already has a very strong advise to leave it at
>VMSPLIT_3G.

I have not had yet any problems with VMSPLIT_3G_OPT ever since I
used it -- which dates back to when it was a feature of Con
Kolivas's patchset (known as LOWMEM1G), [even] before it got
merged in mainline.

It only seem to break the VMware compilation process, but what
they do in the makefiles is not really standard anyhow.

>Anyway, I don't want to stress this further: I'm happy enough with my
>Kconfig that has "if EMBEDDED" removed for the prompt. 



	-`J'
-- 

Re: Why is "Memory split" Kconfig option only for EMBEDDED?

[Jan Engelhardt]

On Dec 6 2006 22:00, Jan Engelhardt wrote:
>
>I have not had yet any problems with VMSPLIT_3G_OPT ever since I
>used it -- which dates back to when it was a feature of Con
>Kolivas's patchset (known as LOWMEM1G), [even] before it got
>merged in mainline.

(Excluding the cases Adrian Bunk listed: WINE, which I don't use, and 
also 'some Java programs' which I have not seen.)


	-`J'
-- 

Re: Why is "Memory split" Kconfig option only for EMBEDDED?

[Alejandro Riveira Fernández]

El Wed, 6 Dec 2006 14:10:03 +0100
Adrian Bunk <bunk@stusta.de> escribió:

> On Wed, Dec 06, 2006 at 01:19:08PM +0100, Norbert Kiesel wrote:
> > On Wed, 2006-12-06 at 12:58 +0100, Arjan van de Ven wrote:
> > > On Wed, 2006-12-06 at 12:42 +0100, Norbert Kiesel wrote:
> > > > Hi,
> > > > 
> > > > I remember reading on LKML some time ago that using VMSPLIT_3G_OPT would
> > > > be optimal for a machine with exactly 1GB memory (like my current
> > > > desktop). Why is that option only prompted for after selecting EMBEDDED
> > > > (which I normally don't select for desktop machines
> > > 
> > > because it changes the userspace ABI and has some other caveats.... this
> > > is not something you should muck with lightly 
> > > 
> > 
> > Hmm, but it's also marked EXPERIMENTAL. Would that not be the
> > sufficient?  Assuming I don't use any external/binary drivers and a
> > self-compiled kernel w//o any additional patches: is there really any
> > downside?
> 
> - Wine doesn't work (I'm not sure about VMSPLIT_3G_OPT, but
>                      VMSPLIT_2G definitely breaks Wine)

 I use VMSPLIT_3G_OPT=y and wine works just fine (only tested with one
 program). Edgy + 2.6.19-rc1



> - AFAIR some people reported problems with some Java programs
>   after fiddling with the vmsplit options
> 
> EMBEDDED isn't exactly the right way to hide it, but the vmsplit options 
> aren't something you can safely change.
> 
> > </nk>
> 
> cu
> Adrian
> 

Re: Why is "Memory split" Kconfig option only for EMBEDDED?

[Norbert Kiesel]

On Sat, 2006-12-09 at 13:27 +0100, Alejandro Riveira Fernández wrote:
> El Wed, 6 Dec 2006 14:10:03 +0100
> Adrian Bunk <bunk@stusta.de> escribió:
> 
> > On Wed, Dec 06, 2006 at 01:19:08PM +0100, Norbert Kiesel wrote:
> > > On Wed, 2006-12-06 at 12:58 +0100, Arjan van de Ven wrote:
> > > > On Wed, 2006-12-06 at 12:42 +0100, Norbert Kiesel wrote:
> > > > > Hi,
> > > > > 
> > > > > I remember reading on LKML some time ago that using VMSPLIT_3G_OPT would
> > > > > be optimal for a machine with exactly 1GB memory (like my current
> > > > > desktop). Why is that option only prompted for after selecting EMBEDDED
> > > > > (which I normally don't select for desktop machines
> > > > 
> > > > because it changes the userspace ABI and has some other caveats.... this
> > > > is not something you should muck with lightly 
> > > > 
> > > 
> > > Hmm, but it's also marked EXPERIMENTAL. Would that not be the
> > > sufficient?  Assuming I don't use any external/binary drivers and a
> > > self-compiled kernel w//o any additional patches: is there really any
> > > downside?
> > 
> > - Wine doesn't work (I'm not sure about VMSPLIT_3G_OPT, but
> >                      VMSPLIT_2G definitely breaks Wine)
> 
>  I use VMSPLIT_3G_OPT=y and wine works just fine (only tested with one
>  program). Edgy + 2.6.19-rc1
> 
> 
> 
> > - AFAIR some people reported problems with some Java programs
> >   after fiddling with the vmsplit options
> > 
> > EMBEDDED isn't exactly the right way to hide it, but the vmsplit options 
> > aren't something you can safely change.
> > 

So far all first-hand experiences I heard of were positive (i.e. I did
not get an emaail from anyone saying: It had a negative effect for me),
so I propose to apply the patch from Con Kolivas. The wording in the
description still very strongly recommends to not change that value, and
it's still dependent on EXPERIMENTAL. I append the patch just because
it's short, it's also available from
http://www.kernel.org/pub/linux/kernel/people/ck/patches/2.6/2.6.19/2.6.19-ck2/patches/kconfig-expose_vmsplit_option.patch

The options to alter the vmsplit to enable more lowmem are hidden behind the
embedded option. Make it more exposed for -ck users and make the help menu
more explicit about what each option means.

Signed-off-by: Con Kolivas <kernel@kolivas.org>

---
 arch/i386/Kconfig |   10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

Index: linux-2.6.19-ck2/arch/i386/Kconfig
===================================================================
--- linux-2.6.19-ck2.orig/arch/i386/Kconfig	2006-11-30 11:30:32.000000000 +1100
+++ linux-2.6.19-ck2/arch/i386/Kconfig	2006-12-09 09:01:36.000000000 +1100
@@ -500,7 +500,7 @@ endchoice
 
 choice
 	depends on EXPERIMENTAL
-	prompt "Memory split" if EMBEDDED
+	prompt "Memory split"
 	default VMSPLIT_3G
 	help
 	  Select the desired split between kernel and user memory.
@@ -519,14 +519,14 @@ choice
 	  option alone!
 
 	config VMSPLIT_3G
-		bool "3G/1G user/kernel split"
+		bool "Default 896MB lowmem (3G/1G user/kernel split)"
 	config VMSPLIT_3G_OPT
 		depends on !HIGHMEM
-		bool "3G/1G user/kernel split (for full 1G low memory)"
+		bool "1GB lowmem (3G/1G user/kernel split)"
 	config VMSPLIT_2G
-		bool "2G/2G user/kernel split"
+		bool "2GB lowmem (2G/2G user/kernel split)"
 	config VMSPLIT_1G
-		bool "1G/3G user/kernel split"
+		bool "3GB lowmem (1G/3G user/kernel split)"
 endchoice
 
 config PAGE_OFFSET

Re: Why is "Memory split" Kconfig option only for EMBEDDED?

[Adrian Bunk]

On Sat, Dec 09, 2006 at 04:45:04PM +0100, Norbert Kiesel wrote:
> On Sat, 2006-12-09 at 13:27 +0100, Alejandro Riveira Fernández wrote:
> > El Wed, 6 Dec 2006 14:10:03 +0100
> > Adrian Bunk <bunk@stusta.de> escribió:
> > 
> > > On Wed, Dec 06, 2006 at 01:19:08PM +0100, Norbert Kiesel wrote:
> > > > On Wed, 2006-12-06 at 12:58 +0100, Arjan van de Ven wrote:
> > > > > On Wed, 2006-12-06 at 12:42 +0100, Norbert Kiesel wrote:
> > > > > > Hi,
> > > > > > 
> > > > > > I remember reading on LKML some time ago that using VMSPLIT_3G_OPT would
> > > > > > be optimal for a machine with exactly 1GB memory (like my current
> > > > > > desktop). Why is that option only prompted for after selecting EMBEDDED
> > > > > > (which I normally don't select for desktop machines
> > > > > 
> > > > > because it changes the userspace ABI and has some other caveats.... this
> > > > > is not something you should muck with lightly 
> > > > > 
> > > > 
> > > > Hmm, but it's also marked EXPERIMENTAL. Would that not be the
> > > > sufficient?  Assuming I don't use any external/binary drivers and a
> > > > self-compiled kernel w//o any additional patches: is there really any
> > > > downside?
> > > 
> > > - Wine doesn't work (I'm not sure about VMSPLIT_3G_OPT, but
> > >                      VMSPLIT_2G definitely breaks Wine)
> > 
> >  I use VMSPLIT_3G_OPT=y and wine works just fine (only tested with one
> >  program). Edgy + 2.6.19-rc1
> > 
> > 
> > 
> > > - AFAIR some people reported problems with some Java programs
> > >   after fiddling with the vmsplit options
> > > 
> > > EMBEDDED isn't exactly the right way to hide it, but the vmsplit options 
> > > aren't something you can safely change.
> > > 
> 
> So far all first-hand experiences I heard of were positive (i.e. I did
> not get an emaail from anyone saying: It had a negative effect for me),
>...

As I said, VMSPLIT_2G does break Wine for me.

cu
Adrian

-- 

       "Is there not promise of rain?" Ling Tan asked suddenly out
        of the darkness. There had been need of rain for many days.
       "Only a promise," Lao Er said.
                                       Pearl S. Buck - Dragon Seed

Re: Why is "Memory split" Kconfig option only for EMBEDDED?

[Pavel Machek]

Hi!

> So far all first-hand experiences I heard of were positive (i.e. I did
> not get an emaail from anyone saying: It had a negative effect for me),
> so I propose to apply the patch from Con Kolivas. The wording in the
> description still very strongly recommends to not change that value, and
> it's still dependent on EXPERIMENTAL. I append the patch just because

There's a big difference between 'experimental' and 'known to broke
obscure userspace apps'.

-- 
Thanks for all the (sleeping) penguins.

Re: Why is "Memory split" Kconfig option only for EMBEDDED?

[Norbert Kiesel]

On Thu, 2006-12-14 at 14:47 +0000, Pavel Machek wrote:
> Hi!
> 
> > So far all first-hand experiences I heard of were positive (i.e. I did
> > not get an emaail from anyone saying: It had a negative effect for me),
> > so I propose to apply the patch from Con Kolivas. The wording in the
> > description still very strongly recommends to not change that value, and
> > it's still dependent on EXPERIMENTAL. I append the patch just because
> 
> There's a big difference between 'experimental' and 'known to broke
> obscure userspace apps'.

True, but abusing EMBEDDED for "only do that if you know what you are
doing and if it breaks, you have to keep the pieces" is not good
either. 
Some other places that seem to fall into the same category are
IPX_INTERN ("...This might break existing applications...") which is
neither EMBEDDED nor EXPERIMENTAL or RMW_INSNS ("...It is very likely
that this will cause serious problems on any Amiga...") that is
dependent on ADVANCED.

Anyway, perhaps I should just select EMBEDDED although I don't have a
small system (though a 6 year old 1Ghz K7 with 1GB mem might be
considered small by some people these days :-), and ignore all the other
options that pop up through this.

Best,
  Norbert

Re: Why is "Memory split" Kconfig option only for EMBEDDED?

[linux]

> I have not had yet any problems with VMSPLIT_3G_OPT ever since I
> used it -- which dates back to when it was a feature of Con
> Kolivas's patchset (known as LOWMEM1G), [even] before it got
> merged in mainline.
>
> (Excluding the cases Adrian Bunk listed: WINE, which I don't use, and 
> also 'some Java programs' which I have not seen.)

Seconded.  I have several servers with 1G of memory, and appreciate the
option very much; I maintained it as a custom patch long before it
became a CONFIG option.

Turning on CONFIG_EMBEDDED makes it a bit annoying to be sure not to
play with any of the other far more dangerous options that enables.
(I suppose I could just maintain a local patch to remove that from Kconfig.)

The last I remember hearing, the vm system wasn't very happy with highem
much smaller than lowmem (128M/896M = 1/7) anyway.

There's nothing wrong with a stern warning, but I'd think that disabling
CONFIG_NET would break a lot more user-space programs, and that's not
protected.


How about the following (which also fixes a bug if you select VMSPLIT_2G
and HIGHEM; with 64-bit page tables, the split must be on a 1G boundary):

choice
	depends on EXPERIMENTAL
	prompt "Memory split"
	default VMSPLIT_3G
	help
	  Select the desired split between kernel and user memory.
	  If you are not absolutely sure what you are doing, leave this
	  option alone!

	  There are important efficiency reasons why the user address
	  space and the kernel address space must both fit into the 4G
	  linear virtual address space provided by the x86 architecture.

	  Normally, Linux divides this into 3G for user virtual memory
	  and 1G for kernel memory, which holds up to 896M of RAM plus all
	  memory-mapped peripheral (e.g. PCI) devices.	Excess RAM is ignored.

	  If the "High memory support" options are enabled, the excess memory
	  is available as "high memory", which can be used for user data,
	  including file system caches, but not kernel data structures.
	  However, accessing high memory from the kernel is slightly more
	  costly than low memory, as it has to be mapped into the kernel
	  address range first.

	  This option lets systems choose to have a larger "low memory" space,
	  either to avoid the need for high memory support entirely, or for
	  workloads which require particularly large kernel data structures.

	  The downside is that the available user address space is reduced.
	  While most programs do not care, this is an incompatible change
	  to the kernel binary interface, and must be made with caution.
	  Some programs that process a lot of data will work more slowly or
	  fail, and some programs that do clever things with virtual memory
	  will crash immediately.

	  In particular, changing this option from the default breaks valgrind
	  version 3.1.0, VMware, and some Java virtual machines.

	config VMSPLIT_3G
		bool "Default 896MB lowmem (3G/1G user/kernel split)"
	config VMSPLIT_3G_OPT
		depends on !HIGHMEM
		bool "1G lowmem (2.75G/1.25G user/kernel split)   CAUTION"
	config VMSPLIT_2G
		bool "1.875G lowmem (2G/2G user/kernel split)     CAUTION"
	config VMSPLIT_2G_OPT
		depends on !HIGHMEM
		bool "2G lowmem (1.875G/2.125G user/kernel split) CAUTION"
	config VMSPLIT_1G
		bool "2.875G lowmem (1G/3G user/kernel split)     CAUTION"
	config VMSPLIT_1G_OPT
		depends on !HIGHMEM
		bool "3G lowmem (896M/3.125G user/kernel split)   CAUTION"
endchoice

config PAGE_OFFSET
        hex
        default 0xB0000000 if VMSPLIT_3G_OPT
        default 0x80000000 if VMSPLIT_2G
        default 0x78000000 if VMSPLIT_2G_OPT
        default 0x40000000 if VMSPLIT_1G
        default 0x38000000 if VMSPLIT_1G_OPT
        default 0xC0000000

(Copyright on the above abandoned to the public domain.)