LKML Archive on lore.kernel.org
help / color / mirror / Atom feed
* Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
@ 2007-02-08 13:10 Etienne Lorrain
  2007-02-09  5:59 ` Vivek Goyal
  0 siblings, 1 reply; 14+ messages in thread
From: Etienne Lorrain @ 2007-02-08 13:10 UTC (permalink / raw)
  To: vgoyal; +Cc: H. Peter Anvin, linux-kernel, Eric W. Biederman

> >  I assume you care about this ELF header because you are also a user of
> > the ELF file vmlinux, aren't you?
>
> Yes I am. I use kexec boot loader which has capability to load ELF kernel
> images (vmlinux). That's why I am concerned about linking real mode code
> in vmlinux as for kdump case I shall have to be aware that kernel vmlinux
> might contain a special PT_LOAD type program header which will contain
> real mode code and it does not have to be loaded. Then I will run into
> guessing business which one is that real mode PT_LOAD  program header and
> my assumption might very well break in next few kernel release.

 So is kexec able to relocate this vmlinux? If kexec and Gujin do approximately
the same thing they should do it the same way. If you cannot get a PT_LOAD
section, maybe we can put a simple system in NOTE, or just create a
PT_LOAD16 if the linker accepts other values.
 I do not really like to relocate after vmlinux has been linked at a fixed address,
because I am not sure you can guess each address to relocate or not:
you can define permanent address in the linker file by simply "symbol = address"
and those should never be relocated. For instance the very high addresses
on ia32 may point to registers or FPU instead of memory, so may or may not
have to be relocated. I also would better like you not to relocate the real-mode
addresses.
 
> >  And you do not want to write protect the kernel code (if the CPU write protection
> > is not working, the hardware is not working so debug will be difficult, and a simple
> > CRC32 can tell kernel memory failure) and use twice the same code memory
> > (with different data area or saving kernel data elsewhere before reload).
> >  Is that related to module loading or instruction set detection/patch or multiprocessor?
>
> If I understand it right, you seem to be suggesting that I don't have to 
> reload the kernel text and I can only reload the data for second kernel?
> 
> We run the whole of the kernel from a mutually execlusive location from
> first kernel to mitigate the concerns that first kernel's ongoing DMA might
> corrupt second kernel. That's why first kernel's text can't be reused.

  So in some exception handler, you detect something is wrong and then
 jump to a new kernel. Maybe this class of bug (DMA or hardware bit flipped)
 should be detected even without this double kernel environment, by running a
 CRC32 on the kernel text section in this exception handler, displayed that in
 the crash dump.

> Secondly, it gives flexibility to user that either he can choose to use 
> the production kernel as capture kernel or an entirely different custom
> kernel can be used as capture kernel.

  IHMO if it is just capturing the memory, I would have a kernel without
 any modules (usb-storage drivers linked-in) and save to USB key the
 core. Automatically loading modules after the crash may be a problem.

> If real mode code is linked with vmlinux, then kdump will be broken.

 I do not want to break anything for fun. I need either a reason or a bug.

> bzImage is relocatable. If a new kernel image format is introduced
> (compressed ELF), then I will prefer it to be a relocatable one
> (if possible).

 It may be possible to do some kind of "ld -r -o vmlinux", but delaying
the decision where to run the kernel is just delaying: someone has to
decide, and even if Gujin do not decide It will ask the user who most
of the time will have no clue... and on which basis do you want the
bootloader to decide... maybe this SDRAM is faulty and usually the
SDRAM are 512 Mbytes so run the kernel at 512 + 16 Mbytes, but
only if there is more than 1024 Mbytes of SDRAM?

  Thanks,
  Etienne.


	

	
		
___________________________________________________________________________ 
Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! 
Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses 
http://fr.answers.yahoo.com

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
  2007-02-08 13:10 Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3 Etienne Lorrain
@ 2007-02-09  5:59 ` Vivek Goyal
  2007-02-09 13:04   ` Etienne Lorrain
  0 siblings, 1 reply; 14+ messages in thread
From: Vivek Goyal @ 2007-02-09  5:59 UTC (permalink / raw)
  To: Etienne Lorrain; +Cc: H. Peter Anvin, linux-kernel, Eric W. Biederman

On Thu, Feb 08, 2007 at 01:10:58PM +0000, Etienne Lorrain wrote:
> > >  I assume you care about this ELF header because you are also a user of
> > > the ELF file vmlinux, aren't you?
> >
> > Yes I am. I use kexec boot loader which has capability to load ELF kernel
> > images (vmlinux). That's why I am concerned about linking real mode code
> > in vmlinux as for kdump case I shall have to be aware that kernel vmlinux
> > might contain a special PT_LOAD type program header which will contain
> > real mode code and it does not have to be loaded. Then I will run into
> > guessing business which one is that real mode PT_LOAD  program header and
> > my assumption might very well break in next few kernel release.
> 
>  So is kexec able to relocate this vmlinux? If kexec and Gujin do approximately
> the same thing they should do it the same way.

No, as of today, kexec does not relocate vmlinux. At compilation time,
vmlinux is compiled for a fixed address and vmlinux is loaded at that
address. This compile address can be controlled with CONFIG_PHYSICAL_START,
as you already mentioned in your patchset.

So in the case of kdump, if one wishes to use vmlinux instead of a relocatable
bzImage, he will re-compile the kernel for a different address and then use
that vmlinux.

OTOH, boot loader can possibly relocate vmlinux because all the relocation
information is retained in vmlinux (Using ld option --emit-relocs). But
probably self relocating image should be a better option as all the
boot-loaders out there don't have to be modifed.

> If you cannot get a PT_LOAD
> section, maybe we can put a simple system in NOTE, or just create a
> PT_LOAD16 if the linker accepts other values.

My guess is that PT_LOAD16 is not an acceptable value. Putting information
in PT_NOTE seems interesting (As Eric already mentioned).

>  I do not really like to relocate after vmlinux has been linked at a fixed address,
> because I am not sure you can guess each address to relocate or not:
> you can define permanent address in the linker file by simply "symbol = address"
> and those should never be relocated. For instance the very high addresses
> on ia32 may point to registers or FPU instead of memory, so may or may not
> have to be relocated. I also would better like you not to relocate the real-mode
> addresses.
>  

I think there will have to be an upper limit on the address where you can
relocate your image to. Anyway, given i386 virtual memory address space
constraint, I think you can not load kernel image in high memory (896MB).
It has to run anywhere below that.

I think you are referring to defining absolute symbols. Already there are
some absolute symbols generated when kernel is compiled and these are
not relocated while image is being loaded at a non-compiled address. So if
a boot loader decides to relocate a vmlinux image, it will have to skip
relocations present w.r.t absolute symbols. (Unfortunately, ld does generate
relocation entries even for absolute symbols).
 
> > >  And you do not want to write protect the kernel code (if the CPU write protection
> > > is not working, the hardware is not working so debug will be difficult, and a simple
> > > CRC32 can tell kernel memory failure) and use twice the same code memory
> > > (with different data area or saving kernel data elsewhere before reload).
> > >  Is that related to module loading or instruction set detection/patch or multiprocessor?
> >
> > If I understand it right, you seem to be suggesting that I don't have to 
> > reload the kernel text and I can only reload the data for second kernel?
> > 
> > We run the whole of the kernel from a mutually execlusive location from
> > first kernel to mitigate the concerns that first kernel's ongoing DMA might
> > corrupt second kernel. That's why first kernel's text can't be reused.
> 
>   So in some exception handler, you detect something is wrong and then
>  jump to a new kernel.

In a nutshell, Yes.

>  Maybe this class of bug (DMA or hardware bit flipped)
>  should be detected even without this double kernel environment, by running a
>  CRC32 on the kernel text section in this exception handler, displayed that in
>  the crash dump.

We already do checksum verification to make sure newly loaded kernel has
not been corrupted before we jump to it. What's the point in doing the
verification existing kernel text. Because even that is intact, and you
reload your data section, that data section will have to be reloaded at
the same place where previous data section was. Now after you re-loaded 
the data section, some DMA might corrupt it now (Because we never stopped
DMAs). So loading a fresh copy of text and data section at a reserved 
memory location seems safer. 

> 
> > Secondly, it gives flexibility to user that either he can choose to use 
> > the production kernel as capture kernel or an entirely different custom
> > kernel can be used as capture kernel.
> 
>   IHMO if it is just capturing the memory, I would have a kernel without
>  any modules (usb-storage drivers linked-in) and save to USB key the
>  core. Automatically loading modules after the crash may be a problem.

As a user you can very well do it. Distros are also doing something 
similar. They just insert 1-2 modules from initrd (custom initrd created
based on choice of dump saving destination) and capture the dump either
to a local storage or over network and then system is booted back to
production kernel.

> 
> > If real mode code is linked with vmlinux, then kdump will be broken.
> 
>  I do not want to break anything for fun. I need either a reason or a bug.
> 
> > bzImage is relocatable. If a new kernel image format is introduced
> > (compressed ELF), then I will prefer it to be a relocatable one
> > (if possible).
> 
>  It may be possible to do some kind of "ld -r -o vmlinux", but delaying
> the decision where to run the kernel is just delaying: someone has to
> decide, and even if Gujin do not decide It will ask the user who most
> of the time will have no clue... and on which basis do you want the
> bootloader to decide... maybe this SDRAM is faulty and usually the
> SDRAM are 512 Mbytes so run the kernel at 512 + 16 Mbytes, but
> only if there is more than 1024 Mbytes of SDRAM?
> 

Interesting question, How does a boot loader/user decide where to load
the relocatable image? I think it depends on the new interesting usages
of the relocatable kernel. As of today, kexec knows where is reserved
memory region (Read from /proc/iomem) and it loads the image at the
start of that reserved region (Meeting alignment restrictions, if any). So
in this case boot loader takes the decision. May be a user option also
can be created, something like --load-address=0xXYZ and then people
can have fun loading same image at various addresses.

I think this is interesting usage if boot loader can somehow decide
that first few MB is faulty and load kernel at a memory address higher
than that. If first whole 1G is faulty then you can't do much on i386,
but on 64bit arches like x86_64, this should not be an issue. 

Thanks
Vivek

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
  2007-02-09  5:59 ` Vivek Goyal
@ 2007-02-09 13:04   ` Etienne Lorrain
  2007-02-09 19:42     ` Eric W. Biederman
  0 siblings, 1 reply; 14+ messages in thread
From: Etienne Lorrain @ 2007-02-09 13:04 UTC (permalink / raw)
  To: vgoyal; +Cc: H. Peter Anvin, linux-kernel, Eric W. Biederman

--- Vivek Goyal <vgoyal@in.ibm.com> wrote:
> No, as of today, kexec does not relocate vmlinux. At compilation time,
> vmlinux is compiled for a fixed address and vmlinux is loaded at that
> address. This compile address can be controlled with CONFIG_PHYSICAL_START,
> as you already mentioned in your patchset.
> 
> So in the case of kdump, if one wishes to use vmlinux instead of a relocatable
> bzImage, he will re-compile the kernel for a different address and then use
> that vmlinux.
> 
> OTOH, boot loader can possibly relocate vmlinux because all the relocation
> information is retained in vmlinux (Using ld option --emit-relocs). But
> probably self relocating image should be a better option as all the
> boot-loaders out there don't have to be modifed.

  Well, a self relocating image cannot be an ELF file because the code
 to relocate the ELF cannot be executed at the wrong place.
 If relocation is needed, I would better like not to link vmlinux at a
 fixed address first. In fact I wonder if we are talking of the same
 kind of relocation: you seem to talk about "ld --pic-executable" while
 I am thinking of "ld -r" to "locate" it at the bootloader loading time.
  The main problem I see is that I do not have the code for that, and
 I am going deeper/earlier into the generation of vmlinux, while comments
 are "already you are too early, loading an ELF file is too complex for
 a bootloader". The solution I have already is working.

> > If you cannot get a PT_LOAD
> > section, maybe we can put a simple system in NOTE, or just create a
> > PT_LOAD16 if the linker accepts other values.
> 
> My guess is that PT_LOAD16 is not an acceptable value. Putting information
> in PT_NOTE seems interesting (As Eric already mentioned).

 In fact, thinking more about that, I am going back to my implementation
 of it, because on ia32 the interrupt vectors are at address zero and it is
 obviouly an invalid address to load an ELF for this architecture.
 But for the linker, it is the right address to link it (being an offset
 into a non-null segment in real mode), and because the entry point has
 to be zero (I cannot use the ELF entry value) the program header base
 address has to be zero.
 Anyway, your loader in (probably) written in C, so a test against zero
 is a simple thing to do, and should be done anyway to check for an
 incorrect ELF program header. I wonder if this NOTE program header is
 not simply designed as an "end" marker, it does not seem to contain
 anything, so me defining the realmode after that program header may
 be a good idea.

> We already do checksum verification to make sure newly loaded kernel has
> not been corrupted before we jump to it. What's the point in doing the
> verification existing kernel text. Because even that is intact, and you
> reload your data section, that data section will have to be reloaded at
> the same place where previous data section was. Now after you re-loaded 
> the data section, some DMA might corrupt it now (Because we never stopped
> DMAs). So loading a fresh copy of text and data section at a reserved 
> memory location seems safer.

 If you really are tring to catch an erroneous DMA into the kernel,
 is it better to keep an exact copy of the kernel you are using somewhere
 else to do a bit-to-bit comparisson after the crash, and so no relocate.
 Anyway if the DMA crash has crashed the exception handling area the system
 is dead anyway.

> Interesting question, How does a boot loader/user decide where to load
> the relocatable image? I think it depends on the new interesting usages
> of the relocatable kernel. As of today, kexec knows where is reserved
> memory region (Read from /proc/iomem) and it loads the image at the
> start of that reserved region (Meeting alignment restrictions, if any). So
> in this case boot loader takes the decision. May be a user option also
> can be created, something like --load-address=0xXYZ and then people
> can have fun loading same image at various addresses.

 I think that you are asking too much for the bootloader user, and that
is a decision he has to take *before* the crash; even me, I would select
one address like 16 Mbytes and stick with it...
 If the running Linux kernel do not erase Gujin from memory, it could
also go back to real mode and do a "longjmp()" to return to the
Gujin interface - but most of the times the system had a reason to
crash (for instance a ventilator stopped working) and you can plan
whatever you want in software...

  Thanks,
  Etienne.


	

	
		
___________________________________________________________________________ 
Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! 
Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses 
http://fr.answers.yahoo.com

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
  2007-02-09 13:04   ` Etienne Lorrain
@ 2007-02-09 19:42     ` Eric W. Biederman
  2007-02-11 13:23       ` RE : " Etienne Lorrain
  0 siblings, 1 reply; 14+ messages in thread
From: Eric W. Biederman @ 2007-02-09 19:42 UTC (permalink / raw)
  To: Etienne Lorrain; +Cc: vgoyal, H. Peter Anvin, linux-kernel

Etienne Lorrain <etienne_lorrain@yahoo.fr> writes:

>   Well, a self relocating image cannot be an ELF file because the code
>  to relocate the ELF cannot be executed at the wrong place.
>  If relocation is needed, I would better like not to link vmlinux at a
>  fixed address first. In fact I wonder if we are talking of the same
>  kind of relocation: you seem to talk about "ld --pic-executable" while
>  I am thinking of "ld -r" to "locate" it at the bootloader loading time.
>   The main problem I see is that I do not have the code for that, and
>  I am going deeper/earlier into the generation of vmlinux, while comments
>  are "already you are too early, loading an ELF file is too complex for
>  a bootloader". The solution I have already is working.

Being very clear.  ld --pic-executable or ld -shared is essentially
what we are talking when we are discussing building a relocatable
kernel.  Something with the properties of an ELF ET_DYN executable
that does not use an interpreter.  ld.so is the only common executable
of this type in linux.

Loading an ELF executable is very much:
- Walk through the program headers and for each PT_LOAD segment load
  it at the address it requests.
- Jump to e_entry from the ELF header.

If you are working with a relocatable ELF object the rules become:
- Walk through the program header once to find the size and alignment
  of the chunk of memory that the linux kernel needs.
- Find a hole in the memory map that meets those requirements.
- Compute the offset 
- Walk through the program headers and for each PT_LOAD segment
  add offset to the addresses and load the segment like normal.
- Jump to offset + e_entry.

This is within the scope of what a bootloader can reasonably do, and
I have implemented it in etherboot as well as /sbin/kexec. 

>> > If you cannot get a PT_LOAD
>> > section, maybe we can put a simple system in NOTE, or just create a
>> > PT_LOAD16 if the linker accepts other values.
>> 
>> My guess is that PT_LOAD16 is not an acceptable value. Putting information
>> in PT_NOTE seems interesting (As Eric already mentioned).
>
>  In fact, thinking more about that, I am going back to my implementation
>  of it, because on ia32 the interrupt vectors are at address zero and it is
>  obviouly an invalid address to load an ELF for this architecture.

No special games no special rules with the well defined ELF components
either add a note that you can define all of the semantics yourself
or don't do it.  That is what the notes are there for.

>  But for the linker, it is the right address to link it (being an offset
>  into a non-null segment in real mode), and because the entry point has
>  to be zero (I cannot use the ELF entry value) the program header base
>  address has to be zero.

Agreed. When the object file is linked using offset 0, and letting the
real mode segments do have different bases to do your relocation is fine.

>  Anyway, your loader in (probably) written in C, so a test against zero
>  is a simple thing to do, and should be done anyway to check for an
>  incorrect ELF program header. I wonder if this NOTE program header is
>  not simply designed as an "end" marker, it does not seem to contain
>  anything, so me defining the realmode after that program header may
>  be a good idea.

We have been very sparse on the usage of ELF notes but yes they exist
and yes people do look at them.  Please dig up a copy of the ELF spec
and read up on them or look at etherboot for an example.

>  If you really are tring to catch an erroneous DMA into the kernel,
>  is it better to keep an exact copy of the kernel you are using somewhere
>  else to do a bit-to-bit comparisson after the crash, and so no relocate.
>  Anyway if the DMA crash has crashed the exception handling area the system
>  is dead anyway.

No.  We are not trying to catch a erroneous DMA in that sense.  We
do not shutdown any drivers when switching to the new kernel from
panic(), because we don't know what is broken.   Any single bit
of kernel code of could cause problems not just the exception table.
Running in an area that we have never used for DMA and is completely
reserved gives us freedom to not worry about it.  I.e. This is not
error detection but future error prevention and it works.

Before starting the new kernel we do a sha256 checksum test on the
new kernel and on our code that is running the checksum.  All of which
comes from /sbin/kexec.  Not compiled into the running kernel.  The
policy is in user space.

>> Interesting question, How does a boot loader/user decide where to load
>> the relocatable image? I think it depends on the new interesting usages
>> of the relocatable kernel. As of today, kexec knows where is reserved
>> memory region (Read from /proc/iomem) and it loads the image at the
>> start of that reserved region (Meeting alignment restrictions, if any). So
>> in this case boot loader takes the decision. May be a user option also
>> can be created, something like --load-address=0xXYZ and then people
>> can have fun loading same image at various addresses.
>
>  I think that you are asking too much for the bootloader user, and that
> is a decision he has to take *before* the crash; even me, I would select
> one address like 16 Mbytes and stick with it.

Yes.  Unfortunately there is no one value that works on all machines,
which is why we are moving to a relocatable kernel.

Currently we specify crashkernel=size@location on the kernel command
line to reserve the memory.  Hopefully we can reduce this to just
crashkernel=size and have the kernel find a reasonable whole in
the memory map to reserve.  Where that hole is, is exported to
userspace so /sbin/kexec just puts the kernel in that hole.

>  If the running Linux kernel do not erase Gujin from memory, it could
> also go back to real mode and do a "longjmp()" to return to the
> Gujin interface - but most of the times the system had a reason to
> crash (for instance a ventilator stopped working) and you can plan
> whatever you want in software...

True, and no solution is perfect.   The target is to catch the maximum
number of situations that can be caught.

Going back to real mode generally doesn't work because the BIOS get's
confused with the changes in hardware state from linux running.  Some
systems it does work on though.  Going back to real mode especially
don't work well when the kernel doesn't do a clean shutdown.

In a normal context there are two practical advantages to a bootloader
speaking ELF.
1) The load address is no longer fixed at 1MB.  So if (for example) we
   want to get the performance advantages of 4MB pages all we have to
   do is tweak the  alignment and the load address to be at 4MB.
2) Being able to choose somewhere else in the memory map that works.
   If we have a big (32MB uncompressed) kernel with all of the modules
   compiled in and there is a memory hole at 15MB-16MB.  The normal
   load address won't work so the bootloader can pick another address.
   Similarly problems appear when people place acpi tables at lower
   addresses.

For a 64bit kernel I have thought placing the kernel above 4GB has
several interesting advantages for making more memory available for
DMA accesses without needed an IOMMU.

Then we get into the cases like Xen, which have no real mode to go
through so need completely different bootloaders.

Eric

^ permalink raw reply	[flat|nested] 14+ messages in thread

* RE : Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
  2007-02-09 19:42     ` Eric W. Biederman
@ 2007-02-11 13:23       ` Etienne Lorrain
  2007-02-11 20:49         ` Eric W. Biederman
  0 siblings, 1 reply; 14+ messages in thread
From: Etienne Lorrain @ 2007-02-11 13:23 UTC (permalink / raw)
  To: Eric W. Biederman; +Cc: vgoyal, H. Peter Anvin, linux-kernel

Eric W. Biederman wrote:
> Etienne Lorrain writes:
> >   Well, a self relocating image cannot be an ELF file because the code
> >  to relocate the ELF cannot be executed at the wrong place.
> >  If relocation is needed, I would better like not to link vmlinux at a
> >  fixed address first. In fact I wonder if we are talking of the same
> >  kind of relocation: you seem to talk about "ld --pic-executable" while
> >  I am thinking of "ld -r" to "locate" it at the bootloader loading time.
> >   The main problem I see is that I do not have the code for that, and
> >  I am going deeper/earlier into the generation of vmlinux, while comments
> >  are "already you are too early, loading an ELF file is too complex for
> >  a bootloader". The solution I have already is working.
> 
> Being very clear.  ld --pic-executable or ld -shared is essentially
> what we are talking when we are discussing building a relocatable
> kernel.  Something with the properties of an ELF ET_DYN executable
> that does not use an interpreter.  ld.so is the only common executable
> of this type in linux.

 So we are talking of two different things.
 You want to _compile_ the totality of the kernel as code and data
relocatable - right now only modules are compiled that way.
 On IA32 architecture, there is no problem having the code relocatable,
because most/all calls and jumps are PC relative, so very few instructions
need to be different.
 The problem is having the data relocatable, then every access to memory
has to be relative to a register, and so the register EBX is reserved
for that by the compiler.
 The IA32 processor does not have enough registers already, removing one
of them makes the assembly seriously worse - mostly when dealing with
64 bits "long long" - you can only get two of them in register and a
simple addition produces long and slow code.
 Having the data relocatable could be done by using the base address of
IA32 segments, but this does not seem to be the preferred method because
segments use seems to slow down the processor a lot more.

 What I was talking of was "loading" a non relocatable kernel (compiled
as usual without -fPIC/-shared) at any address, so not inducing slow
down whatever the base address used. I said that it is possible, maybe
quite easy to do so, but I do not have the code for it - and people
say right now that an ELF bootloader is not what they want to.

 Now, if people wants to switch to a completely relocatable kernel,
because they do not mind the slow down (that I am not really able
to quantify) - and because anyway most of their kernel is modules,
then Gujin can relocate that without _any_ problem; in fact it
does already load in intermediate (HIMEM malloc'ed) memory under
DOS to relocate it later (called 'late relocation' in Gujin code,
done every times the code is not at the right address in memory,
interrupt disabled and in protected mode).
 Loading and executing at whatever address is not a problem,
and should be doable already in the real-mode function of the
kernel called by Gujin, by modifying few elements of the
structure passed in parameter of this real-mode function:
it should be done after loading the E820 memory map for the
BIOS parameter page.
I am reading right now Gujin source and note that you may want
to access array elf_reloc[] and variable nb_elf_reloc in this
Linux function - will be added in next Gujin version.
 You already can access type LOADER_str parameter "loader"
where you can get runadr which can overwrite the ELF entry
point address. The old ELF header is at loader.load_address
if it is needed.

 The main problem Gujin has, is to decide where to load a
relocatable kernel, so this policy is left to the kernel and
not to the user. For instance and as you made me think of, if you
do not want for security reasons that the kernel memory could
be modified by an attacker by using the legacy DMA, then you
want to load it at an address which cannot be modified this way,
i.e. over 16 Mbytes. Note that some chipset had an extension I/O
address to extend this legacy DMA to 32 bits addresses, but it
has not been documented and probably not implemented widely.

 I do not know how many people are like me, but I am currently
typing on a monolitic kernel because I find it a _lot_ simpler
to do a ".config" once and make it slowly evolve with new kernel
versions (answering No to most of the questions), than managing
hundreds of module files each time I regenerate/remove a kernel.
 The kernel I am working on is not relocatable because - well
I do not have crash to debug...

 So I think that from the Gujin point of view, everything you need
is done and working (but the access to elf_reloc)- please explain what
you want more if not. If you want a solution to load now, you can
use patch linux-2.6.20-rc5 in sourceforge, which uses an intermediate
binary dump of the ELF file, and can be relocated the non ELF way.

> >  In fact, thinking more about that, I am going back to my implementation
> >  of it, because on ia32 the interrupt vectors are at address zero and it is
> >  obviouly an invalid address to load an ELF for this architecture.
> 
> No special games no special rules with the well defined ELF components
> either add a note that you can define all of the semantics yourself
> or don't do it.  That is what the notes are there for.

 So, why do I want an ELF file loaded by a bootloader?
 The main reason is to be able to use all the usual ELF tools to do
all sort of things, like showing segments size and base address with
"objdump" and "readelf", adding and extracting sections with "strip"
and "objcopy", and dumping section content with either "objdump -m i8086"
or "objdump" alone to display the assembly instructions after link,
for real-mode or protected-mode, with symbols.
 For all that, I need a 100% compatible ELF file structure - that is
what I have already.
 I do _not_ need to use the Linux loader at the prompt of a Xterm for
this file, if I try I have this very informative error message:
[etienne@localhost linux-2.6.20]$ ./vmlinux.elf 
Killed
[etienne@localhost linux-2.6.20]$ 
 I know that to load this ELF kernel file the user will need a different
loader like Gujin or your kexec. I know that I am asking you not to treat
an impossible action (loading a section at address 0 over the interrupt
table), and I say please.

> >  But for the linker, it is the right address to link it (being an offset
> >  into a non-null segment in real mode), and because the entry point has
> >  to be zero (I cannot use the ELF entry value) the program header base
> >  address has to be zero.
> 
> Agreed. When the object file is linked using offset 0, and letting the
> real mode segments do have different bases to do your relocation is fine.

 Nice to agree.

> We have been very sparse on the usage of ELF notes but yes they exist
> and yes people do look at them.  Please dig up a copy of the ELF spec
> and read up on them or look at etherboot for an example.

 I've read it again from cover to cover, version 1.2, pretty boring
 stuff, isn't it?
 So technically - and if I understand you correctly (feel free to correct
 me) - you ask me to put the real mode code into the NOTE section, by
 adding a few words header and concatening the code.

 There is two way to do so, the first being simply to replace in
vmlinux.lds.S (in the patch I sent) the lines:
  .realmode 0 : AT (ADDR(.bss) + SIZEOF(.bss) - LOAD_OFFSET) {
  ..... *(.init.text16) ...
  } :text16 = 0x9090
by something like:
  .realmode 0 : AT (ADDR(.bss) + SIZEOF(.bss) - LOAD_OFFSET) {
  LONG(8)
  LONG(SIZEOF (.realmode))
  LONG(1)
  BYTE('R') BYTE('E') BYTE('A') BYTE('L')
  BYTE('M') BYTE('O') BYTE('D') BYTE('E')
  ..... *(.init.text16) ...
  } :note = 0x9090
That has the main advantage to keep producing a single link, but the
major disadvantage that this code is no more linked at 0 but offseted
by the header of 5 longs, the call to the function has to be offseted,
and it may not work nicely when another developper comes in few years
and add another few Kbytes note - everything will work if his note
is added after mine but break badly if it is added before mine.
 The order it is processed is under the linker control, and anybody
could add a note in C by declaring a structure in the .note section.
 You will probably agree with me that it is not an acceptable solution.

 The other way to do is to produce an independant link for the real-mode
program (linked at base address 0), convert it to binary and include
the resulting block into the NOTE segment. First that is a lot more
complex (it can be done but is more complex), and two I loose all
the debuging information I wanted in the ELF file.
 I can still see the real-mode assembly by forcing objdump to interpret
the NOTE section as real-mode code, but the offset calculated for
assembly calls and jumps is wrong, I lost all symbols, in short it
goes against what I wanted initially.

  Thanks,
  Etienne.


	

	
		
___________________________________________________________________________ 
Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! 
Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses 
http://fr.answers.yahoo.com

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: RE : Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
  2007-02-11 13:23       ` RE : " Etienne Lorrain
@ 2007-02-11 20:49         ` Eric W. Biederman
  2007-02-12 10:42           ` Etienne Lorrain
  0 siblings, 1 reply; 14+ messages in thread
From: Eric W. Biederman @ 2007-02-11 20:49 UTC (permalink / raw)
  To: Etienne Lorrain; +Cc: vgoyal, H. Peter Anvin, linux-kernel

Etienne Lorrain <etienne_lorrain@yahoo.fr> writes:

> Eric W. Biederman wrote:
>> Etienne Lorrain writes:
>> >   Well, a self relocating image cannot be an ELF file because the code
>> >  to relocate the ELF cannot be executed at the wrong place.
>> >  If relocation is needed, I would better like not to link vmlinux at a
>> >  fixed address first. In fact I wonder if we are talking of the same
>> >  kind of relocation: you seem to talk about "ld --pic-executable" while
>> >  I am thinking of "ld -r" to "locate" it at the bootloader loading time.
>> >   The main problem I see is that I do not have the code for that, and
>> >  I am going deeper/earlier into the generation of vmlinux, while comments
>> >  are "already you are too early, loading an ELF file is too complex for
>> >  a bootloader". The solution I have already is working.
>> 
>> Being very clear.  ld --pic-executable or ld -shared is essentially
>> what we are talking when we are discussing building a relocatable
>> kernel.  Something with the properties of an ELF ET_DYN executable
>> that does not use an interpreter.  ld.so is the only common executable
>> of this type in linux.
>
>  So we are talking of two different things.
>  You want to _compile_ the totality of the kernel as code and data
> relocatable - right now only modules are compiled that way.
>  On IA32 architecture, there is no problem having the code relocatable,
> because most/all calls and jumps are PC relative, so very few instructions
> need to be different.
>  The problem is having the data relocatable, then every access to memory
> has to be relative to a register, and so the register EBX is reserved
> for that by the compiler.
>  The IA32 processor does not have enough registers already, removing one
> of them makes the assembly seriously worse - mostly when dealing with
> 64 bits "long long" - you can only get two of them in register and a
> simple addition produces long and slow code.
>  Having the data relocatable could be done by using the base address of
> IA32 segments, but this does not seem to be the preferred method because
> segments use seems to slow down the processor a lot more.

There are two more options in addition to segments:
- Running at a fixed virtual address.
- Processing relocation entries to modify the initial code/data to
  live where we want.  We don't have the that shared libraries do of wanting
  to share text/data pages loaded at different addresses.

>  What I was talking of was "loading" a non relocatable kernel (compiled
> as usual without -fPIC/-shared) at any address, so not inducing slow
> down whatever the base address used. I said that it is possible, maybe
> quite easy to do so, but I do not have the code for it - and people
> say right now that an ELF bootloader is not what they want to.

The thing to pick up on is that we really don't want the gcc flag
-fPIC but if ld wasn't an idiot we would want the linker flag -shared.
Anyway we have working code merged that processes all of the
relocation entries and modifies the kernel to run properly where we
load it at.  This has all been done and merged at this point.  It's
only available to a bzImage loader but it is all there.  Currently the
only overhead is something like 5 or 10 percent overhead to hold the
relocation entries.

The only thing currently compiled with -fPIC is the kernel gzip
decompressor and it is carefully coded so it doesn't generate any
relocation entries that need processing.  It is just throw away code
so we don't much care about it's efficiency.

>  Now, if people wants to switch to a completely relocatable kernel,
> because they do not mind the slow down (that I am not really able
> to quantify) - and because anyway most of their kernel is modules,
> then Gujin can relocate that without _any_ problem; in fact it
> does already load in intermediate (HIMEM malloc'ed) memory under
> DOS to relocate it later (called 'late relocation' in Gujin code,
> done every times the code is not at the right address in memory,
> interrupt disabled and in protected mode).

Yes.  That is roughly what we are saying.  We want that at least
as a well defined option.

>  Loading and executing at whatever address is not a problem,
> and should be doable already in the real-mode function of the
> kernel called by Gujin, by modifying few elements of the
> structure passed in parameter of this real-mode function:
> it should be done after loading the E820 memory map for the
> BIOS parameter page.
> I am reading right now Gujin source and note that you may want
> to access array elf_reloc[] and variable nb_elf_reloc in this
> Linux function - will be added in next Gujin version.
>  You already can access type LOADER_str parameter "loader"
> where you can get runadr which can overwrite the ELF entry
> point address. The old ELF header is at loader.load_address
> if it is needed.
>
>  The main problem Gujin has, is to decide where to load a
> relocatable kernel, so this policy is left to the kernel and
> not to the user. For instance and as you made me think of, if you
> do not want for security reasons that the kernel memory could
> be modified by an attacker by using the legacy DMA, then you
> want to load it at an address which cannot be modified this way,
> i.e. over 16 Mbytes. Note that some chipset had an extension I/O
> address to extend this legacy DMA to 32 bits addresses, but it
> has not been documented and probably not implemented widely.
>
>  I do not know how many people are like me, but I am currently
> typing on a monolitic kernel because I find it a _lot_ simpler
> to do a ".config" once and make it slowly evolve with new kernel
> versions (answering No to most of the questions), than managing
> hundreds of module files each time I regenerate/remove a kernel.
>  The kernel I am working on is not relocatable because - well
> I do not have crash to debug...

Yes.   In this sense I'm a tool builder.  I build general capabilities
that can be used to solve specific problems.  A relocatable kernel
is a general capability and the only interesting problem to date
has been taking crash dumps.  To reduce the maintenance overhead I
try and remove the cases where you need an extra config option for
the new functionality.  If there is not overhead a config option just
makes things harder to test.

>  So I think that from the Gujin point of view, everything you need
> is done and working (but the access to elf_reloc)- please explain what
> you want more if not. If you want a solution to load now, you can
> use patch linux-2.6.20-rc5 in sourceforge, which uses an intermediate
> binary dump of the ELF file, and can be relocated the non ELF way.

This requires a more detailed examination.  The previous conversation
indicated that if we could replace setup.S, video.S and edd.S with
C versions it would be very interesting.  If we happen to change the
link so they are available in an ELF executable so much the better.

If we wind up adding two versions of the real mode code one written
in assembly and one written in C accessible by different means
by different bootloaders that is not interesting.

>  So, why do I want an ELF file loaded by a bootloader?

>  The main reason is to be able to use all the usual ELF tools to do
> all sort of things, like showing segments size and base address with
> "objdump" and "readelf", adding and extracting sections with "strip"
> and "objcopy", and dumping section content with either "objdump -m i8086"
> or "objdump" alone to display the assembly instructions after link,
> for real-mode or protected-mode, with symbols.
>  For all that, I need a 100% compatible ELF file structure - that is
> what I have already.

Yes.  That is nice. 

>  I know that to load this ELF kernel file the user will need a different
> loader like Gujin or your kexec. I know that I am asking you not to treat
> an impossible action (loading a section at address 0 over the interrupt
> table), and I say please.

It is my belief that we can solve this without hacks, and still have
a useable structure.

[snip]
>  So technically - and if I understand you correctly (feel free to correct
>  me) - you ask me to put the real mode code into the NOTE section, by
>  adding a few words header and concatening the code.
>
>  There is two way to do so, the first being simply to replace in
> vmlinux.lds.S (in the patch I sent) the lines:
>   .realmode 0 : AT (ADDR(.bss) + SIZEOF(.bss) - LOAD_OFFSET) {
>   ..... *(.init.text16) ...
>   } :text16 = 0x9090
> by something like:
>   .realmode 0 : AT (ADDR(.bss) + SIZEOF(.bss) - LOAD_OFFSET) {
>   LONG(8)
>   LONG(SIZEOF (.realmode))
>   LONG(1)
>   BYTE('R') BYTE('E') BYTE('A') BYTE('L')
>   BYTE('M') BYTE('O') BYTE('D') BYTE('E')
>   ..... *(.init.text16) ...
>   } :note = 0x9090

That would certainly get the job done.

>  The other way to do is to produce an independant link for the real-mode
> program (linked at base address 0), convert it to binary and include
> the resulting block into the NOTE segment. First that is a lot more
> complex (it can be done but is more complex), and two I loose all
> the debuging information I wanted in the ELF file.
>  I can still see the real-mode assembly by forcing objdump to interpret
> the NOTE section as real-mode code, but the offset calculated for
> assembly calls and jumps is wrong, I lost all symbols, in short it
> goes against what I wanted initially.

So I really don't care if we move whole real mode section into a note
or if we just put a pointer to it into a note.  What is important is
that we use a note to find it.

Which means that we could do something goofy in the linker script
like we do with the current vdso.  So we could give it a virtual
address of 0 and a physical address in the init code section.
Allowing us to load it normally and then just drop it later.  Not
perfect but something that would not need special treatment by
a bootloader unless it wanted to copy the data from there into
some location in low memory and execute it.

For me the objective is not so much reusing the existing tools
(although that is a plus) but more to be able to build a unified
binary that can be used for everything, and will give us the freedom
to do interesting things with the kernel in the future, and hopefully
something that is more or less usable by portable bootloaders.  Having
a different file format and different rules for different
architectures is a pain.

Eric

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
  2007-02-11 20:49         ` Eric W. Biederman
@ 2007-02-12 10:42           ` Etienne Lorrain
  2007-02-12 12:29             ` Eric W. Biederman
  0 siblings, 1 reply; 14+ messages in thread
From: Etienne Lorrain @ 2007-02-12 10:42 UTC (permalink / raw)
  To: Eric W. Biederman; +Cc: vgoyal, H. Peter Anvin, linux-kernel

--- "Eric W. Biederman" wrote:
> So I really don't care if we move whole real mode section into a note
> or if we just put a pointer to it into a note.  What is important is
> that we use a note to find it.

 Well, we can advertise by a note the section number or the section
name which contains the real-mode code, but finding the section of
type SHT_PROGBITS having SHF_EXECINSTR flags linked at zero is not
that difficult to do: it is what Gujin does right now when it does
not find the program header linked at zero.

> Which means that we could do something goofy in the linker script
> like we do with the current vdso.  So we could give it a virtual
> address of 0 and a physical address in the init code section.

 Gujin loads at the physical address, i.e. kernel is loaded at
0x100000 and not 0xC0100000, is that wrong?
 I am not sure playing these games with addresses is cleaner than
not loading a section which is not in the program header.

> For me the objective is not so much reusing the existing tools
> (although that is a plus) but more to be able to build a unified
> binary that can be used for everything, and will give us the freedom
> to do interesting things with the kernel in the future, and hopefully
> something that is more or less usable by portable bootloaders.  Having
> a different file format and different rules for different
> architectures is a pain.

 For me the objective is to have a reasonable bootloader,
I will not have the time to port back and test with every other
bootloader some transfert of code from Gujin to an ELF looking
like bzImage file.

  Etienne.


	

	
		
___________________________________________________________________________ 
Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! 
Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses 
http://fr.answers.yahoo.com

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
  2007-02-12 10:42           ` Etienne Lorrain
@ 2007-02-12 12:29             ` Eric W. Biederman
  2007-02-12 13:58               ` Etienne Lorrain
  2007-02-12 14:06               ` H. Peter Anvin
  0 siblings, 2 replies; 14+ messages in thread
From: Eric W. Biederman @ 2007-02-12 12:29 UTC (permalink / raw)
  To: Etienne Lorrain; +Cc: vgoyal, H. Peter Anvin, linux-kernel

Etienne Lorrain <etienne_lorrain@yahoo.fr> writes:

> --- "Eric W. Biederman" wrote:
>> So I really don't care if we move whole real mode section into a note
>> or if we just put a pointer to it into a note.  What is important is
>> that we use a note to find it.
>
>  Well, we can advertise by a note the section number or the section
> name which contains the real-mode code, but finding the section of
> type SHT_PROGBITS having SHF_EXECINSTR flags linked at zero is not
> that difficult to do: it is what Gujin does right now when it does
> not find the program header linked at zero.

What I was thinking is that in the not we place the physical address
and length that we load the real mode code at.  My assumption being
that we have marked the real mode code __init or the equivalent,
so we always load and just ignore it later on.

>> Which means that we could do something goofy in the linker script
>> like we do with the current vdso.  So we could give it a virtual
>> address of 0 and a physical address in the init code section.
>
>  Gujin loads at the physical address, i.e. kernel is loaded at
> 0x100000 and not 0xC0100000, is that wrong?

No.  For practical purposes we can say virtual addresses are for
the code and the kernel.  Physical addresses are for the loader.

>  I am not sure playing these games with addresses is cleaner than
> not loading a section which is not in the program header.

It is some cleaner.  I won't call it perfectly clean, but it is
better.  It keeps everything but argument generation generic,
and there is already precedent in things like the vdso.

Playing the games with the addresses does allow the existing debugging
tools to work without problem because the end users of the code all
examine the virtual not the physical address.

>> For me the objective is not so much reusing the existing tools
>> (although that is a plus) but more to be able to build a unified
>> binary that can be used for everything, and will give us the freedom
>> to do interesting things with the kernel in the future, and hopefully
>> something that is more or less usable by portable bootloaders.  Having
>> a different file format and different rules for different
>> architectures is a pain.
>
>  For me the objective is to have a reasonable bootloader,
> I will not have the time to port back and test with every other
> bootloader some transfert of code from Gujin to an ELF looking
> like bzImage file.

I agree I want a reasonable bootloader as well.

With ELF we get a single file format that works on multiple
architectures and for multiple OS-s, with well understood rules.  This
allows for a broader audience who can review and maintain the code.
In addition to the real possibility of multi-architecture or multi-os
bootloaders.

What ELF cannot standardize is the how we pass the initial arguments.
ELF does standardize the initial parameters for unix executables but
the requirements of arguments to kernels, and the years of practice
are quite different.

ELF should make it much easier for people implementing simple
stand-alone executables for testing.  As you don't even need a linker
script or any fancy games if you don't take arguments.   Similarly
the switch from linux to multi-boot that almost standard is just
supporting a different argument passing format.

Eric

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
  2007-02-12 12:29             ` Eric W. Biederman
@ 2007-02-12 13:58               ` Etienne Lorrain
  2007-02-12 14:06               ` H. Peter Anvin
  1 sibling, 0 replies; 14+ messages in thread
From: Etienne Lorrain @ 2007-02-12 13:58 UTC (permalink / raw)
  To: Eric W. Biederman; +Cc: vgoyal, H. Peter Anvin, linux-kernel

--- "Eric W. Biederman" <ebiederm@xmission.com> wrote:
> What I was thinking is that in the not we place the physical address
> and length that we load the real mode code at.  My assumption being
> that we have marked the real mode code __init or the equivalent,
> so we always load and just ignore it later on.

 But Gujin need the real-mode part linked at zero and there isn't
anything linked at this address in the current Linux kernel, no
section at all. If you ask to do a separate link then I loose all
the symbols and the ELF standard tools are a lot less useable.

> Playing the games with the addresses does allow the existing debugging
> tools to work without problem because the end users of the code all
> examine the virtual not the physical address.

 The linker also resolve relocation in virtual addresses so this part
 needs virtual address = 0.

> I agree I want a reasonable bootloader as well.
>  [snip]
> ELF should make it much easier for people implementing simple
> stand-alone executables for testing.  As you don't even need a linker
> script or any fancy games if you don't take arguments.   Similarly
> the switch from linux to multi-boot that almost standard is just
> supporting a different argument passing format.

 There is plenty of possible future, but right now I have a simple
(OK, not perfect, but code is shown) solution which works.
I can modify bits, but there is no point complexifying the system
for possible theoretical problems - by experience you always miss
important future problem while overdesigning for some problem which
never appear.
If someone can show me a real problem - then I'd like to hear from him,
but ELF compatibility will not be up to the point where the user will
be able to run the kernel from a Xterm command line.

  Etienne.


	

	
		
___________________________________________________________________________ 
Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! 
Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses 
http://fr.answers.yahoo.com

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
  2007-02-12 12:29             ` Eric W. Biederman
  2007-02-12 13:58               ` Etienne Lorrain
@ 2007-02-12 14:06               ` H. Peter Anvin
  2007-02-12 19:47                 ` Eric W. Biederman
  1 sibling, 1 reply; 14+ messages in thread
From: H. Peter Anvin @ 2007-02-12 14:06 UTC (permalink / raw)
  To: Eric W. Biederman; +Cc: Etienne Lorrain, vgoyal, linux-kernel

Eric W. Biederman wrote:
> 
> With ELF we get a single file format that works on multiple
> architectures and for multiple OS-s, with well understood rules.  This
> allows for a broader audience who can review and maintain the code.
> In addition to the real possibility of multi-architecture or multi-os
> bootloaders.
> 

Well understood MAYBE (although the "virtual" versus "physical" address 
stuff in ELF is clearly misspecified in the extreme), but they are 
clearly not sufficient for presenting a well-defined entry condition. 
So it's not really anywhere near as straightforward as you make it sound.

	-hpa


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
  2007-02-12 14:06               ` H. Peter Anvin
@ 2007-02-12 19:47                 ` Eric W. Biederman
  0 siblings, 0 replies; 14+ messages in thread
From: Eric W. Biederman @ 2007-02-12 19:47 UTC (permalink / raw)
  To: H. Peter Anvin; +Cc: Etienne Lorrain, vgoyal, linux-kernel

"H. Peter Anvin" <hpa@zytor.com> writes:

> Eric W. Biederman wrote:
>>
>> With ELF we get a single file format that works on multiple
>> architectures and for multiple OS-s, with well understood rules.  This
>> allows for a broader audience who can review and maintain the code.
>> In addition to the real possibility of multi-architecture or multi-os
>> bootloaders.
>>
>
> Well understood MAYBE (although the "virtual" versus "physical" address stuff in
> ELF is clearly misspecified in the extreme), but they are clearly not sufficient
> for presenting a well-defined entry condition. So it's not really anywhere near
> as straightforward as you make it sound.

About the entry condition I completely agree.

The fact that the generic tools almost do what you need and then fall short is
also problematic.

However it still appears a reasonable approach to consider.

Eric

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Re : Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
  2007-02-06 20:55 ` Eric W. Biederman
@ 2007-02-06 21:00   ` H. Peter Anvin
  0 siblings, 0 replies; 14+ messages in thread
From: H. Peter Anvin @ 2007-02-06 21:00 UTC (permalink / raw)
  To: Eric W. Biederman; +Cc: Etienne Lorrain, vgoyal, linux-kernel

Eric W. Biederman wrote:
> 
>>  Modifying the linux real mode assembler, nobody could even want to.
> 
> I have several times.  It's just code.  But I do agree it will likely
> be more maintainable if we can convert it to C.
> 
> On that same token I wrote a compiler in romcc in another context so I
> didn't have to write so much assembly.  And for maintainability and
> comprehensibility it was a big help.
> 

I've long wished that someone would do a proper 16-bit x86 port of gcc; 
however, the .code16gcc is usually good enough, although it produces 
code which is a lot bigger than it needs to be.

	-hpa


^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re: Re : Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
  2007-02-06 20:30 Re : Re : " Etienne Lorrain
@ 2007-02-06 20:55 ` Eric W. Biederman
  2007-02-06 21:00   ` H. Peter Anvin
  0 siblings, 1 reply; 14+ messages in thread
From: Eric W. Biederman @ 2007-02-06 20:55 UTC (permalink / raw)
  To: Etienne Lorrain; +Cc: H. Peter Anvin, vgoyal, linux-kernel

Etienne Lorrain <etienne_lorrain@yahoo.fr> writes:

>  I am not sure to understand: Gujin calls a function inside the ELF real
> mode program header of the Linux kernel. All the system is currently
> in real mode. There isn't any limitation in this function of the kernel to
> decide to print some message and not return at all, this function can
> selectively do so by reading which bootloader ID it is using (another
> parameter). It is not done mostly because "printf" is difficult to do
> in assembler, Gujin has no problem.

This code is currently completely Gujin specific.  If concieved as a
replacement for setup.S it has a chance of passing review.

>> Is your real mode C code section relative such that it can be loaded
>> at different addresses and still work?
>
>   The code is relative - but not the data (first 4 Kbytes at %ds:0 and stack
>  available) - but the whole lot at any segment boundary, i.e. every 16 bytes.
>  Else Gujin would not work under DOS.

Ok.  Similar restrictions to the current real mode code.

>> The program header is for executable loaders the section header is for
>> linkers and the section header is optional in PT_LOAD and PT_DYN
>> executables.  Use of the section header by a loader is a bug.
>
>   Unless if there is a problem, Gujin uses only the program header;
>  it has a look in the section header just in case - that can be removed
>  easily. I was wondering about a relocatable image there - but I do
>  not know enough for that.

When you are relocatable no relocations are processed and all addresses in
the program header are slid by some fixed amount (preserving alignment)
after that the relocated object has to work out the relocations by itself.

>> >   Well, you can generate with the proposed patch and boot with SYSLINUX,
>> >  Grub and LILO, but trying to clean Linux real-mode code is where I begun.
>>
>> I haven't looked yet.  But I believe this is something that we can do,
>> so long as cleaning and feature enhancement are not mixed in a bad way.
>> Phrasing this another way.  What we can do with the interface is
>> determined our interface to existing bootloaders and what bootloaders
>> exist.  Basically it is the rule that we need to preserve backwards
>> compatibility and changes generally need to be evolutionary ones.
>
>   Well, if you want to preserve compatibility with other bootloaders,
>  it is probably possible to put some source around the ELF kernel file,
>  mostly taken from Gujin if you want (GPL), but I wonder why you would
>  like to be compatible with LILO.

LILO is a lot saner then Grub, and it still supports more filesystems...
Just because it memorizes it all before you shut down the system for
simplicity doesn't mean lilo is worse.

>  Modifying the linux real mode assembler, nobody could even want to.

I have several times.  It's just code.  But I do agree it will likely
be more maintainable if we can convert it to C.

On that same token I wrote a compiler in romcc in another context so I
didn't have to write so much assembly.  And for maintainability and
comprehensibility it was a big help.

Eric

^ permalink raw reply	[flat|nested] 14+ messages in thread

* Re : Re : Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3
@ 2007-02-06 20:30 Etienne Lorrain
  2007-02-06 20:55 ` Eric W. Biederman
  0 siblings, 1 reply; 14+ messages in thread
From: Etienne Lorrain @ 2007-02-06 20:30 UTC (permalink / raw)
  To: Eric W. Biederman; +Cc: H. Peter Anvin, vgoyal, linux-kernel

Eric W. Biederman wrote:
> Etienne Lorrain <etienne_lorrain@yahoo.fr> writes:
>>  Well, if the function called at offset 0 in the real-mode section return
>>  non zero, that is probably an error - that is Gujin interface, but do not ask
>>  me to change other bootloaders to handle that.
>>  This function is called with few parameters, one is a string pointer
>>  if the function do not want to print the problem report itself.
>>  The function is called with an offset of zero in its own intel
>>  segment.
>
> So I think there are some backwards compatibility issues with your
> real mode code interface.  Setting a new flag that says to return instead
> of printing an error message and halting would be one way to handle
> this.

 I am not sure to understand: Gujin calls a function inside the ELF real
mode program header of the Linux kernel. All the system is currently
in real mode. There isn't any limitation in this function of the kernel to
decide to print some message and not return at all, this function can
selectively do so by reading which bootloader ID it is using (another
parameter). It is not done mostly because "printf" is difficult to do
in assembler, Gujin has no problem.

> Is your real mode C code section relative such that it can be loaded
> at different addresses and still work?

  The code is relative - but not the data (first 4 Kbytes at %ds:0 and stack
 available) - but the whole lot at any segment boundary, i.e. every 16 bytes.
 Else Gujin would not work under DOS.

> The program header is for executable loaders the section header is for
> linkers and the section header is optional in PT_LOAD and PT_DYN
> executables.  Use of the section header by a loader is a bug.

  Unless if there is a problem, Gujin uses only the program header;
 it has a look in the section header just in case - that can be removed
 easily. I was wondering about a relocatable image there - but I do
 not know enough for that.

> There have been limitations but mostly with respect to page table size
> and the like but they were not limitations a bootloader had to deal with.

Was talking about this comment, but it is old:
/* 0x28000*16 = 2.5 MB, conservative estimate for the current maximum */
http://lxr.linux.no/source/arch/i386/boot/tools/build.c?v=2.4.28

> >   Well, you can generate with the proposed patch and boot with SYSLINUX,
> >  Grub and LILO, but trying to clean Linux real-mode code is where I begun.
>
> I haven't looked yet.  But I believe this is something that we can do,
> so long as cleaning and feature enhancement are not mixed in a bad way.
> Phrasing this another way.  What we can do with the interface is
> determined our interface to existing bootloaders and what bootloaders
> exist.  Basically it is the rule that we need to preserve backwards
> compatibility and changes generally need to be evolutionary ones.

  Well, if you want to preserve compatibility with other bootloaders,
 it is probably possible to put some source around the ELF kernel file,
 mostly taken from Gujin if you want (GPL), but I wonder why you would
 like to be compatible with LILO.
 Modifying the linux real mode assembler, nobody could even want to.

 Etienne.






	

	
		
___________________________________________________________________________ 
Découvrez une nouvelle façon d'obtenir des réponses à toutes vos questions ! 
Profitez des connaissances, des opinions et des expériences des internautes sur Yahoo! Questions/Réponses 
http://fr.answers.yahoo.com

^ permalink raw reply	[flat|nested] 14+ messages in thread

end of thread, other threads:[~2007-02-12 19:47 UTC | newest]

Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2007-02-08 13:10 Re : [PATCH] Compressed ia32 ELF file generation for loading by Gujin 1/3 Etienne Lorrain
2007-02-09  5:59 ` Vivek Goyal
2007-02-09 13:04   ` Etienne Lorrain
2007-02-09 19:42     ` Eric W. Biederman
2007-02-11 13:23       ` RE : " Etienne Lorrain
2007-02-11 20:49         ` Eric W. Biederman
2007-02-12 10:42           ` Etienne Lorrain
2007-02-12 12:29             ` Eric W. Biederman
2007-02-12 13:58               ` Etienne Lorrain
2007-02-12 14:06               ` H. Peter Anvin
2007-02-12 19:47                 ` Eric W. Biederman
  -- strict thread matches above, loose matches on Subject: below --
2007-02-06 20:30 Re : Re : " Etienne Lorrain
2007-02-06 20:55 ` Eric W. Biederman
2007-02-06 21:00   ` H. Peter Anvin

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).