Re: BOOT_CS

Re: BOOT_CS

[Etienne Lorrain]

me wrote:
> Keep initrd few Mb after kernel because I do not know where exactly
> to uncompress it - anyway the kernel will itself move it where it
> wants it.

hpa wrote:
> If you absolutely want to do this -- for pretty much no reason -- you
> can either decompress it twice, decompress it to nowhere (after all,
> the kernel will decompress it when it starts) or move it into place
> before starting the kernel.

  There is another reason to keep the two parts together, but I
 acknowledge this one is fadding away: the problem appears when loading
 Linux (+ initrd) from a DOS floppy or from operating system dated 98
 "reboot to DOS" menu.
 You then usually have an environment whith EMM386 active and the
 processor in virtual 86 mode + paged memory. Most of the times you
 also have a disk cache software loaded first just after the BIOS
 ROM, at the place you want to load Linux.
 You can still have hope: it has always been possible to do DMA in a
 XMS allocated memory (himem.sys managed) so the physical address is
 equal to the virtual address in these blocks.

 What I did is simply load and uncompress Linux to a legally allocated
 HIMEM block (so that is a 100 % compatible DOS software, you can use
 a network disk or a disk cache for that) and check everything is right
 before disabling interruption, switch back to real mode with
 4 Gb segments of non paged memory, copy the block at the right place, 
 and start Linux in protected mode.
 You have to remember keeping the code short in between the back switch
 to real mode and the start of Linux because the data (Linux kernel)
 is at a clear physical address, but your code itself is in a 4 Kbyte
 page which is available - but the page after it may not be loaded
 so no more code...

 When the problem of loading the initrd happen, you have two choice:
 load another XMS block or use the same one as the kernel.
 And here is the problem: if you load another XMS block with the
 initrd, which is usually smaller than the kernel (Kb size wise),
 you may get it in a block of XMS at the _beginning_ of extended
 memory in physical address - because of the first fit algorithm of
 himem.sys. Then, to move the kernel at its final position, you have
 to first move the initrd out of the way to not overwrite it, it
 begins to be complex to write this bit of assembler and debug at
 this point is not so easy.
 That is the short version why in some cases I did not try to load the
 initrd at the end of memory (just allocating one big block of XMS memory
 with a hole of few Mbytes in between the two - and then move the
 complete block - the complete block could not be of the size of the
 total memory). And if it is not always done, to move initrd at end,
 no need to do it only sometimes.
 Note also that in this corner case the size of the RAM may be tricky
 and could be given as a Linux command line parameter, so the position
 of the initrd is not trivial. I do not want to load the initrd after the
 RAM itself...

  Was just for info, not for flamewar,
  Etienne.

Yahoo! Mail - Votre e-mail personnel et gratuit qui vous suit partout !
 Créez votre adresse sur http://mail.yahoo.fr

Re: BOOT_CS

[H. Peter Anvin]

Followup to:  <20040225103043.44010.qmail@web11807.mail.yahoo.com>
By author:    =?iso-8859-1?q?Etienne=20Lorrain?= <etienne_lorrain@yahoo.fr>
In newsgroup: linux.dev.kernel
> 
>  What I did is simply load and uncompress Linux to a legally allocated
>  HIMEM block (so that is a 100 % compatible DOS software, you can use
>  a network disk or a disk cache for that) and check everything is right
>  before disabling interruption, switch back to real mode with
>  4 Gb segments of non paged memory, copy the block at the right place, 
>  and start Linux in protected mode.
>  You have to remember keeping the code short in between the back switch
>  to real mode and the start of Linux because the data (Linux kernel)
>  is at a clear physical address, but your code itself is in a 4 Kbyte
>  page which is available - but the page after it may not be loaded
>  so no more code...
> 

[...]

There is a hook in the kernel immediately after enteing protected mode
for *exactly* this reason -- it was added to support LOADLIN.  The
whole point is that your boot loader obtains control at that point so
you can put things back where they need to go (such as 0x100000 for
the main part of the kernel, which you will *never* get from an
HMA-enabled DOS.)  The algorithm for that is pretty straightforward;
you can even deal with the case where you have scattered pages all
over memory.

	-hpa

Re: BOOT_CS

[Etienne Lorrain]

> >  What I did is simply load and uncompress Linux to a legally allocated
> >  HIMEM block (so that is a 100 % compatible DOS software, you can use
> >  a network disk or a disk cache for that) and check everything is right
> >  before disabling interruption, switch back to real mode with
> >  4 Gb segments of non paged memory, copy the block at the right place, 
> >  and start Linux in protected mode.
> 
> [...]
> 
> There is a hook in the kernel immediately after enteing protected mode
> for *exactly* this reason -- it was added to support LOADLIN.  The
> whole point is that your boot loader obtains control at that point so
> you can put things back where they need to go (such as 0x100000 for
> the main part of the kernel, which you will *never* get from an
> HMA-enabled DOS.)  The algorithm for that is pretty straightforward;
> you can even deal with the case where you have scattered pages all
> over memory.

  This interface is nice when the VCPI is loaded and running, but if
 only EMM386 is loaded and VCPI not active you cannot use it.

 Extract of "manual.txt" of "lodlin16.tgz":
>>>>>>>>>>>>
 Therefore:
 - You must be in 86 real mode (no EMS driver, no WINDOWS, no Windows 95
Graphics mode ...). or
 - You must have the VCPI server enabled in your EMS driver (still not
running WINDOWS or Windows 95, however).
   Using EMM386 please avoid the NOVCPI-option, but NOEMS doesn't hurt.
-------------
 May be that your VCPI server does garbage collection before entering
 protected mode, so please BE PATIENT, especially on systems with many
 mega bytes ! 
<<<<<<<<<<<<<<

  If the user of this boot floppy has not allocated some VCPI memory,
 usual for a boot floppy, the VCPI server is not active - only the
 GEMMIS interface is useable.

 Note that the user does not usually know in which mode the VCPI
 server is - and on this floppy lying in this box there is still maybe
 a EMM386/compatible software who cannot do VCPI.

 Moreover starting the VCPI server for just using the interface to
 exit it to real mode is really slow, the 4K pages have to be
 copied all other the place (the logical pages address increase
 with physical two lowest bit decreasing, pages are numbered
 0x13 0x12 0x11 0x10 0x17 0x16 0x15 0x14 ...).
 At last you are not guarantied some of the kernel pages have not been
 pushed to the swapping device (VCPI can do that if low memory).

 The GEMMIS interface is always available when VCPI is not present
 because that is the one the operating system dated 95/98 uses.

 With the latter interface you just do the XMS allocation and relocate
 like my bootloader. Once this is done, you do not need more work to
 support the case when the VCPI is running: the XMS allocation is
 still there and contigous memory.

 begining to be seriously off topic,
 Etienne.





	

	
		
Yahoo! Mail : votre e-mail personnel et gratuit qui vous suit partout ! 
Créez votre Yahoo! Mail sur http://fr.benefits.yahoo.com/

Dialoguez en direct avec vos amis grâce à Yahoo! Messenger !Téléchargez Yahoo! Messenger sur http://fr.messenger.yahoo.com

Re: BOOT_CS

[Denis Vlasenko]

On Thursday 26 February 2004 14:17, Etienne Lorrain wrote:
>   If the user of this boot floppy has not allocated some VCPI memory,
>  usual for a boot floppy, the VCPI server is not active - only the
>  GEMMIS interface is useable.

What's GEMMIS?
--
vda

Re: BOOT_CS

[Etienne Lorrain]

 --- Denis Vlasenko wrote:
> On Thursday 26 February 2004 14:17, Etienne Lorrain wrote:
> >   If the user of this boot floppy has not allocated some VCPI memory,
> >  usual for a boot floppy, the VCPI server is not active - only the
> >  GEMMIS interface is useable.
> 
> What's GEMMIS?

  Checked if it was still search-able and available while writing
 the message - seems that it not enough - provide clickable area:
http://search.yahoo.com/search?p=GEMMIS
  gives as hit nb four:
http://dgi_il.tripod.com/gemmis.txt

 Google needs "GEMMIS PC" to give:
http://www.nondot.org/sabre/os/files/ProtectedMode/GEMMIS.txt
 Altavista gives:
http://www.nondot.org/sabre/os/articles
 Even MSN search:
http://search.msn.com/pass/results.aspx?q=GEMMIS+PC&FORM=SMCRT
 Gives you a pointer to a pointer to it:
http://www.cyberscriptorium.com/bookmarks.html

  Enough?


	

	
		
Yahoo! Mail : votre e-mail personnel et gratuit qui vous suit partout ! 
Créez votre Yahoo! Mail sur http://fr.benefits.yahoo.com/

Dialoguez en direct avec vos amis grâce à Yahoo! Messenger !Téléchargez Yahoo! Messenger sur http://fr.messenger.yahoo.com

Re: BOOT_CS

[H. Peter Anvin]

Followup to:  <20040226121713.21924.qmail@web11804.mail.yahoo.com>
By author:    =?iso-8859-1?q?Etienne=20Lorrain?= <etienne_lorrain@yahoo.fr>
In newsgroup: linux.dev.kernel
> 
>   This interface is nice when the VCPI is loaded and running, but if
>  only EMM386 is loaded and VCPI not active you cannot use it.
> 

This has nothing to do with anything.  You're totally confused.

I'm referring to a hook in the kernel; it has nothing to do with VCPI,
EMM386 or LOADLIN, except that the latter uses the kernel interface
when using VCPI.

	-hpa

Re: BOOT_CS

[Etienne Lorrain]

> H. Peter Anvin writes:
> > Eric W. Biederman wrote:
> > for those folks who want to place their ramdisk as low
> > in memory as possible?
> The problem is that you don't know until it's too late, since it can 
> depend on dynamic factors.  This is part of why your insistence of 
> putting the ramdisk in the "most incorrect" position is simply wrong.

  The other problem is for the people who want to check the validity
 of the RAM disk before starting Linux - for instance by checking
 the CRC32 of the decompressed RAM disk - and stop the boot process
 before it is too late - i.e. in the bootloader when you can select
 another kernel version / initrd to load.
  You cannot place the decompressed initrd at a maximum address before
 knowing its decompressed size - the address to place it is the max
 address (or the end of free RAM) minus ramdisk size if I remember
 correctly. That is working for so long loading the decompressed
 initrd after few Mb after the last kernel byte (so that the kernel
 will move it where it wants - no need to move it twice) that I do
 not remember the details. Did you changed this part?

  Etienne.

Yahoo! Mail - Votre e-mail personnel et gratuit qui vous suit partout !
 Créez votre adresse sur http://mail.yahoo.fr

Re: BOOT_CS

[H. Peter Anvin]

Followup to:  <20040224100530.68794.qmail@web11805.mail.yahoo.com>
By author:    =?iso-8859-1?q?Etienne=20Lorrain?= <etienne_lorrain@yahoo.fr>
In newsgroup: linux.dev.kernel
> 
>   The other problem is for the people who want to check the validity
>  of the RAM disk before starting Linux - for instance by checking
>  the CRC32 of the decompressed RAM disk - and stop the boot process
>  before it is too late - i.e. in the bootloader when you can select
>  another kernel version / initrd to load.
>   You cannot place the decompressed initrd at a maximum address before
>  knowing its decompressed size - the address to place it is the max
>  address (or the end of free RAM) minus ramdisk size if I remember
>  correctly. That is working for so long loading the decompressed
>  initrd after few Mb after the last kernel byte (so that the kernel
>  will move it where it wants - no need to move it twice) that I do
>  not remember the details. Did you changed this part?
> 

If you absolutely want to do this -- for pretty much no reason -- you
can either decompress it twice, decompress it to nowhere (after all,
the kernel will decompress it when it starts) or move it into place
before starting the kernel.

	-hpa

BOOT_CS

[H. Peter Anvin]

Anyone happen to know of any legitimate reason not to reload %cs in
head.S?  I think the following would be a lot cleaner, as well as a
lot safer (the jump and indirect branch aren't guaranteed to have the
proper effects, although technically neither should be required due to
the %cr0 write):

@@ -117,10 +147,7 @@
        movl %cr0,%eax
        orl $0x80000000,%eax
        movl %eax,%cr0          /* ..and set paging (PG) bit */
-       jmp 1f                  /* flush the prefetch-queue */
-1:
-       movl $1f,%eax
-       jmp *%eax               /* make sure eip is relocated */
+       ljmp $__BOOT_CS,$1f     /* Clear prefetch and normalize %eip
*/
 1:
        /* Set up the stack pointer */
        lss stack_start,%esp


I've been doing some cleanups in head.S after making the early page
tables dynamic.

	-hpa

Re: BOOT_CS

[Coywolf Qi Hunt]

H. Peter Anvin wrote:
> Anyone happen to know of any legitimate reason not to reload %cs in
> head.S?  I think the following would be a lot cleaner, as well as a
> lot safer (the jump and indirect branch aren't guaranteed to have the
> proper effects, although technically neither should be required due to
> the %cr0 write):
> 
> @@ -117,10 +147,7 @@
>         movl %cr0,%eax
>         orl $0x80000000,%eax
>         movl %eax,%cr0          /* ..and set paging (PG) bit */
> -       jmp 1f                  /* flush the prefetch-queue */
> -1:
> -       movl $1f,%eax
> -       jmp *%eax               /* make sure eip is relocated */
> +       ljmp $__BOOT_CS,$1f     /* Clear prefetch and normalize %eip
> */
>  1:
>         /* Set up the stack pointer */
>         lss stack_start,%esp
> 
> 
> I've been doing some cleanups in head.S after making the early page
> tables dynamic.
> 
> 	-hpa
> -

IMHO, why bother to re-reload %cs again?

In setup.S, %cs is reloaded already. The enable paging code maps the
address identically, so %cs already contains the proper selector.


Coywolf


-- 
Coywolf Qi Hunt
Admin of http://GreatCN.org and http://LoveCN.org

Re: BOOT_CS

[Jamie Lokier]

Coywolf Qi Hunt wrote:
> >(the jump and indirect branch aren't guaranteed to have the
> >proper effects, although technically neither should be required due to
> >the %cr0 write):
>
> 
> IMHO, why bother to re-reload %cs again?
> 
> In setup.S, %cs is reloaded already. The enable paging code maps the
> address identically, so %cs already contains the proper selector.

It's to flush the instruction prefetch queue: that's one of the side
effects of ljmp.

I recall an Intel manual that said ljmp is required when switching
between real and protected modes, to flush the prefetch queue.

Unfortunately I don't remember what that manual said about just setting PG.

I'd guess that current generation CPUs don't care about ljmp when
changing modes, but older ones certainly do.

-- Jamie

Re: BOOT_CS

[Eric W. Biederman]

hpa@zytor.com (H. Peter Anvin) writes:

> Anyone happen to know of any legitimate reason not to reload %cs in
> head.S?  

Other than the fact it is strongly rude and error prone to depend on
the contents of a global descriptor table you did not setup?

If we did the lgdt boot_gdt before hand I don't see any problems
though.  

But at the point we could as easily do lgdt cpu_gdt_descr, and use
__KERNEL_CS which is better anyway.

> I think the following would be a lot cleaner, as well as a
> lot safer (the jump and indirect branch aren't guaranteed to have the
> proper effects, although technically neither should be required due to
> the %cr0 write):
> 
> @@ -117,10 +147,7 @@
>         movl %cr0,%eax
>         orl $0x80000000,%eax
>         movl %eax,%cr0          /* ..and set paging (PG) bit */
> -       jmp 1f                  /* flush the prefetch-queue */
> -1:
> -       movl $1f,%eax
> -       jmp *%eax               /* make sure eip is relocated */
> +       ljmp $__BOOT_CS,$1f     /* Clear prefetch and normalize %eip
> */
>  1:
>         /* Set up the stack pointer */
>         lss stack_start,%esp
> 
> 
> I've been doing some cleanups in head.S after making the early page
> tables dynamic.

That is almost nice.  Care to export where the bottom of the page
tables or even better where the bottom of the kernel is for those
folks who want to place their ramdisk as low in memory as possible?

Eric

Re: BOOT_CS

[H. Peter Anvin]

Eric W. Biederman wrote:
> hpa@zytor.com (H. Peter Anvin) writes:
> 
>>Anyone happen to know of any legitimate reason not to reload %cs in
>>head.S?  
> 
> Other than the fact it is strongly rude and error prone to depend on
> the contents of a global descriptor table you did not setup?
> 

We already do that, as you might have noticed (we set all the data 
registers to __BOOT_DS; CS is the only that is changed.)

> 
> That is almost nice.  Care to export where the bottom of the page
> tables or even better where the bottom of the kernel is for those
> folks who want to place their ramdisk as low in memory as possible?
> 

The problem is that you don't know until it's too late, since it can 
depend on dynamic factors.  This is part of why your insistence of 
putting the ramdisk in the "most incorrect" position is simply wrong.

	-hpa

Re: BOOT_CS

[Eric W. Biederman]

"H. Peter Anvin" <hpa@zytor.com> writes:

> Eric W. Biederman wrote:
> > hpa@zytor.com (H. Peter Anvin) writes:
> >
> >>Anyone happen to know of any legitimate reason not to reload %cs in
> >> head.S?
> > Other than the fact it is strongly rude and error prone to depend on
> > the contents of a global descriptor table you did not setup?
> >
> 
> We already do that, as you might have noticed (we set all the data registers to
> __BOOT_DS; CS is the only that is changed.)

Right and it would be a cleanup not to touch __BOOT_DS.  We have
already reloaded it in  arch/i386/boot/compressed/head.S anyway.

> > That is almost nice.  Care to export where the bottom of the page
> > tables or even better where the bottom of the kernel is for those
> > folks who want to place their ramdisk as low in memory as possible?
> >
> 
> The problem is that you don't know until it's too late, since it can depend on
> dynamic factors.  This is part of why your insistence of putting the ramdisk in
> the "most incorrect" position is simply wrong.

Nope.  On other architectures where the bootloader has access to
vmlinux this works just fine, all dynamic factors can be contained in
the bss.  We don't go very long before we reserve memory after all.
It is only on x86 where there is not enough information that it is
problematic. 

Putting the ramdisk right after the kernel (text + data + bss) is the
"most correct" position.  Anything else is likely to break when the
firmware changes.  This has already happened 2 or 3 times on x86.

When putting the ramdisk right after the kernel if anything breaks you
will notice it immediately, and the kernel will be fixed.

If I truly put an initrd at the top of memory the kernel would not
even be able to read the ramdisk.

Eric