Linux OS boilerplate

Linux OS boilerplate

[Scott Long]

I've been poring over the x86 boot code for a while now and I've been
considering writing a FAQ on the boot process (mostly for my own use,
but maybe others will be interested). This would include all relevant
information on setting up the x86 hardware for a boot (timers, PIC, A20,
protected mode, GDT, initial page tables, initial TSS, etc).

The motivation behind this is that I would like to use the Linux boot
system as a boilerplate booter for some experimental code. It's probably
much cleaner and more robust than any boot loader I might come up with.

Does there exist an outline (detailed or not) of the boot process from
the point of BIOS bootsector load to when the kernel proper begins
execution? If not would anyone be willing to help me understand
bootsect.S and setup.S enough so that I might write such an outline?

If no one can help me, would you consider it appropriate for me to send
email to the people listed in bootsect.S and setup.S asking for
assistance?

Thanks,
Scott

Re: Linux OS boilerplate

[Jeremy Jackson]

Scott Long wrote:

> Does there exist an outline (detailed or not) of the boot process from
> the point of BIOS bootsector load to when the kernel proper begins
> execution? If not would anyone be willing to help me understand
> bootsect.S and setup.S enough so that I might write such an outline?

I have been over it, and would be willing to help.  You should first read all

of the LILO documentation, and check out some of the LinuxBIOS
project at www.linuxbios.org.

Re: Linux OS boilerplate

[Rick Hohensee]

Have you seen Janet_Reno? 

ftp://linux01.gwdg.de/pub/cLIeNUX/interim/Janet_Reno.tgz IIRC.
Janet is an x86 bootsector that gets into protected mode and can
use the AT BIOS in pmode interrupts. It's written with a bunch of
m4 macros I call asmacs that I'm currently basing an assembler in 
Bash on. That's shasm in the same directory as Janet.

Rick Hohensee
www.cLIeNUX.com

Re: Linux OS boilerplate

[TeknoDragon]

On Sun, 18 Feb 2001, Scott Long wrote:

> Does there exist an outline (detailed or not) of the boot process from
> the point of BIOS bootsector load to when the kernel proper begins
> execution? If not would anyone be willing to help me understand
> bootsect.S and setup.S enough so that I might write such an outline?

It might be a little fundamental but there is *a* boot loading process
documented here: http://www.eecs.wsu.edu/~cs460/

Look over all the lecture notes and lab assignments up to the booter lab.

I'd offer up my own explanation, but I'm just starting to learn this
stuff.


-karl

Re: Linux OS boilerplate

[Alan Cox]

> I've been poring over the x86 boot code for a while now and I've been
> considering writing a FAQ on the boot process (mostly for my own use,
> but maybe others will be interested). This would include all relevant
> information on setting up the x86 hardware for a boot (timers, PIC, A20,
> protected mode, GDT, initial page tables, initial TSS, etc).

It would certainly be a valuable piece for the kernel Documentation dir.
Paticularly as people with embedded x86 grow keener and keener to boot 
biosless to save money and flash.

Re: Linux OS boilerplate

[Paul Gortmaker]

Scott Long wrote:
> 
> I've been poring over the x86 boot code for a while now and I've been
> considering writing a FAQ on the boot process (mostly for my own use,

[...]

> Does there exist an outline (detailed or not) of the boot process from
> the point of BIOS bootsector load to when the kernel proper begins

IIRC, there is some useful info contained within loadlin.  Also, I
found a doc by hpa called "THE LINUX/I386 BOOT PROTOCOL" in my local
archive of cruft -  I just assumed it was in Documentation/ but
apparently it never made it there (yet).

Paul.


_________________________________________________________
Do You Yahoo!?
Get your free @yahoo.com address at http://mail.yahoo.com

Re: Linux OS boilerplate

[Eric W. Biederman]

Alan Cox <alan@lxorguk.ukuu.org.uk> writes:

> > I've been poring over the x86 boot code for a while now and I've been
> > considering writing a FAQ on the boot process (mostly for my own use,
> > but maybe others will be interested). This would include all relevant
> > information on setting up the x86 hardware for a boot (timers, PIC, A20,
> > protected mode, GDT, initial page tables, initial TSS, etc).
> 
> It would certainly be a valuable piece for the kernel Documentation dir.
> Paticularly as people with embedded x86 grow keener and keener to boot 
> biosless to save money and flash.

Actually biosless is nice.  You don't actually need to do things like enable
a20 as there isn't a BIOS to disable it...

As for working code see the linuxBIOS project.  http://www.linuxbios.org
There aren't a lot of chipsets supported yet but progress is being made.
And you do need something like a BIOS to enable memory and cache
before you jump to the linux kernel.

Alan working on all booting x86 biosless, keeps making me chuckle over
your hatred towards BIOSes, it is absolutely amazing what linux
assumes the BIOS will setup correctly.  With linux-2.4 able to do a
complete PCI bus setup it isn't as bad it used to be, but it's still
pretty significant.

You wouldn't happen to know a good place to put drivers for superio
chips, and pci-isa bridges with the superio functions integrated?

I should have a proposal written up (and the code to implement it) in
the next month, on how to change the boot process to be more friendly
to BIOS less situations.   What we do with the ``empty_zero_page'' is
nasty, and it isn't even the empty_zero_page.

And if anyone has any questions on how the boot process works,
I'd be glad to answer specific questions.  As I've traced it fairly
thoroughly. 

Eric

Re: Linux OS boilerplate

[H. Peter Anvin]

Paul Gortmaker wrote:
> 
> Scott Long wrote:
> >
> > I've been poring over the x86 boot code for a while now and I've been
> > considering writing a FAQ on the boot process (mostly for my own use,
> 
> [...]
> 
> > Does there exist an outline (detailed or not) of the boot process from
> > the point of BIOS bootsector load to when the kernel proper begins
> 
> IIRC, there is some useful info contained within loadlin.  Also, I
> found a doc by hpa called "THE LINUX/I386 BOOT PROTOCOL" in my local
> archive of cruft -  I just assumed it was in Documentation/ but
> apparently it never made it there (yet).
> 

It's in there (Documentation/i386/boot.txt).

	-hpa

-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt

Re: Linux OS boilerplate

[Tim Wright]

On Mon, Feb 19, 2001 at 03:07:33AM -0700, Eric W. Biederman wrote:
> With linux-2.4 able to do a complete PCI bus setup it isn't as bad it used
> to be, but it's still pretty significant.
 
For an incomplete subset of chipsets. Serverworks doesn't work correctly for
a start (see the threads relating to having to kill the Serverworks fixup code
and rely on the BIOS to see all the PCI busses on larger systems). Of course,
this is due to Serverworks refusal to release documentation (yes, I've heard
the excuses regarding protection of IP), and it's a worrisome. What else are
we potentially failing to setup on this chipset ?

Tim

-- 
Tim Wright - timw@splhi.com or timw@aracnet.com or twright@us.ibm.com
IBM Linux Technology Center, Beaverton, Oregon
Interested in Linux scalability ? Look at http://lse.sourceforge.net/
"Nobody ever said I was charming, they said "Rimmer, you're a git!"" RD VI

Re: Linux OS boilerplate

[H. Peter Anvin]

Followup to:  <3A90E16D.DB868F2@yahoo.com>
By author:    Paul Gortmaker <p_gortmaker@yahoo.com>
In newsgroup: linux.dev.kernel
> 
> > Does there exist an outline (detailed or not) of the boot process from
> > the point of BIOS bootsector load to when the kernel proper begins
> 
> IIRC, there is some useful info contained within loadlin.  Also, I
> found a doc by hpa called "THE LINUX/I386 BOOT PROTOCOL" in my local
> archive of cruft -  I just assumed it was in Documentation/ but
> apparently it never made it there (yet).
> 

Documentation/i386/boot.txt

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt

Linux scalability?

[Sasi Peter]

Hi!

I am just writing an essay, an have mentioned TUX as a performance and
scalability linearity recort holder with TUX, referencing the specweb99
website summary page:

http://www.spec.org/osg/web99/results/web99.html

However, taking a closer look, it turns out, that the above statement
holds true only for 1 and 2 processor machines. Scalability already
suffers at 4 processors, and at 8 processors, TUX 2.0 (7500) gets beaten
by IIS 5.0 (8001), and these were measured on the same kind of box!

How come, TUX is soooo good at the lowend (1 and 2 CPUs), and scales this
bad?

-- 
SaPE - Peter, Sasi - mailto:sape@sch.hu - http://sape.iq.rulez.org/

Re: Linux scalability?

[reiser.angus]

> However, taking a closer look, it turns out, that the above statement
> holds true only for 1 and 2 processor machines. Scalability already
> suffers at 4 processors, and at 8 processors, TUX 2.0 (7500) gets beaten
> by IIS 5.0 (8001), and these were measured on the same kind of box!
not really the same box
look at the disk subsystem
7 x 9GB 10KRPM Drives and 1 x 18GB 15KRPM (html+log & os) for Win2000
5 x 9GB 10KRPM Drives (html+log+os) for TUX 2.0

this is sufficient for a such difference

-David

Re: Linux scalability?

[Sean Hunter]

Why would you want to run a web server with 8 processors rather than four
webservers with 2 each?

Sean

On Fri, May 18, 2001 at 09:24:48AM +0200, Sasi Peter wrote:
> Hi!
> 
> I am just writing an essay, an have mentioned TUX as a performance and
> scalability linearity recort holder with TUX, referencing the specweb99
> website summary page:
> 
> http://www.spec.org/osg/web99/results/web99.html
> 
> However, taking a closer look, it turns out, that the above statement
> holds true only for 1 and 2 processor machines. Scalability already
> suffers at 4 processors, and at 8 processors, TUX 2.0 (7500) gets beaten
> by IIS 5.0 (8001), and these were measured on the same kind of box!
> 
> How come, TUX is soooo good at the lowend (1 and 2 CPUs), and scales this
> bad?
> 
> -- 
> SaPE - Peter, Sasi - mailto:sape@sch.hu - http://sape.iq.rulez.org/
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: Linux scalability?

[reiser.angus]

> I read an article about TUX in the dutch C'T a few months ago (nov/dec
> 2000, I think) - the real difference (according to the article) was the
> 2.2.x kernel used in TUX. Look at the stats of the website, they used
> Redhat 7.0 as base, with kernel 2.2.16.

TUX does not exist on 2.2 kernel
They use a RedHat 7.0 with a 2.4 kernel patched by RedHat (with TUX,
zerocopy, etc..)

-David

Re: Linux scalability?

[Ronald Bultje]

On 18 May 2001 10:12:34 +0200, reiser.angus wrote:
> > However, taking a closer look, it turns out, that the above statement
> > holds true only for 1 and 2 processor machines. Scalability already
> > suffers at 4 processors, and at 8 processors, TUX 2.0 (7500) gets beaten
> > by IIS 5.0 (8001), and these were measured on the same kind of box!
> not really the same box
> look at the disk subsystem
> 7 x 9GB 10KRPM Drives and 1 x 18GB 15KRPM (html+log & os) for Win2000
> 5 x 9GB 10KRPM Drives (html+log+os) for TUX 2.0
> 
> this is sufficient for a such difference

I read an article about TUX in the dutch C'T a few months ago (nov/dec
2000, I think) - the real difference (according to the article) was the
2.2.x kernel used in TUX. Look at the stats of the website, they used
Redhat 7.0 as base, with kernel 2.2.16. In the C'T, they also used a
2.4-test kernel for TUX, and this one didn't have these "scalibility
problems". The problem seemed to be SMP problems with systems with more
than two cpus in the 2.2.x-based kernel series. 2.4.x kernels didn't
seem to have this problem.

And as far as I know, TUX with 2.4.x kernel was faster than win2k on all
SMP-combinations.

--
Ronald

Re: Linux scalability?

[Ronald Bultje]

On 18 May 2001 10:30:40 +0200, reiser.angus wrote:
> TUX does not exist on 2.2 kernel
> They use a RedHat 7.0 with a 2.4 kernel patched by RedHat (with TUX,
> zerocopy, etc..)

I am pretty sure the C'T article mentioned that TUX did use a 2.2.x
kernel - so it does exist. How else could they make a 2.2.x-kernel based
TUX vs. 2.4-test-kernel based TUX comparison?

Too bad the website doesn't mention the kernel number.

--
Ronald

[OT] Re: Linux scalability?

[J Sloan]

Ronald Bultje wrote:

> On 18 May 2001 10:12:34 +0200, reiser.angus wrote:
> > > However, taking a closer look, it turns out, that the above statement
> > > holds true only for 1 and 2 processor machines. Scalability already
> > > suffers at 4 processors, and at 8 processors, TUX 2.0 (7500) gets beaten
> > > by IIS 5.0 (8001), and these were measured on the same kind of box!
> > not really the same box
> > look at the disk subsystem
> > 7 x 9GB 10KRPM Drives and 1 x 18GB 15KRPM (html+log & os) for Win2000
> > 5 x 9GB 10KRPM Drives (html+log+os) for TUX 2.0
> >
> > this is sufficient for a such difference
>
> I read an article about TUX in the dutch C'T a few months ago (nov/dec
> 2000, I think) - the real difference (according to the article) was the
> 2.2.x kernel used in TUX. Look at the stats of the website, they used
> Redhat 7.0 as base, with kernel 2.2.16. In the C'T, they also used a
> 2.4-test kernel for TUX, and this one didn't have these "scalibility
> problems". The problem seemed to be SMP problems with systems with more
> than two cpus in the 2.2.x-based kernel series. 2.4.x kernels didn't
> seem to have this problem.

All Tux webservers have run on a 2.4 or 2.4-pre kernel.

> And as far as I know, TUX with 2.4.x kernel was faster than win2k on all
> SMP-combinations.

Tux held the record for most of the time since last
summer, when Linux vaulted into 1st place

Microsoft finally managed to get a better result using
an all-out, "bet the farm", "benchmark buster" setup
with a special web cache in front of iis.

However, they haven't heard the last of Linux either.

cu

jjs

Re: [OT] Re: Linux scalability?

[David S. Miller]

J Sloan writes:
 > Microsoft finally managed to get a better result using
 > an all-out, "bet the farm", "benchmark buster" setup
 > with a special web cache in front of iis.

I haven't heard anyone talk about the fact that their 8-cpu numbers
got disqualified and aren't even mentioned on the SPEC site on the
main tables anymore.

Later,
David S. Miller
davem@redhat.com

Re: [OT] Re: Linux scalability?

[Peter Rival]

"David S. Miller" wrote:

> J Sloan writes:
>  > Microsoft finally managed to get a better result using
>  > an all-out, "bet the farm", "benchmark buster" setup
>  > with a special web cache in front of iis.
>
> I haven't heard anyone talk about the fact that their 8-cpu numbers
> got disqualified and aren't even mentioned on the SPEC site on the
> main tables anymore.
>

Really?  I just checked and it's still there from what I see.  We're talking
about the Dell 8450/700 w/ IIS & SWC 3.0 result, right?  I'm hoping that
they're deemed NC, but I don't see it yet...

 - Pete

Re: [OT] Re: Linux scalability?

[David S. Miller]

Peter Rival writes:
 > Really?  I just checked and it's still there from what I see.  We're talking
 > about the Dell 8450/700 w/ IIS & SWC 3.0 result, right?  I'm hoping that
 > they're deemed NC, but I don't see it yet...

Sorry, they are there in the table, but marked as NC.

Maybe you need to hit reload in your browser :-)

Later,
David S. Miller
davem@redhat.com

Re: [OT] Re: Linux scalability?

[Peter Rival]

"David S. Miller" wrote:

> Peter Rival writes:
>  > Really?  I just checked and it's still there from what I see.  We're talking
>  > about the Dell 8450/700 w/ IIS & SWC 3.0 result, right?  I'm hoping that
>  > they're deemed NC, but I don't see it yet...
>
> Sorry, they are there in the table, but marked as NC.
>
> Maybe you need to hit reload in your browser :-)
>

Yup, one of them is marked as NC.  But the other one is still there (and I hit
reload and even shift-reload).  So either you're missing the second one or
something is not behaving nicely with our web proxies here.  While I'd probably be
more inclined to believe the latter... ;)

 - Pete

Re: [OT] Re: Linux scalability?

[David S. Miller]

Peter Rival writes:
 > Yup, one of them is marked as NC.  But the other one is still there

Right, I missed the second one.

That's an interesting trick, the old one gets disqualified because of
the 3-month rule, so just submit new results :-)

Later,
David S. Miller
davem@redhat.com

Re: Linux scalability?

[Rodger Donaldson]

On Fri, May 18, 2001 at 09:17:11AM +0100, Sean Hunter wrote:

[Discussion of SPECWeb results]

> Why would you want to run a web server with 8 processors rather than four
> webservers with 2 each?

Because you want to win benchmarketing exercises, not demonstrate that your
architecture has any value in the real world whatsoever.  Because you know
that you can induce people with financial approval to make stupid and
irrational decisions based on irrelevant data.

-- 
Rodger Donaldson		rodgerd@diaspora.gen.nz
Klingons do *not* make good programmers.  They make good PFWs and abuse
staff, though.  "You have dishonoured our Ascend!  I should kill you
where you stand!"	-- Malcolm Ray

Re: [OT] Re: Linux scalability?

[J Sloan]

Peter Rival wrote:

> "David S. Miller" wrote:
>
> > J Sloan writes:
> >  > Microsoft finally managed to get a better result using
> >  > an all-out, "bet the farm", "benchmark buster" setup
> >  > with a special web cache in front of iis.
> >
> > I haven't heard anyone talk about the fact that their 8-cpu numbers
> > got disqualified and aren't even mentioned on the SPEC site on the
> > main tables anymore.
> >
>
> Really?  I just checked and it's still there from what I see.  We're talking
> about the Dell 8450/700 w/ IIS & SWC 3.0 result, right?  I'm hoping that
> they're deemed NC, but I don't see it yet...
>

IIRC they did have some results disqualified, but
them these latest results have been submitted
since then - perhaps they will be disqualified as
well, once the facts come to light...

cu

jjs

Re: Linux scalability?

[Sasi Peter]

On 18 May 2001, reiser.angus wrote:

> not really the same box
> look at the disk subsystem
> 7 x 9GB 10KRPM Drives and 1 x 18GB 15KRPM (html+log & os) for Win2000
> 5 x 9GB 10KRPM Drives (html+log+os) for TUX 2.0
> this is sufficient for a such difference

Don't you think that all the really needed stuff could just fit in the
enormous ram of the boxes?

-- 
SaPE - Peter, Sasi - mailto:sape@sch.hu - http://sape.iq.rulez.org/

Re: Linux scalability?

[Sasi Peter]

On Fri, 18 May 2001, Sean Hunter wrote:

> Why would you want to run a web server with 8 processors rather than four
> webservers with 2 each?

As you might already know, after the interviews to Mingo I assumed, that a
major portion of the achievements was enabled by the 2.4 scalability
enhacements. That is why I wrote to LKML, to ask about the 2.4
scalability, if anybody out there could tell us about the linux kernel's
scalability possibily compared to W2k scalability...

-- 
SaPE - Peter, Sasi - mailto:sape@sch.hu - http://sape.iq.rulez.org/

Re: Linux scalability?

[Sean Hunter]

Yup.  The problem is that you're trying to measure scalability in performance
of an i/o-bound task by comparing a machine with greater i/o resource but less
processing power with one with greater processing but poorer i/o.  Surprisingly
enough, the one with the best i/o wins.  This isn't really a fair comparison
between the two platforms.

If you put the same disk array on both machines and got the same results, then
you'd have a point.

My point was that in the real world having this configuration for a webserver
is unlikely to be sensible at all.

Sean

On Sat, May 19, 2001 at 10:31:01AM +0200, Sasi Peter wrote:
> On Fri, 18 May 2001, Sean Hunter wrote:
> 
> > Why would you want to run a web server with 8 processors rather than four
> > webservers with 2 each?
> 
> As you might already know, after the interviews to Mingo I assumed, that a
> major portion of the achievements was enabled by the 2.4 scalability
> enhacements. That is why I wrote to LKML, to ask about the 2.4
> scalability, if anybody out there could tell us about the linux kernel's
> scalability possibily compared to W2k scalability...
> 
> -- 
> SaPE - Peter, Sasi - mailto:sape@sch.hu - http://sape.iq.rulez.org/
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>