PCI coprocessors

PCI coprocessors

[Andre Bonin]

Hey all,
I'me building an FPGA based pci board for a degree project.  In theory 
this board could be used as a custom, field programmable coprocessor (to 
accelerate processes).  At which point, it might be nice to be able to 
support it as a processor under the kernel.

Yes bandwidth, yes it should be PCI-Express but it is still just a 
degree project, 33mhz is fast enough for the proof of concept.

Which leads me to my questions:

1) Is their support for having two different 'machine types' within one 
kernel? that is for example, certain executables for intel would get run 
on an intel processor, and others would get run on processor with type XXXX.

I heard once someone put native "java" .class support within the kernel 
(it would call the jvm run time if i remember).  I could maby do this 
with my own set of libraries and driver.  But differentiating between 
the types of executable might be hard.

2) Is their kernel support for PCI coprocessors for thread allocation 
etc.  I couldn't find any but i can try looking through the code again.

Re: PCI coprocessors

[Alan Cox]

On Mer, 2004-09-15 at 13:55, Andre Bonin wrote:
> 1) Is their support for having two different 'machine types' within one 
> kernel? that is for example, certain executables for intel would get run 
> on an intel processor, and others would get run on processor with type XXXX.

The kernel provides everything you need to run userspace apps on the
co-processor - which is very little indeed. It provides binfmt_misc
which allows other binary types to be revectored to user applications.
That is how the example you remember worked.

Your application gets the program to run, you run it, you throw it at
the coprocessor and you need to take any traps back for syscalls (which
might need a little driver kernel side if it involves interrupts).

There are then the hard bits (mmap, ptrace, scheduling...) 8)

> 2) Is their kernel support for PCI coprocessors for thread allocation 
> etc.  I couldn't find any but i can try looking through the code again.

We don't deal at all with the question of scheduling stuff on different
processor types. 

Alan

Re: PCI coprocessors

[Matti Aarnio]

On Wed, Sep 15, 2004 at 08:55:47AM -0400, Andre Bonin wrote:
> Hey all,
> I'me building an FPGA based pci board for a degree project.  In theory 
> this board could be used as a custom, field programmable coprocessor
> (to accelerate processes).  At which point, it might be nice to be able
> to support it as a processor under the kernel.
> 
> Yes bandwidth, yes it should be PCI-Express but it is still just a 
> degree project, 33mhz is fast enough for the proof of concept.
> 
> Which leads me to my questions:
> 
> 1) Is their support for having two different 'machine types' within one 
> kernel? that is for example, certain executables for intel would get run 
> on an intel processor, and others would get run on processor with type XXXX.

Depending...   If you think of computation intensive things, such have
been implemented in FPGA, usually in "slave mode".  Consider algorithms
that you want to run all the time with heavy load.  (some brute-force
crypto breakings, Seti@Home, ...)  the math that the hardware does must
(by necessity) be rather simple (or broken into simple steps.)

Things that are useful and have in the past been done in co-processors
do include:

  - RSA exponentation
  - 3DES
  - FIR/IIR filters
  - FFT engines
  - Matrix math engines

Depending on used FPGA (RAM-based I presume, anti-fuse stuff is not
reprogrammable), there might exist internal multipliers, and distributed
RAM blocks for intermediate results.  Fairly complicated signal processing
could be done in such a card.

Cheaper FPGAs can be used to implement complexish logic to do strange
serial IO protocols with strict timing requirements, for example.
"Send this word to the remote device", "read back device's lattest
reply".  (e.g. implement a VGA-LCD controller.)


> I heard once someone put native "java" .class support within the kernel 
> (it would call the jvm run time if i remember).  I could maby do this 
> with my own set of libraries and driver.  But differentiating between 
> the types of executable might be hard.

Sure a "micro-java-engine" can be implemented, and then ran.
However a 50-60 M java intructions (byte-codes) per second isn't
all that fancy, especially if you need to access memory rather
frequently...  Your host will likely be able to run JVM faster
than that.  (A P-IV-XEON running at 3+ GHz..)


If you have a deeper look into how e.g. intel e100 ethernet cards
are driven, you will quickly notice that they are given rather simple
task lists -- except in case of "setup" operation, which sends big
magic blob of data to the controller microsequencer.

You want to do something similar, if you can do bus-mastering.
You might introduce "must be 32-bit aligned" requirement, or
whatever your application fancies.  (A 1000x1000 matrix inversion...)

You shall make sure, that you won't be offloading tasks to the external
engine, tasks that your internal engine could do faster (except if you
want to spend the freed time at something else.


> 2) Is their kernel support for PCI coprocessors for thread allocation 
> etc.  I couldn't find any but i can try looking through the code again.


There is nothing such for situations where processors execute different
binary language (and thus different data layouts).

There is some support to offload heavyish cryptographic tasks to
physical co-processors (PCI-cards) .. but going twice over the
32 bit * 33 MHz bus does limit the amount of data processable in
the co-processor.


/Matti Aarnio

Re: PCI coprocessors

[Richard B. Johnson]

On Wed, 15 Sep 2004, Andre Bonin wrote:

> Hey all,
> I'me building an FPGA based pci board for a degree project.  In theory
> this board could be used as a custom, field programmable coprocessor (to
> accelerate processes).  At which point, it might be nice to be able to
> support it as a processor under the kernel.
>
> Yes bandwidth, yes it should be PCI-Express but it is still just a
> degree project, 33mhz is fast enough for the proof of concept.
>

Typically, such a board would use a standard PCI interface chip
like those made by PLX. This would be connected to a FPGA plus
whatever else was needed to perform the needed functions.

If you attempt to make your own PCI interface using a FPGA,
you are going to devote a lot of time to that, alone. You
probably won't even get to your coprocessor until graduation.

> Which leads me to my questions:
>
> 1) Is their support for having two different 'machine types' within one
> kernel? that is for example, certain executables for intel would get run
> on an intel processor, and others would get run on processor with type XXXX.
>

The support is for whatever driver you provide. For instance,
the Analogic's Sky Computer Division produces array processors
with their own CPUs. The Sky-code runs in those processors.
It doesn't (can't) affect the Intel processor kernel code in
the host.

> I heard once someone put native "java" .class support within the kernel
> (it would call the jvm run time if i remember).  I could maby do this
> with my own set of libraries and driver.  But differentiating between
> the types of executable might be hard.
>

You could certainly make a Java engine run on your coprocessor
board or use Intel code, whatever is better at that instant.
This is done with a library that you provide.

> 2) Is their kernel support for PCI coprocessors for thread allocation
> etc.  I couldn't find any but i can try looking through the code again.
>

Things that go on the PCI bus use drivers (modules) for interface.
The kernel doesn't directly determine what functions are handled
by kernel code and what ones are handled by your PCI interface
coprocessor. Your (or the standard) runtime libraries do this.

Cheers,
Dick Johnson
Penguin : Linux version 2.4.26 on an i686 machine (5570.56 BogoMips).
            Note 96.31% of all statistics are fiction.

Re: PCI coprocessors

[Tomasz Rola]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 15 Sep 2004, Andre Bonin wrote:

> Hey all,
[...]
> Which leads me to my questions:
> 
> 1) Is their support for having two different 'machine types' within one 
> kernel? that is for example, certain executables for intel would get run 
> on an intel processor, and others would get run on processor with type XXXX.

There are probably no impossible things - some may be unthinkable but once
they are thought of, they can be done too. But this one thing may be
rather difficult (just my opinion).

How about porting kernel and gcc to your fp-cpu and use pci as a
kind of fast network-like interconnect? After loading a kernel into it
somehow, boot it with nfs root and run the rest from nfs server that would
be provided by a host Intel machine.

That would require less changes to a kernel, probably. A module for a
host, for example - some "pci-net". And port of a kernel to your fp-cpu
which should be easier than putting support for heterogenous
multiprocessors...

bye
T.

- --
** A C programmer asked whether computer had Buddha's nature.      **
** As the answer, master did "rm -rif" on the programmer's home    **
** directory. And then the C programmer became enlightened...      **
**                                                                 **
** Tomasz Rola          mailto:tomasz_rola@bigfoot.com             **

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 5.0i for non-commercial use
Charset: noconv

iQA/AwUBQUhX/xETUsyL9vbiEQKAlACg9Rv6rD8INCQFItk1/s5OfZbXjukAn2Mp
PGjv6ihFXwTInSn8nu3ZOKpu
=E5XU
-----END PGP SIGNATURE-----

Re: PCI coprocessors

[Andre Bonin]

Thank you for all your input on this.  Of course i could do many things 
on the board.  The idea of the library is that a programmer would create 
the fpga image file by him/herself and then put it in the library.  The 
kernel driver and the library does the decision if its worth putting on 
the chip or not (that's optional, you could force it on chip).

Tomasz Rola wrote:

>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA1
>  
>
>>Which leads me to my questions:
>>
>>1) Is their support for having two different 'machine types' within one 
>>kernel? that is for example, certain executables for intel would get run 
>>on an intel processor, and others would get run on processor with type XXXX.
>>    
>>
>
>There are probably no impossible things - some may be unthinkable but once
>they are thought of, they can be done too. But this one thing may be
>rather difficult (just my opinion).
>  
>
Yes i think so too, especially since it will be my first kernel module 
(and anxiously waiting the next release of the o'reilly book for 
refference).

>How about porting kernel and gcc to your fp-cpu and use pci as a
>kind of fast network-like interconnect? After loading a kernel into it
>somehow, boot it with nfs root and run the rest from nfs server that would
>be provided by a host Intel machine.
>  
>
System-on-chips have been done before, and could be integrated into the 
kernel like you said.  But RAM becomes a problem.  Since its only a 
student project, we have a limit on the adressing width for the ram 
(32bit addressing becomes 32 wires, that's a lot of wirewraping :)  ).

>That would require less changes to a kernel, probably. A module for a
>host, for example - some "pci-net". And port of a kernel to your fp-cpu
>which should be easier than putting support for heterogenous
>multiprocessors...
>  
>

I agree but i think that goes beyond the scope of the project.  Though i 
will consider it.  Thanks for your input!

>- --
>** A C programmer asked whether computer had Buddha's nature.      **
>** As the answer, master did "rm -rif" on the programmer's home    **
>** directory. And then the C programmer became enlightened...      **
>**                                                                 **
>** Tomasz Rola          mailto:tomasz_rola@bigfoot.com             **
>
>-----BEGIN PGP SIGNATURE-----
>Version: PGPfreeware 5.0i for non-commercial use
>Charset: noconv
>
>iQA/AwUBQUhX/xETUsyL9vbiEQKAlACg9Rv6rD8INCQFItk1/s5OfZbXjukAn2Mp
>PGjv6ihFXwTInSn8nu3ZOKpu
>=E5XU
>-----END PGP SIGNATURE-----
>
>
>
>
>  
>

Re: PCI coprocessors

[Tonnerre]

[-- Attachment #1: Type: text/plain, Size: 489 bytes --]

Salut,

On Wed, Sep 15, 2004 at 04:55:53PM +0200, Tomasz Rola wrote:
> After loading  a kernel into it  somehow, boot it with  nfs root and
> run the rest from nfs server  that would be provided by a host Intel
> machine.

I'd rather not do that via nfs. Rather some special "hostfs" port over
PCI.

But  anyway, reading  his original  post  he seems  to have  something
completely different  in mind than booting  a second PC on  his PC: to
boot a supportive processor..

			    Tonnerre

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: PCI coprocessors

[Tomasz Rola]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Wed, 15 Sep 2004, Tonnerre wrote:

> Salut,
> 
> On Wed, Sep 15, 2004 at 04:55:53PM +0200, Tomasz Rola wrote:
> > After loading  a kernel into it  somehow, boot it with  nfs root and
> > run the rest from nfs server  that would be provided by a host Intel
> > machine.
> 
> I'd rather not do that via nfs. Rather some special "hostfs" port over
> PCI.

Well, "there is more than one way of doing this", it seems.

> But  anyway, reading  his original  post  he seems  to have  something
> completely different  in mind than booting  a second PC on  his PC: to
> boot a supportive processor..

Erm, somehow I came to thinking he wanted to schedule binaries on cpus of
different types.

Anyway, Andre, you are welcome :-).

One more thing - I remember reading on one Polish newsgroup devoted to
electronics, that making PCI cards is rather difficult (actually, nobody
wanted to try it) while one can do ISA cards "at home" or something like
this. I know ISA is out of business, but for "proof of concept" it may be
a better choice. Or maybe you (Andre) can do it with USB chip - it should
be even easier to make such device with USB and connect it to your host
this way. There are quite a few controllers available. I've searched for
ATMEL's and found this:

http://www.atmel.com/dyn/products/devices.asp?family_id=655

Of course, there must be others, too.

bye
T.

- --
** A C programmer asked whether computer had Buddha's nature.      **
** As the answer, master did "rm -rif" on the programmer's home    **
** directory. And then the C programmer became enlightened...      **
**                                                                 **
** Tomasz Rola          mailto:tomasz_rola@bigfoot.com             **


-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 5.0i for non-commercial use
Charset: noconv

iQA/AwUBQUjaaBETUsyL9vbiEQLi1wCg8stbROOHy36NUN15uZ2XW1/LGp0An0fy
m0ynDnyoOkf+aRUxt0v502vk
=qHKK
-----END PGP SIGNATURE-----

Re: PCI coprocessors

[Jeff Garzik]

Andre Bonin wrote:
> 1) Is their support for having two different 'machine types' within one 
> kernel? that is for example, certain executables for intel would get run 
> on an intel processor, and others would get run on processor with type 
> XXXX.
> 
> I heard once someone put native "java" .class support within the kernel 
> (it would call the jvm run time if i remember).  I could maby do this 
> with my own set of libraries and driver.  But differentiating between 
> the types of executable might be hard.
> 
> 2) Is their kernel support for PCI coprocessors for thread allocation 
> etc.  I couldn't find any but i can try looking through the code again.


You can mix and match processor types all you want...  just treat them 
as completely asynchronous and independent entities.

I have long dreamed of being able to add a PCI card to my x86 system, a 
PCI card containing a processor (of any type), RAM, and an ethernet 
interface.  I would use this for routing, or iSCSI, or network offload...

	Jeff

Re: PCI coprocessors

[Tonnerre]

[-- Attachment #1: Type: text/plain, Size: 427 bytes --]

Salut,

On Wed, Sep 15, 2004 at 12:01:25PM -0400, Jeff Garzik wrote:
> I have long dreamed of being able to add a PCI card to my x86 system, a 
> PCI card containing a processor (of any type), RAM, and an ethernet 
> interface.  I would use this for routing, or iSCSI, or network offload...

Such as  the i386 co-computer  card for older Macintosh  computers? (I
can't remember what it was called.)

			    Tonnerre

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: PCI coprocessors

[Andre Tomt]

Tonnerre wrote:
> Salut,
> 
> On Wed, Sep 15, 2004 at 12:01:25PM -0400, Jeff Garzik wrote:
> 
>>I have long dreamed of being able to add a PCI card to my x86 system, a 
>>PCI card containing a processor (of any type), RAM, and an ethernet 
>>interface.  I would use this for routing, or iSCSI, or network offload...
> 
> 
> Such as  the i386 co-computer  card for older Macintosh  computers? (I
> can't remember what it was called.)

I've recently seen several ia32 PCI boards, with network, cpu, ram, etc. 
that works in modern PC's. Can't recall any names just now though. Not 
sure if they had any communication with the host over PCI either.

Re: PCI coprocessors

[Tony Lee]

On Fri, 17 Sep 2004 12:16:19 +0200, Andre Tomt <andre@tomt.net> wrote:
> Tonnerre wrote:
> > Salut,
> >
> > On Wed, Sep 15, 2004 at 12:01:25PM -0400, Jeff Garzik wrote:
> >
> >>I have long dreamed of being able to add a PCI card to my x86 system, a
> >>PCI card containing a processor (of any type), RAM, and an ethernet
> >>interface.  I would use this for routing, or iSCSI, or network offload...
> >
> >
> > Such as  the i386 co-computer  card for older Macintosh  computers? (I
> > can't remember what it was called.)
> 
> I've recently seen several ia32 PCI boards, with network, cpu, ram, etc.
> that works in modern PC's. Can't recall any names just now though. Not
> sure if they had any communication with the host over PCI either.


If you can get access to SDK for Broadcom 5704 (570x?) , you have GIGE nic
card with 2 133MHz MIPS CPU, one for TX, onr for RX.

Most of the NIC cards with BCM chip should be < $50.

Maybe enough of us bugging Broadcom, they will open up the SDK.

-- 
-Tony
Having a lot of fun with Xilinx Virtex Pro II reconfigurable HW + ppc + Linux