Sleep and Wake up

Sleep and Wake up

[Abu Rasheda]

I am testing my driver on much faster host processor and facing
following issues:

My host is too powerful and it can fill up device buffer queue very fast.

I get best performance when I do busy wait, but this is not desirable
and is bad design.

I need to sleep and wake up quickly and predictability. Indication
from device that queue has space, is coming in form of memory write
(device writes to a memory location of i86 processor).

I tried using wait_event_interruptible_timeout, I am depending on 2nd
parameter of the function but it wake up is too slow, even tried using
value of 1.

Any suggestions ?

Sleep and Wake up

[Greg KH]

On Mon, May 02, 2011 at 11:32:32AM -0700, Abu Rasheda wrote:
> I am testing my driver on much faster host processor and facing
> following issues:
> 
> My host is too powerful and it can fill up device buffer queue very fast.

What kind of device is this?

> I get best performance when I do busy wait, but this is not desirable
> and is bad design.
> 
> I need to sleep and wake up quickly and predictability. Indication
> from device that queue has space, is coming in form of memory write
> (device writes to a memory location of i86 processor).
> 
> I tried using wait_event_interruptible_timeout, I am depending on 2nd
> parameter of the function but it wake up is too slow, even tried using
> value of 1.

Why not try increasing the buffer in your driver to handle any amount of
data needed?

thanks,

greg k-h

Sleep and Wake up

[Abu Rasheda]

On Mon, May 2, 2011 at 12:28 PM, Greg KH <greg@kroah.com> wrote:
> On Mon, May 02, 2011 at 11:32:32AM -0700, Abu Rasheda wrote:
>> I am testing my driver on much faster host processor and facing
>> following issues:
>>
>> My host is too powerful and it can fill up device buffer queue very fast.
>
> What kind of device is this?

its a networking device

>> I get best performance when I do busy wait, but this is not desirable
>> and is bad design.
>>
>> I need to sleep and wake up quickly and predictability. Indication
>> from device that queue has space, is coming in form of memory write
>> (device writes to a memory location of i86 processor).
>>
>> I tried using wait_event_interruptible_timeout, I am depending on 2nd
>> parameter of the function but it wake up is too slow, even tried using
>> value of 1.
>
> Why not try increasing the buffer in your driver to handle any amount of
> data needed?

Problem come from the fact that remote end can be slow, so device /
driver cannot do much. If I increase buffer size, than running on
(even) faster host processor will bring back same issue. I am looking
for solution which will scale.

I check for available buffer size if space is low, I sleep and retry.
This is causing lot of CPU to be wasted.

How can I optimize this ?

>
> thanks,
>
> greg k-h

Sleep and Wake up

[Greg KH]

On Mon, May 02, 2011 at 02:02:34PM -0700, Abu Rasheda wrote:
> On Mon, May 2, 2011 at 12:28 PM, Greg KH <greg@kroah.com> wrote:
> > On Mon, May 02, 2011 at 11:32:32AM -0700, Abu Rasheda wrote:
> >> I am testing my driver on much faster host processor and facing
> >> following issues:
> >>
> >> My host is too powerful and it can fill up device buffer queue very fast.
> >
> > What kind of device is this?
> 
> its a networking device

It sounds like a broken device, you need to be able to handle large data
streams, right?  Are you running Linux in this device and that is the
issue you are having?

> >> I get best performance when I do busy wait, but this is not desirable
> >> and is bad design.
> >>
> >> I need to sleep and wake up quickly and predictability. Indication
> >> from device that queue has space, is coming in form of memory write
> >> (device writes to a memory location of i86 processor).
> >>
> >> I tried using wait_event_interruptible_timeout, I am depending on 2nd
> >> parameter of the function but it wake up is too slow, even tried using
> >> value of 1.
> >
> > Why not try increasing the buffer in your driver to handle any amount of
> > data needed?
> 
> Problem come from the fact that remote end can be slow, so device /
> driver cannot do much. If I increase buffer size, than running on
> (even) faster host processor will bring back same issue. I am looking
> for solution which will scale.
> 
> I check for available buffer size if space is low, I sleep and retry.
> This is causing lot of CPU to be wasted.
> 
> How can I optimize this ?

Use NAPI for your network driver?

good luck,

greg k-h

Fwd: Sleep and Wake up

[Abu Rasheda]

> >> I am testing my driver on much faster host processor and facing
> >> following issues:
> >>
> >> My host is too powerful and it can fill up device buffer queue very fast.
> >
> > What kind of device is this?
>
>> its a networking device

>It sounds like a broken device, you need to be able to handle large data
>streams, right? ?Are you running Linux in this device and that is the
>issue you are having?

I am not running Linux on the device. As a said, if remote host does
not send syn for TCP packets, my device queue will continue to grow.

My question is simple, what would be correct way of handling things in
this situation in software or if device is broken how would device
behave differently ?

I am here to learn.

Kernel compilation

[Ezequiel García]

Greetings,After reading Linux Kernel in a Nutshell (by the way, great book), I now have my own kernel configured for my hardware needs.?
However, I have some really strange behavior. On tty1, the same tty that launches X11, I have to press twice each keyboard key.?Anyone has any idea what's going on? I am using a customized Debian Squeeze + OpenBox.
Thanks in advance, and thanks Greg for writing the book.
Ezequiel.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110502/4f88e012/attachment.html 

Kernel compilation

[Pico Geyer]

[Forgot to cc list]

2011/5/2 Ezequiel Garc?a <elezegarcia@yahoo.com.ar>
>
> Greetings,
> After reading Linux Kernel in a Nutshell (by the way, great book), I now have my own kernel configured for my hardware needs.
> However, I have some really strange behavior. On tty1, the same tty that launches X11, I have to press twice each keyboard key.?Anyone has any idea what's going on? I am using a customized Debian Squeeze + OpenBox.
> Thanks in advance, and thanks Greg for writing the book.
> Ezequiel.

Hi Ezequiel

I'm new on this list as well but I don't think anyone will be able to
help you with the limited info that you have provided.
When you say you have a kernel configured for your hardware, did you:

A) Download the kernel from http://www.kernel.org/ ? In that case you
need to tell us which version you're using.
B) Get the kernel from git a git tree http://git.kernel.org/? We still
need to know exactly which version
C) Recompile your debian kernel? (2.6.32?)

If this bug is present with only a change in kernel config, it would
be interesting to see the difference between the working and broken
config.

I hope that helps,
Pico

Sleep and Wake up

[Mulyadi Santosa]

Hi Abu...

On Tue, May 3, 2011 at 01:32, Abu Rasheda <rcpilot2010@gmail.com> wrote:
> I am testing my driver on much faster host processor and facing
> following issues:
>
> My host is too powerful and it can fill up device buffer queue very fast.
>
> I get best performance when I do busy wait, but this is not desirable
> and is bad design.

For me, busy waiting could be the best for your case... in fact, NAPI
sometimes do that too....in most cases.

Actually, devices usually fills the in-RAM buffer with the help of
interrupt(s). So whenever in-device buffer has something new, it
interrupts...then soft irq follows up by copying them to RAM buffe
(circular buffer usually). To make it fast, DMA  or other CPU less
operation kicks in here.

So I think that;s the key...by relying on the interrupts...then you
will know when your buffer is about to be full.

Other than that, I would still say, busy waiting could be your best bet.

PS: here's crazy idea, if you can accept it. Put a guard page, right
at the tail of your buffer,. Make it unwritable even by kernel mode.
Right when you hit it, page fault should kicks in. Then you know
buffer is full. This is inspired by the stack guard..usual method in
user space to reduce buffer overflow risk.

-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

Sleep and Wake up

[Peter Teoh]

On Tue, May 3, 2011 at 2:32 AM, Abu Rasheda <rcpilot2010@gmail.com> wrote:

> I am testing my driver on much faster host processor and facing
> following issues:
>
> My host is too powerful and it can fill up device buffer queue very fast.
>
> I get best performance when I do busy wait, but this is not desirable
> and is bad design.
>
> I need to sleep and wake up quickly and predictability. Indication
> from device that queue has space, is coming in form of memory write
> (device writes to a memory location of i86 processor).
>
> I tried using wait_event_interruptible_timeout, I am depending on 2nd
> parameter of the function but it wake up is too slow, even tried using
> value of 1.
>
> Any suggestions ?
>
>
Many of us are just newbies in this area, and therefore, get it working is
much more important than to optimize it - u can safely said that the
hardcore kernel developer has already optimize many of these problems away,
and so if they cannot do it, there must be a reason...try to probe more
first perhaps.

High speed networking device has many special hardware features: IP/UDP/TCP
checksum offload etc.

http://www.fenrus.org/how-to-not-write-a-device-driver-paper.pdf
http://www.sun.com/products/networking/infiniband/ibhcaPCI-E/docs/datasheet.pdf

Read this about NAPI:

http://www.linuxfoundation.org/collaborate/workgroups/networking/napi

<http://www.linuxfoundation.org/collaborate/workgroups/networking/napi>Eg,
"Interrupt mitigation" - whereby interrupt mechanism is disabled (in
particular my desktop PC's r8196.c is using this feature) and polling takes
over instead - but u have to implemented complicated mechanism to reinject
the interrupt if necessary (read r8196.c).

And if u really ready....this is a good writeup:

http://datatag.web.cern.ch/datatag/howto/tcp.html

Other possible suggestion/features:

Jumbo frames:

http://www.cyberciti.biz/faq/rhel-centos-debian-ubuntu-jumbo-frames-configuration/

PCI posting (pdf paper above and r8196.c).

Disabling TCP software checksum (and use the hardware instead):

http://www.linuxquestions.org/questions/linux-enterprise-47/how-to-disable-tcp-checksumming-690745/

And for loads of other ideas (eg, TCP bypass):

*
http://ttthebear.blogspot.com/2008/07/linux-kernel-bypass-and-performance.html
*

Generally a lot of these ideas can be found in the kernel source codes -
just search and copy the implementation.....the highest network data
transfer is achieved in infiniband-based Mellanox card (in some China
supercomputer), and this involved the use of GPU technology etc....

https://lists.sdsc.edu/pipermail/npaci-rocks-discussion/2009-May/039639.html

-- 
Regards,
Peter Teoh
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.kernelnewbies.org/pipermail/kernelnewbies/attachments/20110510/4263ed73/attachment.html 

Kernel compilation

[scott]

I'm new to the list and would like to say hello to everyone. Now on to 
the meat.

I have tried to compile a kernel with a grsec patch 5 times now using a
couple of different methods and I keep getting a "No init found" and
"Couldn't support optional features" errors every time at bootup.
I'm trying to use a vanilla 2.6.32.27 kernel on 10.04 Ubuntu and it 
compiles fine,
just will not boot.

I've tried "CONCURRENCY_LEVEL=`getconf _NPROCESSORS_ONLN` fakeroot make-kpkg
--initrd --append-to-version=-custom kernel_image kernel_headers",
"CONCURRENCY_LEVEL=3 make-kpkg --initrd --append-to-version=-586
kernel_image kernel_headers modules_image".

I have all the reqs for building kernels such as build-essentials, fakeroot,
kernel-package, kernel-wedge, nurces-dev and qt for xconfig.

These are the steps I've taken. DL the kernel image from kernels.org, get
the patch from grsec.net, untar the kernel, cd into the directory created,
do a 'cp /boot/config-`uname -r` .config', 'make oldconfig', apply the
patch, then either 'make menuconfig' or 'make xconfig'. Then I do a
'make-kpkg clean' and run one of the commands in the first paragraph above.

I get the debs built and can install them just fine, but I keep getting the
same errors. I've cut out as many "Experimental" options I thought might
cause a problem and still get a kernel panic at boot.

Anyone have a clue what I'm doing wrong?
Scott

Kernel compilation

[Mulyadi Santosa]

Hi Scott :)

On Wed, Jan 5, 2011 at 12:34, scott <redhowlingwolves@nc.rr.com> wrote:
> I'm new to the list and would like to say hello to everyone. Now on to
> the meat.
>
> I have tried to compile a kernel with a grsec patch 5 times now using a
> couple of different methods and I keep getting a "No init found" and
> "Couldn't support optional features" errors every time at bootup.
> I'm trying to use a vanilla 2.6.32.27 kernel on 10.04 Ubuntu and it
> compiles fine,
> just will not boot.

My first question is: did the patching go successfully? were you
applying the correct grsec version against a matching vanilla kernel
version?

And last, maybe you can attach your kernel .config content...and one
of fine people here will try to decipher what's wrong with it :)

-- 
regards,

Mulyadi Santosa
Freelance Linux trainer and consultant

blog: the-hydra.blogspot.com
training: mulyaditraining.blogspot.com

Kernel compilation

[scott]

On 01/05/2011 01:46 AM, Mulyadi Santosa wrote:
> Hi Scott :)
>
> On Wed, Jan 5, 2011 at 12:34, scott<redhowlingwolves@nc.rr.com>  wrote:
>> I'm new to the list and would like to say hello to everyone. Now on to
>> the meat.
>>
>> I have tried to compile a kernel with a grsec patch 5 times now using a
>> couple of different methods and I keep getting a "No init found" and
>> "Couldn't support optional features" errors every time at bootup.
>> I'm trying to use a vanilla 2.6.32.27 kernel on 10.04 Ubuntu and it
>> compiles fine,
>> just will not boot.
> My first question is: did the patching go successfully? were you
> applying the correct grsec version against a matching vanilla kernel
> version?
>
> And last, maybe you can attach your kernel .config content...and one
> of fine people here will try to decipher what's wrong with it :)
>
The patch was matched and was successful. It compiled with no errors. 
Just the 'no init' at boot.
I'll post my .config later today as it's late here and have to work. I'm 
beginning to think it may have something to do with the limits on file 
size as I got a few warnings about out-of-range sizes. I have it set to 
1024.

Thanks for your reply,
Scott

Thanks for your reply.

Kernel compilation

[Robert P. J. Day]

On Wed, 5 Jan 2011, scott wrote:

> I'm new to the list and would like to say hello to everyone. Now on
> to the meat.
>
> I have tried to compile a kernel with a grsec patch 5 times now
> using a couple of different methods and I keep getting a "No init
> found" and "Couldn't support optional features" errors every time at
> bootup. I'm trying to use a vanilla 2.6.32.27 kernel on 10.04 Ubuntu
> and it compiles fine, just will not boot.
>
> I've tried "CONCURRENCY_LEVEL=`getconf _NPROCESSORS_ONLN` fakeroot
> make-kpkg --initrd --append-to-version=-custom kernel_image
> kernel_headers", "CONCURRENCY_LEVEL=3 make-kpkg --initrd
> --append-to-version=-586 kernel_image kernel_headers modules_image".
>
> I have all the reqs for building kernels such as build-essentials,
> fakeroot, kernel-package, kernel-wedge, nurces-dev and qt for
> xconfig.
>
> These are the steps I've taken. DL the kernel image from
> kernels.org, get the patch from grsec.net, untar the kernel, cd into
> the directory created, do a 'cp /boot/config-`uname -r` .config',
> 'make oldconfig', apply the patch, then either 'make menuconfig' or
> 'make xconfig'. Then I do a 'make-kpkg clean' and run one of the
> commands in the first paragraph above.
>
> I get the debs built and can install them just fine, but I keep
> getting the same errors. I've cut out as many "Experimental" options
> I thought might cause a problem and still get a kernel panic at
> boot.
>
> Anyone have a clue what I'm doing wrong?

  you are clearly using ubuntu, so here's a simpler recipe i've used
successfully.

  1) configure and build your kernel from the source
  2) "sudo make modules_install"
  3) "sudo make install"
  4) "sudo update-initramfs -c -k <kernel version goes here>"
  5) "sudo update-grub"

the drawback to the above is that your kernel is not built as an
official *package* but for testing purposes, it should do.  you'll get
your kernel installed under /boot, your modules installed, a new
initramfs image, and your grub file updated.  can you try it this way
and see if it makes a difference?

rday

-- 

========================================================================
Robert P. J. Day                               Waterloo, Ontario, CANADA
                        http://crashcourse.ca

Twitter:                                       http://twitter.com/rpjday
LinkedIn:                               http://ca.linkedin.com/in/rpjday
========================================================================