Linux kernel thread with Linux 2.6.x

Linux kernel thread with Linux 2.6.x

[Laurent Lagrange]

Hello everybody,

I'm writing a custom network driver based on a MPC8260 FCC device for a
2.6.9 linux kernel.
In this driver, I need to use specific buffer allocation functions which use
down and up semaphore functions.

As the interrupt handler can't be scheduled, I have made a kernel thread
which waits forever on a semaphore.
This semaphore is set when a received packet interrupt occured. All that
mechanism works but the performances
are very poor.

Is there any solution to increase the kthread priority in the scheduler ?
Is there another solution than the semaphore to quickly wake up a kthread ?

Thanks a lot for your help.
Laurent

Re: Linux kernel thread with Linux 2.6.x

[Thiago Galesi]

>
> As the interrupt handler can't be scheduled, I have made a kernel thread
> which waits forever on a semaphore.
> This semaphore is set when a received packet interrupt occured.

You should look into tasklets for this. Not that your system is not
OK, but, as you said it, it's not fast enough.

Note that you still have some limitations using tasklets, but it's
more flexible than Interrupt handlers.

RE: Linux kernel thread with Linux 2.6.x

[Laurent Lagrange]

Hi,

Thanks for your answer, but a tasklet runs in interrupt context
(in_interrupt() != 0) so it doesn't support schedule() call
included in "down" semaphore function.

Any other idea ?

Laurent.

> -----Message d'origine-----
> De : Thiago Galesi [mailto:thiagogalesi@gmail.com]
> Envoyé : lun. 29 mai 2006 18:13
> À : Laurent Lagrange
> Cc : linuxppc-embedded@ozlabs.org
> Objet : Re: Linux kernel thread with Linux 2.6.x
>
>
> >
> > As the interrupt handler can't be scheduled, I have made a
> kernel thread
> > which waits forever on a semaphore.
> > This semaphore is set when a received packet interrupt occured.
>
> You should look into tasklets for this. Not that your system is not
> OK, but, as you said it, it's not fast enough.
>
> Note that you still have some limitations using tasklets, but it's
> more flexible than Interrupt handlers.
>
>

Re: Linux kernel thread with Linux 2.6.x

[Jörn Engel]

On Tue, 30 May 2006 11:46:09 +0200, Laurent Lagrange wrote:
> 
> Thanks for your answer, but a tasklet runs in interrupt context
> (in_interrupt() != 0) so it doesn't support schedule() call
> included in "down" semaphore function.

Do you have code you can show?

Jörn

-- 
Data expands to fill the space available for storage.
-- Parkinson's Law

Re: Linux kernel thread with Linux 2.6.x

[almoeli]

Hi,

do you have to use the semaphore or does a spinlock also meet your needs?
If you are in kernel 2.4 you can use one of the task_queues (e.g the
scheduler queue, timer and immediate queue also run at irq context) but
that won't have a better speed than your solution because it also gets
the processor when tasks are scheduled (all 10ms I think). In 2.6 it
would be the work_queue (schedule_work()).
So the best way is to use spinlocks and do the work in a tasklet at
interrupt context.

Oliver

Jörn Engel schrieb:
> On Tue, 30 May 2006 11:46:09 +0200, Laurent Lagrange wrote:
>> Thanks for your answer, but a tasklet runs in interrupt context
>> (in_interrupt() != 0) so it doesn't support schedule() call
>> included in "down" semaphore function.
> 
> Do you have code you can show?
> 
> Jörn
> 

Re: Linux kernel thread with Linux 2.6.x

[Andy Fleming]

Couldn't you use a work_queue?  For the PHY Layer, the MDIO  
transactions can't occur at interrupt time (they need to be  
interruptible), so the interrupt handler calls schedule_work() to get  
the interrupt handled.  I'm not sure how much faster it is (if at  
all), but it's another method, and sounds less complicated than  
waiting on a semaphore.

Of course, I would suggest trying to avoid the buffer allocation  
functions you are using, or defer the work using NAPI, but it's  
impossible for me to tell whether that's a reasonable solution for you.

On May 29, 2006, at 10:35, Laurent Lagrange wrote:

>
> Hello everybody,
>
> I'm writing a custom network driver based on a MPC8260 FCC device  
> for a
> 2.6.9 linux kernel.
> In this driver, I need to use specific buffer allocation functions  
> which use
> down and up semaphore functions.
>
> As the interrupt handler can't be scheduled, I have made a kernel  
> thread
> which waits forever on a semaphore.
> This semaphore is set when a received packet interrupt occured. All  
> that
> mechanism works but the performances
> are very poor.
>
> Is there any solution to increase the kthread priority in the  
> scheduler ?
> Is there another solution than the semaphore to quickly wake up a  
> kthread ?
>
> Thanks a lot for your help.
> Laurent
>
>
> _______________________________________________
> Linuxppc-embedded mailing list
> Linuxppc-embedded@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-embedded

RE: Linux kernel thread with Linux 2.6.x

[Laurent Lagrange]

I have tried many solutions, the two last solutions (work_queue and
kernel_thread)
seem to have the same poor performances.

Does anyone have another idea ?
Thanks in advance.
Laurent

PS : eth_init_rx_bd function can allocate a new skbuff (YES parameter) or
re-use
the previous skbuff allocated to a rx buffer descriptor(NO parameter).

1) tasklet : runs in interrupt context and
then doesn't support semaphore down function.

2) work_queue : doesn't run in interrupt context
and supports semaphore down function.

static irqreturn_t eth_interrupt (int irq, void * dev_id, struct pt_regs *
regs)
{
    DRV_ETH_INFO    *info       = (DRV_ETH_INFO *)dev_id;
    unsigned short  events;

    /* get pending events */
    events = info->eth_reg->fcc_fcce & info->eth_reg->fcc_fccm;

    /* ack pending events */
    info->eth_reg->fcc_fcce = events;

    /* mask interrupts */
    info->eth_reg->fcc_fccm = ~events;

    /* check events */
    if (BTST(FCC_ENET_RXF, events))
tasklet_schedule(&Info_low->recvTasklet);
    if (BTST(FCC_ENET_TXE|FCC_ENET_TXB, events))
tasklet_schedule(&Info_low->xmitEndTasklet);

    return(IRQ_HANDLED);
}

static void eth_rx_event (DRV_ETH_INFO *info)
{
    ETH_BD_MGT_TABLE    *bd_table   = &info->eth_bd_table;
    ETH_BD_MGT          *bd_mgt     = &bd_table->rx[bd_table->rx_out];
    struct sk_buff      *skb;

    /* check empty status */
    while(BTST(BD_ENET_RX_EMPTY, bd_mgt->bd->cbd_sc) == 0) {
        /* check error and upper layer status */
        if
(BTST((BD_ENET_RX_LG|BD_ENET_RX_SH|BD_ENET_RX_NO|BD_ENET_RX_CR|BD_ENET_RX_OV
|BD_ENET_RX_CL), bd_mgt->bd->cbd_sc))    {
            /* re-enable bd with current buffer */
            eth_init_rx_bd(info, bd_mgt, NO); /* always return without error
*/
        } else {
            /* save current segment */
            skb         = (struct sk_buff *)bd_mgt->segment;
            skb->len    = bd_mgt->bd->cbd_datlen;

            /* get a new buffer for this bd */
            if (eth_init_rx_bd(info, bd_mgt, YES) == BRG_NO_ERROR) {
                /* fill sk_buff parameters */
	          skb->dev      = info->dev;
		    skb->protocol = eth_type_trans(skb, info->dev);

		    /* send frame to upper layers that use semaphores */
		    netif_packet(skb);
		} else {
                /* re-enable bd with current buffer */
                eth_init_rx_bd(info, bd_mgt, NO); /* always return without
error */
            }
        }

        /* set next index */
        bd_table->rx_out = (bd_table->rx_out == (ETH_RX_BD_NUMBER -
1))?0:(bd_table->rx_out + 1);

        /* get next bd mgt pointer */
        bd_mgt = &bd_table->rx[bd_table->rx_out];
    }

    /* enable interruption*/
    info->eth_reg->fcc_fccm |= FCC_ENET_RXF;

    return;
}

3) kernel_thread : doesn't run in interrupt context
and supports semaphore down function. I have tried to
change many task parameters :
    task->policy		= SCHED_RR;
    task->static_prio	= 100;
    task->prio		= 100;
    task->rt_priority	= 1;

To synchronize the rx interrruption and the thread I have tried completion
and semaphore mecanism without any difference

static irqreturn_t eth_interrupt (int irq, void * dev_id, struct pt_regs *
regs)
{
    DRV_ETH_INFO    *info       = (DRV_ETH_INFO *)dev_id;
    USHORT          events;

    /* get pending events */
    events = info->eth_reg->fcc_fcce & info->eth_reg->fcc_fccm;

    /* ack pending events */
    info->eth_reg->fcc_fcce = events;

    /* mask interrupts */
    info->eth_reg->fcc_fccm = ~events;

    /* check events */
    if (BTST(FCC_ENET_RXF, events)) osSemSignal(Info_low->recvSem);
    if (BTST(FCC_ENET_TXE|FCC_ENET_TXB, events))
osSemSignal(Info_low->xmitEndSem);

    return(IRQ_HANDLED);
}

unsigned eth_recv_task (void* data)
{
    DRV_ETH_INFO        *info       = (DRV_ETH_INFO
*)Info_low->drv_eth_info;
    ETH_BD_MGT_TABLE    *bd_table   = &info->eth_bd_table;
    ETH_BD_MGT          *bd_mgt     = &bd_table->rx[bd_table->rx_out];

    while(1) {
        /* wait interrupt */
        osSemWait(Info_low->recvSem, 0);

        /* check empty status */
        while(BTST(BD_ENET_RX_EMPTY, bd_mgt->bd->cbd_sc) == 0) {
            /* check error and upper layer status */
            if
(BTST((BD_ENET_RX_LG|BD_ENET_RX_SH|BD_ENET_RX_NO|BD_ENET_RX_CR|BD_ENET_RX_OV
|BD_ENET_RX_CL), bd_mgt->bd->cbd_sc))    {
                /* re-enable bd with current buffer */
                eth_init_rx_bd(info, bd_mgt, NO); /* always return without
error */
            } else {
                /* save current segment */
                skb         = (struct sk_buff *)bd_mgt->segment;
                skb->len    = bd_mgt->bd->cbd_datlen - 4;

            /* get a new buffer for this bd */
            if (eth_init_rx_bd(info, bd_mgt, YES) == BRG_NO_ERROR) {
                /* fill sk_buff parameters */
	          skb->dev      = info->dev;
		    skb->protocol = eth_type_trans(skb, info->dev);

		    /* send frame to upper layers that use semaphores */
		    netif_packet(skb);
		} else {
                /* re-enable bd with current buffer */
                eth_init_rx_bd(info, bd_mgt, NO); /* always return without
error */
            }

            /* set next index */
            bd_table->rx_out = (bd_table->rx_out == (ETH_RX_BD_NUMBER -
1))?0:(bd_table->rx_out + 1);

            /* get next bd mgt pointer */
            bd_mgt = &bd_table->rx[bd_table->rx_out];
        }

        /* enable interrupt */
        info->eth_reg->fcc_fccm |= FCC_ENET_RXF;
    }
    return(0);
}