Re: Questions on kernel skb send / netdev queue monitoring

[Andre Schwarz <andre.schwarz@matrix-vision.de>]

Eric,

thanks for your reply, but there are still open questions.


Eric Dumazet schrieb:
> Andre Schwarz a écrit :
>> Hi,
>>
>> we're running 2.6.27 on a MPC8343 based board.
>> The board is working as a camera and is supposed to stream image data
>> over 1000M Ethernet.
>>
>> Ethernet is connected via 2x Vitesse VSC8601 RGMII PHY, i.e. "eth0" and
>> "eth1" present.
>>
>> Basically the system is running fine for quite some time - starting with
>> kernel 2.6.19.
>> Lately I have some trouble regarding performance and errors.
>>
>> Obviously I'm doing something wrong ... hopefully someone can
>> enlighten me.
>>
>>
>> How the system works :
>>
>> - Kernel driver allocates static list of skb to hold a complete image.
>> This can be up to 4k skb depending on mtu.
>> - Imaging device (FPGA @ PCI) initiates DMA into skb.
>> - driver sends the skb out.
>>
>>
>> 1. Sending
>>
>> This is my "inner loop" send function and is called for every skb in the
>> list.
>>
>> static inline int gevss_send_get_ehdr(TGevStream *gevs, struct
>> sk_buff *skb)
>> {
>>         int result;
>>         struct sk_buff *slow_skb = skb_clone(skb, GFP_ATOMIC);
>>
>>         atomic_inc(&slow_skb->users);
>>         result = gevs->rt->u.dst.output(slow_skb);
>>         kfree_skb(slow_skb);
>>
>>         return result;
>> }
>>
>> Is there really any need for cloning each skb before sending ?
>> I'd really like to send the static skb without consuming it. How can
>> this be done ?
>>
>
> You have replied to yourself... you clone skb because you want to keep
> skb.
>
As long as this is the fastest way. Wouldn't be a simple use count
increment better ?
Why is incrementing the user count before sending not working ?
>> Is "gevs->rt->u.dst.output(slow_skb)" reasonable ?
>> What about "hard_start_xmit" and/or "dev_queue_xmit" inside netdev ?
>> Are these functions supposed to be used by other drivers ?
>>
>> What result can I expect if there's a failure, i.e. the HW-queue is
>> full ?
>> How should this be handled ? retry,i.e. send again after a while ?
>> Can I query the xmit queue size/usage ?
>>
>> Actually I'm checking for NETDEV_TX_OK and NETDEV_TX_BUSY.
>> Is this reasonable ?
>>
>>
>> 2. "overruns"
>>
>> I've never seen that before. The overrun counter is incrementing quite
>> fast even during proper operation.
>> Looks like this is also an issue with not throttling the sender when the
>> xmit queue is full ...  :-( How can I avoid this ?
>>
>> eth0      Link encap:Ethernet  HWaddr 00:0C:8D:30:40:25
>>           inet addr:192.168.65.55  Bcast:192.168.65.255 
>> Mask:255.255.255.0
>>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>>           RX packets:929 errors:0 dropped:0 overruns:0 frame:0
>>           TX packets:180937 errors:0 dropped:0 overruns:54002
>> carrier:0             collisions:0 txqueuelen:1000
>>           RX bytes:65212 (63.6 KiB)  TX bytes:262068658 (249.9 MiB)
>>           Base address:0xa000
>
> If your driver has to push 4096 skb at once, and you dont want to
> handle overruns,
> you might need to change eth0 settings
>
> ifconfig eth0 txqueuelen 5000
>
ok - but I definitely want to handle overruns or at least get notified
when the queue is almost full.
Querying the queue status would be fine, also.

Since I have data sources capable of delivering much more than 125MB
_any_ queue will overflow sooner or later without some kind of flow control.
Can you tell me how the txqueue can be monitored ?


regards,
Andre



MATRIX VISION GmbH, Talstraße 16, DE-71570 Oppenweiler  - Registergericht: Amtsgericht Stuttgart, HRB 271090
Geschäftsführer: Gerhard Thullner, Werner Armingeon, Uwe Furtner