command ip -details -statistics link show wlan0

command ip -details -statistics link show wlan0

[Ernast Sevo]

Hello,

 I am quite new to socket can and I had a question about the output of
the command: ip -details -statistics link show can0. When I run the
above command I get the following output:

re-started bus-errors arbit-lost error-warn error-pass bus-off
    0          0          0          0          0          0
    RX: bytes  packets  errors  dropped overrun mcast
    3956387    948121    11      0           11          0
    TX: bytes  packets  errors  dropped carrier collsns
    100        44       0       0       0       0


I see that I received 11 errors packets (all of them being rx overrun
errors) and that no packets were dropped. Does the dropped field refer
to packets lost in userspace (i.e because they weren't being read fast
enough and the socket receive buffer overflowed) or does the dropped
packet field refer to dropped packets being dropped somewhere other
than userspace?

Thanks,

Ernast

Re: command ip -details -statistics link show wlan0

[Wolfgang Grandegger]

Hi Ernast,

On 05/15/2013 08:58 PM, Ernast Sevo wrote:
> Hello,
> 
>  I am quite new to socket can and I had a question about the output of
> the command: ip -details -statistics link show can0. When I run the
> above command I get the following output:
> 
> re-started bus-errors arbit-lost error-warn error-pass bus-off
>     0          0          0          0          0          0
>     RX: bytes  packets  errors  dropped overrun mcast
>     3956387    948121    11      0           11          0
>     TX: bytes  packets  errors  dropped carrier collsns
>     100        44       0       0       0       0
> 
> 
> I see that I received 11 errors packets (all of them being rx overrun
> errors) and that no packets were dropped. Does the dropped field refer
> to packets lost in userspace (i.e because they weren't being read fast
> enough and the socket receive buffer overflowed) or does the dropped
> packet field refer to dropped packets being dropped somewhere other
> than userspace?

They refer to packets dropped in the driver for some reason, maily
because memory is short (ENOMEM). The overrun errors above are reported
by the hardware due to messages arrived but there was no space to store
it. Messages dropped because the socket buffer was full can be reported
with the candump option "-d" as shown here:

https://gitorious.org/linux-can/can-utils/blobs/master/candump.c#line549

Hope that helps.

Wolfgang.

command ip -details -statistics link show wlan0

[Ernast Sevo]

On Thu, May 16, 2013 at 3:02 AM, Wolfgang Grandegger <wg@grandegger.com> wrote:
> Hi Ernast,
>
> On 05/15/2013 08:58 PM, Ernast Sevo wrote:
>> Hello,
>>
>>  I am quite new to socket can and I had a question about the output of
>> the command: ip -details -statistics link show can0. When I run the
>> above command I get the following output:
>>
>> re-started bus-errors arbit-lost error-warn error-pass bus-off
>>     0          0          0          0          0          0
>>     RX: bytes  packets  errors  dropped overrun mcast
>>     3956387    948121    11      0           11          0
>>     TX: bytes  packets  errors  dropped carrier collsns
>>     100        44       0       0       0       0
>>
>>
>> I see that I received 11 errors packets (all of them being rx overrun
>> errors) and that no packets were dropped. Does the dropped field refer
>> to packets lost in userspace (i.e because they weren't being read fast
>> enough and the socket receive buffer overflowed) or does the dropped
>> packet field refer to dropped packets being dropped somewhere other
>> than userspace?
>
> They refer to packets dropped in the driver for some reason, maily
> because memory is short (ENOMEM). The overrun errors above are reported
> by the hardware due to messages arrived but there was no space to store
> it. Messages dropped because the socket buffer was full can be reported
> with the candump option "-d" as shown here:
>
> https://gitorious.org/linux-can/can-utils/blobs/master/candump.c#line549
>
> Hope that helps.
>
> Wolfgang.



Hi Wolfgang,

That definitely clears things up a bit. One more question though. Does
the number in
overrun or the number in errors from the command above then directly
refer to the number
of packets lost on the hardware level due to there being no space to
store it (i.e 11 overrun or
errors means 11 packets lost) ?

Thanks again,

Ernast

Re: command ip -details -statistics link show wlan0

[Marc Kleine-Budde]

[-- Attachment #1: Type: text/plain, Size: 779 bytes --]

On 05/16/2013 03:43 PM, Ernast Sevo wrote:
> That definitely clears things up a bit. One more question though. Does
> the number in
> overrun or the number in errors from the command above then directly
> refer to the number
> of packets lost on the hardware level due to there being no space to
> store it (i.e 11 overrun or
> errors means 11 packets lost) ?

This depends on the hardware/driver. But usually that means 11 overrun
events, which means _at_least_ 11 CAN frames lost.

Marc

-- 
Pengutronix e.K.                  | Marc Kleine-Budde           |
Industrial Linux Solutions        | Phone: +49-231-2826-924     |
Vertretung West/Dortmund          | Fax:   +49-5121-206917-5555 |
Amtsgericht Hildesheim, HRA 2686  | http://www.pengutronix.de   |


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

Re: command ip -details -statistics link show wlan0

[Ernast Sevo]

On Thu, May 16, 2013 at 10:59 AM, Wolfgang Grandegger <wg@grandegger.com> wrote:
> On 05/16/2013 03:43 PM, Ernast Sevo wrote:
>> On Thu, May 16, 2013 at 3:02 AM, Wolfgang Grandegger <wg@grandegger.com> wrote:
>>> Hi Ernast,
>>>
>>> On 05/15/2013 08:58 PM, Ernast Sevo wrote:
>>>> Hello,
>>>>
>>>>  I am quite new to socket can and I had a question about the output of
>>>> the command: ip -details -statistics link show can0. When I run the
>>>> above command I get the following output:
>>>>
>>>> re-started bus-errors arbit-lost error-warn error-pass bus-off
>>>>     0          0          0          0          0          0
>>>>     RX: bytes  packets  errors  dropped overrun mcast
>>>>     3956387    948121    11      0           11          0
>>>>     TX: bytes  packets  errors  dropped carrier collsns
>>>>     100        44       0       0       0       0
>>>>
>>>>
>>>> I see that I received 11 errors packets (all of them being rx overrun
>>>> errors) and that no packets were dropped. Does the dropped field refer
>>>> to packets lost in userspace (i.e because they weren't being read fast
>>>> enough and the socket receive buffer overflowed) or does the dropped
>>>> packet field refer to dropped packets being dropped somewhere other
>>>> than userspace?
>>>
>>> They refer to packets dropped in the driver for some reason, maily
>>> because memory is short (ENOMEM). The overrun errors above are reported
>>> by the hardware due to messages arrived but there was no space to store
>>> it. Messages dropped because the socket buffer was full can be reported
>>> with the candump option "-d" as shown here:
>>>
>>> https://gitorious.org/linux-can/can-utils/blobs/master/candump.c#line549
>>>
>>> Hope that helps.
>>>
>>> Wolfgang.
>>
>>
>>
>> Hi Wolfgang,
>>
>> That definitely clears things up a bit. One more question though. Does
>> the number in
>> overrun or the number in errors from the command above then directly
>> refer to the number
>> of packets lost on the hardware level due to there being no space to
>> store it (i.e 11 overrun or
>> errors means 11 packets lost) ?
>
> Well, not sure if the hardware realizes any packet lost. This typically
> happens when the software does not empty the hardware buffers quickly
> enough? What hardware are you using? MCP251x?
>
> Wolfgang.
>
> Wolfgang.
>
>

Yes I am using MCP2515. I am getting a couple of rx buffer overflow errors
so I was wondering how these errors corresponded to packet loss,
whether this is a 1:1 relationship or 1:N relationship (if i am
interpretting Marc's post
correctly should I assume 1:1 for best case and 1:N for worst case? ).
Essentially I am trying to determine at what speed i can have stuff
coming in before
I start losing packets for any reason. I know how to account packets
lost in userspace
 and packets lost by the driver (which you mentioned above) and now I
am trying to
account for packets lost at the hardware level. Knowing how these rx buffer
overflow/overrun errors correspond to packet loss would help make my
data more accurate and reliable.

Thanks,

Ernast

Re: command ip -details -statistics link show wlan0

[Marc Kleine-Budde]

[-- Attachment #1: Type: text/plain, Size: 1628 bytes --]

On 05/16/2013 05:32 PM, Ernast Sevo wrote:

>> Well, not sure if the hardware realizes any packet lost. This typically
>> happens when the software does not empty the hardware buffers quickly
>> enough? What hardware are you using? MCP251x?

> Yes I am using MCP2515. I am getting a couple of rx buffer overflow errors
> so I was wondering how these errors corresponded to packet loss,
> whether this is a 1:1 relationship or 1:N relationship (if i am
> interpretting Marc's post
> correctly should I assume 1:1 for best case and 1:N for worst case? ).

Yes, exactly.

> Essentially I am trying to determine at what speed i can have stuff
> coming in before
> I start losing packets for any reason. I know how to account packets
> lost in userspace
>  and packets lost by the driver (which you mentioned above) and now I
> am trying to
> account for packets lost at the hardware level. Knowing how these rx buffer
> overflow/overrun errors correspond to packet loss would help make my
> data more accurate and reliable.

If you don't get any dropped frames reported via SO_RXQ_OVFL (see
Wolfgang mail) you can be sure they are lost at the hardware level. I
suggest don't use the mcp251x, it's really slow and puts heavy load on
the machine. However there might be some knobs to tune in your SPI
controller/driver.

Marc

-- 
Pengutronix e.K.                  | Marc Kleine-Budde           |
Industrial Linux Solutions        | Phone: +49-231-2826-924     |
Vertretung West/Dortmund          | Fax:   +49-5121-206917-5555 |
Amtsgericht Hildesheim, HRA 2686  | http://www.pengutronix.de   |


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 263 bytes --]

Re: command ip -details -statistics link show wlan0

[Wolfgang Grandegger]

Hi Ernast,

please don't frop the CC to the mailing list...

On 05/16/2013 05:32 PM, Ernast Sevo wrote:
> On Thu, May 16, 2013 at 10:59 AM, Wolfgang Grandegger <wg@grandegger.com> wrote:
>> On 05/16/2013 03:43 PM, Ernast Sevo wrote:
>>> On Thu, May 16, 2013 at 3:02 AM, Wolfgang Grandegger <wg@grandegger.com> wrote:
>>>> Hi Ernast,
>>>>
>>>> On 05/15/2013 08:58 PM, Ernast Sevo wrote:
>>>>> Hello,
>>>>>
>>>>>  I am quite new to socket can and I had a question about the output of
>>>>> the command: ip -details -statistics link show can0. When I run the
>>>>> above command I get the following output:
>>>>>
>>>>> re-started bus-errors arbit-lost error-warn error-pass bus-off
>>>>>     0          0          0          0          0          0
>>>>>     RX: bytes  packets  errors  dropped overrun mcast
>>>>>     3956387    948121    11      0           11          0
>>>>>     TX: bytes  packets  errors  dropped carrier collsns
>>>>>     100        44       0       0       0       0
>>>>>
>>>>>
>>>>> I see that I received 11 errors packets (all of them being rx overrun
>>>>> errors) and that no packets were dropped. Does the dropped field refer
>>>>> to packets lost in userspace (i.e because they weren't being read fast
>>>>> enough and the socket receive buffer overflowed) or does the dropped
>>>>> packet field refer to dropped packets being dropped somewhere other
>>>>> than userspace?
>>>>
>>>> They refer to packets dropped in the driver for some reason, maily
>>>> because memory is short (ENOMEM). The overrun errors above are reported
>>>> by the hardware due to messages arrived but there was no space to store
>>>> it. Messages dropped because the socket buffer was full can be reported
>>>> with the candump option "-d" as shown here:
>>>>
>>>> https://gitorious.org/linux-can/can-utils/blobs/master/candump.c#line549
>>>>
>>>> Hope that helps.
>>>>
>>>> Wolfgang.
>>>
>>>
>>>
>>> Hi Wolfgang,
>>>
>>> That definitely clears things up a bit. One more question though. Does
>>> the number in
>>> overrun or the number in errors from the command above then directly
>>> refer to the number
>>> of packets lost on the hardware level due to there being no space to
>>> store it (i.e 11 overrun or
>>> errors means 11 packets lost) ?
>>
>> Well, not sure if the hardware realizes any packet lost. This typically
>> happens when the software does not empty the hardware buffers quickly
>> enough? What hardware are you using? MCP251x?
>>
>> Wolfgang.
>>
>> Wolfgang.
>>
>>
> 
> Yes I am using MCP2515. I am getting a couple of rx buffer overflow errors
> so I was wondering how these errors corresponded to packet loss,
> whether this is a 1:1 relationship or 1:N relationship (if i am
> interpretting Marc's post
> correctly should I assume 1:1 for best case and 1:N for worst case? ).

That would be my guess as well.

> Essentially I am trying to determine at what speed i can have stuff
> coming in before
> I start losing packets for any reason. I know how to account packets
> lost in userspace
>  and packets lost by the driver (which you mentioned above) and now I
> am trying to
> account for packets lost at the hardware level. Knowing how these rx buffer
> overflow/overrun errors correspond to packet loss would help make my
> data more accurate and reliable.

You may try to increase the priority of the MCP2515 IRQ thread.

Recently I also stumbled over the following optimized driver for the
MCP2515:

  http://clientes.netvisao.pt/anbadeol/mcp2515.html

But that would required some effort to update and test it.

Wolfgang.

Re: command ip -details -statistics link show wlan0

[Ernast Sevo]

Great! Thank you very much for all your help! And I will take a closer
look at that optimized driver.

Best,

 Ernast

On Thu, May 16, 2013 at 11:43 AM, Wolfgang Grandegger <wg@grandegger.com> wrote:
> Hi Ernast,
>
> please don't frop the CC to the mailing list...
>
> On 05/16/2013 05:32 PM, Ernast Sevo wrote:
>> On Thu, May 16, 2013 at 10:59 AM, Wolfgang Grandegger <wg@grandegger.com> wrote:
>>> On 05/16/2013 03:43 PM, Ernast Sevo wrote:
>>>> On Thu, May 16, 2013 at 3:02 AM, Wolfgang Grandegger <wg@grandegger.com> wrote:
>>>>> Hi Ernast,
>>>>>
>>>>> On 05/15/2013 08:58 PM, Ernast Sevo wrote:
>>>>>> Hello,
>>>>>>
>>>>>>  I am quite new to socket can and I had a question about the output of
>>>>>> the command: ip -details -statistics link show can0. When I run the
>>>>>> above command I get the following output:
>>>>>>
>>>>>> re-started bus-errors arbit-lost error-warn error-pass bus-off
>>>>>>     0          0          0          0          0          0
>>>>>>     RX: bytes  packets  errors  dropped overrun mcast
>>>>>>     3956387    948121    11      0           11          0
>>>>>>     TX: bytes  packets  errors  dropped carrier collsns
>>>>>>     100        44       0       0       0       0
>>>>>>
>>>>>>
>>>>>> I see that I received 11 errors packets (all of them being rx overrun
>>>>>> errors) and that no packets were dropped. Does the dropped field refer
>>>>>> to packets lost in userspace (i.e because they weren't being read fast
>>>>>> enough and the socket receive buffer overflowed) or does the dropped
>>>>>> packet field refer to dropped packets being dropped somewhere other
>>>>>> than userspace?
>>>>>
>>>>> They refer to packets dropped in the driver for some reason, maily
>>>>> because memory is short (ENOMEM). The overrun errors above are reported
>>>>> by the hardware due to messages arrived but there was no space to store
>>>>> it. Messages dropped because the socket buffer was full can be reported
>>>>> with the candump option "-d" as shown here:
>>>>>
>>>>> https://gitorious.org/linux-can/can-utils/blobs/master/candump.c#line549
>>>>>
>>>>> Hope that helps.
>>>>>
>>>>> Wolfgang.
>>>>
>>>>
>>>>
>>>> Hi Wolfgang,
>>>>
>>>> That definitely clears things up a bit. One more question though. Does
>>>> the number in
>>>> overrun or the number in errors from the command above then directly
>>>> refer to the number
>>>> of packets lost on the hardware level due to there being no space to
>>>> store it (i.e 11 overrun or
>>>> errors means 11 packets lost) ?
>>>
>>> Well, not sure if the hardware realizes any packet lost. This typically
>>> happens when the software does not empty the hardware buffers quickly
>>> enough? What hardware are you using? MCP251x?
>>>
>>> Wolfgang.
>>>
>>> Wolfgang.
>>>
>>>
>>
>> Yes I am using MCP2515. I am getting a couple of rx buffer overflow errors
>> so I was wondering how these errors corresponded to packet loss,
>> whether this is a 1:1 relationship or 1:N relationship (if i am
>> interpretting Marc's post
>> correctly should I assume 1:1 for best case and 1:N for worst case? ).
>
> That would be my guess as well.
>
>> Essentially I am trying to determine at what speed i can have stuff
>> coming in before
>> I start losing packets for any reason. I know how to account packets
>> lost in userspace
>>  and packets lost by the driver (which you mentioned above) and now I
>> am trying to
>> account for packets lost at the hardware level. Knowing how these rx buffer
>> overflow/overrun errors correspond to packet loss would help make my
>> data more accurate and reliable.
>
> You may try to increase the priority of the MCP2515 IRQ thread.
>
> Recently I also stumbled over the following optimized driver for the
> MCP2515:
>
>   http://clientes.netvisao.pt/anbadeol/mcp2515.html
>
> But that would required some effort to update and test it.
>
> Wolfgang.
>

Re: command ip -details -statistics link show wlan0

[Wolfgang Grandegger]

[-- Attachment #1: Type: text/plain, Size: 1954 bytes --]

Hi Ernast,

Please do not drop the CC to the Linux-CAN mailing list...

On 06/07/2013 09:52 PM, Wolfgang Grandegger wrote:
> On 06/07/2013 09:32 PM, Ernast Sevo wrote:
>> Hello,
>>
>>  I apologize for resurrecting an old thread but I finally got around
>> to testing a program which uses
>> socket can that I wrote and I get quite a few overrun errors. I also
>> tried increasing the priority of
>> the  MCP2515 IRQ which made things slightly better for me but not good
>> enough. Are there any other
>> suggestions as to how to fix these overrun errors? or at least improve
>> them a bit more.
> 
> What priority did you use? Could you show the output of
> 
>   # ps -m -eo pid,tid,class,rtprio,ni,pri,psr,pcpu,stat,wchan:14,comm
> 
> when the test app is running? Note that these option are not available
> with the busybox version of ps.
> 
> I also wrote:
> 
>>> Recently I also stumbled over the following optimized driver 
>>> MCP2515:
>>>
>>>   http://clientes.netvisao.pt/anbadeol/mcp2515.html
>>>
>>> But that would required some effort to update and test it.
> 
> Would be interesting if this driver behaves better concerning message
> losses. What kernel version are you using?

Just to give this driver a try, I have attached
"mcp2515-extern-v1.tar.bz2", my out-of-tree version of the MCP2515
driver mentioned above. On my Ubuntu system (running Linux 3.2.0), I
just use the following commands to get the kernel module built:

  $ tar xf mcp2515-extern-v1.tar.bz2
  $ cd mcp2515-extern
  $ ./make-modules.sh

For your embedded environment you need to tweak "make-modules.sh",
though, or use the following command:

  $ CONFIG_CAN_DEBUG_DEVICES=y CONFIG_CAN_MCP2515=m \
    make -C <path-to-your-kernel-tree> SUBDIRS=`pwd` modules

You should then be able to load "mcp2515.ko" on your embedded system
(after removing the old MCP2510 driver):

  # rmmod mcp2510
  # insmod mcp2515.ko

If you are lucky, it will work out of the box.

Wolfgang.


[-- Attachment #2: mcp2515-extern-v1.tar.bz2 --]
[-- Type: application/x-bzip, Size: 6079 bytes --]

Re: command ip -details -statistics link show wlan0

[Ernast Sevo]

Hi Wolfgang,

I will give this a try and let you know as soon as I test it out.

Best,

 Ernast

On Sat, Jun 8, 2013 at 3:42 AM, Wolfgang Grandegger <wg@grandegger.com> wrote:
> Hi Ernast,
>
> Please do not drop the CC to the Linux-CAN mailing list...
>
> On 06/07/2013 09:52 PM, Wolfgang Grandegger wrote:
>> On 06/07/2013 09:32 PM, Ernast Sevo wrote:
>>> Hello,
>>>
>>>  I apologize for resurrecting an old thread but I finally got around
>>> to testing a program which uses
>>> socket can that I wrote and I get quite a few overrun errors. I also
>>> tried increasing the priority of
>>> the  MCP2515 IRQ which made things slightly better for me but not good
>>> enough. Are there any other
>>> suggestions as to how to fix these overrun errors? or at least improve
>>> them a bit more.
>>
>> What priority did you use? Could you show the output of
>>
>>   # ps -m -eo pid,tid,class,rtprio,ni,pri,psr,pcpu,stat,wchan:14,comm
>>
>> when the test app is running? Note that these option are not available
>> with the busybox version of ps.
>>
>> I also wrote:
>>
>>>> Recently I also stumbled over the following optimized driver
>>>> MCP2515:
>>>>
>>>>   http://clientes.netvisao.pt/anbadeol/mcp2515.html
>>>>
>>>> But that would required some effort to update and test it.
>>
>> Would be interesting if this driver behaves better concerning message
>> losses. What kernel version are you using?
>
> Just to give this driver a try, I have attached
> "mcp2515-extern-v1.tar.bz2", my out-of-tree version of the MCP2515
> driver mentioned above. On my Ubuntu system (running Linux 3.2.0), I
> just use the following commands to get the kernel module built:
>
>   $ tar xf mcp2515-extern-v1.tar.bz2
>   $ cd mcp2515-extern
>   $ ./make-modules.sh
>
> For your embedded environment you need to tweak "make-modules.sh",
> though, or use the following command:
>
>   $ CONFIG_CAN_DEBUG_DEVICES=y CONFIG_CAN_MCP2515=m \
>     make -C <path-to-your-kernel-tree> SUBDIRS=`pwd` modules
>
> You should then be able to load "mcp2515.ko" on your embedded system
> (after removing the old MCP2510 driver):
>
>   # rmmod mcp2510
>   # insmod mcp2515.ko
>
> If you are lucky, it will work out of the box.
>
> Wolfgang.
>