Re: [Xenomai] Add cards parameter to rt_e1000e module?

Re: [Xenomai] Add cards parameter to rt_e1000e module?

[Mariusz Janiak]

> >> Certainly a well behaved driver would not touch a device already handled
> >> by a linux driver.  It should be possible to just issue an unbind request
> >> to the linux driver for the desired device and then a bind request to
> >> the rtnet driver to take over that device.
> 
> Thanks for sharing this information.  It sounds like that might work.
> 
> > The problem with the
> > bind/unbind solution is that the driver that will get all the cards
> > first depends on the drivers initialization order. With the "cards"
> > parameter, you do not have this issue.
> 
> Maybe one could create a script that would attempt to unbind both the rt and non-rt drivers (or check to see which one is bound and then unbind it), and then make the desired "bind" calls.  It would be better if the incorrect binding were not made in the first place, however.  I think the ideal solution would be to have a file that contains device/driver pairs that would keep the kernel from passing the specified devices to any other drivers.  I wonder how hard that would be to implement.

The RTnet configuration file rtnet.conf has already instrumentation for binding/unbinding devices. The REBIND_RT_NICS variable has to be set up with the correct NIC PCI addresses. If you have more then one NIC, you can identity PCI address of respective eth's using ethtool

ethtool -i eth0
ethtool -i eth1

etc.

The bus-info field contains device pci addres that you can directly insert to REBIND_RT_NICS. If you don't know which eth menage with physical interface, you can blink LED port of the NIC using, eg.

ethtool -p eth0

Best,
Mariusz

Re: [Xenomai] Add cards parameter to rt_e1000e module?

[Jeff Webb]

On 12/15/2014 02:04 PM, Mariusz Janiak wrote:
>>>> Certainly a well behaved driver would not touch a device already handled
>>>> by a linux driver.  It should be possible to just issue an unbind request
>>>> to the linux driver for the desired device and then a bind request to
>>>> the rtnet driver to take over that device.
>>
>> Thanks for sharing this information.  It sounds like that might work.
>>
>>> The problem with the
>>> bind/unbind solution is that the driver that will get all the cards
>>> first depends on the drivers initialization order. With the "cards"
>>> parameter, you do not have this issue.
>>
>> Maybe one could create a script that would attempt to unbind both the rt and non-rt drivers (or check to see which one is bound and then unbind it), and then make the desired "bind" calls.  It would be better if the incorrect binding were not made in the first place, however.  I think the ideal solution would be to have a file that contains device/driver pairs that would keep the kernel from passing the specified devices to any other drivers.  I wonder how hard that would be to implement.
>
> The RTnet configuration file rtnet.conf has already instrumentation for binding/unbinding devices. The REBIND_RT_NICS variable has to be set up with the correct NIC PCI addresses. If you have more then one NIC, you can identity PCI address of respective eth's using ethtool
>
> ethtool -i eth0
> ethtool -i eth1
>
> etc.
>
> The bus-info field contains device pci addres that you can directly insert to REBIND_RT_NICS. If you don't know which eth menage with physical interface, you can blink LED port of the NIC using, eg.
>
> ethtool -p eth0

(I am cross-posting this message to rtnet-users from the xenomai mailing list, since this is even more relevant there.  I apologize to those who receive duplicate e-mails.)

Thank you again, Mariusz.  All of this information is helpful.  I was following the instructions to start the RTnet core manually (as documented in the README file), since I am not using RTmac in my application.  I saw some references to REBIND_RT_NICS, but I didn't look into it very closely, since rtnet.conf and the "rtnet" script appeared to associated with RTmac.

After looking at the "rtnet" script, I see that it does indeed use bind/unbind in exactly the way we were discussing.  The script works very works well to switch an interface between standard and real-time networking, but unfortunately it does not support running without the RTmac layer.  I made some minor modifications to the script to add this support.  With my modifications, if TDMA_MODE = "none" (instead of "master" or "slave"), the script skips all of the rtmac/rtcfg/tmda initialization.  I have attached patches for both the rtnet and xenomai-3.git/next repositories.  I hope that they will be accepted, since I believe others have asked for this functionality in the past.

Gilles, it appears that the "cards" parameter may not be needed anymore, unless there are some other situations in which bind/unbind won't work.  I can see how that would make maintaining the RT drivers simpler.

-Jeff

-------------- next part --------------
A non-text attachment was scrubbed...
Name: rtnet-no-tdma-git.patch
Type: text/x-patch
Size: 888 bytes
Desc: not available
URL: <http://www.xenomai.org/pipermail/xenomai/attachments/20141216/a7cf8f8a/attachment.bin>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rtnet-no-tdma-xeno3.patch
Type: text/x-patch
Size: 617 bytes
Desc: not available
URL: <http://www.xenomai.org/pipermail/xenomai/attachments/20141216/a7cf8f8a/attachment-0001.bin>

[Xenomai] Add cards parameter to rt_e1000e module?

[Jeff Webb]

The rtnet rt_e1000e module does not implement the 'cards' parameter.  I have attached a patch for the 'next' branch of xenomai-3.git that adds this parameter, and it seems to work fine on my hardware.  The code was ported straight from the rt_e1000 module.  My machine has two e1000e cards, so this is a useful feature for me.  (I actually applied the same fix to the standard e1000e driver, which is even more useful to me, but I know it would be a maintenance nightmare for Xenomai to start maintaining that sort of patching...)

I am trying to learn rtnet, but it seems that the code from the rtnet git repo won't compile with linux-3.14.17/xenomai-2.6.4.  I saw that Gilles has pulled the rtnet code into the Xenomai-3 git repo, so it seemed to me that beta testing the Xenomai-3 rtnet code would be a better use of my time than messing with the old code base.  So far, so good.  My machine's primary ethernet interface is using the standard linux e1000e driver, and the secondary ethernet interface is up and running under rtnet using the rt_e1000e driver.  I can use rtping to talk to another linux box that is connected via a switch, provided I ping in the reverse direction first to set up the route information.

-Jeff
  
-------------- next part --------------
A non-text attachment was scrubbed...
Name: rt_e1000e.patch
Type: text/x-patch
Size: 1462 bytes
Desc: not available
URL: <http://www.xenomai.org/pipermail/xenomai/attachments/20141212/ff3c0806/attachment.bin>

Re: [Xenomai] Add cards parameter to rt_e1000e module?

[Gilles Chanteperdrix]

On Fri, Dec 12, 2014 at 04:10:43PM -0600, Jeff Webb wrote:
> The rtnet rt_e1000e module does not implement the 'cards'
> parameter. I have attached a patch for the 'next' branch of
> xenomai-3.git that adds this parameter, and it seems to work fine
> on my hardware. The code was ported straight from the rt_e1000
> module. My machine has two e1000e cards, so this is a useful
> feature for me. (I actually applied the same fix to the standard
> e1000e driver, which is even more useful to me, but I know it
> would be a maintenance nightmare for Xenomai to start maintaining
> that sort of patching...)
> 
> I am trying to learn rtnet, but it seems that the code from the
> rtnet git repo won't compile with linux-3.14.17/xenomai-2.6.4. I
> saw that Gilles has pulled the rtnet code into the Xenomai-3 git
> repo, so it seemed to me that beta testing the Xenomai-3 rtnet
> code would be a better use of my time than messing with the old
> code base. So far, so good. My machine's primary ethernet
> interface is using the standard linux e1000e driver, and the
> secondary ethernet interface is up and running under rtnet using
> the rt_e1000e driver. I can use rtping to talk to another linux
> box that is connected via a switch, provided I ping in the reverse
> direction first to set up the route information.

You can get rtnet to send an arp request to learn the route by using
the rtroute solicit command.

-- 
					    Gilles.

Re: [Xenomai] Add cards parameter to rt_e1000e module?

[Jeff Webb]

On 12/12/2014 05:56 PM, Gilles Chanteperdrix wrote:
> On Fri, Dec 12, 2014 at 04:10:43PM -0600, Jeff Webb wrote:
>> The rtnet rt_e1000e module does not implement the 'cards'
>> parameter. I have attached a patch for the 'next' branch of
>> xenomai-3.git that adds this parameter, and it seems to work fine
>> on my hardware. The code was ported straight from the rt_e1000
>> module. My machine has two e1000e cards, so this is a useful
>> feature for me. (I actually applied the same fix to the standard
>> e1000e driver, which is even more useful to me, but I know it
>> would be a maintenance nightmare for Xenomai to start maintaining
>> that sort of patching...)
>>
>> I am trying to learn rtnet, but it seems that the code from the
>> rtnet git repo won't compile with linux-3.14.17/xenomai-2.6.4. I
>> saw that Gilles has pulled the rtnet code into the Xenomai-3 git
>> repo, so it seemed to me that beta testing the Xenomai-3 rtnet
>> code would be a better use of my time than messing with the old
>> code base. So far, so good. My machine's primary ethernet
>> interface is using the standard linux e1000e driver, and the
>> secondary ethernet interface is up and running under rtnet using
>> the rt_e1000e driver. I can use rtping to talk to another linux
>> box that is connected via a switch, provided I ping in the reverse
>> direction first to set up the route information.
>
> You can get rtnet to send an arp request to learn the route by using
> the rtroute solicit command.
>

That works great.  Thanks for the ip.

Re: [Xenomai] Add cards parameter to rt_e1000e module?

[Jeff Webb]

On 12/15/2014 09:01 AM, Jeff Webb wrote:
> On 12/12/2014 05:56 PM, Gilles Chanteperdrix wrote:
>> On Fri, Dec 12, 2014 at 04:10:43PM -0600, Jeff Webb wrote:
>>> The rtnet rt_e1000e module does not implement the 'cards'
>>> parameter. I have attached a patch for the 'next' branch of
>>> xenomai-3.git that adds this parameter, and it seems to work fine
>>> on my hardware. The code was ported straight from the rt_e1000
>>> module. My machine has two e1000e cards, so this is a useful
>>> feature for me. (I actually applied the same fix to the standard
>>> e1000e driver, which is even more useful to me, but I know it
>>> would be a maintenance nightmare for Xenomai to start maintaining
>>> that sort of patching...)
>>>
>>> I am trying to learn rtnet, but it seems that the code from the
>>> rtnet git repo won't compile with linux-3.14.17/xenomai-2.6.4. I
>>> saw that Gilles has pulled the rtnet code into the Xenomai-3 git
>>> repo, so it seemed to me that beta testing the Xenomai-3 rtnet
>>> code would be a better use of my time than messing with the old
>>> code base. So far, so good. My machine's primary ethernet
>>> interface is using the standard linux e1000e driver, and the
>>> secondary ethernet interface is up and running under rtnet using
>>> the rt_e1000e driver. I can use rtping to talk to another linux
>>> box that is connected via a switch, provided I ping in the reverse
>>> direction first to set up the route information.
>>
>> You can get rtnet to send an arp request to learn the route by using
>> the rtroute solicit command.
>>
>
> That works great.  Thanks for the ip.
>

Sorry, I meant "tip", not "ip". ;)

Re: [Xenomai] Add cards parameter to rt_e1000e module?

[Gilles Chanteperdrix]

On Fri, Dec 12, 2014 at 04:10:43PM -0600, Jeff Webb wrote:
> The rtnet rt_e1000e module does not implement the 'cards'
> parameter. I have attached a patch for the 'next' branch of
> xenomai-3.git that adds this parameter, and it seems to work fine
> on my hardware. The code was ported straight from the rt_e1000
> module. My machine has two e1000e cards, so this is a useful
> feature for me. (I actually applied the same fix to the standard
> e1000e driver, which is even more useful to me, but I know it
> would be a maintenance nightmare for Xenomai to start maintaining
> that sort of patching...)

I understand the need for the "cards" parameter, but...
Is not there another way to solve this issue ? I do not know,
something in /proc or /sys to tell Linux to not load this driver for
that card? I suppose if we tell Linux to not use its driver for some
cards, then later loading the rtnet driver will be used for these
cards? Adding the cards parameter means that the rtnet drivers
diverge from mainline drivers, and so imply a maintenance cost. On
the other hand, maybe that is a cheaper one than trying and solving
the issue globally.

-- 
					    Gilles.

Re: [Xenomai] Add cards parameter to rt_e1000e module?

[Lennart Sorensen]

On Sat, Dec 13, 2014 at 02:03:58AM +0100, Gilles Chanteperdrix wrote:
> I understand the need for the "cards" parameter, but...
> Is not there another way to solve this issue ? I do not know,
> something in /proc or /sys to tell Linux to not load this driver for
> that card? I suppose if we tell Linux to not use its driver for some
> cards, then later loading the rtnet driver will be used for these
> cards? Adding the cards parameter means that the rtnet drivers
> diverge from mainline drivers, and so imply a maintenance cost. On
> the other hand, maybe that is a cheaper one than trying and solving
> the issue globally.

Certainly a well behaved driver would not touch a device already handled
by a linux driver.  It should be possible to just issue an unbind request
to the linux driver for the desired device and then a bind request to
the rtnet driver to take over that device.

But I have not looked at rtnet, so I don't know how those behave compared
to linux drivers.

-- 
Len Sorensen

Re: [Xenomai] Add cards parameter to rt_e1000e module?

[Gilles Chanteperdrix]

On Fri, Dec 12, 2014 at 09:12:23PM -0500, Lennart Sorensen wrote:
> On Sat, Dec 13, 2014 at 02:03:58AM +0100, Gilles Chanteperdrix wrote:
> > I understand the need for the "cards" parameter, but...
> > Is not there another way to solve this issue ? I do not know,
> > something in /proc or /sys to tell Linux to not load this driver for
> > that card? I suppose if we tell Linux to not use its driver for some
> > cards, then later loading the rtnet driver will be used for these
> > cards? Adding the cards parameter means that the rtnet drivers
> > diverge from mainline drivers, and so imply a maintenance cost. On
> > the other hand, maybe that is a cheaper one than trying and solving
> > the issue globally.
> 
> Certainly a well behaved driver would not touch a device already handled
> by a linux driver.  It should be possible to just issue an unbind request
> to the linux driver for the desired device and then a bind request to
> the rtnet driver to take over that device.
> 
> But I have not looked at rtnet, so I don't know how those behave compared
> to linux drivers.

As all RTDM drivers, RTnet drivers are just linux drivers. The issue
is to have a setup where two cards with the same chip are used by
two different drivers. Linux does not have this problem, because it
rarely has several drivers for the same chip. The problem with the
bind/unbind solution is that the driver that will get all the cards
first depends on the drivers initialization order. With the "cards"
parameter, you do not have this issue.

-- 
					    Gilles.

Re: [Xenomai] Add cards parameter to rt_e1000e module?

[Lennart Sorensen]

On Sat, Dec 13, 2014 at 06:11:51AM +0100, Gilles Chanteperdrix wrote:
> As all RTDM drivers, RTnet drivers are just linux drivers. The issue
> is to have a setup where two cards with the same chip are used by
> two different drivers. Linux does not have this problem, because it
> rarely has several drivers for the same chip. The problem with the
> bind/unbind solution is that the driver that will get all the cards
> first depends on the drivers initialization order. With the "cards"
> parameter, you do not have this issue.

Well I remember multiple drivers in the past for the tulip network chip,
and for a while there where two firewire drivers too.  Also those that
run xen with pci passthrough have the same issue of preventing linux
from grabbing a given device.

It would seem that just using bind/unbind and loading all the drivers,
user space could resolve the issue, but the cards parameter may be more
convinient, at the expense of having to add that to the driver of course.

-- 
Len Sorensen

Re: [Xenomai] Add cards parameter to rt_e1000e module?

[Jeff Webb]

On 12/12/2014 11:11 PM, Gilles Chanteperdrix wrote:
> On Fri, Dec 12, 2014 at 09:12:23PM -0500, Lennart Sorensen wrote:
>> On Sat, Dec 13, 2014 at 02:03:58AM +0100, Gilles Chanteperdrix wrote:
>>> I understand the need for the "cards" parameter, but...
>>> Is not there another way to solve this issue ? I do not know,
>>> something in /proc or /sys to tell Linux to not load this driver for
>>> that card? I suppose if we tell Linux to not use its driver for some
>>> cards, then later loading the rtnet driver will be used for these
>>> cards? Adding the cards parameter means that the rtnet drivers
>>> diverge from mainline drivers, and so imply a maintenance cost. On
>>> the other hand, maybe that is a cheaper one than trying and solving
>>> the issue globally.

I think the "cards" parameter is a pretty crude solution, too.  As you mentioned, this seems like a more general problem that Linux would already have a solution for, but my search for another solution turned up empty.

>> Certainly a well behaved driver would not touch a device already handled
>> by a linux driver.  It should be possible to just issue an unbind request
>> to the linux driver for the desired device and then a bind request to
>> the rtnet driver to take over that device.

Thanks for sharing this information.  It sounds like that might work.

> The problem with the
> bind/unbind solution is that the driver that will get all the cards
> first depends on the drivers initialization order. With the "cards"
> parameter, you do not have this issue.

Maybe one could create a script that would attempt to unbind both the rt and non-rt drivers (or check to see which one is bound and then unbind it), and then make the desired "bind" calls.  It would be better if the incorrect binding were not made in the first place, however.  I think the ideal solution would be to have a file that contains device/driver pairs that would keep the kernel from passing the specified devices to any other drivers.  I wonder how hard that would be to implement.

For what it's worth, I have these same issues with 16550 serial port drivers.  The 8250.nr_uarts parameter is helpful, but doesn't cover all use cases.

-Jeff