[Xenomai-help] How can I make Xenomai share a mii bus with the kernel?

[Xenomai-help] How can I make Xenomai share a mii bus with the kernel?

[Michael Galea]

I wonder if someone can suggest an approach to the following problem.

We need to use xenomai to manage devices on the mii bus.  The mii bus 
has PHYs managed by linux. What is the best way of providing access to 
both linux and xenomai?

I know that before xenomai "loads", I can simply access the mii bus from 
linux.  But after xenomai loads, is the right answer to make the linux 
function that reads the phy migrate into the xenomai domain and contend 
for the bus there?

Re: [Xenomai-help] How can I make Xenomai share a mii bus with the kernel?

[Gilles Chanteperdrix]

Michael Galea wrote:
> I wonder if someone can suggest an approach to the following problem.
> 
> We need to use xenomai to manage devices on the mii bus.  The mii bus 
> has PHYs managed by linux. What is the best way of providing access to 
> both linux and xenomai?
> 
> I know that before xenomai "loads", I can simply access the mii bus from 
> linux.  But after xenomai loads, is the right answer to make the linux 
> function that reads the phy migrate into the xenomai domain and contend 
> for the bus there?

What protects this access to the bus, is this a mutex or a spinlock ? If 
a spinlock, then simply make it an ipipe_spinlock, it will provide 
mutual exclusion between all domains.

I do not think it can be a mutex anyway, since it may certainly possible 
for a driver to have a function that reads the mii status in a timer 
handler to maintain the IFF_RUNNING bit state of the ethernet interface.

-- 
                                                  Gilles.

Re: [Xenomai-help] How can I make Xenomai share a mii bus with the kernel?

[Philippe Gerum]

Michael Galea wrote:
> I wonder if someone can suggest an approach to the following problem.
> 
> We need to use xenomai to manage devices on the mii bus.  The mii bus 
> has PHYs managed by linux. What is the best way of providing access to 
> both linux and xenomai?
> 
> I know that before xenomai "loads", I can simply access the mii bus from 
> linux.  But after xenomai loads, is the right answer to make the linux 
> function that reads the phy migrate into the xenomai domain and contend 
> for the bus there?
>

Problem is that Xenomai does not really compete with Linux, it preempts it
bluntly and rudely when it needs so, regardless of the Linux locking. So regular
Linux users of the PHY layer would start having problems, and your board would
happily jump out of the window.

Therefore, you would have to analyze and fix every callee from net/phy (and on)
to use Xenomai locking, instead of plain Linux locking constructs, at the very
least. Aside of the fact that this may be quite time-consuming and error-prone,
this would also de facto make the critical sections defined in the phy code
non-preemptible RT-wise, which may have some impact on the worst-case latency as
well. Sorry, this is co-kernel neighbourhood: we fancy behaving as masochistic
zombies around here.

Question: Is the Xenomai code calling into the phy layer a real-time beast, with
actual deadlines to be met, or not?

If yes, then you have no other choice than "ironing" the phy management code, so
that it can be entered from both the Linux and real-time spaces in a mutually
exclusive manner.

If there are no deadlines (lucky you), then you probably want to issue a request
(_not_ a function call, but some kind of APC) to the Linux space from your
Xenomai task (see rthal_apc_alloc and friends). But in the latter case, the
Xenomai task would lose any RT properties, because your code would likely want
to wait for completion of that request on the Linux side, most of the time.

The description above assumes your code lives in kernel space. In case that code
could live in userland, then you could use a more straightforward
implementation, based on a small RTDM driver forwarding PHY requests and sending
back the results to userland. Since Xenomai tasks in user-space may switch
automatically back and forth between the Linux and Xenomai domains, you would
not need to handle the quirks of APC handling. Still, you would not have any RT
guarantees for the call site, but maybe that's acceptable.

-- 
Philippe.

Re: [Xenomai-help] How can I make Xenomai share a mii bus with the kernel?

[Philippe Gerum]

Gilles Chanteperdrix wrote:
> Michael Galea wrote:
>> I wonder if someone can suggest an approach to the following problem.
>>
>> We need to use xenomai to manage devices on the mii bus.  The mii bus 
>> has PHYs managed by linux. What is the best way of providing access to 
>> both linux and xenomai?
>>
>> I know that before xenomai "loads", I can simply access the mii bus from 
>> linux.  But after xenomai loads, is the right answer to make the linux 
>> function that reads the phy migrate into the xenomai domain and contend 
>> for the bus there?
> 
> What protects this access to the bus, is this a mutex or a spinlock ? If 
> a spinlock, then simply make it an ipipe_spinlock, it will provide 
> mutual exclusion between all domains.
>

It's a spinlock_bh, but you can't tell actually, since there is a callback
mechanism to perform the actual read depending on the bus implementation.
Additionally, genphy_read may trigger an update from the link status and so on.
I'm afraid this kind of sport may well resemble playing Russian roulette with a
fully loaded gun. This said, if all executed code paths can be identified and
fixed, why not (having fun).

> I do not think it can be a mutex anyway, since it may certainly possible 
> for a driver to have a function that reads the mii status in a timer 
> handler to maintain the IFF_RUNNING bit state of the ethernet interface.
> 

-- 
Philippe.

Re: [Xenomai-help] How can I make Xenomai share a mii bus with the kernel?

[Gilles Chanteperdrix]

Philippe Gerum wrote:
> Gilles Chanteperdrix wrote:
>> Michael Galea wrote:
>>> I wonder if someone can suggest an approach to the following problem.
>>>
>>> We need to use xenomai to manage devices on the mii bus.  The mii bus 
>>> has PHYs managed by linux. What is the best way of providing access to 
>>> both linux and xenomai?
>>>
>>> I know that before xenomai "loads", I can simply access the mii bus from 
>>> linux.  But after xenomai loads, is the right answer to make the linux 
>>> function that reads the phy migrate into the xenomai domain and contend 
>>> for the bus there?
>> What protects this access to the bus, is this a mutex or a spinlock ? If 
>> a spinlock, then simply make it an ipipe_spinlock, it will provide 
>> mutual exclusion between all domains.
>>
> 
> It's a spinlock_bh, but you can't tell actually, since there is a callback
> mechanism to perform the actual read depending on the bus implementation.
> Additionally, genphy_read may trigger an update from the link status and so on.
> I'm afraid this kind of sport may well resemble playing Russian roulette with a
> fully loaded gun. This said, if all executed code paths can be identified and
> fixed, why not (having fun).

I was talking about the actual implementation by the hardware driver, 
not the generic part. This probably depends on the driver.


-- 
                                                  Gilles.

Re: [Xenomai-help] How can I make Xenomai share a mii bus with the kernel?

[Michael Galea]

Gilles Chanteperdrix wrote:
> Michael Galea wrote:
>> I wonder if someone can suggest an approach to the following problem.
>>
>> We need to use xenomai to manage devices on the mii bus.  The mii bus 
>> has PHYs managed by linux. What is the best way of providing access to 
>> both linux and xenomai?
>>
>> I know that before xenomai "loads", I can simply access the mii bus 
>> from linux.  But after xenomai loads, is the right answer to make the 
>> linux function that reads the phy migrate into the xenomai domain and 
>> contend for the bus there?
> 
> What protects this access to the bus, is this a mutex or a spinlock ? If 
> a spinlock, then simply make it an ipipe_spinlock, it will provide 
> mutual exclusion between all domains.
> 
> I do not think it can be a mutex anyway, since it may certainly possible 
> for a driver to have a function that reads the mii status in a timer 
> handler to maintain the IFF_RUNNING bit state of the ethernet interface.
> 
Thanks, access to the bus is provided by a spinlock in 
drivers/net/phy/phy.c, so I will try ipipe_spinlock (when I get hardware 
:-)).

I assume that I can make it an ipipe_spinlock and use that from linux 
even before xenomai is loaded.

Thanks

Re: [Xenomai-help] How can I make Xenomai share a mii bus with the kernel?

[Gilles Chanteperdrix]

Michael Galea wrote:
> Gilles Chanteperdrix wrote:
>> Michael Galea wrote:
>>> I wonder if someone can suggest an approach to the following problem.
>>>
>>> We need to use xenomai to manage devices on the mii bus.  The mii bus 
>>> has PHYs managed by linux. What is the best way of providing access to 
>>> both linux and xenomai?
>>>
>>> I know that before xenomai "loads", I can simply access the mii bus 
>>> from linux.  But after xenomai loads, is the right answer to make the 
>>> linux function that reads the phy migrate into the xenomai domain and 
>>> contend for the bus there?
>> What protects this access to the bus, is this a mutex or a spinlock ? If 
>> a spinlock, then simply make it an ipipe_spinlock, it will provide 
>> mutual exclusion between all domains.
>>
>> I do not think it can be a mutex anyway, since it may certainly possible 
>> for a driver to have a function that reads the mii status in a timer 
>> handler to maintain the IFF_RUNNING bit state of the ethernet interface.
>>
> Thanks, access to the bus is provided by a spinlock in 
> drivers/net/phy/phy.c, so I will try ipipe_spinlock (when I get hardware 
> :-)).
> 
> I assume that I can make it an ipipe_spinlock and use that from linux 
> even before xenomai is loaded.

No, this is not the spinlock I am talking about. I am talking about the 
spinlock (if it exists) used by the driver's mii read method.

-- 
                                                  Gilles.

Re: [Xenomai-help] How can I make Xenomai share a mii bus with the kernel?

[Philippe Gerum]

Gilles Chanteperdrix wrote:
> Philippe Gerum wrote:
>> Gilles Chanteperdrix wrote:
>>> Michael Galea wrote:
>>>> I wonder if someone can suggest an approach to the following problem.
>>>>
>>>> We need to use xenomai to manage devices on the mii bus.  The mii
>>>> bus has PHYs managed by linux. What is the best way of providing
>>>> access to both linux and xenomai?
>>>>
>>>> I know that before xenomai "loads", I can simply access the mii bus
>>>> from linux.  But after xenomai loads, is the right answer to make
>>>> the linux function that reads the phy migrate into the xenomai
>>>> domain and contend for the bus there?
>>> What protects this access to the bus, is this a mutex or a spinlock ?
>>> If a spinlock, then simply make it an ipipe_spinlock, it will provide
>>> mutual exclusion between all domains.
>>>
>>
>> It's a spinlock_bh, but you can't tell actually, since there is a
>> callback
>> mechanism to perform the actual read depending on the bus implementation.
>> Additionally, genphy_read may trigger an update from the link status
>> and so on.
>> I'm afraid this kind of sport may well resemble playing Russian
>> roulette with a
>> fully loaded gun. This said, if all executed code paths can be
>> identified and
>> fixed, why not (having fun).
> 
> I was talking about the actual implementation by the hardware driver,
> not the generic part. This probably depends on the driver.
> 
> 

The drivers don't do much, except IRQ handling. Most of them route their
requests to the generic PHY layer, therefore a parallel implementation would be
needed. Still, concurrency on the bus would not be solved.

-- 
Philippe.

Re: [Xenomai-help] How can I make Xenomai share a mii bus with the kernel?

[Gilles Chanteperdrix]

Philippe Gerum wrote:
> The drivers don't do much, except IRQ handling. Most of them route their
> requests to the generic PHY layer, therefore a parallel implementation would be
> needed. Still, concurrency on the bus would not be solved.

My reference ethernet driver is some proprietary crap, so I should stop 
assuming too much about its compliance to any Linux programming good rule...

-- 
                                                  Gilles.

Re: [Xenomai-help] How can I make Xenomai share a mii bus with the kernel?

[Michael Galea]

Philippe Gerum wrote:
> Michael Galea wrote:
>> I wonder if someone can suggest an approach to the following problem.
>>
>> We need to use xenomai to manage devices on the mii bus.  The mii bus 
>> has PHYs managed by linux. What is the best way of providing access to 
>> both linux and xenomai?
>>
>> I know that before xenomai "loads", I can simply access the mii bus from 
>> linux.  But after xenomai loads, is the right answer to make the linux 
>> function that reads the phy migrate into the xenomai domain and contend 
>> for the bus there?
>>
> 
> Problem is that Xenomai does not really compete with Linux, it preempts it
> bluntly and rudely when it needs so, regardless of the Linux locking. So regular
> Linux users of the PHY layer would start having problems, and your board would
> happily jump out of the window.
> 
> Therefore, you would have to analyze and fix every callee from net/phy (and on)
> to use Xenomai locking, instead of plain Linux locking constructs, at the very
> least. Aside of the fact that this may be quite time-consuming and error-prone,
> this would also de facto make the critical sections defined in the phy code
> non-preemptible RT-wise, which may have some impact on the worst-case latency as
> well. Sorry, this is co-kernel neighbourhood: we fancy behaving as masochistic
> zombies around here.
> 
> Question: Is the Xenomai code calling into the phy layer a real-time beast, with
> actual deadlines to be met, or not?
> 
Yes, it is.  We will be notified via interrupt that the part needs 
servicing and have to access it from xenomai.  Linux could be using the 
bus but mii operations are very quick, and for us, quite rare after boot.

> If yes, then you have no other choice than "ironing" the phy management code, so
> that it can be entered from both the Linux and real-time spaces in a mutually
> exclusive manner.
> 
The system in question is Freescale 8360 based and the mii bus is used 
to manage 5 Marvell 88e1111 Ethernet PHYs, never more, never less.  On 
my system its ucc_geth_phy that manages the bus.  I have looked at that 
and at the Marvell PHY driver source and neither use it doesn't use 
spinlocks.  Is that enough to predict that "ironing" the code is just an 
  ipipe_spinlock?

Where can I learn more about ipipe_spinlock?  I can only see the 
prototype in spinlock_types.h.


> If there are no deadlines (lucky you), then you probably want to issue a request
> (_not_ a function call, but some kind of APC) to the Linux space from your
> Xenomai task (see rthal_apc_alloc and friends). But in the latter case, the
> Xenomai task would lose any RT properties, because your code would likely want
> to wait for completion of that request on the Linux side, most of the time.
> 
> The description above assumes your code lives in kernel space. In case that code
> could live in userland, then you could use a more straightforward
> implementation, based on a small RTDM driver forwarding PHY requests and sending
> back the results to userland. Since Xenomai tasks in user-space may switch
> automatically back and forth between the Linux and Xenomai domains, you would
> not need to handle the quirks of APC handling. Still, you would not have any RT
> guarantees for the call site, but maybe that's acceptable.
> 

Re: [Xenomai-help] How can I make Xenomai share a mii bus with the kernel?

[Michael Galea]

Philippe Gerum wrote:
> Gilles Chanteperdrix wrote:
>> Philippe Gerum wrote:
>>> Gilles Chanteperdrix wrote:
>>>> Michael Galea wrote:
>>>>> I wonder if someone can suggest an approach to the following problem.
>>>>>
>>>>> We need to use xenomai to manage devices on the mii bus.  The mii
>>>>> bus has PHYs managed by linux. What is the best way of providing
>>>>> access to both linux and xenomai?
>>>>>
>>>>> I know that before xenomai "loads", I can simply access the mii bus
>>>>> from linux.  But after xenomai loads, is the right answer to make
>>>>> the linux function that reads the phy migrate into the xenomai
>>>>> domain and contend for the bus there?
>>>> What protects this access to the bus, is this a mutex or a spinlock ?
>>>> If a spinlock, then simply make it an ipipe_spinlock, it will provide
>>>> mutual exclusion between all domains.
>>>>
>>> It's a spinlock_bh, but you can't tell actually, since there is a
>>> callback
>>> mechanism to perform the actual read depending on the bus implementation.
>>> Additionally, genphy_read may trigger an update from the link status
>>> and so on.
>>> I'm afraid this kind of sport may well resemble playing Russian
>>> roulette with a
>>> fully loaded gun. This said, if all executed code paths can be
>>> identified and
>>> fixed, why not (having fun).
>> I was talking about the actual implementation by the hardware driver,
>> not the generic part. This probably depends on the driver.
>>
>>
> 
> The drivers don't do much, except IRQ handling. Most of them route their
> requests to the generic PHY layer, therefore a parallel implementation would be
> needed. Still, concurrency on the bus would not be solved.
> 

My boards driver, ucc_geth_mii, seems to do even less. A write looks like.

/* Setting up the MII Mangement Address Register */
out_be32(&regs->miimadd, (mii_id << MIIMADD_PHY_ADDRESS_SHIFT) | regnum);

/* Setting up the MII Mangement Control Register with the value */
out_be32(&regs->miimcon, value);

/* Wait till MII management write is complete */
while ((in_be32(&regs->miimind)) & MIIMIND_BUSY) cpu_relax();

I imagine that this is what would have to move to a "parallel 
implementation" part.  Could I use the ipipe_spinlock to ensure 
exclusion between xenomai and linux for the bus?

Re: [Xenomai-help] How can I make Xenomai share a mii bus with the kernel?

[Gilles Chanteperdrix]

Michael Galea wrote:
> /* Wait till MII management write is complete */
> while ((in_be32(&regs->miimind)) & MIIMIND_BUSY) cpu_relax();

This is no good. Putting this under spinlock may mean large masking
sections.

-- 


					    Gilles.