Re: RFC on writel and writel_relaxed

[Sinan Kaya <okaya@codeaurora.org>]

On 3/20/2018 10:40 PM, Oliver wrote:
>> I see that network drivers are working around the issue by calling
>> __raw_write() API directly but this also breaks other architectures
>> like SPARC since the semantics of __raw_writel() seems to be system dependent.
> Yeah that's pretty gross. Which drivers are doing this?
> 

Searching for __raw_writel() and BIG_ENDIAN in drivers/net directory
should give you the idea.

In a nutshell, drivers are doing this today.

wmb()
__raw_writel();
mmiowb()

I'm in the process of posting patches ([1], [2], [3]) to various subsystems to
eliminate double barriers by replacing sequences like 

wmb()
writel()
mmiowb()

with

wmb()
writel_relaxed()
mmiowb()

Reviewers pointed out that writel_relaxed() is not __raw_writel() on PPC
and cannot take advantage of the optimization. 

Replacing writel_relaxed() with raw_writel() or vice versa is not correct.

writel_relaxed() needs to have ordering guarantees with respect to the order
device observes writes. 

x86 has compiler barrier inside the relaxed() API so that code does not
get reordered. ARM64 architecturally guarantees device writes to be observed
in order.

I was hoping that PPC could follow x86 and inject compiler barrier into the
relaxed functions. 

BTW, I have no idea what compiler barrier does on PPC and if

wrltel() == compiler barrier() + wrltel_relaxed()

can be said.

Need guidance from the PPC maintainers. 

[1] https://www.spinics.net/lists/netdev/msg490480.html
[2] https://www.spinics.net/lists/arm-kernel/msg642341.html
[3] https://www.spinics.net/lists/arm-kernel/msg642336.html

-- 
Sinan Kaya
Qualcomm Datacenter Technologies, Inc. as an affiliate of Qualcomm Technologies, Inc.
Qualcomm Technologies, Inc. is a member of the Code Aurora Forum, a Linux Foundation Collaborative Project.