[PATCH net-next v3 2/2] net: Adding support for Cavium ThunderX network controller

[PATCH net-next v3 2/2] net: Adding support for Cavium ThunderX network controller

[Paul Bolle]

Just a nit: a license mismatch.

On Fri, 2015-05-15 at 20:36 -0700, Aleksey Makarov wrote:
> --- /dev/null
> +++ b/drivers/net/ethernet/cavium/thunder/nic_main.c

> + * This program is free software; you can redistribute it and/or
> + * modify it under the terms of the GNU General Public License as
> + * published by the Free Software Foundation; either version 2 of
> + * the License, or (at your option) any later version.

This states the license is GPL v2 or later. (The other files used for
this module contain an identical comment.)

> +MODULE_LICENSE("GPL v2");

And, according to include/linux/module.h, this states the license is
just GPL v2. So I think that either the license used in the comment at
the top of those files or the ident used in the MODULE_LICENSE() macro
needs to change.

Ditto for the thunder_bgx.ko and the nicvf.ko module.

Thanks,


Paul Bolle

[PATCH net-next v3 2/2] net: Adding support for Cavium ThunderX network controller

[David Miller]

From: Aleksey Makarov <aleksey.makarov@auriga.com>
Date: Fri, 15 May 2015 20:36:39 -0700

> +/* Register read/write APIs */
> +static void nic_reg_write(struct nicpf *nic, u64 offset, u64 val)
> +{
> +	writeq_relaxed(val, nic->reg_base + offset);
> +}
> +
> +static u64 nic_reg_read(struct nicpf *nic, u64 offset)
> +{
> +	return readq_relaxed(nic->reg_base + offset);
> +}

Are you really sure it's OK to used relaxed ordering for all register
accesses like this?

Personally, I think it's asking for trouble.

Maybe in _extremely_ specific situations in the packet processing
fast path where you can clearly define the ordering needs when
programming the mailbox registers, I'd say it's OK.

But universally across the entire driver?  No way, no way at all.

[PATCH net-next v3 2/2] net: Adding support for Cavium ThunderX network controller

[David Daney]

On 05/18/2015 01:09 PM, David Miller wrote:
> From: Aleksey Makarov <aleksey.makarov@auriga.com>
> Date: Fri, 15 May 2015 20:36:39 -0700
>
>> +/* Register read/write APIs */
>> +static void nic_reg_write(struct nicpf *nic, u64 offset, u64 val)
>> +{
>> +	writeq_relaxed(val, nic->reg_base + offset);
>> +}
>> +
>> +static u64 nic_reg_read(struct nicpf *nic, u64 offset)
>> +{
>> +	return readq_relaxed(nic->reg_base + offset);
>> +}
>
> Are you really sure it's OK to used relaxed ordering for all register
> accesses like this?

Yes.

>
> Personally, I think it's asking for trouble.
>
 > Maybe in _extremely_ specific situations in the packet processing
 > fast path where you can clearly define the ordering needs when
 > programming the mailbox registers, I'd say it's OK.
 >
 > But universally across the entire driver?  No way, no way at all.
 >

I think for an ordinary NIC you would be 100% correct.  However, ...

... there is something that may not be evident from the patch:

The Cavium ThunderX network controller can *only* be found in SoCs 
containing the ThunderX ARM64 CPU implementation.  So, we know quite a 
bit (i.e. everything) about the memory ordering semantics of the system.

A slightly more detailed explanation of why it is safe to use 
writeq_relaxed/readq_relaxed is that all accesses to the device 
registers are implicitly strongly ordered with respect to memory 
accesses, so even though the function names imply otherwise, there is no 
relaxed ordering.  The readq/writeq functions add explicit ordering 
operation which in this case are redundant, and only add overhead.

So this leaves us with the question:  What should we do?

As I see it there are two options:

1) Keep the writeq_relaxed()/readq_relaxed(), but add a comment about 
why they are safe.

2) Change the patch so that we are using writeq()/readq() and suffer a 
decrease in performance.

I/we wouldn't object to either of these, so it is really up to you. 
Please let us know what you think the best way forward is.

Thanks,
David Daney

[PATCH net-next v3 2/2] net: Adding support for Cavium ThunderX network controller

[David Miller]

From: David Daney <ddaney@caviumnetworks.com>
Date: Mon, 18 May 2015 14:52:30 -0700

> As I see it there are two options:
> 
> 1) Keep the writeq_relaxed()/readq_relaxed(), but add a comment about
> why they are safe.
> 
> 2) Change the patch so that we are using writeq()/readq() and suffer a
> decrease in performance.

I want a detailed comment added, so #1 would be my choice.

There are more than one set of register accessors, so make sure you
annotate them all.

Thanks.