Re: bnxt_en: Incorrect tx timestamp report

[Vadim Fedorenko <vadim.fedorenko@linux.dev>]

On 20/03/2025 15:56, Pavan Chebbi wrote:
> On Thu, Mar 20, 2025 at 8:07 PM Kamil Zaripov <zaripov-kamil@avride.ai> wrote:
>>
>> Hi all,
>>
>> I've encountered a bug in the bnxt_en driver and I am unsure about the correct approach to fix it. Every 2^48 nanoseconds (or roughly 78.19 hours) there is a probability that the hardware timestamp for a sent packet may deviate by either 2^48 nanoseconds less or 2^47 nanoseconds more compared to the actual time.
>>
>> This issue likely occurs within the bnxt_async_event_process function when handling the ASYNC_EVENT_CMPL_EVENT_ID_PHC_UPDATE event. It appears that the payload of this event contains bits 48–63 of the PHC timer counter. During event handling, this function reads bits 0–47 of the same counter to combine them and subsequently updates the cycle_last field within the struct timecounter. The relevant code can be found here:
>> https://elixir.bootlin.com/linux/v6.13.7/source/drivers/net/ethernet/broadcom/bnxt/bnxt.c#L2829-L2833
>>
>> The issue arises if bits 48–63 of the PHC counter increment by 1 between sending the ASYNC_EVENT_CMPL_EVENT_ID_PHC_UPDATE event and its actual handling by the driver. In such a case, cycle_last becomes approximately 2^48 nanoseconds behind the real-time value.
>>
>> A possibly related issue involves the BCM57502 network card, which seemingly possesses only a single PHC device. However, the bnxt_en driver creates four PHC Linux devices when operating in quad-port mode. Consequently, clock synchronization daemons like phc2sys attempt to independently synchronize the system clock to each of these four PHC clocks. This scenario can lead to unstable synchronization and might also trigger additional ASYNC_EVENT_CMPL_EVENT_ID_PHC_UPDATE events.
>>
>> Given these issues, I have two questions:
>>
>> 1. Would it be beneficial to modify the bnxt_en driver to create only a single PHC Linux device for network cards that physically have only one PHC?
> 
> It's not clear to me if you are facing this issue when the PHC is
> shared between multiple hosts or if you are running a single host NIC.
> In the cases where a PHC is shared across multiple hosts, the driver
> identifies such a configuration and switches to non-real time PHC
> access mode.
> https://web.git.kernel.org/pub/scm/linux/kernel/git/netdev/net-next.git/commit/drivers/net/ethernet/broadcom/bnxt?id=85036aee1938d65da4be6ae1bc7e5e7e30b567b9
> If you are using a configuration like the multi host, can you please
> make sure you have this patch?
> 
> Let me know if you are not in the multi-host config. Do post the
> ethtool -i output to help know the firmware version.

AFAIU, the setup is single host, but multi port NIC, which exports
several PTP devices, all of them are using RTC mode. But as HW has 
single physical PHC, it's not possible to properly discipline all
of PTP devices in parallel. I think mlx5 was adjusted to export only
single PHC device for multi-port configuration because of the very same
reasons.