From: Ivan Vecera <ivecera@redhat.com>
To: Jakub Kicinski <kuba@kernel.org>
Cc: netdev@vger.kernel.org,
Vadim Fedorenko <vadim.fedorenko@linux.dev>,
Arkadiusz Kubalewski <arkadiusz.kubalewski@intel.com>,
Jiri Pirko <jiri@resnulli.us>, Jonathan Corbet <corbet@lwn.net>,
Shuah Khan <skhan@linuxfoundation.org>,
Randy Dunlap <rdunlap@infradead.org>,
Min Li <min.li@microchip.com>,
Tony Nguyen <anthony.l.nguyen@intel.com>,
Przemek Kitszel <przemyslaw.kitszel@intel.com>,
Andrew Lunn <andrew+netdev@lunn.ch>,
"David S. Miller" <davem@davemloft.net>,
Eric Dumazet <edumazet@kernel.org>,
Paolo Abeni <pabeni@redhat.com>,
Richard Cochran <richardcochran@gmail.com>,
linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
intel-wired-lan@lists.osuosl.org,
Grzegorz Nitka <grzegorz.nitka@intel.com>,
Karol Kolacinski <karol.kolacinski@intel.com>,
Saeed Mahameed <saeedm@nvidia.com>,
Tariq Toukan <tariqt@nvidia.com>, Petr Oros <poros@redhat.com>
Subject: Re: [PATCH net-next v2 0/9] dpll: do not serialize driver callbacks under global dpll_lock
Date: Thu, 19 Nov 2026 13:29:11 +0100 [thread overview]
Message-ID: <dd1d61d7-7bef-4a02-8518-e17d2f50369f@redhat.com> (raw)
In-Reply-To: <20261118174424.32e9f130@kernel.org>
On 11/19/26 02:44, Jakub Kicinski wrote:
> On Wed, 18 Nov 2026 16:29:23 +0100 Ivan Vecera wrote:
>> A step of the zl3073x PHC, which ptp4l and ts2phc do at start and after
>> a large offset, waits for the next 1 PPS edge, up to 1 s, with the
>> channel lock held. A phase step can hold it for up to 3 s while it
>> waits for the previous one. A dpll callback of that channel waits for
>> the lock under dpll_lock, and every other DPLL request and notification
>> in the system waits behind it. With ToD writes running, requests to an
>> ice DPLL took 1448 ms in median and 3.9 s at most. With the series they
>> took 0.2 ms and 0.3 ms, the same as idle.
>>
>> The bigger reason are zl3073x callbacks we are working on, which have
>> to wait for the hardware themselves:
>>
>> - a device reset with resync waits for the fast lock, up to 1 s with a
>> 1 PPS input
>> - a glitchless stop or start of an output completes on an output edge,
>> so it takes up to one period, 1 s for a 1 PPS output
>>
>> Holding the global dpll_lock for that long would stall every DPLL
>> device in the system. For the output state we tried an asynchronous
>> implementation, but then the callback returns success before it knows
>> the result.
>
> Right, for these level of waiting async seems like the right direction,
> at least naively/without knowing the HW. Why does the device wait until
> PPS to apply the step change? Is the programmed time supposed to
> correspond to the pulse?
Yes. The device has no immediate ToD write, it can only write the ToD
on the next internal 1 Hz edge, the next input sync edge or the next
internal 0.5 Hz edge. The written value is the time at that edge, so
the seconds stay aligned with the 1 PPS outputs.
Looking at it again, the zl3073x holds the channel lock while it waits
for the edge, although the dpll callbacks do not touch the ToD, TIE or
phase step registers, and they already run while the device applies
such an operation, because the PTP ops return right after issuing it.
I will look at narrowing the lock in the driver.
For the resync, async is not only about reporting the result. Fast lock
is a setting of the DPLL, not of a reference, and forcing it makes the
DPLL ignore its frequency and phase error monitors. While an
asynchronous resync is pending, everything else done to the DPLL runs
in forced fast lock. A reference switch, whether from a priority or
state change, a manual reference or the device itself in automatic
mode, would follow the fast lock phase rate instead of the configured
phase slope limit, and the resync would then report success on another
reference. A switch to NCO would keep the force bit set until the work
clears it. So every request changing the DPLL configuration would have
to fail with -EBUSY, cancel the resync, or wait for it. With a lock per
chip they simply wait, for at most 1 s.
>> A failure can only be reported by a later notification, and
>> the next request gets -EBUSY until the change completes. With a lock
>> per chip the callback can simply wait, and one chip cannot do these
>> operations in parallel anyway.
>
> Do the failures only happen because of "unforeseen circumstances" / EIO,
> or there may be more common errors reported? If it's just the former
> a devlink health reporter could be an option.
For the output start and stop, only EIO. The device does not report a
stopped output and the driver can only wait for one period, so a health
reporter would work there.
For the resync, no. The wait ends with -ETIMEDOUT when the DPLL does
not enter fast lock, for example because the selected reference lost
its qualification in the meantime. That is a result of the input
signal the user needs in the reply, not a device fault.
Beyond the zl3073x, with the global lock any callback that waits for
its device, for example an mlx5 firmware command with its 60 s timeout,
stalls the DPLL devices of every other driver in the system.
Thanks,
Ivan
next prev parent reply other threads:[~2026-11-19 12:29 UTC|newest]
Thread overview: 21+ messages / expand[flat|nested] mbox.gz Atom feed top
2026-11-11 16:40 [PATCH net-next v2 0/9] dpll: do not serialize driver callbacks under global dpll_lock Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 1/9] ice: dpll: notify TSPLL lock status from the periodic work only Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 2/9] dpll: add domain object Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 3/9] dpll: take domain locks around driver callbacks Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 4/9] dpll: convert dpll_lock to rwsem Ivan Vecera
2026-11-16 8:21 ` Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 5/9] dpll: add KUnit tests for core locking Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 6/9] dpll: reject measured_freq_get for foreign and on-pin registrations Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 7/9] dpll: zl3073x: update locking comments Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 8/9] ice: dpll: " Ivan Vecera
2026-11-16 8:22 ` Ivan Vecera
2026-11-11 16:40 ` [PATCH net-next v2 9/9] dpll: document locking rules Ivan Vecera
2026-11-16 8:23 ` Ivan Vecera
2026-11-16 8:24 ` [PATCH net-next v2 0/9] dpll: do not serialize driver callbacks under global dpll_lock Ivan Vecera
2026-11-17 2:17 ` Jakub Kicinski
2026-11-18 15:30 ` Ivan Vecera
2026-11-17 2:16 ` Jakub Kicinski
2026-11-18 15:29 ` Ivan Vecera
2026-11-19 1:44 ` Jakub Kicinski
2026-11-19 12:29 ` Ivan Vecera [this message]
2026-11-19 21:38 ` Jakub Kicinski
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=dd1d61d7-7bef-4a02-8518-e17d2f50369f@redhat.com \
--to=ivecera@redhat.com \
--cc=andrew+netdev@lunn.ch \
--cc=anthony.l.nguyen@intel.com \
--cc=arkadiusz.kubalewski@intel.com \
--cc=corbet@lwn.net \
--cc=davem@davemloft.net \
--cc=edumazet@kernel.org \
--cc=grzegorz.nitka@intel.com \
--cc=intel-wired-lan@lists.osuosl.org \
--cc=jiri@resnulli.us \
--cc=karol.kolacinski@intel.com \
--cc=kuba@kernel.org \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=min.li@microchip.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=poros@redhat.com \
--cc=przemyslaw.kitszel@intel.com \
--cc=rdunlap@infradead.org \
--cc=richardcochran@gmail.com \
--cc=saeedm@nvidia.com \
--cc=skhan@linuxfoundation.org \
--cc=tariqt@nvidia.com \
--cc=vadim.fedorenko@linux.dev \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox