dbus-sensors issue

dbus-sensors issue

[Jeff9 Chan(陳松儉_ASRockRack)]

[-- Attachment #1: Type: text/plain, Size: 634 bytes --]

Hi all,
It appears that IO wait is very high all the time. This can be viewed by running `top`. It was traced down to dbus-sensors systemd units. When the units are running, IO wait is >90%. This can impact system performance and also impacts CPU utilization telemetry, since the idle task never runs, which normally means CPU is being completely utilized.

My platform uses below services, stop them will free the IO usage, any idea?

xyz.openbmc_project.hwmontempsensor.service
xyz.openbmc_project.fansensor.service
xyz.openbmc_project.adcsensor.service
xyz.openbmc_project.psusensor.service


best regards

Jeff Chan

[-- Attachment #2: Type: text/html, Size: 3913 bytes --]

Re: [External] dbus-sensors issue

[Jian Zhang]

See:
https://lore.kernel.org/lkml/20230716195007.731909670@linuxfoundation.org/
https://bugzilla.kernel.org/show_bug.cgi?id=217699
For services utilizing io_uring, when waiting for I/O, CPU time is counted as I/O rather than idle time.

Jian.
> From: "Jeff9 Chan(陳松儉_ASRockRack)"<Jeff9_Chan@asrockrack.com>
> Date:  Fri, Feb 7, 2025, 19:42
> Subject:  [External] dbus-sensors issue
> To: "openbmc@lists.ozlabs.org"<openbmc@lists.ozlabs.org>
> Hi all,
> It appears that IO wait is very high all the time. This can be viewed by running `top`. It was traced down to dbus-sensors systemd units. When the units are running, IO
 wait is >90%. This can impact system performance and also impacts CPU utilization telemetry, since the idle task never runs, which normally means CPU is being completely utilized.
>  
> My platform uses below services, stop them will free the IO usage, any idea?
>  
> xyz.openbmc_project.hwmontempsensor.service
> xyz.openbmc_project.fansensor.service
> xyz.openbmc_project.adcsensor.service
> xyz.openbmc_project.psusensor.service
>  
>  
> best regards
>  
> Jeff Chan

Re: [External] dbus-sensors issue

[Patrick Williams]

[-- Attachment #1: Type: text/plain, Size: 1729 bytes --]

On Fri, Feb 07, 2025 at 07:54:28PM +0800, Jian Zhang wrote:
> See:
> https://lore.kernel.org/lkml/20230716195007.731909670@linuxfoundation.org/
> https://bugzilla.kernel.org/show_bug.cgi?id=217699
> For services utilizing io_uring, when waiting for I/O, CPU time is counted as I/O rather than idle time.

Yep.  So to be clear, this is just an accounting difference.  It has no
real measurement of a performance difference.  "io_wait" is effectively
"idle" now.

For similar reasons, phosphor-health-monitor doesn't consider 'iowait'
part of the CPU utilization for determining if the CPU has been too busy
for too long:

https://github.com/openbmc/phosphor-health-monitor/blob/ba29dbdd04a96a46e1972641c20bf5a8e930ed8f/health_metric_collection.cpp#L73

> 
> Jian.
> > From: "Jeff9 Chan(陳松儉_ASRockRack)"<Jeff9_Chan@asrockrack.com>
> > Date:  Fri, Feb 7, 2025, 19:42
> > Subject:  [External] dbus-sensors issue
> > To: "openbmc@lists.ozlabs.org"<openbmc@lists.ozlabs.org>
> > Hi all,
> > It appears that IO wait is very high all the time. This can be viewed by running `top`. It was traced down to dbus-sensors systemd units. When the units are running, IO
>  wait is >90%. This can impact system performance and also impacts CPU utilization telemetry, since the idle task never runs, which normally means CPU is being completely utilized.
> >  
> > My platform uses below services, stop them will free the IO usage, any idea?
> >  
> > xyz.openbmc_project.hwmontempsensor.service
> > xyz.openbmc_project.fansensor.service
> > xyz.openbmc_project.adcsensor.service
> > xyz.openbmc_project.psusensor.service
> >  
> >  
> > best regards
> >  
> > Jeff Chan

-- 
Patrick Williams

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

RE: [External] dbus-sensors issue

[Jeff9 Chan(陳松儉_ASRockRack)]

Thanks a lot!

best regards

Jeff Chan

-----Original Message-----
From: patrick@stwcx.xyz <patrick@stwcx.xyz> 
Sent: Saturday, February 8, 2025 1:04 AM
To: Jian Zhang <zhangjian.3032@bytedance.com>
Cc: Jeff9 Chan(陳松儉_ASRockRack) <Jeff9_Chan@asrockrack.com>; openbmc@lists.ozlabs.org
Subject: Re: [External] dbus-sensors issue

On Fri, Feb 07, 2025 at 07:54:28PM +0800, Jian Zhang wrote:
> See:
> https://lore.kernel.org/lkml/20230716195007.731909670@linuxfoundation.
> org/
> https://bugzilla.kernel.org/show_bug.cgi?id=217699
> For services utilizing io_uring, when waiting for I/O, CPU time is counted as I/O rather than idle time.

Yep.  So to be clear, this is just an accounting difference.  It has no real measurement of a performance difference.  "io_wait" is effectively "idle" now.

For similar reasons, phosphor-health-monitor doesn't consider 'iowait'
part of the CPU utilization for determining if the CPU has been too busy for too long:

https://github.com/openbmc/phosphor-health-monitor/blob/ba29dbdd04a96a46e1972641c20bf5a8e930ed8f/health_metric_collection.cpp#L73

> 
> Jian.
> > From: "Jeff9 Chan(陳松儉_ASRockRack)"<Jeff9_Chan@asrockrack.com>
> > Date:  Fri, Feb 7, 2025, 19:42
> > Subject:  [External] dbus-sensors issue
> > To: "openbmc@lists.ozlabs.org"<openbmc@lists.ozlabs.org>
> > Hi all,
> > It appears that IO wait is very high all the time. This can be 
> > viewed by running `top`. It was traced down to dbus-sensors systemd 
> > units. When the units are running, IO
>  wait is >90%. This can impact system performance and also impacts CPU utilization telemetry, since the idle task never runs, which normally means CPU is being completely utilized.
> >  
> > My platform uses below services, stop them will free the IO usage, any idea?
> >  
> > xyz.openbmc_project.hwmontempsensor.service
> > xyz.openbmc_project.fansensor.service
> > xyz.openbmc_project.adcsensor.service
> > xyz.openbmc_project.psusensor.service
> >  
> >  
> > best regards
> >  
> > Jeff Chan

--
Patrick Williams