Seeking guidance on detecting mount point inactivity

Seeking guidance on detecting mount point inactivity

[Ali Nasrolahi]

[-- Attachment #1.1: Type: text/plain, Size: 1277 bytes --]

Hello everyone,

I am working on a  problem involving managing some number of filesystem
mounts (~1000+) and determining when a mount becomes inactive to do some
maintenance.
To be specific, the goal is to detect when a mounted filesystem has no
remaining active file usage, and after it remains inactive for some period
(e.g. 10 seconds),
transition it into a state where new file opens are blocked so that
maintenance operations can safely proceed.
It must handle concurrent access from arbitrary applications, so
correctness under races is quite challenging.

I have looked into userspace based approaches and tools for observing file
usage, but they seem difficult to make fully correct in edge cases such
as process forking and descriptor inheritance, which makes it hard to
reliably implement the “last reference + grace period + safe transition”.

At this point I suspect this problem should be tied to kernel-level mount
lifecycle semantics rather than something that can be reliably done in
userspace alone.

However, I am not sure which mailing-list (if any) is appropriate to
present this kind of problem/question.
So, I am mainly looking for guidance on where to take this discussion to
seek some sort of advice.

Thanks a lot,
Ali

[-- Attachment #1.2: Type: text/html, Size: 1393 bytes --]

[-- Attachment #2: Type: text/plain, Size: 170 bytes --]

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Seeking guidance on detecting mount point inactivity

[Richard]

Hi,

On 25/06/2026 16:37, Ali Nasrolahi wrote:
> Hello everyone,
> 
> I am working on a  problem involving managing some number of filesystem 
> mounts (~1000+) and determining when a mount becomes inactive to do some 
Honestly this does not sound like a good/sane setup to start with. Maybe 
zooming out and solving the overall requirement with a different setup 
might be smarter.

> maintenance.
> To be specific, the goal is to detect when a mounted filesystem has no 
> remaining active file usage, and after it remains inactive for some 
> period (e.g. 10 seconds),
> transition it into a state where new file opens are blocked so that 
> maintenance operations can safely proceed.
> It must handle concurrent access from arbitrary applications, so 
> correctness under races is quite challenging.

Why do you care about the filesystem level? If this is about device 
maintendance device activity might be more relevant than fs activity. 
(Remember there is caching and lots of other abstraction). It might also 
be easier to track bio requests for a specific device on the device level.

I don't know tools from the top of my head but I'm sure there are files 
for this in /sys/. There is also iotop and you can see its source code 
where it gets its info from. I'd also recommend reading brendan greggs 
System Performance book. It has 2 chapters on disks and block I/O, I'm 
sure you'll find good diagnostic tools there.

Best,
-- Richard

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Seeking guidance on detecting mount point inactivity

[Ali Nasrolahi]

Hi

On Tue, Jun 30, 2026 at 2:21 PM Richard <richard_siegfried@systemli.org> wrote:
>
> Hi,
>
> On 25/06/2026 16:37, Ali Nasrolahi wrote:
> > Hello everyone,
> >
> > I am working on a  problem involving managing some number of filesystem
> > mounts (~1000+) and determining when a mount becomes inactive to do some
> Honestly this does not sound like a good/sane setup to start with. Maybe
> zooming out and solving the overall requirement with a different setup
> might be smarter.
>
> > To be specific, the goal is to detect when a mounted filesystem has no
> > remaining active file usage, and after it remains inactive for some
> > period (e.g. 10 seconds),
> > transition it into a state where new file opens are blocked so that
> > maintenance operations can safely proceed.
> > It must handle concurrent access from arbitrary applications, so
> > correctness under races is quite challenging.
>
> Why do you care about the filesystem level? If this is about device
> maintendance device activity might be more relevant than fs activity.
> (Remember there is caching and lots of other abstraction). It might also
> be easier to track bio requests for a specific device on the device level.
>
> I don't know tools from the top of my head but I'm sure there are files
> for this in /sys/. There is also iotop and you can see its source code
> where it gets its info from. I'd also recommend reading brendan greggs
> System Performance book. It has 2 chapters on disks and block I/O, I'm
> sure you'll find good diagnostic tools there.
>
> Best,
> -- Richard

Thank you for taking the time to reply.

My maintenance is actually tied to the VFS rather than the block layer.
Ultimately I have to determine when an `umount()` is likely to succeed,
which isn't necessarily the same as "no block I/O."

For example, a process may still hold an open file or other kernel reference
on the mount while generating no bio activity at all, in which case
`umount()` would still fail.
On the other hand, even with determining the no block activity, I
still need some mechanism
to prevent new accesses from racing with the maintenance process.

Also, the maintenance itself is also not necessarily related to the
underlying block device.
In my use case, I often don't need to take the device offline at all.
For example, I want to unmount a logical volume on one node and
activate it on another node in a cluster,
or temporarily unmount a loopback-backed filesystem so that the
backing file can be moved or backed up.

Also, I do appreciate your suggestion on System Performance, I would
surely check it out.

Regards,
Ali

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Seeking guidance on detecting mount point inactivity

[Raka Gunarto]

On Fri, Jul 3, 2026 at 10:05 AM Ali Nasrolahi <a.nasrolahi01@gmail.com> wrote:
>
>
> Thank you for taking the time to reply.
>
> My maintenance is actually tied to the VFS rather than the block layer.
> Ultimately I have to determine when an `umount()` is likely to succeed,
> which isn't necessarily the same as "no block I/O."
>
> For example, a process may still hold an open file or other kernel reference
> on the mount while generating no bio activity at all, in which case
> `umount()` would still fail.
> On the other hand, even with determining the no block activity, I
> still need some mechanism
> to prevent new accesses from racing with the maintenance process.
>

Is moving the mount to another path or lazy unmount not an option?

After moving the mount, you can wait until there are no more open
handles (since all subsequent open calls would be blocked). Then you
can do your operations and then restore the mount at the original
location.

Lazy unmounting would work the same way, wait until it properly
unmounts, and then you remount at your maintenance location.

If you don't want to disrupt currently running operations, you could
use the fanotify API and keep track of events and determine an
activity threshold to move the mount / lazy unmount. Or perhaps an
eBPF program that hooks into the FS tracepoints?

If your workloads can afford to wait indefinitely during an open()
call, you can also just use the fanotify API to delay any calls to
open() until your maintenance program finishes too, this wouldn't
require moving the mount or unmounting.

Raka

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Seeking guidance on detecting mount point inactivity

[Ali Nasrolahi]

Thank you for the suggestions.

On 26/07/03 10:54AM, Raka Gunarto wrote:
> Is moving the mount to another path or lazy unmount not an option?
> 
> After moving the mount, you can wait until there are no more open
> handles (since all subsequent open calls would be blocked). Then you
> can do your operations and then restore the mount at the original
> location.
> 
> Lazy unmounting would work the same way, wait until it properly
> unmounts, and then you remount at your maintenance location.

Lazy unmounting, moving the mount to another location, or even
techniques such as `chmod 000` (for non-root users) would all eventually
drain the mount of references. My concern is slightly different. I don't
want to start draining the mount immediately after observing that it has
no references at a particular instant, because that alone is not strong
evidence that the workload is actually inactive or ,more accurately,
likely to be inactive.

For example, a workload may repeatedly open a file, perform a small
amount of work, close it, sleep for a short time, and then repeat. If I
immediately started draining the mount every time the reference count
reached zero, the system would constantly interrupt some workload.

That is the whole reason behind the grace period. Rather than treating
"reference count reached zero" as sufficient, I only consider the mount
a possible maintenance candidate if it has remained without active
references for some time. The goal is not to prove inactivity, but to
make it much more likely that the workload has genuinely gone idle.

> If you don't want to disrupt currently running operations, you could
> use the fanotify API and keep track of events and determine an
> activity threshold to move the mount / lazy unmount. Or perhaps an
> eBPF program that hooks into the FS tracepoints?

Well, this is actually my direction. I even wrote a small PoC eBPF
program that attaches to LSM hooks to account for opens and reject
new ones during maintenance. However, it has its own challenges,
specifically, some nasty races. For example, if monitoring begins after
some file objects are already open, the accounting is immediately off. As
a result, the design becomes closer to a best-effort mechanism than one
that can guarantee a successful `umount()`.

These kind of challenges are exactly the points I hope to discuss.

Regards,
Ali

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Seeking guidance on detecting mount point inactivity

[Raka Gunarto]

On Sun, Jul 5, 2026 at 6:04 AM Ali Nasrolahi <a.nasrolahi01@gmail.com> wrote:
> For example, a workload may repeatedly open a file, perform a small
> amount of work, close it, sleep for a short time, and then repeat. If I
> immediately started draining the mount every time the reference count
> reached zero, the system would constantly interrupt some workload.

Are your workloads okay to wait indefinitely on a call to open()? I
believe you can use the fanotify permission events (FAN_OPEN_PERM) to
queue the call and then poll for active handles until they close.
After the maintenance operations then respond FAN_ALLOW to the
requests in the queue.

Raka

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies

Re: Seeking guidance on detecting mount point inactivity

[Ali Nasrolahi]

On 26/07/05 05:34PM, Raka Gunarto wrote:
> Are your workloads okay to wait indefinitely on a call to open()? I
> believe you can use the fanotify permission events (FAN_OPEN_PERM) to
> queue the call and then poll for active handles until they close.
> After the maintenance operations then respond FAN_ALLOW to the
> requests in the queue.

To be honest, I don't have a concrete idea of how different workloads
would behave if an `open()` were blocked for an extended period, but I
think they should be alright.

Could you elaborate a bit on what you mean by polling the active
handles?  My understanding is that, regardless of the enforcement
mechanism (fanotify, eBPF, or something else) the challenges of
determining that a mount has actually become inactive (and remained so
for the grace period) are still there, right? same ebpf issues, what if
the daemon didn't count the existing handles or somehow crashes and lose
the count. These challenges are common; or am I overlooking something?

--Ali

_______________________________________________
Kernelnewbies mailing list
Kernelnewbies@kernelnewbies.org
https://lists.kernelnewbies.org/mailman/listinfo/kernelnewbies