trim calls not occurring during device removal operation

[Eric Levy <contact@ericlevy.name>]

I recently removed a device from a filesystem. All the devices in the 
file system were thin-provisioned LUNs. From the LUN manager, I was 
able to monitor the space allocated for each LUN. I noticed that during 
the operation, as the file system consumed increasing amounts of space 
from the other devices (from transferring the data off the one being 
removed), none of the deallocated space was freed on the backend for 
the device targeted for removal. Apparently, the file system was not 
calling a hardware trim operation for freed space.

For physical devices, such behavior might not be problematic. However, 
for logical devices backed on storage with usage approaching capacity, 
it is preferred to free space as soon as possible. If a single 
operation consumes increasing space on the backend but frees none, then 
the space on the backed may become depleted before the operation 
completes.

The file system had been mounted with the `discard` option set to 
`async`. In addition to noticing that the option was not causing trim 
calls to be made during the remove operation, I also noticed that calls 
to the `fstrim` system utility targeting the mounted file system did 
not cause any of the space to be trimmed.

In order to avoid overconsumption of space on the backend, it would be 
helpful to trim space as it is freed on a device targeted for an 
ongoing removal operation. Even if the backend space is not imediately 
needed, freeing the space as part of the removal operation would leave 
the device in a state that clearly signals to the LUN adminstrator that 
the LUN is no longer storing data, and so may be safely destroyed. It 
reduces the chance of data loss by human error if an administrator may 
understand that a LUN may be safely destroyed if and only if it is not 
showing as having allocated space.

Would you consider adding more robust support for trim operations as 
part of device removal?

All of my observations occurred under kernel 5.15.