* [linux-lvm] lvm2: zeroing free space
@ 2011-02-04 14:12 Bryn M. Reeves
2011-02-04 15:54 ` Mike Snitzer
0 siblings, 1 reply; 6+ messages in thread
From: Bryn M. Reeves @ 2011-02-04 14:12 UTC (permalink / raw)
To: LVM general discussion and development
Hi Folks,
Recently had a query from someone using an array supporting thin provisioning
with zero page reclaim[1].
They wanted a method to write zeros to all the unallocated space in a VG to
trigger a reclaim of unused regions in the VG's PVs.
My suggestion was to lvcreate a -l100%FREE LV named "filler" and overwrite it
from /dev/zero.
It seems like this might something a lot of users want as this functionality
gets to be more common.
I wondered if it could be worth adding a script to automate this or even an
option to vgchange (e.g. --zero-free-space)?
Didn't have time to try putting anything together yet but it seems such a
feature could be useful.
Regards,
Bryn.
[1] http://blogs.hds.com/hu/2009/03/zero-page-reclaim.html
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [linux-lvm] lvm2: zeroing free space
2011-02-04 14:12 [linux-lvm] lvm2: zeroing free space Bryn M. Reeves
@ 2011-02-04 15:54 ` Mike Snitzer
2011-02-04 16:21 ` hansbkk
2011-02-04 16:37 ` Bryn M. Reeves
0 siblings, 2 replies; 6+ messages in thread
From: Mike Snitzer @ 2011-02-04 15:54 UTC (permalink / raw)
To: LVM general discussion and development
On Fri, Feb 04 2011 at 9:12am -0500,
Bryn M. Reeves <bmr@redhat.com> wrote:
> Hi Folks,
>
> Recently had a query from someone using an array supporting thin provisioning
> with zero page reclaim[1].
>
> They wanted a method to write zeros to all the unallocated space in a VG to
> trigger a reclaim of unused regions in the VG's PVs.
>
> My suggestion was to lvcreate a -l100%FREE LV named "filler" and overwrite it
> from /dev/zero.
>
> It seems like this might something a lot of users want as this functionality
> gets to be more common.
>
> I wondered if it could be worth adding a script to automate this or even an
> option to vgchange (e.g. --zero-free-space)?
>
> Didn't have time to try putting anything together yet but it seems such a
> feature could be useful.
Do these HDS arrays also support discard? Would seem to me that issuing
discards for the free space would be better (more standard) than writing
zeros to accomplish the same.
This variant would be: vgchange --discard-free-space
Mike
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [linux-lvm] lvm2: zeroing free space
2011-02-04 15:54 ` Mike Snitzer
@ 2011-02-04 16:21 ` hansbkk
2011-02-04 16:37 ` Bryn M. Reeves
1 sibling, 0 replies; 6+ messages in thread
From: hansbkk @ 2011-02-04 16:21 UTC (permalink / raw)
To: LVM general discussion and development
On Fri, Feb 4, 2011 at 10:54 PM, Mike Snitzer <snitzer@redhat.com> wrote:
> On Fri, Feb 04 2011 at �9:12am -0500,
> Bryn M. Reeves <bmr@redhat.com> wrote:
>> They wanted a method to write zeros to all the unallocated space in a VG to
>> trigger a reclaim of unused regions in the VG's PVs.
Probably not directly relevant, but perhaps of interest to those who
haven't come across it before.
Zerofree is designed to do this for loopback filesystems; I came
across it in the context of union/aufs overlays, via Grml's changelog
http://intgat.tigress.co.uk/rmy/uml/sparsify.html
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [linux-lvm] lvm2: zeroing free space
2011-02-04 15:54 ` Mike Snitzer
2011-02-04 16:21 ` hansbkk
@ 2011-02-04 16:37 ` Bryn M. Reeves
2011-02-04 20:02 ` Martin K. Petersen
2011-02-08 2:10 ` wayne.berthiaume
1 sibling, 2 replies; 6+ messages in thread
From: Bryn M. Reeves @ 2011-02-04 16:37 UTC (permalink / raw)
To: LVM general discussion and development; +Cc: Mike Snitzer
On 02/04/2011 03:54 PM, Mike Snitzer wrote:
> On Fri, Feb 04 2011 at 9:12am -0500,
> Bryn M. Reeves <bmr@redhat.com> wrote:
>
>> Hi Folks,
>>
>> Recently had a query from someone using an array supporting thin provisioning
>> with zero page reclaim[1].
>>
>> They wanted a method to write zeros to all the unallocated space in a VG to
>> trigger a reclaim of unused regions in the VG's PVs.
>>
>> My suggestion was to lvcreate a -l100%FREE LV named "filler" and overwrite it
>> from /dev/zero.
>>
>> It seems like this might something a lot of users want as this functionality
>> gets to be more common.
>>
>> I wondered if it could be worth adding a script to automate this or even an
>> option to vgchange (e.g. --zero-free-space)?
>>
>> Didn't have time to try putting anything together yet but it seems such a
>> feature could be useful.
>
> Do these HDS arrays also support discard? Would seem to me that issuing
> discards for the free space would be better (more standard) than writing
> zeros to accomplish the same.
>
> This variant would be: vgchange --discard-free-space
There are two variants of reclamation currently supported by external storage
products.
The WRITE_SAME/TRIM based explicit discard support is superior when the entire
storage stack supports it and I agree it would be good to support this mode of
reclaim too (superior in that you lose the zero-write overhead and associated
reclaim lag and can get better space utilisation since the host "knows" better
which blocks are unused).
Several vendors (at least HDS/EMC that I know of) also support an implicit
reclaim mode known as zero page/space reclaim in which the storage de-allocates
blocks that are overwritten with zeros.
I suppose the ideal solution would be to detect whether the storage supports one
or the other automatically but I don't know that there is a way to achieve this
right now (TRIM/WRITE_SAME possibly but I think the zero page stuff is entirely
transparent).
Regards,
Bryn.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [linux-lvm] lvm2: zeroing free space
2011-02-04 16:37 ` Bryn M. Reeves
@ 2011-02-04 20:02 ` Martin K. Petersen
2011-02-08 2:10 ` wayne.berthiaume
1 sibling, 0 replies; 6+ messages in thread
From: Martin K. Petersen @ 2011-02-04 20:02 UTC (permalink / raw)
To: LVM general discussion and development; +Cc: Mike Snitzer
>>>>> "Bryn" == Bryn M Reeves <bmr@redhat.com> writes:
Bryn> I suppose the ideal solution would be to detect whether the
Bryn> storage supports one or the other automatically but I don't know
Bryn> that there is a way to achieve this right now (TRIM/WRITE_SAME
Bryn> possibly but I think the zero page stuff is entirely transparent).
The problem with the zero-detection stuff is that it predates all the
thin provisioning goo in SCSI and we have no way of enabling
automatically. So it would have to be a manual per-device override.
However, all the vendors I have talked to prefer to use the
standards-based way now that it's there. And that's why I removed the
"write zeroes" code path before the original TP support went
upstream. It was there in the first few iterations of patches.
If people think it's a good idea I'll be happy to wire it back up. I
have a new set of TP patches going out shortly.
--
Martin K. Petersen Oracle Linux Engineering
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [linux-lvm] lvm2: zeroing free space
2011-02-04 16:37 ` Bryn M. Reeves
2011-02-04 20:02 ` Martin K. Petersen
@ 2011-02-08 2:10 ` wayne.berthiaume
1 sibling, 0 replies; 6+ messages in thread
From: wayne.berthiaume @ 2011-02-08 2:10 UTC (permalink / raw)
To: linux-lvm; +Cc: snitzer
You might be able to detect at least explicit via the VPD page.
Regards,
Wayne.
-----Original Message-----
From: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com] On Behalf Of Bryn M. Reeves
Sent: Friday, February 04, 2011 11:37 AM
To: LVM general discussion and development
Cc: Mike Snitzer
Subject: Re: [linux-lvm] lvm2: zeroing free space
On 02/04/2011 03:54 PM, Mike Snitzer wrote:
> On Fri, Feb 04 2011 at 9:12am -0500,
> Bryn M. Reeves <bmr@redhat.com> wrote:
>
>> Hi Folks,
>>
>> Recently had a query from someone using an array supporting thin provisioning
>> with zero page reclaim[1].
>>
>> They wanted a method to write zeros to all the unallocated space in a VG to
>> trigger a reclaim of unused regions in the VG's PVs.
>>
>> My suggestion was to lvcreate a -l100%FREE LV named "filler" and overwrite it
>> from /dev/zero.
>>
>> It seems like this might something a lot of users want as this functionality
>> gets to be more common.
>>
>> I wondered if it could be worth adding a script to automate this or even an
>> option to vgchange (e.g. --zero-free-space)?
>>
>> Didn't have time to try putting anything together yet but it seems such a
>> feature could be useful.
>
> Do these HDS arrays also support discard? Would seem to me that issuing
> discards for the free space would be better (more standard) than writing
> zeros to accomplish the same.
>
> This variant would be: vgchange --discard-free-space
There are two variants of reclamation currently supported by external storage
products.
The WRITE_SAME/TRIM based explicit discard support is superior when the entire
storage stack supports it and I agree it would be good to support this mode of
reclaim too (superior in that you lose the zero-write overhead and associated
reclaim lag and can get better space utilisation since the host "knows" better
which blocks are unused).
Several vendors (at least HDS/EMC that I know of) also support an implicit
reclaim mode known as zero page/space reclaim in which the storage de-allocates
blocks that are overwritten with zeros.
I suppose the ideal solution would be to detect whether the storage supports one
or the other automatically but I don't know that there is a way to achieve this
right now (TRIM/WRITE_SAME possibly but I think the zero page stuff is entirely
transparent).
Regards,
Bryn.
_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2011-02-08 2:10 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-02-04 14:12 [linux-lvm] lvm2: zeroing free space Bryn M. Reeves
2011-02-04 15:54 ` Mike Snitzer
2011-02-04 16:21 ` hansbkk
2011-02-04 16:37 ` Bryn M. Reeves
2011-02-04 20:02 ` Martin K. Petersen
2011-02-08 2:10 ` wayne.berthiaume
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).