drill: Make a file sparse without using extra disk space

drill: Make a file sparse without using extra disk space

[Rodrigo Campos]

Hi!

I've written a small simple tool to make a file sparse without using extra disk
space, it just digs holes on the file when possible. To dig the hole uses the
Linux-specific fallocate(2) syscall with FALLOC_FL_PUNCH_HOLE. So, also, it only
works on some file-systems.

Do you think that util-linux would be a good place for such a simple, linux-only
program ? The case I see it can be useful is to use on VM images file backed (or
I wrote it with this in mind, at least :)).

The code is quite simple, it's written in plain C (C99 actually) and doesn't
have any external dependencies. You can see the gitweb here:

	http://sdfg.com.ar/git/?p=drill.git

or just clone it from:

	git://sdfg.com.ar/drill.git


Given that there is a fallocate tool that uses the other mode of fallocate(2)
already, all the detection for fallocate is already there. So it should, I
guess, be easy to integrate to the code-base, adding also the #ifndef
fallocate.c has and that compatibility stuff.

Also, as you can see in the commits, I just started it some days ago. So if
you think it should have some other feature, or something else, please let me
know :)

I'm not sure how this work, but I can write a patch against util-linux if you
think it can be added. If you know there is something I should particulary take
into account for that, I will really appreciate if you let me know :)




Thanks a lot,
Rodrigo

Re: drill: Make a file sparse without using extra disk space

[Pádraig Brady]

On 01/23/2014 09:28 PM, Rodrigo Campos wrote:
> Hi!
> 
> I've written a small simple tool to make a file sparse without using extra disk
> space, it just digs holes on the file when possible. To dig the hole uses the
> Linux-specific fallocate(2) syscall with FALLOC_FL_PUNCH_HOLE. So, also, it only
> works on some file-systems.
> 
> Do you think that util-linux would be a good place for such a simple, linux-only
> program ? The case I see it can be useful is to use on VM images file backed (or
> I wrote it with this in mind, at least :)).
> 
> The code is quite simple, it's written in plain C (C99 actually) and doesn't
> have any external dependencies. You can see the gitweb here:
> 
> 	http://sdfg.com.ar/git/?p=drill.git
> 
> or just clone it from:
> 
> 	git://sdfg.com.ar/drill.git
> 
> 
> Given that there is a fallocate tool that uses the other mode of fallocate(2)
> already, all the detection for fallocate is already there. So it should, I
> guess, be easy to integrate to the code-base, adding also the #ifndef
> fallocate.c has and that compatibility stuff.
> 
> Also, as you can see in the commits, I just started it some days ago. So if
> you think it should have some other feature, or something else, please let me
> know :)
> 
> I'm not sure how this work, but I can write a patch against util-linux if you
> think it can be added. If you know there is something I should particulary take
> into account for that, I will really appreciate if you let me know :)
> 
> 
> 
> 
> Thanks a lot,
> Rodrigo
> --
> To unsubscribe from this list: send the line "unsubscribe util-linux" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

This functionality is useful.
Would it be possible to just add to fallocate with --punch-hole=sparse or something?
Also dd has the conv=sparse option, so maybe this functionality can be added there.
We're currently discussing that on the GNU coreutils mailing list.

thanks,
Pádraig.

Re: drill: Make a file sparse without using extra disk space

[Rodrigo Campos]

On Fri, Jan 24, 2014 at 01:00:55AM +0000, Pádraig Brady wrote:
> On 01/23/2014 09:28 PM, Rodrigo Campos wrote:
> > Hi!
> > 
> > I've written a small simple tool to make a file sparse without using extra disk
> > space, it just digs holes on the file when possible. To dig the hole uses the
> > Linux-specific fallocate(2) syscall with FALLOC_FL_PUNCH_HOLE. So, also, it only
> > works on some file-systems.
> > 
> > Do you think that util-linux would be a good place for such a simple, linux-only
> > program ? The case I see it can be useful is to use on VM images file backed (or
> > I wrote it with this in mind, at least :)).
> > 
> This functionality is useful.
> Would it be possible to just add to fallocate with --punch-hole=sparse or something?

Totally, that's an option too :)

> Also dd has the conv=sparse option, so maybe this functionality can be added there.

But how are you going to do a hole in-place in a portable fashion ? I don't know
of any

> We're currently discussing that on the GNU coreutils mailing list.

You mean the thread "making a file sparse - in-place?" started after this thread ?




Thanks a lot,
Rodrigo

Re: drill: Make a file sparse without using extra disk space

[Karel Zak]

On Fri, Jan 24, 2014 at 02:16:02AM +0000, Rodrigo Campos wrote:
> On Fri, Jan 24, 2014 at 01:00:55AM +0000, Pádraig Brady wrote:
> > On 01/23/2014 09:28 PM, Rodrigo Campos wrote:
> > > I've written a small simple tool to make a file sparse without using extra disk
> > > space, it just digs holes on the file when possible.

 Nice idea.

> > > To dig the hole uses the
> > > Linux-specific fallocate(2) syscall with FALLOC_FL_PUNCH_HOLE. So, also, it only
> > > works on some file-systems.
> > > 
> > > Do you think that util-linux would be a good place for such a simple, linux-only
> > > program ? The case I see it can be useful is to use on VM images file backed (or
> > > I wrote it with this in mind, at least :)).
> > > 
> > This functionality is useful.
> > Would it be possible to just add to fallocate with --punch-hole=sparse or something?
> 
> Totally, that's an option too :)

 It would be the best way for util-linux, I really don't want to add
 another util for fallocate.

 Maybe we can use a new option --dig-holes [ --length <size> ] or so.

> > Also dd has the conv=sparse option, so maybe this functionality can be added there.

 IMHO would be possible to add this functionality to both places ;-)

    Karel

-- 
 Karel Zak  <kzak@redhat.com>
 http://karelzak.blogspot.com

Re: drill: Make a file sparse without using extra disk space

[Rodrigo Campos]

On Fri, Jan 24, 2014 at 10:34:48AM +0100, Karel Zak wrote:
> On Fri, Jan 24, 2014 at 02:16:02AM +0000, Rodrigo Campos wrote:
> > On Fri, Jan 24, 2014 at 01:00:55AM +0000, Pádraig Brady wrote:
> > > On 01/23/2014 09:28 PM, Rodrigo Campos wrote:
> > > > I've written a small simple tool to make a file sparse without using extra disk
> > > > space, it just digs holes on the file when possible.
> 
>  Nice idea.

Thanks! Not really original, though :)

> 
> > > > To dig the hole uses the
> > > > Linux-specific fallocate(2) syscall with FALLOC_FL_PUNCH_HOLE. So, also, it only
> > > > works on some file-systems.
> > > > 
> > > > Do you think that util-linux would be a good place for such a simple, linux-only
> > > > program ? The case I see it can be useful is to use on VM images file backed (or
> > > > I wrote it with this in mind, at least :)).
> > > > 
> > > This functionality is useful.
> > > Would it be possible to just add to fallocate with --punch-hole=sparse or something?
> > 
> > Totally, that's an option too :)
> 
>  It would be the best way for util-linux, I really don't want to add
>  another util for fallocate.

Great!

> 
>  Maybe we can use a new option --dig-holes [ --length <size> ] or so.

Sounds fine, but needs to be mutually exclusive with the options it takes now, I
think. Or maybe something can be shared...

Will try to come up with a patch during the weekend or next week. Thanks!

> > > Also dd has the conv=sparse option, so maybe this functionality can be added there.
> 
>  IMHO would be possible to add this functionality to both places ;-)

:-D





Thanks a lot,
Rodrigo