Can moving data to a subvolume not take as long as a fully copy?

Can moving data to a subvolume not take as long as a fully copy?

[Marc MERLIN]

I made a mistake and copied data in the root of a new btrfs filesystem.
I created a subvolume, and used mv to put everything in there.
Something like:
cd /mnt
btrfs subvolume create dir
mv * dir

Except it's been running for over a day now (ok, it's 5TB of data)

Looks like mv is really copying all the data as if it were an entirely
different filesystem.

Is there not a way to short circuit this and only update the metadata?

Thanks,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: Can moving data to a subvolume not take as long as a fully copy?

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1015 bytes --]

On Mon, Jan 14, 2013 at 09:32:50AM -0800, Marc MERLIN wrote:
> I made a mistake and copied data in the root of a new btrfs filesystem.
> I created a subvolume, and used mv to put everything in there.
> Something like:
> cd /mnt
> btrfs subvolume create dir
> mv * dir
> 
> Except it's been running for over a day now (ok, it's 5TB of data)
> 
> Looks like mv is really copying all the data as if it were an entirely
> different filesystem.
> 
> Is there not a way to short circuit this and only update the metadata?

   I guess the best way of doing this in this case is to teach mv to
do cp --reflink=always then unlink the origin.

   Clearly that won't work over mount boundaries (where a copy of the
data is the best you're going to get), but that's not what you've got
here.

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
       --- He's playing Schubert.  I think Schubert is losing. ---       

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: Can moving data to a subvolume not take as long as a fully copy?

[Marc MERLIN]

[-- Attachment #1: Type: text/plain, Size: 1350 bytes --]

On Mon, Jan 14, 2013 at 05:41:55PM +0000, Hugo Mills wrote:
> On Mon, Jan 14, 2013 at 09:32:50AM -0800, Marc MERLIN wrote:
> > I made a mistake and copied data in the root of a new btrfs filesystem.
> > I created a subvolume, and used mv to put everything in there.
> > Something like:
> > cd /mnt
> > btrfs subvolume create dir
> > mv * dir
> > 
> > Except it's been running for over a day now (ok, it's 5TB of data)
> > 
> > Looks like mv is really copying all the data as if it were an entirely
> > different filesystem.
> > 
> > Is there not a way to short circuit this and only update the metadata?
> 
>    I guess the best way of doing this in this case is to teach mv to
> do cp --reflink=always then unlink the origin.
> 
>    Clearly that won't work over mount boundaries (where a copy of the
> data is the best you're going to get), but that's not what you've got
> here.

Mmmh, this made me think:
It seems that I could have done cp --reflink without duplicating the data
and running out of space.
Then, I could have deleted the originals?

Is that correct?

Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 308 bytes --]

Re: Can moving data to a subvolume not take as long as a fully copy?

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 1541 bytes --]

On Mon, Jan 14, 2013 at 10:00:41AM -0800, Marc MERLIN wrote:
> On Mon, Jan 14, 2013 at 05:41:55PM +0000, Hugo Mills wrote:
> > On Mon, Jan 14, 2013 at 09:32:50AM -0800, Marc MERLIN wrote:
> > > I made a mistake and copied data in the root of a new btrfs filesystem.
> > > I created a subvolume, and used mv to put everything in there.
> > > Something like:
> > > cd /mnt
> > > btrfs subvolume create dir
> > > mv * dir
> > > 
> > > Except it's been running for over a day now (ok, it's 5TB of data)
> > > 
> > > Looks like mv is really copying all the data as if it were an entirely
> > > different filesystem.
> > > 
> > > Is there not a way to short circuit this and only update the metadata?
> > 
> >    I guess the best way of doing this in this case is to teach mv to
> > do cp --reflink=always then unlink the origin.
> > 
> >    Clearly that won't work over mount boundaries (where a copy of the
> > data is the best you're going to get), but that's not what you've got
> > here.
> 
> Mmmh, this made me think:
> It seems that I could have done cp --reflink without duplicating the data
> and running out of space.
> Then, I could have deleted the originals?
> 
> Is that correct?

   Yup, exactly what I just said above. :)

   Hugo.

-- 
=== Hugo Mills: hugo@... carfax.org.uk | darksatanic.net | lug.org.uk ===
  PGP key: 515C238D from wwwkeys.eu.pgp.net or http://www.carfax.org.uk
      --- Your problem is that you've got too much taste to be ---       
                            a web developer.                             

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 828 bytes --]

Re: Can moving data to a subvolume not take as long as a fully copy?

[David Brown]

Marc MERLIN <marc@merlins.org> writes:

> I made a mistake and copied data in the root of a new btrfs filesystem.
> I created a subvolume, and used mv to put everything in there.
> Something like:
> cd /mnt
> btrfs subvolume create dir
> mv * dir
>
> Except it's been running for over a day now (ok, it's 5TB of data)
>
> Looks like mv is really copying all the data as if it were an entirely
> different filesystem.
>
> Is there not a way to short circuit this and only update the metadata?

Why not make a snapshot of the root volume, and then delete the files
you want to move from the original root, and delete the rest of root
from the snapshot?

David

Re: Can moving data to a subvolume not take as long as a fully copy?

[Marc MERLIN]

On Mon, Jan 14, 2013 at 10:48:50PM -0800, David Brown wrote:
> Why not make a snapshot of the root volume, and then delete the files
> you want to move from the original root, and delete the rest of root
> from the snapshot?

Are a snapshot of the root volume and a subvolume effectively the same thing
as far as btrfs sees them?
Once I have that snapshot which I'll treat as a subvolume, can I then
snapshot that snapshot/subvolume further?

If so, that's definitely a good way of doing this for next time.

Thanks,
Marc
-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
Microsoft is to operating systems ....
                                      .... what McDonalds is to gourmet cooking
Home page: http://marc.merlins.org/  

Re: Can moving data to a subvolume not take as long as a fully copy?

[Mitch Harder]

On Tue, Jan 15, 2013 at 8:49 AM, Marc MERLIN <marc@merlins.org> wrote:
> On Mon, Jan 14, 2013 at 10:48:50PM -0800, David Brown wrote:
>> Why not make a snapshot of the root volume, and then delete the files
>> you want to move from the original root, and delete the rest of root
>> from the snapshot?
>
> Are a snapshot of the root volume and a subvolume effectively the same thing
> as far as btrfs sees them?
> Once I have that snapshot which I'll treat as a subvolume, can I then
> snapshot that snapshot/subvolume further?
>

Yes, the product of the btrfs snapshot command is a subvolume.