Copying between lzo compressed BtrFS's: de/re-compressing.

Copying between lzo compressed BtrFS's: de/re-compressing.

[Diagon]

I'm copying a large number of files between two lzo compressed BtrFS
filesystems on different drives mounted on the same machine. It appears
that the files are being de/re-compressed. Is there a way avoid this?

Thanks
/D

Re: Copying between lzo compressed BtrFS's: de/re-compressing.

[Timofey Titovets]

2016-01-15 12:00 GMT+03:00 Diagon <kernel.boxy@xoxy.net>:
> I'm copying a large number of files between two lzo compressed BtrFS
> filesystems on different drives mounted on the same machine. It appears
> that the files are being de/re-compressed. Is there a way avoid this?

If you just copy files, files will be decompressed while reading and
recompressed while writing
For avoiding this, you must use send receive feature


-- 
Have a nice day,
Timofey.

Re: Copying between lzo compressed BtrFS's: de/re-compressing.

[Filipe Manana]

On Fri, Jan 15, 2016 at 9:56 AM, Timofey Titovets <nefelim4ag@gmail.com> wrote:
> 2016-01-15 12:00 GMT+03:00 Diagon <kernel.boxy@xoxy.net>:
>> I'm copying a large number of files between two lzo compressed BtrFS
>> filesystems on different drives mounted on the same machine. It appears
>> that the files are being de/re-compressed. Is there a way avoid this?
>
> If you just copy files, files will be decompressed while reading and
> recompressed while writing
> For avoiding this, you must use send receive feature

No, send/receive does not avoid decompression on the send side nor
re-compression at the receiving side. The receiving side writes the
data from a send stream, which is uncompressed, to the destination
filesystem using standard system calls like write/pwrite.

>
>
> --
> Have a nice day,
> Timofey.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html



-- 
Filipe David Manana,

"Reasonable men adapt themselves to the world.
 Unreasonable men adapt the world to themselves.
 That's why all progress depends on unreasonable men."

Re: Copying between lzo compressed BtrFS's: de/re-compressing.

[Timofey Titovets]

Thanks, Filipe ^_^

2016-01-15 15:25 GMT+03:00 Filipe Manana <fdmanana@gmail.com>:
> On Fri, Jan 15, 2016 at 9:56 AM, Timofey Titovets <nefelim4ag@gmail.com> wrote:
>> 2016-01-15 12:00 GMT+03:00 Diagon <kernel.boxy@xoxy.net>:
>>> I'm copying a large number of files between two lzo compressed BtrFS
>>> filesystems on different drives mounted on the same machine. It appears
>>> that the files are being de/re-compressed. Is there a way avoid this?
>>
>> If you just copy files, files will be decompressed while reading and
>> recompressed while writing
>> For avoiding this, you must use send receive feature
>
> No, send/receive does not avoid decompression on the send side nor
> re-compression at the receiving side. The receiving side writes the
> data from a send stream, which is uncompressed, to the destination
> filesystem using standard system calls like write/pwrite.
>
>>
>>
>> --
>> Have a nice day,
>> Timofey.
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>
>
> --
> Filipe David Manana,
>
> "Reasonable men adapt themselves to the world.
>  Unreasonable men adapt the world to themselves.
>  That's why all progress depends on unreasonable men."



-- 
Have a nice day,
Timofey.

Re: Copying between lzo compressed BtrFS's: de/re-compressing.

[Diagon]

On 01/15/2016 04:25 AM, Filipe Manana wrote:
> On Fri, Jan 15, 2016 at 9:56 AM, Timofey Titovets <nefelim4ag@gmail.com> wrote:

>> 2016-01-15 12:00 GMT+03:00 Diagon <kernel.boxy@xoxy.net>:

>>> I'm copying a large number of files between two lzo compressed BtrFS
>>> filesystems on different drives mounted on the same machine. It appears
>>> that the files are being de/re-compressed. Is there a way avoid this?
>>
>> If you just copy files, files will be decompressed while reading and
>> recompressed while writing
>> For avoiding this, you must use send receive feature
> 
> No, send/receive does not avoid decompression on the send side nor
> re-compression at the receiving side. The receiving side writes the
> data from a send stream, which is uncompressed, to the destination
> filesystem using standard system calls like write/pwrite.

So, just to spell it out, I understand that de/re-compression is
unavoidable?  (And I'll post that to my stackexchange question.)

>> Have a nice day,
>> Timofey.

Thanks
/D.

Re: Copying between lzo compressed BtrFS's: de/re-compressing.

[Chris Murphy]

On Sun, Jan 17, 2016 at 1:42 PM, Diagon <kernel.boxy@xoxy.net> wrote:
> On 01/15/2016 04:25 AM, Filipe Manana wrote:
>> On Fri, Jan 15, 2016 at 9:56 AM, Timofey Titovets <nefelim4ag@gmail.com> wrote:
>
>>> 2016-01-15 12:00 GMT+03:00 Diagon <kernel.boxy@xoxy.net>:
>
>>>> I'm copying a large number of files between two lzo compressed BtrFS
>>>> filesystems on different drives mounted on the same machine. It appears
>>>> that the files are being de/re-compressed. Is there a way avoid this?
>>>
>>> If you just copy files, files will be decompressed while reading and
>>> recompressed while writing
>>> For avoiding this, you must use send receive feature
>>
>> No, send/receive does not avoid decompression on the send side nor
>> re-compression at the receiving side. The receiving side writes the
>> data from a send stream, which is uncompressed, to the destination
>> filesystem using standard system calls like write/pwrite.
>
> So, just to spell it out, I understand that de/re-compression is
> unavoidable?  (And I'll post that to my stackexchange question.)

It's unavoidable when replicating data across file systems
(send/receive, rsync, cp). It's avoidable if you use a seed device,
add a new device, then delete the seed. While the volume UUID changes
with this process, the duplicate volume is essentially the same as the
source (the process copies chunks), which means the chunk profile is
also preserved.

-- 
Chris Murphy

Re: Copying between lzo compressed BtrFS's: de/re-compressing.

[Diagon]

On 01/17/2016 01:27 PM, Chris Murphy - lists@colorremedies.com wrote:
> On Sun, Jan 17, 2016 at 1:42 PM, Diagon <kernel.boxy@xoxy.net> wrote:
>> On 01/15/2016 04:25 AM, Filipe Manana wrote:
>>> On Fri, Jan 15, 2016 at 9:56 AM, Timofey Titovets <nefelim4ag@gmail.com> wrote:
>>
>>>> 2016-01-15 12:00 GMT+03:00 Diagon <kernel.boxy@xoxy.net>:
>>
>>>>> I'm copying a large number of files between two lzo compressed BtrFS
>>>>> filesystems on different drives mounted on the same machine. It appears
>>>>> that the files are being de/re-compressed. Is there a way avoid this?
>>>>
>>>> If you just copy files, files will be decompressed while reading and
>>>> recompressed while writing
>>>> For avoiding this, you must use send receive feature
>>>
>>> No, send/receive does not avoid decompression on the send side nor
>>> re-compression at the receiving side. The receiving side writes the
>>> data from a send stream, which is uncompressed, to the destination
>>> filesystem using standard system calls like write/pwrite.
>>
>> So, just to spell it out, I understand that de/re-compression is
>> unavoidable?  (And I'll post that to my stackexchange question.)
> 
> It's unavoidable when replicating data across file systems
> (send/receive, rsync, cp). 

Got it.

> It's avoidable if you use a seed device,
> add a new device, then delete the seed. While the volume UUID changes
> with this process, the duplicate volume is essentially the same as the
> source (the process copies chunks), which means the chunk profile is
> also preserved.

Fantastic!  I could have actually used this on a new build by using the
installer to build into a new subvolume and then moving the files over.

I'll know next time.  Much appreciated.

/D