* btrfs send / receive via netcat, fails halfway?
@ 2021-01-10 10:34
2021-01-10 11:07 ` Forza
` (2 more replies)
0 siblings, 3 replies; 6+ messages in thread
From: @ 2021-01-10 10:34 UTC (permalink / raw)
To: linux-btrfs
I'm trying to transfer a btrfs snapshot via the network.
First attempt: Both NC programs don't exit after the transfer is complete. When I ctrl-C the sending side, the receiving side exits OK.
btrfs subvolume delete /mnt/rec/snapshots/*
receive side:
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At subvol 0
sending side:
# btrfs send /mnt/send/snapshots/0 | nc -v 127.0.0.1 6790
At subvol /mnt/send/snapshots/0
localhost [127.0.0.1] 6790 (hnmp) open
Second attempt: both nc programs exit ok at snapshot 0,1,2, but snapshot3 fails halfway, and 4 fails, as 3 is not complete.
receive side:
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At subvol 0
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At snapshot 1
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At snapshot 2
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At snapshot 3
read(net): Connection reset by peer
ERROR: short read from stream: expected 49183 read 10450
# nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
At snapshot 4
ERROR: cannot find parent subvolume
write(stdout): Broken pipe
sending side:
# btrfs send /mnt/send/snapshots/0 | nc -v -c 127.0.0.1 6790
At subvol /mnt/send/snapshots/0
localhost [127.0.0.1] 6790 (hnmp) open
# btrfs send -p /mnt/send/snapshots/0 /mnt/send/snapshots/1 | nc -v -c 127.0.0.1 6790
At subvol /mnt/send/snapshots/1
localhost [127.0.0.1] 6790 (hnmp) open
# btrfs send -p /mnt/send/snapshots/1 /mnt/send/snapshots/2 | nc -v -c 127.0.0.1 6790
At subvol /mnt/send/snapshots/2
localhost [127.0.0.1] 6790 (hnmp) open
# btrfs send -p /mnt/send/snapshots/2 /mnt/send/snapshots/3 | nc -v -c 127.0.0.1 6790
At subvol /mnt/send/snapshots/3
localhost [127.0.0.1] 6790 (hnmp) open
# btrfs send -p /mnt/send/snapshots/3 /mnt/send/snapshots/4 | nc -v -c 127.0.0.1 6790
At subvol /mnt/send/snapshots/4
localhost [127.0.0.1] 6790 (hnmp) open
write(net): Connection reset by peer
---
Take your mailboxes with you. Free, fast and secure Mail & Cloud: https://www.eclipso.eu - Time to change!
^ permalink raw reply [flat|nested] 6+ messages in thread* Re: btrfs send / receive via netcat, fails halfway?
2021-01-10 10:34 btrfs send / receive via netcat, fails halfway?
@ 2021-01-10 11:07 ` Forza
2021-01-10 11:37 ` Roman Mamedov
2021-01-10 11:47 ` Hugo Mills
2 siblings, 0 replies; 6+ messages in thread
From: Forza @ 2021-01-10 11:07 UTC (permalink / raw)
To: Cedric.dewijs, linux-btrfs
On 2021-01-10 11:34, Cedric.dewijs@eclipso.eu wrote:
> I'm trying to transfer a btrfs snapshot via the network.
>
> First attempt: Both NC programs don't exit after the transfer is complete. When I ctrl-C the sending side, the receiving side exits OK.
>
> btrfs subvolume delete /mnt/rec/snapshots/*
> receive side:
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At subvol 0
>
> sending side:
> # btrfs send /mnt/send/snapshots/0 | nc -v 127.0.0.1 6790
> At subvol /mnt/send/snapshots/0
> localhost [127.0.0.1] 6790 (hnmp) open
>
>
>
> Second attempt: both nc programs exit ok at snapshot 0,1,2, but snapshot3 fails halfway, and 4 fails, as 3 is not complete.
> receive side:
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At subvol 0
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At snapshot 1
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At snapshot 2
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At snapshot 3
> read(net): Connection reset by peer
> ERROR: short read from stream: expected 49183 read 10450
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At snapshot 4
> ERROR: cannot find parent subvolume
> write(stdout): Broken pipe
>
> sending side:
> # btrfs send /mnt/send/snapshots/0 | nc -v -c 127.0.0.1 6790
> At subvol /mnt/send/snapshots/0
> localhost [127.0.0.1] 6790 (hnmp) open
> # btrfs send -p /mnt/send/snapshots/0 /mnt/send/snapshots/1 | nc -v -c 127.0.0.1 6790
> At subvol /mnt/send/snapshots/1
> localhost [127.0.0.1] 6790 (hnmp) open
> # btrfs send -p /mnt/send/snapshots/1 /mnt/send/snapshots/2 | nc -v -c 127.0.0.1 6790
> At subvol /mnt/send/snapshots/2
> localhost [127.0.0.1] 6790 (hnmp) open
> # btrfs send -p /mnt/send/snapshots/2 /mnt/send/snapshots/3 | nc -v -c 127.0.0.1 6790
> At subvol /mnt/send/snapshots/3
> localhost [127.0.0.1] 6790 (hnmp) open
> # btrfs send -p /mnt/send/snapshots/3 /mnt/send/snapshots/4 | nc -v -c 127.0.0.1 6790
> At subvol /mnt/send/snapshots/4
> localhost [127.0.0.1] 6790 (hnmp) open
> write(net): Connection reset by peer
>
>
>
Hi,
Haven't used netcat for btrfs send, so I am not sure why you have these
issues. Have you tried mbuffer instead? This is what I use when I don't
need to go over ssh.
Mbuffer[1] allows you to specify remote host and port. It also has an
async buffer so it should be a bit faster too. In fact, it is useful on
a localhost too, for example when sending snaps to a slower local backup
media.
Receiving side:
# mbuffer -I port | btrfs receive /mnt/rec/snapshots
Sending side:
# btrfs send /mnt/send/snapshots/0 | mbuffer -O remotehost:port
Good Luck!
//Forza
[1]https://www.maier-komor.de/mbuffer.html
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: btrfs send / receive via netcat, fails halfway?
2021-01-10 10:34 btrfs send / receive via netcat, fails halfway?
2021-01-10 11:07 ` Forza
@ 2021-01-10 11:37 ` Roman Mamedov
2021-01-10 12:38 `
2021-01-10 11:47 ` Hugo Mills
2 siblings, 1 reply; 6+ messages in thread
From: Roman Mamedov @ 2021-01-10 11:37 UTC (permalink / raw)
To: ; +Cc: linux-btrfs
On Sun, 10 Jan 2021 11:34:27 +0100
" " <Cedric.dewijs@eclipso.eu> wrote:
> I'm trying to transfer a btrfs snapshot via the network.
>
> First attempt: Both NC programs don't exit after the transfer is complete. When I ctrl-C the sending side, the receiving side exits OK.
It is a common annoyance that NC doesn't exit in such scenario and needs to be
Ctrl-C'ed after verifying that the transfer is over.
Instead, at host2 try:
ssh host1 "btrfs send ..." | btrfs receive ...
Also much more secure.
--
With respect,
Roman
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Re: btrfs send / receive via netcat, fails halfway?
2021-01-10 11:37 ` Roman Mamedov
@ 2021-01-10 12:38 `
0 siblings, 0 replies; 6+ messages in thread
From: @ 2021-01-10 12:38 UTC (permalink / raw)
To: Roman Mamedov; +Cc: linux-btrfs
It is a common annoyance that NC doesn't exit in such scenario and needs
to be
Ctrl-C'ed after verifying that the transfer is over.
Instead, at host2 try:
ssh host1 "btrfs send ..." | btrfs receive ...
Also much more secure.
--
With respect,
Roman
I was practicing on a test system, the real transfer was supposed to be between two systems on a LAN. I didn't want encryption, as my system then only is capable of sending 30MB/sec, instead of the 110MB/sec over the 1GB network.
---
Take your mailboxes with you. Free, fast and secure Mail & Cloud: https://www.eclipso.eu - Time to change!
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: btrfs send / receive via netcat, fails halfway?
2021-01-10 10:34 btrfs send / receive via netcat, fails halfway?
2021-01-10 11:07 ` Forza
2021-01-10 11:37 ` Roman Mamedov
@ 2021-01-10 11:47 ` Hugo Mills
2021-01-10 13:27 ` Hugo Mills
2 siblings, 1 reply; 6+ messages in thread
From: Hugo Mills @ 2021-01-10 11:47 UTC (permalink / raw)
To: Cedric.dewijs; +Cc: linux-btrfs
On Sun, Jan 10, 2021 at 11:34:27AM +0100, wrote:
> I'm trying to transfer a btrfs snapshot via the network.
>
> First attempt: Both NC programs don't exit after the transfer is complete. When I ctrl-C the sending side, the receiving side exits OK.
>
> btrfs subvolume delete /mnt/rec/snapshots/*
> receive side:
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At subvol 0
>
> sending side:
> # btrfs send /mnt/send/snapshots/0 | nc -v 127.0.0.1 6790
> At subvol /mnt/send/snapshots/0
> localhost [127.0.0.1] 6790 (hnmp) open
Use -q 15 on the sending side. That will exit after 15 seconds of
no activity from the send process.
> Second attempt: both nc programs exit ok at snapshot 0,1,2, but snapshot3 fails halfway, and 4 fails, as 3 is not complete.
> receive side:
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At subvol 0
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At snapshot 1
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At snapshot 2
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At snapshot 3
> read(net): Connection reset by peer
> ERROR: short read from stream: expected 49183 read 10450
This failed because of a network disconnect.
> # nc -l -p 6790 | btrfs receive /mnt/rec/snapshots
> At snapshot 4
> ERROR: cannot find parent subvolume
> write(stdout): Broken pipe
This is expected because the previous one failed.
There's no btrfs problem here. You just need better error handling
(to retry a failed transfer, for example).
Hugo.
--
Hugo Mills | How do you become King? You stand in the marketplace
hugo@... carfax.org.uk | and announce you're going to tax everyone. If you
http://carfax.org.uk/ | get out alive, you're King.
PGP: E2AB1DE4 | Harry Harrison
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2021-01-10 13:28 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2021-01-10 10:34 btrfs send / receive via netcat, fails halfway?
2021-01-10 11:07 ` Forza
2021-01-10 11:37 ` Roman Mamedov
2021-01-10 12:38 `
2021-01-10 11:47 ` Hugo Mills
2021-01-10 13:27 ` Hugo Mills
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox