xfs_freeze -f misbehaving under lenny / xfsprogs 2.9.8

xfs_freeze -f misbehaving under lenny / xfsprogs 2.9.8

[Stuart Rowan]

Hi,

We have a backup script running on another machine that ssh's in to the 
affected server and does the following:
mkdir -p /tmp/foo;
/usr/sbin/xfs_freeze -f /home;
/sbin/lvcreate -s -L 20G -n snap-shot home ;
/usr/sbin/xfs_freeze -u /home;
mount -o nouuid,ro /dev/data/snap-shot /tmp/foo;

It then rsyncs (over ssh) the data to the backup store from /tmp/foo

The above command set hangs at running "/sbin/lvcreate -s -L 20G -n 
snap-shot home;"

All I/O to /home is of course blocked at this point so for example exim 
starts queueing up all the mail.

As soon as I manually log in some hours later and run /usr/sbin/xfs_freeze 
-u /home; ... the lvcreate succeeds, the backup runs etc.


We recently upgraded this server to lenny from etch. It is still using the 
same kernel as it did with etch. The kernel is 2.6.26-bpo.1-amd64.

So this has never happened before and it occurring directly coincides with 
my upgrade to lenny on Friday.

The error occurred on Friday night
Backup proceeded normally on Saturday night
The error occurred on Sunday night
This suggests to me that there's some sort of race going on?

Previous version of xfsprogs (using etch): 2.8.11-1
Current version of xfsprogs (using lenny): 2.9.8-1lenny1

Anyone have any thoughts / is this a known issue with the 2.9.8 release?

Kind regards,
Stu.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: xfs_freeze -f misbehaving under lenny / xfsprogs 2.9.8

[Eric Sandeen]

Stuart Rowan wrote:
> Hi,
> 
> We have a backup script running on another machine that ssh's in to the 
> affected server and does the following:
> mkdir -p /tmp/foo;
> /usr/sbin/xfs_freeze -f /home;
> /sbin/lvcreate -s -L 20G -n snap-shot home ;
> /usr/sbin/xfs_freeze -u /home;
> mount -o nouuid,ro /dev/data/snap-shot /tmp/foo;
> 
> It then rsyncs (over ssh) the data to the backup store from /tmp/foo
> 
> The above command set hangs at running "/sbin/lvcreate -s -L 20G -n 
> snap-shot home;"
> 
> All I/O to /home is of course blocked at this point so for example exim 
> starts queueing up all the mail.


lvcreate now does the fs freeze on its own via the snapshot ioctl, so if
you run freeze manually first, you are stuck behind that first freeze.

Just drop the xfs_freeze's from the above, and all should be well.

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: xfs_freeze -f misbehaving under lenny / xfsprogs 2.9.8

[Stuart Rowan]

Eric Sandeen wrote, on 30/03/09 14:45:
> Stuart Rowan wrote:
>> Hi,
>>
>> We have a backup script running on another machine that ssh's in to the 
>> affected server and does the following:
>> mkdir -p /tmp/foo;
>> /usr/sbin/xfs_freeze -f /home;
>> /sbin/lvcreate -s -L 20G -n snap-shot home ;
>> /usr/sbin/xfs_freeze -u /home;
>> mount -o nouuid,ro /dev/data/snap-shot /tmp/foo;
>>
>> It then rsyncs (over ssh) the data to the backup store from /tmp/foo
>>
>> The above command set hangs at running "/sbin/lvcreate -s -L 20G -n 
>> snap-shot home;"
>>
>> All I/O to /home is of course blocked at this point so for example exim 
>> starts queueing up all the mail.
> 
> 
> lvcreate now does the fs freeze on its own via the snapshot ioctl, so if
> you run freeze manually first, you are stuck behind that first freeze.
> 
> Just drop the xfs_freeze's from the above, and all should be well.
> 
> -Eric
Eric,

Many thanks for your prompt reply and explanation :-)

It's good to know that there's an easy solution ... except we now have to 
differentiate the commands to run in the backup script based on the version 
of lvm on the remote system :-$

OOI when implementing the freeze ioctl, what made the developers decide 
that a freeze can't succeed on an already frozen filesystem ... you'd 
expect it to just be a no-op really?

Cheers,
Stu.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: xfs_freeze -f misbehaving under lenny / xfsprogs 2.9.8

[Eric Sandeen]

Stuart Rowan wrote:
> Eric Sandeen wrote, on 30/03/09 14:45:
>> Stuart Rowan wrote:
>>> Hi,
>>>
>>> We have a backup script running on another machine that ssh's in to the 
>>> affected server and does the following:
>>> mkdir -p /tmp/foo;
>>> /usr/sbin/xfs_freeze -f /home;
>>> /sbin/lvcreate -s -L 20G -n snap-shot home ;
>>> /usr/sbin/xfs_freeze -u /home;
>>> mount -o nouuid,ro /dev/data/snap-shot /tmp/foo;
>>>
>>> It then rsyncs (over ssh) the data to the backup store from /tmp/foo
>>>
>>> The above command set hangs at running "/sbin/lvcreate -s -L 20G -n 
>>> snap-shot home;"
>>>
>>> All I/O to /home is of course blocked at this point so for example exim 
>>> starts queueing up all the mail.
>>
>> lvcreate now does the fs freeze on its own via the snapshot ioctl, so if
>> you run freeze manually first, you are stuck behind that first freeze.
>>
>> Just drop the xfs_freeze's from the above, and all should be well.
>>
>> -Eric
> Eric,
> 
> Many thanks for your prompt reply and explanation :-)
> 
> It's good to know that there's an easy solution ... except we now have to 
> differentiate the commands to run in the backup script based on the version 
> of lvm on the remote system :-$

can't be *that* bad ... ;)

> OOI when implementing the freeze ioctl, what made the developers decide 
> that a freeze can't succeed on an already frozen filesystem ... you'd 
> expect it to just be a no-op really?

To complicate matters more, on newer upstream kernels w/ the freeze
ioctl exposed for all filesystems, nested freezes are in fact supported.

I'm not sure why sequential freezes were serialized initially, TBH...

-Eric

> Cheers,
> Stu.
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: xfs_freeze -f misbehaving under lenny / xfsprogs 2.9.8

[Michael Monnerie]

On Montag 30 März 2009 Stuart Rowan wrote:
> OOI when implementing the freeze ioctl, what made the developers
> decide that a freeze can't succeed on an already frozen filesystem
> ... you'd expect it to just be a no-op really?

So you would break the unfreeze:

xfs_freeze -f
lvcreate -s (lvcreate does freeze;work;unfreeze)
*now you are unfrozen*
xfs_freeze -u

The problem would be if you expect to be frozen still after the lvcreate 
command and do some processing, that might be bad.

mfg zmi
-- 
// Michael Monnerie, Ing.BSc    -----      http://it-management.at
// Tel: 0660 / 415 65 31                      .network.your.ideas.
// PGP Key:         "curl -s http://zmi.at/zmi.asc | gpg --import"
// Fingerprint: AC19 F9D5 36ED CD8A EF38  500E CE14 91F7 1C12 09B4
// Keyserver: wwwkeys.eu.pgp.net                  Key-ID: 1C1209B4

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs