Re: [linux-lvm] LVM snapshot setup for mysql-zrm backups problem

[Marian Csontos <mcsontos@redhat.com>]

On 01/29/2015 01:18 AM, Philip Amadeo Saeli wrote:
> Thanks for the response.  More info and questions following.
>
> * Marian Csontos <mcsontos@redhat.com> [2015-01-28 04:53]:
>> On 01/28/2015 06:46 AM, Philip Amadeo Saeli wrote:
>
>> What does `lvs` say? Is there any snapshot of the LV where your
>> database lives? Copy if to backup destination and try to check the
>> content - I presume there is a tool to check the backup...
>
> The backups appear to be OK.  However, my suspicion is that they're
> being made from the main database LVs rather than from a freshly-made
> snapshot.

That's exactly question for mysql-list.

You could monitor the snapshot with for example iotop or something to 
see if it is being read from but...

>
> I am able to manually make a snapshot LV and it looks OK.  I have also
> upped the LVM log level so I can better monitor any LVM errors or
> warnings.
>
> However, I have another question:
>
> When I run any of the various LVM cmds with the "-v" flag, I get a
> "DEGRADED MODE. Incomplete RAID LVs will be processed." message in the
> output.  I find this surprising since none of the LVM vols on that
> system are RAID vols.  They are simply single PV vols created for the
> sole purpose of enabling snapshots for consistent point-in-time database
> backups.  Any ideas why it is printing that?

Post the output of the command with -vvvv, please.
And may be lvmdump.

>
>> Just a word of warning:
>>
>> if the script is using old-style snapshots, you want to copy and
>> delete it ASAP, or you will see severely degraded performance, and
>> much worse once you have multiple snapshots in the system.
>
> What do you mean by "old-style" snapshots?  Does that mean non-thin
> snapshots?  Or something else?

Yes, non-thin snapshots.

 > What type of degraded performance?  Any
> references?

For example I were unable to squeeze more than few hundred IOPS on 
rather high-end hardware from old-snapshots (but it was on RHEL6 kernel 
missing few recent performance optimizations.)

And the more snapshots of the origin LV you have the write performance 
is divided by the number of them as the original value must be written 
to each of them. (And there is no optimization for that. So remember to 
remove the snapshots early.)

HTH

Marian

>
> Note that the snapshots I'm using are ephemeral and are to exist only
> long enough for the actual database backup operation to complete.
>
>> Also once the space allocated for snapshot gets full, the snapshot
>> becomes invalid, so hurry up!
>>
>> -- Martian
>
> Thanks!
>
> --Phil
>
>>> the appropriate place to ask this.  This is my first time setting up
>>> LVM snapshots for mysql-zrm backups.
>>>
>>> CentOS-6.6
>>> MySQL-5.6.22
>>> MySQL-zrm-3.0 (EPEL).
>>>
>>> Short version:
>>>
>>> I am unable to tell if the MySQL-zrm LVM snapshots are indeed taking
>>> place, or if the backup is being taken directly from the main database
>>> LVs.  The log file info isn't giving me enough info to tell one way or
>>> another.
>>>
>>> Some more info:
>>>
>>> I am setting up a database server with all of the database files on LVM
>>> LVs and am using mysql-zrm for logical backups using LVM snapshots to
>>> reduce the database lock times.
>>>
>>> I have followed all of the various setup instructions very closely, and
>>> am using the default "lvm-snapshot.pl" plugin.  However, the only log
>>> file lines mentioning snapshot are the following:
>>>
>>>    mysql:backup:INFO: Running snapshot verification using command /usr/share/mysql-zrm/plugins/lvm-snapshot.pl --action verify-config 2>/home/tmp/xH8KKpsF2O
>>>
>>> There are no other log entries following this one that seem to have
>>> anything to do with snapshots.  There are no errors.  The backups
>>> succeed.  Both "--verbose" and "--noquiet" are enabled.
>>>
>>> Here are the salient entries from the mysql-zrm.conf file:
>>>
>>> backup-level=0
>>> backup-mode=logical
>>> lvm-snapshot=20G
>>> snapshot-plugin="/usr/share/mysql-zrm/plugins/lvm-snapshot.pl"
>>> backup-type=regular
>>> destination=/home/mysql-zrm
>>> retention-policy=4W
>>> compress=1
>>>
>>> Any info or help would be much appreciated!
>>>
>>> Thanks,
>>>
>>> --Phil
>