Re: btrfs issue with mariadb incremental backup

[A L <crimsoncottage@gmail.com>]

On 8/13/2017 12:52 PM, siranee.ja@tpc.co.th wrote:
> Hi "A L",
>
> [root@backuplogC7 ~]# btrfs sub show /var/lib/mariadb/mysql
> /var/lib/mariadb/mysql
>          Name:                   mysql
>          UUID:                   92f319c5-e132-3249-9b13-d39ee77a2b44
>          Parent UUID:            -
>          Received UUID:          3ad0334a-4063-654c-add6-b1cbcdeaa639
>          Creation time:          2017-06-21 13:27:41 +0700
>          Subvolume ID:           257
>          Generation:             539
>          Gen at creation:        9
>          Parent ID:              5
>          Top level ID:           5
>          Flags:                  -
>          Snapshot(s):
>                                  mysql_201708060830
>                                  mysql_201708070830
>                                  mysql_201708080830
>                                  mysql_201708090830
>                                  mysql_201708100830
>                                  mysql_201708110830
>                                  mysql_201708120830
>                                  mysql_201708130830
>
> yes I think it has Received UUID because I restored the source from snapshot
> mysql_201708040830 for prove that the local snapshot was work.
>
> How to clear the Received UUID ?
> What to do next?
You need to make a read-write snapshot of  /var/lib/mariadb/mysql and 
then remove the old subvolume and all its snapshots.

Example from https://github.com/digint/btrbk/blob/master/doc/FAQ.md

# cd /mnt/btr_pool
# mv mysubvolume mysubvolume.broken
# btrfs subvolume snapshot mysubvolume.broken mysubvolume

You can do the same with each of your snapshots too, and send them as 
full snapshots (without -p).

~A

> Best Regards,
> Siranee Jaraswachirakul.
>
>> Have you checked that there is no Received UUID on the source subvolume?
>>
>> # btrfs sub show volume/mysql/
>> volume/mysql
>>           Name:                   mysql
>>           UUID:                   8a94524e-a956-c14b-bb8d-d453627f27d5
>>           Parent UUID:            -
>>           Received UUID:          -
>>           Creation time:          2017-04-17 11:46:20 +0200
>>           Subvolume ID:           1469
>>           Generation:             122934
>>           Gen at creation:        78671
>>           Parent ID:              5
>>           Top level ID:           5
>>           Flags:                  -
>>           Snapshot(s):
>>
>> There is no Received UUID here. If it has, then btrfs send-receive will
>> have problems, since all snapshots of the source subvolume will have the
>> same Received UUID and it can't tell the differences between the snapshots.
>>
>> On 8/13/2017 5:40 AM, siranee.ja@tpc.co.th wrote:
>>> Hi Chris,
>>>
>>> I started as your suggestion again. The diff occured since snapshot
>>> mysql_201708090830 manually send. What should I do next?
>>>
>>> - delete all the bad/mismatching snapshots only on the destination computer.
>>> [root@joytest ~]# date
>>> Sun Aug 13 10:27:23 ICT 2017
>>> [root@joytest ~]# cd /var/lib/mariadb
>>> [root@joytest mariadb]# btrfs sub list .
>>> ID 313 gen 220 top level 5 path mysql_201708070830
>>> ID 316 gen 199 top level 5 path mysql_201708080830
>>> ID 318 gen 205 top level 5 path mysql_201708090830
>>> ID 320 gen 211 top level 5 path mysql_201708100830
>>> ID 322 gen 219 top level 5 path mysql_201708110830
>>> ID 323 gen 219 top level 5 path mysql_201708120830
>>> ID 324 gen 224 top level 5 path mysql_201708130830
>>> ID 325 gen 225 top level 5 path mysql
>>> [root@joytest mariadb]# btrfs sub del mysql_201708130830
>>> Delete subvolume (no-commit): '/var/lib/mariadb/mysql_201708130830'
>>> [root@joytest mariadb]# btrfs sub del mysql_201708120830
>>> Delete subvolume (no-commit): '/var/lib/mariadb/mysql_201708120830'
>>> [root@joytest mariadb]# btrfs sub del mysql_201708110830
>>> Delete subvolume (no-commit): '/var/lib/mariadb/mysql_201708110830'
>>> [root@joytest mariadb]# btrfs sub del mysql_201708100830
>>> Delete subvolume (no-commit): '/var/lib/mariadb/mysql_201708100830'
>>> [root@joytest mariadb]# btrfs sub del mysql_201708090830
>>> Delete subvolume (no-commit): '/var/lib/mariadb/mysql_201708090830'
>>> [root@joytest mariadb]# btrfs sub sync .
>>> [root@joytest mariadb]# systemctl status mariadb
>>> â— mariadb.service - MariaDB database server
>>>      Loaded: loaded (/usr/lib/systemd/system/mariadb.service; enabled; vendor
>>> preset:
>>> disabled)
>>>      Active: failed (Result: exit-code) since Sun 2017-08-13 09:07:00 ICT; 1h 24min
>>> ago
>>>     Process: 19871 ExecStartPost=/usr/libexec/mariadb-wait-ready $MAINPID
>>> (code=exited, status=1/FAILURE)
>>>     Process: 19870 ExecStart=/usr/bin/mysqld_safe --basedir=/usr (code=exited,
>>> status=0/SUCCESS)
>>>     Process: 19842 ExecStartPre=/usr/libexec/mariadb-prepare-db-dir %n
>>> (code=exited,
>>> status=0/SUCCESS)
>>>    Main PID: 19870 (code=exited, status=0/SUCCESS)
>>>
>>> Aug 13 09:06:58 joytest systemd[1]: Starting MariaDB database server...
>>> Aug 13 09:06:58 joytest mysqld_safe[19870]: 170813 09:06:58 mysqld_safe Logging to
>>> '/var/log/mariadb/mariadb.log'.
>>> Aug 13 09:06:58 joytest mysqld_safe[19870]: 170813 09:06:58 mysqld_safe Starting
>>> mysqld daemon with databases from /var/lib/mariadb/mysql
>>> Aug 13 09:07:00 joytest systemd[1]: mariadb.service: control process exited,
>>> code=exited status=1
>>> Aug 13 09:07:00 joytest systemd[1]: Failed to start MariaDB database server.
>>> Aug 13 09:07:00 joytest systemd[1]: Unit mariadb.service entered failed state.
>>> Aug 13 09:07:00 joytest systemd[1]: mariadb.service failed.
>>> [root@joytest mariadb]# btrfs sub list .
>>> ID 313 gen 220 top level 5 path mysql_201708070830
>>> ID 316 gen 199 top level 5 path mysql_201708080830
>>> ID 325 gen 225 top level 5 path mysql
>>> [root@joytest mariadb]#
>>>
>>> - The most recent good snapshot pair, which rsync shows origin and
>>> destination match, is mysql_201708080830 so you can keep that one on
>>> both sides.
>>>
>>> [root@backuplogC7 ~]# btrfs sub list /var/lib/mariadb
>>> ID 257 gen 538 top level 5 path mysql
>>> ID 316 gen 498 top level 5 path mysql_201708060830
>>> ID 317 gen 503 top level 5 path mysql_201708070830
>>> ID 318 gen 507 top level 5 path mysql_201708080830
>>> ID 319 gen 514 top level 5 path mysql_201708090830
>>> ID 320 gen 524 top level 5 path mysql_201708100830
>>> ID 321 gen 529 top level 5 path mysql_201708110830
>>> ID 322 gen 533 top level 5 path mysql_201708120830
>>> ID 323 gen 538 top level 5 path mysql_201708130830
>>> [root@backuplogC7 ~]# rsync -avnc /var/lib/mariadb/mysql_201708070830/
>>> root@192.168.45.166://var/lib/mariadb/mysql_201708070830/
>>> sending incremental file list
>>> ./
>>>
>>> sent 3773 bytes  received 19 bytes  842.67 bytes/sec
>>> total size is 718361496  speedup is 189441.32 (DRY RUN)
>>> [root@backuplogC7 ~]# rsync -avnc /var/lib/mariadb/mysql_201708080830/
>>> root@192.168.45.166://var/lib/mariadb/mysql_201708080830/
>>> sending incremental file list
>>> ./
>>>
>>> sent 3769 bytes  received 19 bytes  841.78 bytes/sec
>>> total size is 718361496  speedup is 189641.37 (DRY RUN)
>>> [root@backuplogC7 ~]# date
>>> Sun Aug 13 10:34:05 ICT 2017
>>> [root@backuplogC7 ~]#
>>>
>>> - manually do incremental send/receive, starting with
>>> mysql_201708090830/, to make the destination current again with the
>>> origin.
>>>
>>> [root@backuplogC7 ~]# btrfs send -p /var/lib/mariadb/mysql_201708080830
>>> /var/lib/mariadb/mysql_201708090830 | ssh 192.168.45.166 btrfs receive
>>> /var/lib/mariadb
>>> At subvol /var/lib/mariadb/mysql_201708090830
>>> At snapshot mysql_201708090830
>>> [root@backuplogC7 ~]# rsync -avnc /var/lib/mariadb/mysql_201708090830/
>>> root@192.168.45.166://var/lib/mariadb/mysql_201708090830/
>>> sending incremental file list
>>> ./
>>> ib_logfile1
>>> ibdata1
>>>
>>> sent 3779 bytes  received 25 bytes  507.20 bytes/sec
>>> total size is 718361496  speedup is 188843.72 (DRY RUN)
>>> [root@backuplogC7 ~]#
>>>
>>> Best Regards,
>>>
>>> Siranee Jaraswachirakul.
>>>
>>>> On Sat, Aug 12, 2017 at 8:20 PM,  <siranee.ja@tpc.co.th> wrote:
>>>>
>>>>> [root@backuplogC7 ~]# rsync -avnc /var/lib/mariadb/mysql_201708090830
>>>>> root@192.168.45.166://var/lib/mariadb/mysql_201708090830
>>>> You need trailing / for the first directory with -a option.
>>>>
>>>> rsync -a dir dir
>>>>
>>>> is not the same command as
>>>>
>>>> rsync -a dir/ dir
>>>>
>>>> It's confusing but your command is trying to create mysql_201708090830
>>>> directory on the source, in the mysql_201708090830 on the destination.
>>>> That is why everything mismatches. To make it mean "contents of" you
>>>> need trailing slash on at least the origin.
>>>>
>>>>
>>>>
>>>> --
>>>> Chris Murphy
>>>>
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>