Fwd: First 12Mb of data missing after accidental deletion of first drive (4 2TB raid5)

Fwd: First 12Mb of data missing after accidental deletion of first drive (4 2TB raid5)

[Marek]

Hi,

I'd like to ask about recovering partitions saved as images on eg a raid.
How do I recover such image - I tried to run test disk and it found
lots of images which I wasn't able to view via P (NFTS) and ext3
partitions but I'd like to save them to another harddrive, I tried
photo tree with only ext3 superblock and image file enabled but it
wouldn't find the partition in was looki for and found via testdisk.
What would you recommend me to do?

Marek

Re: First 12Mb of data missing after accidental deletion of first drive (4 2TB raid5)

[Alexander Afonyashin]

Hi Marek,

Do you mean you have files (on raid) that were used as loop devices?

Regards,
Alexander

On Tue, Oct 13, 2015 at 4:44 PM, Marek <mlf.conv@gmail.com> wrote:
> Hi,
>
> I'd like to ask about recovering partitions saved as images on eg a raid.
> How do I recover such image - I tried to run test disk and it found
> lots of images which I wasn't able to view via P (NFTS) and ext3
> partitions but I'd like to save them to another harddrive, I tried
> photo tree with only ext3 superblock and image file enabled but it
> wouldn't find the partition in was looki for and found via testdisk.
> What would you recommend me to do?
>
> Marek
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: First 12Mb of data missing after accidental deletion of first drive (4 2TB raid5)

[Marek]

no, i had a working raid consisting of 4 2TB drives (raid5)
sda,sdb,sdc,sdb ( no partitions were created on drives) - then i
accidentally deleted /dev/sda by installing ubuntu. when inspecting
dev/md127 with a hex editor i discovered first 12mb are missing.

On Tue, Oct 13, 2015 at 3:53 PM, Alexander Afonyashin
<a.afonyashin@madnet-team.ru> wrote:
> Hi Marek,
>
> Do you mean you have files (on raid) that were used as loop devices?
>
> Regards,
> Alexander
>
> On Tue, Oct 13, 2015 at 4:44 PM, Marek <mlf.conv@gmail.com> wrote:
>> Hi,
>>
>> I'd like to ask about recovering partitions saved as images on eg a raid.
>> How do I recover such image - I tried to run test disk and it found
>> lots of images which I wasn't able to view via P (NFTS) and ext3
>> partitions but I'd like to save them to another harddrive, I tried
>> photo tree with only ext3 superblock and image file enabled but it
>> wouldn't find the partition in was looki for and found via testdisk.
>> What would you recommend me to do?
>>
>> Marek
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: First 12Mb of data missing after accidental deletion of first drive (4 2TB raid5)

[Marek]

sorry i pasted the wrong email:
Hi,

I have a problem with my 4 2TB disk Raid5 (most likely btrfs) after I
accidentally deleted first drive by installing Ubuntu on it (/dev/sda
automatic choice) then I discovered the raid isn't working and by
closer inspecting it hexdump I discovered first ~12Mb of data are
missing. Is it possible to recover it somehow? What I did was to zero
superblock then add the first drive - everything seems fine according
to /proc/mdstat.

On Tue, Oct 13, 2015 at 4:04 PM, Marek <mlf.conv@gmail.com> wrote:
> no, i had a working raid consisting of 4 2TB drives (raid5)
> sda,sdb,sdc,sdb ( no partitions were created on drives) - then i
> accidentally deleted /dev/sda by installing ubuntu. when inspecting
> dev/md127 with a hex editor i discovered first 12mb are missing.
>
> On Tue, Oct 13, 2015 at 3:53 PM, Alexander Afonyashin
> <a.afonyashin@madnet-team.ru> wrote:
>> Hi Marek,
>>
>> Do you mean you have files (on raid) that were used as loop devices?
>>
>> Regards,
>> Alexander
>>
>> On Tue, Oct 13, 2015 at 4:44 PM, Marek <mlf.conv@gmail.com> wrote:
>>> Hi,
>>>
>>> I'd like to ask about recovering partitions saved as images on eg a raid.
>>> How do I recover such image - I tried to run test disk and it found
>>> lots of images which I wasn't able to view via P (NFTS) and ext3
>>> partitions but I'd like to save them to another harddrive, I tried
>>> photo tree with only ext3 superblock and image file enabled but it
>>> wouldn't find the partition in was looki for and found via testdisk.
>>> What would you recommend me to do?
>>>
>>> Marek
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: First 12Mb of data missing after accidental deletion of first drive (4 2TB raid5)

[Alexander Afonyashin]

Hi,

Does /proc/mdstat show that raid is ok? If so, why did you make an
assumption that data of first 12MB are gone? Zeroed? Missed files on
filesystem?

Regards,
Alexander

On Tue, Oct 13, 2015 at 5:04 PM, Marek <mlf.conv@gmail.com> wrote:
> no, i had a working raid consisting of 4 2TB drives (raid5)
> sda,sdb,sdc,sdb ( no partitions were created on drives) - then i
> accidentally deleted /dev/sda by installing ubuntu. when inspecting
> dev/md127 with a hex editor i discovered first 12mb are missing.
>
> On Tue, Oct 13, 2015 at 3:53 PM, Alexander Afonyashin
> <a.afonyashin@madnet-team.ru> wrote:
>> Hi Marek,
>>
>> Do you mean you have files (on raid) that were used as loop devices?
>>
>> Regards,
>> Alexander
>>
>> On Tue, Oct 13, 2015 at 4:44 PM, Marek <mlf.conv@gmail.com> wrote:
>>> Hi,
>>>
>>> I'd like to ask about recovering partitions saved as images on eg a raid.
>>> How do I recover such image - I tried to run test disk and it found
>>> lots of images which I wasn't able to view via P (NFTS) and ext3
>>> partitions but I'd like to save them to another harddrive, I tried
>>> photo tree with only ext3 superblock and image file enabled but it
>>> wouldn't find the partition in was looki for and found via testdisk.
>>> What would you recommend me to do?
>>>
>>> Marek
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: First 12Mb of data missing after accidental deletion of first drive (4 2TB raid5)

[Marek]

yes first 12Mb seem to be zeroed

On Tue, Oct 13, 2015 at 4:40 PM, Alexander Afonyashin
<a.afonyashin@madnet-team.ru> wrote:
> Hi,
>
> Does /proc/mdstat show that raid is ok? If so, why did you make an
> assumption that data of first 12MB are gone? Zeroed? Missed files on
> filesystem?
>
> Regards,
> Alexander
>
> On Tue, Oct 13, 2015 at 5:04 PM, Marek <mlf.conv@gmail.com> wrote:
>> no, i had a working raid consisting of 4 2TB drives (raid5)
>> sda,sdb,sdc,sdb ( no partitions were created on drives) - then i
>> accidentally deleted /dev/sda by installing ubuntu. when inspecting
>> dev/md127 with a hex editor i discovered first 12mb are missing.
>>
>> On Tue, Oct 13, 2015 at 3:53 PM, Alexander Afonyashin
>> <a.afonyashin@madnet-team.ru> wrote:
>>> Hi Marek,
>>>
>>> Do you mean you have files (on raid) that were used as loop devices?
>>>
>>> Regards,
>>> Alexander
>>>
>>> On Tue, Oct 13, 2015 at 4:44 PM, Marek <mlf.conv@gmail.com> wrote:
>>>> Hi,
>>>>
>>>> I'd like to ask about recovering partitions saved as images on eg a raid.
>>>> How do I recover such image - I tried to run test disk and it found
>>>> lots of images which I wasn't able to view via P (NFTS) and ext3
>>>> partitions but I'd like to save them to another harddrive, I tried
>>>> photo tree with only ext3 superblock and image file enabled but it
>>>> wouldn't find the partition in was looki for and found via testdisk.
>>>> What would you recommend me to do?
>>>>
>>>> Marek
>>>> --
>>>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>>> the body of a message to majordomo@vger.kernel.org
>>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: First 12Mb of data missing after accidental deletion of first drive (4 2TB raid5)

[Marek]

Could it be that the raid wasnt synced at the time i overwrote
/dev/sda ? i have run the command you suggested and will report after
its done. would it still be possible to recover the raid?

On Tue, Oct 13, 2015 at 4:36 PM, Weedy <weedy2887@gmail.com> wrote:
>
> On 13 Oct 2015 10:06 am, "Marek" <mlf.conv@gmail.com> wrote:
>>
>> no, i had a working raid consisting of 4 2TB drives (raid5)
>> sda,sdb,sdc,sdb ( no partitions were created on drives) - then i
>> accidentally deleted /dev/sda by installing ubuntu. when inspecting
>> dev/md127 with a hex editor i discovered first 12mb are missing.
>
> Wait, if it's a raid5 can't you just resync it?
>
> Also I believe 1.2 metadata is at the end of a partition. If your saying the
> raids hasn't noticed you can run a check and it should fix itself.
>
> echo check > /sys/block/mdX/md/sync_action

Re: First 12Mb of data missing after accidental deletion of first drive (4 2TB raid5)

[Benjamin ESTRABAUD]

Hi Marek,

On 13/10/15 16:04, Marek wrote:
> Could it be that the raid wasnt synced at the time i overwrote
> /dev/sda ? i have run the command you suggested and will report after
> its done. would it still be possible to recover the raid?
>
So what likely happened is that you wrote ontop of a RAID member drive 
while the RAID was likely not assembled (during your install).

If you do a resync it won't unfortunately *recover* the previously 
written data, it will go over each stripe and recalculate the parity. It 
is true that it *might* have been possible to do the opposite operation 
by not touching anything and figuring out the previous data with the 
data and parity blocks from the remaining RAID members (assuming they 
weren't overwritten), but it would have been a tricky operation. 
However, since you've done the resync now, the previous parity blocks 
have been overwritten leaving you with a consistent RAID, but with part 
of the data on a disk that you want to revert.

At this stage you should recover the old data from a backup.

> On Tue, Oct 13, 2015 at 4:36 PM, Weedy <weedy2887@gmail.com> wrote:
>>
>> On 13 Oct 2015 10:06 am, "Marek" <mlf.conv@gmail.com> wrote:
>>>
>>> no, i had a working raid consisting of 4 2TB drives (raid5)
>>> sda,sdb,sdc,sdb ( no partitions were created on drives) - then i
>>> accidentally deleted /dev/sda by installing ubuntu. when inspecting
>>> dev/md127 with a hex editor i discovered first 12mb are missing.
>>
>> Wait, if it's a raid5 can't you just resync it?
>>
>> Also I believe 1.2 metadata is at the end of a partition. If your saying the
>> raids hasn't noticed you can run a check and it should fix itself.
>>
>> echo check > /sys/block/mdX/md/sync_action
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: First 12Mb of data missing after accidental deletion of first drive (4 2TB raid5)

[Adam Goryachev]

On 14/10/15 05:09, Benjamin ESTRABAUD wrote:
> Hi Marek,
>
> On 13/10/15 16:04, Marek wrote:
>> Could it be that the raid wasnt synced at the time i overwrote
>> /dev/sda ? i have run the command you suggested and will report after
>> its done. would it still be possible to recover the raid?
>>
> So what likely happened is that you wrote ontop of a RAID member drive 
> while the RAID was likely not assembled (during your install).
>
> If you do a resync it won't unfortunately *recover* the previously 
> written data, it will go over each stripe and recalculate the parity. 
> It is true that it *might* have been possible to do the opposite 
> operation by not touching anything and figuring out the previous data 
> with the data and parity blocks from the remaining RAID members 
> (assuming they weren't overwritten), but it would have been a tricky 
> operation. However, since you've done the resync now, the previous 
> parity blocks have been overwritten leaving you with a consistent 
> RAID, but with part of the data on a disk that you want to revert.
>
Or better action would be to stop the array, start the array excluding 
sda (not sure best way to manually mark it failed, or maybe physically 
remove it, or zero superblock should work too). Then you should have 
clean degraded array with all data. Now you could add sda back, and it 
will re-write correct data. (Assuming no writes were done to sdb/sdc/sdd).

Depending on what you actually did, and what order, it might still work....

Regards,
Adam

> At this stage you should recover the old data from a backup.
>
>> On Tue, Oct 13, 2015 at 4:36 PM, Weedy <weedy2887@gmail.com> wrote:
>>>
>>> On 13 Oct 2015 10:06 am, "Marek" <mlf.conv@gmail.com> wrote:
>>>>
>>>> no, i had a working raid consisting of 4 2TB drives (raid5)
>>>> sda,sdb,sdc,sdb ( no partitions were created on drives) - then i
>>>> accidentally deleted /dev/sda by installing ubuntu. when inspecting
>>>> dev/md127 with a hex editor i discovered first 12mb are missing.
>>>
>>> Wait, if it's a raid5 can't you just resync it?
>>>
>>> Also I believe 1.2 metadata is at the end of a partition. If your 
>>> saying the
>>> raids hasn't noticed you can run a check and it should fix itself.
>>>
>>> echo check > /sys/block/mdX/md/sync_action
>> -- 
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
>
> -- 
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html


-- 
Adam Goryachev Website Managers www.websitemanagers.com.au

Re: First 12Mb of data missing after accidental deletion of first drive (4 2TB raid5)

[Benjamin ESTRABAUD]

On 14/10/15 01:42, Adam Goryachev wrote:
> On 14/10/15 05:09, Benjamin ESTRABAUD wrote:
>> Hi Marek,
>>
>> On 13/10/15 16:04, Marek wrote:
>>> Could it be that the raid wasnt synced at the time i overwrote
>>> /dev/sda ? i have run the command you suggested and will report after
>>> its done. would it still be possible to recover the raid?
>>>
>> So what likely happened is that you wrote ontop of a RAID member drive
>> while the RAID was likely not assembled (during your install).
>>
>> If you do a resync it won't unfortunately *recover* the previously
>> written data, it will go over each stripe and recalculate the parity.
>> It is true that it *might* have been possible to do the opposite
>> operation by not touching anything and figuring out the previous data
>> with the data and parity blocks from the remaining RAID members
>> (assuming they weren't overwritten), but it would have been a tricky
>> operation. However, since you've done the resync now, the previous
>> parity blocks have been overwritten leaving you with a consistent
>> RAID, but with part of the data on a disk that you want to revert.
>>
> Or better action would be to stop the array, start the array excluding
> sda (not sure best way to manually mark it failed, or maybe physically
> remove it, or zero superblock should work too). Then you should have
> clean degraded array with all data. Now you could add sda back, and it
> will re-write correct data. (Assuming no writes were done to sdb/sdc/sdd).
>
I agree with Adam, in this particular case the data would be recovered 
on "sda". Doing a plain and simple resync with the array not degraded 
would do another operation which is precisely what you don't want, i.e. 
make sure that the parity are recalculated from the current data.

> Depending on what you actually did, and what order, it might still work....
>
If the resync didn't run while sda was in the array it should, otherwise 
the data recalculated on sda will be identical to the current data.

> Regards,
> Adam
>
>> At this stage you should recover the old data from a backup.
>>
>>> On Tue, Oct 13, 2015 at 4:36 PM, Weedy <weedy2887@gmail.com> wrote:
>>>>
>>>> On 13 Oct 2015 10:06 am, "Marek" <mlf.conv@gmail.com> wrote:
>>>>>
>>>>> no, i had a working raid consisting of 4 2TB drives (raid5)
>>>>> sda,sdb,sdc,sdb ( no partitions were created on drives) - then i
>>>>> accidentally deleted /dev/sda by installing ubuntu. when inspecting
>>>>> dev/md127 with a hex editor i discovered first 12mb are missing.
>>>>
>>>> Wait, if it's a raid5 can't you just resync it?
>>>>
>>>> Also I believe 1.2 metadata is at the end of a partition. If your
>>>> saying the
>>>> raids hasn't noticed you can run a check and it should fix itself.
>>>>
>>>> echo check > /sys/block/mdX/md/sync_action
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>