* [Qemu-devel] Please help corrupt filesystem image
@ 2019-02-17 3:54 Alexander Marx
2019-02-18 20:54 ` John Snow
0 siblings, 1 reply; 6+ messages in thread
From: Alexander Marx @ 2019-02-17 3:54 UTC (permalink / raw)
To: qemu-devel
Dear List!
I have a big problem and hope you can help me.
I built a new windows 2016 domain with virtual servers. 2 dc and 9 rds
hosts.
I was nearly finished with the setup and ready to migrate the users from
old to new domain.
Then i had to restart the physical servers. Unfortunately 1 dc and 2 rds
hosts could not be startet anymore.
Error says:
qemu-img: Could not open 'vm-150-disk-0.qcow2': Could not read qcow2
header: Input/output error
Even worst, a qemu-img check also gave that error. and i was not able
to do anything i found in the internet to fix it.
always that error or the qemu-img could not determine the format.
Is there a chance to fix these images? What can i do?
I have no backup because the domain was not fully configured. But if i
have to rebuild the whole domain, i will need several weeks for it.
Any help is appreciated.
Thank you
Alex
^ permalink raw reply [flat|nested] 6+ messages in thread
* [Qemu-devel] Please help corrupt filesystem image
@ 2019-02-17 4:13 Alexander Marx
2019-02-18 9:51 ` Kevin Wolf
0 siblings, 1 reply; 6+ messages in thread
From: Alexander Marx @ 2019-02-17 4:13 UTC (permalink / raw)
To: qemu-devel
Dear List!
I have a big problem and hope you can help me.
I built a new windows 2016 domain with virtual servers. 2 dc and 9 rds
hosts.
I was nearly finished with the setup and ready to migrate the users from
old to new domain.
Then i had to restart the physical servers. Unfortunately 1 dc and 2 rds
hosts could not be startet anymore.
Error says:
qemu-img: Could not open 'vm-150-disk-0.qcow2': Could not read qcow2
header: Input/output error
Even worst, a qemu-img check also gave that error. and i was not able
to do anything i found in the internet to fix it.
always that error or the qemu-img could not determine the format.
Is there a chance to fix these images? What can i do?
I have no backup because the domain was not fully configured. But if i
have to rebuild the whole domain, i will need several weeks for it.
Any help is appreciated.
Thank you
Alex
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Qemu-devel] Please help corrupt filesystem image
2019-02-17 4:13 [Qemu-devel] Please help corrupt filesystem image Alexander Marx
@ 2019-02-18 9:51 ` Kevin Wolf
2019-02-18 10:31 ` Alexander Marx
0 siblings, 1 reply; 6+ messages in thread
From: Kevin Wolf @ 2019-02-18 9:51 UTC (permalink / raw)
To: Alexander Marx; +Cc: qemu-devel, qemu-block
Am 17.02.2019 um 05:13 hat Alexander Marx geschrieben:
> Dear List!
>
> I have a big problem and hope you can help me.
> I built a new windows 2016 domain with virtual servers. 2 dc and 9 rds
> hosts.
> I was nearly finished with the setup and ready to migrate the users from
> old to new domain.
>
> Then i had to restart the physical servers. Unfortunately 1 dc and 2 rds
> hosts could not be startet anymore.
> Error says:
>
> qemu-img: Could not open 'vm-150-disk-0.qcow2': Could not read qcow2
> header: Input/output error
The very first thing the qcow2 driver in QEMU does is reading in the
image file header, and this is the step that produces the error message
you quoted. This is before even looking at any content of the file. So
it looks to me as if the image file was corrupted on the file system
level or even had bad sectors on the physical hard disk.
You can try running 'qemu-img check' under 'strace -f' and double check
which syscall returns an error before it errors out. I think it will be
the first pread64() call, returning EIO. If not, please post the output
you're getting.
Another thing you could try is whether the image file is still
accessible for tools like cp, dd or hexdump.
If these tests confirm that there is a more general problem with the
file, running fsck for your host file system that contains the image
might be a good idea.
> Even worst, a qemu-img check also gave that error. and i was not able
> to do anything i found in the internet to fix it.
> always that error or the qemu-img could not determine the format.
>
> Is there a chance to fix these images? What can i do?
>
> I have no backup because the domain was not fully configured. But if i
> have to rebuild the whole domain, i will need several weeks for it.
> Any help is appreciated.
If only the header is inaccessible, reconstructing it may be possible.
Otherwise, probably not.
We can look into this if your results for the above show that you can
still access the image outside of QEMU, and what exactly is even going
wrong.
Kevin
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Qemu-devel] Please help corrupt filesystem image
2019-02-18 9:51 ` Kevin Wolf
@ 2019-02-18 10:31 ` Alexander Marx
0 siblings, 0 replies; 6+ messages in thread
From: Alexander Marx @ 2019-02-18 10:31 UTC (permalink / raw)
To: Kevin Wolf; +Cc: qemu-devel, qemu-block
Thank you for replying.
The error was in the underlying filesystem. I had to setup new servers.
Alex
Am 18.02.19 um 10:51 schrieb Kevin Wolf:
> Am 17.02.2019 um 05:13 hat Alexander Marx geschrieben:
>> Dear List!
>>
>> I have a big problem and hope you can help me.
>> I built a new windows 2016 domain with virtual servers. 2 dc and 9 rds
>> hosts.
>> I was nearly finished with the setup and ready to migrate the users from
>> old to new domain.
>>
>> Then i had to restart the physical servers. Unfortunately 1 dc and 2 rds
>> hosts could not be startet anymore.
>> Error says:
>>
>> qemu-img: Could not open 'vm-150-disk-0.qcow2': Could not read qcow2
>> header: Input/output error
> The very first thing the qcow2 driver in QEMU does is reading in the
> image file header, and this is the step that produces the error message
> you quoted. This is before even looking at any content of the file. So
> it looks to me as if the image file was corrupted on the file system
> level or even had bad sectors on the physical hard disk.
>
> You can try running 'qemu-img check' under 'strace -f' and double check
> which syscall returns an error before it errors out. I think it will be
> the first pread64() call, returning EIO. If not, please post the output
> you're getting.
>
> Another thing you could try is whether the image file is still
> accessible for tools like cp, dd or hexdump.
>
> If these tests confirm that there is a more general problem with the
> file, running fsck for your host file system that contains the image
> might be a good idea.
>
>> Even worst, a qemu-img check also gave that error. and i was not able
>> to do anything i found in the internet to fix it.
>> always that error or the qemu-img could not determine the format.
>>
>> Is there a chance to fix these images? What can i do?
>>
>> I have no backup because the domain was not fully configured. But if i
>> have to rebuild the whole domain, i will need several weeks for it.
>> Any help is appreciated.
> If only the header is inaccessible, reconstructing it may be possible.
> Otherwise, probably not.
>
> We can look into this if your results for the above show that you can
> still access the image outside of QEMU, and what exactly is even going
> wrong.
>
> Kevin
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Qemu-devel] Please help corrupt filesystem image
2019-02-17 3:54 Alexander Marx
@ 2019-02-18 20:54 ` John Snow
2019-02-19 4:05 ` Alexander Marx
0 siblings, 1 reply; 6+ messages in thread
From: John Snow @ 2019-02-18 20:54 UTC (permalink / raw)
To: Alexander Marx, qemu-devel; +Cc: Qemu-block
On 2/16/19 10:54 PM, Alexander Marx wrote:
> Dear List!
>
> I have a big problem and hope you can help me.
> I built a new windows 2016 domain with virtual servers. 2 dc and 9 rds
> hosts.
> I was nearly finished with the setup and ready to migrate the users from
> old to new domain.
>
> Then i had to restart the physical servers. Unfortunately 1 dc and 2 rds
> hosts could not be startet anymore.
> Error says:
>
> qemu-img: Could not open 'vm-150-disk-0.qcow2': Could not read qcow2
> header: Input/output error
>
> Even worst, a qemu-img check also gave that error. and i was not able
> to do anything i found in the internet to fix it.
> always that error or the qemu-img could not determine the format.
>
> Is there a chance to fix these images? What can i do?
>
> I have no backup because the domain was not fully configured. But if i
> have to rebuild the whole domain, i will need several weeks for it.
> Any help is appreciated.
>
> Thank you
>
> Alex
>
What versions of QEMU were you running?
What filesystem were these images stored on?
What happened at the time of the corruption, is there some detail that
might help explain the nature of the corruption? Did you shut down the
physical host while the VMs were still running?
There may not be a way to recover these images, but we should probably
make sure that your configuration isn't unsafe to prevent this kind of
problem in the future.
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Qemu-devel] Please help corrupt filesystem image
2019-02-18 20:54 ` John Snow
@ 2019-02-19 4:05 ` Alexander Marx
0 siblings, 0 replies; 6+ messages in thread
From: Alexander Marx @ 2019-02-19 4:05 UTC (permalink / raw)
To: John Snow, qemu-devel; +Cc: Qemu-block
Thank you for replying.
Well i am using latest PROXMOX in a cluster of 4 physical servers.
during the weekend i had to stop all hosts because electricians had to
work on the fuse box.
i shutted down all vm's then powered off all physical hosts. One of them
took very long.
this host had a raid5 of 27 TB on it (XFS) where some vm images were saved.
i waited long time but i knew that at weekends the raid does a patrol
read. So i switched that host hard off.
After rebooting, all hosts and nearly all vms came back online without
problems. only these two images could not be started.
As i told before, they were not critical as they were part of a new
domain which was not used in productuction.
Then i found out that the XFS Filesystem has problems. i stopped all
vm's again and did a xfs_check on that storage.
Some Errors were correted which causes 2 other vm's to die.
These vm's i could restore from backup, so no problem. The other 2
Win216 servers i had to delete and install new.
Thank you very much for your interest on my problem.
It is solved.
Have a nice week and please go on developing that great piece of software!
Am 18.02.19 um 21:54 schrieb John Snow:
>
> On 2/16/19 10:54 PM, Alexander Marx wrote:
>> Dear List!
>>
>> I have a big problem and hope you can help me.
>> I built a new windows 2016 domain with virtual servers. 2 dc and 9 rds
>> hosts.
>> I was nearly finished with the setup and ready to migrate the users from
>> old to new domain.
>>
>> Then i had to restart the physical servers. Unfortunately 1 dc and 2 rds
>> hosts could not be startet anymore.
>> Error says:
>>
>> qemu-img: Could not open 'vm-150-disk-0.qcow2': Could not read qcow2
>> header: Input/output error
>>
>> Even worst, a qemu-img check also gave that error. and i was not able
>> to do anything i found in the internet to fix it.
>> always that error or the qemu-img could not determine the format.
>>
>> Is there a chance to fix these images? What can i do?
>>
>> I have no backup because the domain was not fully configured. But if i
>> have to rebuild the whole domain, i will need several weeks for it.
>> Any help is appreciated.
>>
>> Thank you
>>
>> Alex
>>
> What versions of QEMU were you running?
> What filesystem were these images stored on?
> What happened at the time of the corruption, is there some detail that
> might help explain the nature of the corruption? Did you shut down the
> physical host while the VMs were still running?
>
> There may not be a way to recover these images, but we should probably
> make sure that your configuration isn't unsafe to prevent this kind of
> problem in the future.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-02-19 4:06 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2019-02-17 4:13 [Qemu-devel] Please help corrupt filesystem image Alexander Marx
2019-02-18 9:51 ` Kevin Wolf
2019-02-18 10:31 ` Alexander Marx
-- strict thread matches above, loose matches on Subject: below --
2019-02-17 3:54 Alexander Marx
2019-02-18 20:54 ` John Snow
2019-02-19 4:05 ` Alexander Marx
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).