[RFH] Partion table recovery

[RFH] Partion table recovery

[Al Boldi]

As always, a good friend of mine managed to scratch my partion table by 
cat'ing /dev/full into /dev/sda.  I was able to push him out of the way, but 
at least the first 100MB are gone.  I can probably live without the first 
partion, but there are many partitions after that, which I hope should 
easily be recoverable.

I tried parted, but it's not working out for me.  Does anybody know of a 
simple partition recovery tool, that would just scan the disk for lost 
partions?


Thanks!

--
Al

Re: [RFH] Partion table recovery

[Dave Young]

>On 7/20/07, Al Boldi <a1426z@gawab.com> wrote:
> As always, a good friend of mine managed to scratch my partion table by
> cat'ing /dev/full into /dev/sda.  I was able to push him out of the way, but
/dev/null ?
> at least the first 100MB are gone.  I can probably live without the first
> partion, but there are many partitions after that, which I hope should
> easily be recoverable.
>
> I tried parted, but it's not working out for me.  Does anybody know of a
> simple partition recovery tool, that would just scan the disk for lost
> partions?
>
The best way is to backup you partition table before destroyed.

Re: [RFH] Partion table recovery

[Dave Young]

On 7/20/07, Dave Young <hidave.darkstar@gmail.com> wrote:
> >On 7/20/07, Al Boldi <a1426z@gawab.com> wrote:
> > As always, a good friend of mine managed to scratch my partion table by
> > cat'ing /dev/full into /dev/sda.  I was able to push him out of the way, but
> /dev/null ?
withdraw my wrong comment.

> > at least the first 100MB are gone.  I can probably live without the first
> > partion, but there are many partitions after that, which I hope should
> > easily be recoverable.
> >
> > I tried parted, but it's not working out for me.  Does anybody know of a
> > simple partition recovery tool, that would just scan the disk for lost
> > partions?
> >
> The best way is to backup you partition table before destroyed.
>

And another method is use loop device with the offset option.

Regards
dave

Re: [RFH] Partition table recovery

[Al Boldi]

Dave Young wrote:
> >On 7/20/07, Al Boldi <a1426z@gawab.com> wrote:
> > As always, a good friend of mine managed to scratch my partion table by
> > cat'ing /dev/full into /dev/sda.  I was able to push him out of the way,
> > but
>
> /dev/null ?
>
> > at least the first 100MB are gone.  I can probably live without the
> > first partion, but there are many partitions after that, which I hope
> > should easily be recoverable.
> >
> > I tried parted, but it's not working out for me.  Does anybody know of a
> > simple partition recovery tool, that would just scan the disk for lost
> > partions?
>
> The best way is to backup you partition table before destroyed.

Very true! 

# sfdisk -d 

is a real saviour.  But make sure you don't save it on the same disk you are 
trying to recover.


Thanks!

--
Al

Re: [RFH] Partion table recovery

[James Lamanna]

On 7/19/07, Al Boldi <a1426z@gawab.com> wrote:
> As always, a good friend of mine managed to scratch my partion table by
> cat'ing /dev/full into /dev/sda.  I was able to push him out of the way, but
> at least the first 100MB are gone.  I can probably live without the first
> partion, but there are many partitions after that, which I hope should
> easily be recoverable.
>
> I tried parted, but it's not working out for me.  Does anybody know of a
> simple partition recovery tool, that would just scan the disk for lost
> partions?

Tried gpart?
http://www.stud.uni-hannover.de/user/76201/gpart/


>
>
> Thanks!
>
> --
> Al

-- James

Re: [RFH] Partion table recovery

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 1173 bytes --]

On Thu, 2007-07-19 22:25:49 -0700, James Lamanna <jlamanna@gmail.com> wrote:
> On 7/19/07, Al Boldi <a1426z@gawab.com> wrote:
> >
> > I tried parted, but it's not working out for me.  Does anybody know of a
> > simple partition recovery tool, that would just scan the disk for lost
> > partions?
> 
> Tried gpart?
> http://www.stud.uni-hannover.de/user/76201/gpart/

Right, gpart should be the way to go. I used that several times and
it always worked great. You'll probably loose the first partition at
first (it searches for filesystem signatures and may fail to find the
first partition's filesystem), but it should find all the rest.

Once you've written the guessed partition table, just use cfdisk (or
any other fdisk you're comfy with) to create a maximum-sized partition
in the first free area. Depending on the filesystem used there, you
may even be able to get back the remaining data.

MfG, JBG

-- 
      Jan-Benedict Glaw      jbglaw@lug-owl.de              +49-172-7608481
Signature of:             God put me on earth to accomplish a certain number of
the second  :            things. Right now I am so far behind I will never die.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFH] Partition table recovery

[Al Boldi]

James Lamanna wrote:
> On 7/19/07, Al Boldi <a1426z@gawab.com> wrote:
> > As always, a good friend of mine managed to scratch my partion table by
> > cat'ing /dev/full into /dev/sda.  I was able to push him out of the way,
> > but at least the first 100MB are gone.  I can probably live without the
> > first partion, but there are many partitions after that, which I hope
> > should easily be recoverable.
> >
> > I tried parted, but it's not working out for me.  Does anybody know of a
> > simple partition recovery tool, that would just scan the disk for lost
> > partions?
>
> Tried gpart?
> http://www.stud.uni-hannover.de/user/76201/gpart/

This definitely looks like the ticket.  And also rescuept from util-linux.  
There is only one small problem; I have been regularly adding / deleting / 
resizing partitions, which kind of confuses the scanner.  But still, it's 
better than nothing.

Anton Altaparmakov wrote:
> parted and its derivatives are pile of crap...  They cause corruption
> to totally healthy systems at the best of times.  Don't go near them.
>
> Use TestDisk (http://www.cgsecurity.org/wiki/TestDisk) and be happy.
> (-:

This one really worked best, without getting confused about older partitions.


Thanks everybody!


BTW, what's a partion table?

--
Al

Re: [RFH] Partition table recovery

[Anton Altaparmakov]

On 20 Jul 2007, at 12:29, Al Boldi wrote:
> Anton Altaparmakov wrote:
>> parted and its derivatives are pile of crap...  They cause corruption
>> to totally healthy systems at the best of times.  Don't go near them.
>>
>> Use TestDisk (http://www.cgsecurity.org/wiki/TestDisk) and be happy.
>> (-:
>
> This one really worked best, without getting confused about older  
> partitions.

Yes, TestDisk is very good.

> Thanks everybody!

You are welcome.

> BTW, what's a partion table?

Are you kidding?  If not read those two and you shall be enlightened  
(or utterly confused (-;)...

http://en.wikipedia.org/wiki/Partition_(computing)
http://www.ata-atapi.com/hiwtab.htm

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer, http://www.linux-ntfs.org/

Re: [RFH] Partion table recovery

[Willy Tarreau]

On Fri, Jul 20, 2007 at 08:13:03AM +0300, Al Boldi wrote:
> As always, a good friend of mine managed to scratch my partion table by 
> cat'ing /dev/full into /dev/sda.  I was able to push him out of the way, but 
> at least the first 100MB are gone.  I can probably live without the first 
> partion, but there are many partitions after that, which I hope should 
> easily be recoverable.
> 
> I tried parted, but it's not working out for me.  Does anybody know of a 
> simple partition recovery tool, that would just scan the disk for lost 
> partions?

The best one is simply "fdisk", because you can manually enter your
cylinders numbers. You have to find by hand the beginning of each partition,
and for this, you have to remember what filesystems you used and see how to
identify them (using a magic). Then with an hex editor, you scan the disk to
find such entries and note the possible sectors on a paper. Then comes fdisk.
You create the part, exit and try to mount it. If it fails, fdisk again and
try other values.

I've saved many disks that way, it may sound harder than it really is. It
should not take you more than half an hour to get the first part. Knowing
your approximate partitions size will help too.

Good luck!
Willy

Re: [RFH] Partion table recovery

[Dave Young]

>On 7/20/07, Willy Tarreau <w@1wt.eu> wrote:
> On Fri, Jul 20, 2007 at 08:13:03AM +0300, Al Boldi wrote:
> > As always, a good friend of mine managed to scratch my partion table by
> > cat'ing /dev/full into /dev/sda.  I was able to push him out of the way, but
> > at least the first 100MB are gone.  I can probably live without the first
> > partion, but there are many partitions after that, which I hope should
> > easily be recoverable.
> >
> > I tried parted, but it's not working out for me.  Does anybody know of a
> > simple partition recovery tool, that would just scan the disk for lost
> > partions?
>
> The best one is simply "fdisk", because you can manually enter your
> cylinders numbers. You have to find by hand the beginning of each partition,
> and for this, you have to remember what filesystems you used and see how to
> identify them (using a magic). Then with an hex editor, you scan the disk to
> find such entries and note the possible sectors on a paper. Then comes fdisk.
> You create the part, exit and try to mount it. If it fails, fdisk again and
> try other values.
One more possible solution about this, you can find the filesystem
magic position, if ext2/ext3 , /dev/sda can be mounted with option
sb=blocknumber
>
> I've saved many disks that way, it may sound harder than it really is. It
> should not take you more than half an hour to get the first part. Knowing
> your approximate partitions size will help too.
>
> Good luck!
> Willy

Re: [RFH] Partion table recovery

[Jan Engelhardt]

On Jul 20 2007 07:35, Willy Tarreau wrote:
>On Fri, Jul 20, 2007 at 08:13:03AM +0300, Al Boldi wrote:
>> As always, a good friend of mine managed to scratch my partion table by 
>> cat'ing /dev/full into /dev/sda.  I was able to push him out of the way, but 
>> at least the first 100MB are gone.  I can probably live without the first 
>> partion, but there are many partitions after that, which I hope should 
>> easily be recoverable.

Go use GPT, it's got a backup copy of the ptable at then end of the disk ;-)

>> 
>> I tried parted, but it's not working out for me.  Does anybody know of a 
>> simple partition recovery tool, that would just scan the disk for lost 
>> partions?
>
>The best one is simply "fdisk", because you can manually enter your
>cylinders numbers. You have to find by hand the beginning of each partition,
>and for this, you have to remember what filesystems you used and see how to
>identify them (using a magic). Then with an hex editor, you scan the disk to
>find such entries and note the possible sectors on a paper. Then comes fdisk.
>You create the part, exit and try to mount it. If it fails, fdisk again and
>try other values.

Pretty easy:
"XFSB" (offset +0),
"ReIsErFs2" (offset +0x10034),
"SWAPSPACE2" (offset +0xff6),
"FAT32" (offset +0x52h, maybe harder) / "mkdofs" (+0x3)
ext2/3 is TOUGH. (sequence 0x53ef at +0x438 - quite ambiguous!)

>
>I've saved many disks that way, it may sound harder than it really is. It
>should not take you more than half an hour to get the first part. Knowing
>your approximate partitions size will help too.

>
>Good luck!
>Willy
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/
>

	Jan
-- 

Re: [RFH] Partition table recovery

[Al Boldi]

Jan Engelhardt wrote:
> On Jul 20 2007 07:35, Willy Tarreau wrote:
> >On Fri, Jul 20, 2007 at 08:13:03AM +0300, Al Boldi wrote:
> >> As always, a good friend of mine managed to scratch my partion table by
> >> cat'ing /dev/full into /dev/sda.  I was able to push him out of the
> >> way, but at least the first 100MB are gone.  I can probably live
> >> without the first partion, but there are many partitions after that,
> >> which I hope should easily be recoverable.
>
> Go use GPT, it's got a backup copy of the ptable at then end of the disk
> ;-)

Would that be compatible with md?

> >> I tried parted, but it's not working out for me.  Does anybody know of
> >> a simple partition recovery tool, that would just scan the disk for
> >> lost partions?
> >
> >The best one is simply "fdisk", because you can manually enter your
> >cylinders numbers. You have to find by hand the beginning of each
> > partition, and for this, you have to remember what filesystems you used
> > and see how to identify them (using a magic). Then with an hex editor,
> > you scan the disk to find such entries and note the possible sectors on
> > a paper. Then comes fdisk. You create the part, exit and try to mount
> > it. If it fails, fdisk again and try other values.
>
> Pretty easy:
> "XFSB" (offset +0),
> "ReIsErFs2" (offset +0x10034),
> "SWAPSPACE2" (offset +0xff6),
> "FAT32" (offset +0x52h, maybe harder) / "mkdofs" (+0x3)
> ext2/3 is TOUGH. (sequence 0x53ef at +0x438 - quite ambiguous!)

Actually, there is an easier way to do this with cfdisk.  Once you know where 
a partition starts, enter it into cfdisk and set the size to something 
short, then save and restart cfdisk. If the position is correct, then cfdisk 
will identify the fs type.  Now exit again and do an fsck, which will error 
out complaining about a dev-size/fs-size mismatch.  Adjust the cfdisk 
partition accordingly, and cycle through the rest of the partitions.

But, I want something much more automated.  And the partition table backup 
per partition entry isn't really a bad idea.


Thanks!

--
Al

Re: [RFH] Partition table recovery

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 481 bytes --]

On Fri, 2007-07-20 14:29:34 +0300, Al Boldi <a1426z@gawab.com> wrote:
> 
> But, I want something much more automated.  And the partition table backup 
> per partition entry isn't really a bad idea.

That's called `gpart'.

MfG, JBG

-- 
      Jan-Benedict Glaw      jbglaw@lug-owl.de              +49-172-7608481
Signature of:              Fortschritt bedeutet, einen Schritt so zu machen,
the second  :                   daß man den nächsten auch noch machen kann.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFH] Partition table recovery

[Al Boldi]

Jan-Benedict Glaw wrote:
> On Fri, 2007-07-20 14:29:34 +0300, Al Boldi <a1426z@gawab.com> wrote:
> > But, I want something much more automated.  And the partition table
> > backup per partition entry isn't really a bad idea.
>
> That's called `gpart'.

Oh, gpart is great, but if we had a backup copy of the partition table on 
every partition location on disk, then this backup copy could easily be 
reused to reconstruct the original partition table without further 
searching.  Just like the NetWare approach, and in some respect like the 
ext2/3 superblock backups.


Thanks!

--
Al

Re: [RFH] Partition table recovery

[Rene Herman]

On 07/20/2007 02:22 PM, Al Boldi wrote:

> Oh, gpart is great, but if we had a backup copy of the partition table on
> every partition location on disk, then this backup copy could easily be
> reused to reconstruct the original partition table without further 
> searching.

As long as you don't reboot you have a "backup copy" in kernel. Admittedly 
not in a very nice format, but /sys/block/<disk>/<part>/start and size have 
saved my butt a few times...

Rene.

Re: [RFH] Partition table recovery

[Theodore Tso]

On Fri, Jul 20, 2007 at 03:22:17PM +0300, Al Boldi wrote:
> 
> Oh, gpart is great, but if we had a backup copy of the partition table on 
> every partition location on disk, then this backup copy could easily be 
> reused to reconstruct the original partition table without further 
> searching.  Just like the NetWare approach, and in some respect like the 
> ext2/3 superblock backups.

It wouldn't be that hard to put a backup partition table at the
beginning of an ext2/3 filesystem.  No one is currently using the
space between offset 512 and 1023 bytes, and it would be an easy place
to stash a backup copy of the partition table.  We wouldn't be able to
use the MBR format, since information about extended partitions are
stored scattered across the disk.

So for the sake of argument, I'll propose the following partition
backup, starting at offset 512 and going for 512 bytes to byte #1023:

offset 
from 512  Description
-------   -----------
0..7      Signature ASCII: "PARTBAK1"
8..9      Part-type: 1=MBR
10	  # of heads
11	  # sectors
12	  # of partitions in the backup
13..15    Reserved (must be zero)
16..31	  first part entry
...
496..511 31st partition entry

Partition entry (16 bytes)

offset    description
------    -----------
0	  Flags
1	  Type
2..3	  Reserved (must be zero)
4..11	  Start LBA (little endian)
12..15	  # of LBA in partition

Obviously this won't work for LVM or EFI volume partition tables, but
they have enough redundancy into their on-disk formats that it
shouldn't be an issue.

Someone want to write a stand-alone function which pulls the
information from the MBR (and extended MBR's) and put it into a 512
byte buffer?  If so, I'll integrate it into mke2fs and e2fsck (so that
when the user upgrades to a new enough e2fsprogs, it will
automatically back up the partition information just before the
superblock.

						- Ted

Re: [RFH] Partition table recovery

[Rene Herman]

On 07/20/2007 06:06 PM, Theodore Tso wrote:

> It wouldn't be that hard to put a backup partition table at the beginning
> of an ext2/3 filesystem.  No one is currently using the space between
> offset 512 and 1023 bytes, and it would be an easy place to stash a
> backup copy of the partition table.  We wouldn't be able to use the MBR
> format, since information about extended partitions are stored scattered
> across the disk.

I was looking into this, but I'm afraid I believe it's not a very good idea.

> So for the sake of argument, I'll propose the following partition
> backup, starting at offset 512 and going for 512 bytes to byte #1023:
> 
> offset 
> from 512  Description
> -------   -----------
> 0..7      Signature ASCII: "PARTBAK1"
> 8..9      Part-type: 1=MBR

Not sure what you mean with this type field. You were suggesting to only 
backup "Plain Ol' Partition" tables anyway weren't you?

> 10	  # of heads
> 11	  # sectors

This is a problem. Today the CHS fields in the partition entries don't mean 
much of anything anymore and Linux happily ignores them but DOS and (hence) 
Windows 9x do not. From time to time I still have the Windows 98 install 
that's sitting in a corner of my disk throw a fit just by having set the 
BIOS from LBA to Large (meaning the geometry the BIOS pretends the disk has 
changes) for example. Old DOS installs that I keep around for the purpose of 
hardware testing with the originally supplied drivers make for even more of 
a "don't touch, don't touch!" thing -- various version of DOS throw fits for 
various reasons.

As such, really the only functional thing to do is backup _whatever_ is 
there in the CHS fields in the entries currently and unfortunately this can 
vary between entries, and certainly between runs of whatever program creates 
them. For example, the user may have adjusted the layout in the BIOS or some 
or all partitions may have been created while the disk was in a different 
machine all together. That is to say, really no attempt should be made at 
being smart about the CHS values -- a backup should just store the actual 
(16-byte) partition table entry.

> 12	  # of partitions in the backup
> 13..15    Reserved (must be zero)
> 16..31	  first part entry
> ...
> 496..511 31st partition entry

And this is the biggest problem. IDE for example allows 63 partitions and 
while in practice not many people will actually also have used that many, 31 
isn't hugely roomy either especially due to the way extended partitions are 
kept.

An extended partition is a partition with type (0x05 || 0x0F || 0x85) the 
first sector of which contains another partition table. That second-level 
partition table generally consists of two entries; the first for the logical 
(actual) partition and the second being yet another extended partition, 
where walking the list gets you all.

However, there can be more than just 1 extended partition on a disk and this 
isn't (or wasn't...) in fact uncommon. Windows 9x at least used to create 
multiple partitions as 1 primary and the rest (even if just 1 as well) as 
logicals inside one 0x05/0x0F extended. When you then installed Linux onto 
the same disk, needed more than the 3 MBR entries left, and wanted to make 
sure that Windows wouldn't trip over any non-fat entries in the list of 
extended partitions (which it did) you'd create 1 0x85 "Linux only" extended 
  in the MBR that Windows just ignores -- not a problem since it couldn't 
read any of the filesystems on them anyway.

This already means you need to keep more information that just the logicals 
(the actual partitions) since you can't reliably restore the tables from a 
list of them alone.

Moveover, the "generally consists of two entries" above isn't guaranteed 
either. The second-level table _can_ contain more than 1 logical. Linux 
never minded much of anything and back when I was experimenting with more 
operating systems (on one disk) they did -- I used to mostly create my 
partition tables with Norton Diskedit at the time...

A bullet-proof backup then has to really store these actual second-level 
tables verbatim again as you can't assume too much about them meaning you're 
going to be storing a complete 4-entry table for every logical which 
ofcourse eats space in the available 512 bytes / 31 entries _way_ too fast.

Moreover once more -- why stop at "normal" logicals? I for example have a 
MINIX 2 partition sitting on this disk and it uses a subpartitioning scheme 
through the same DOS-style second-level partition-table setup. First entry 
in the second-level table is generally used for / and third for /usr meaning 
two additional partitions. And if two aren't bad enough, from time to time 
(when after buying a new disk my ogg vorbis collection hasn't yet grown to 
fill it again yet) I keep a FreeBSD install around which means 6 or so 
additional "slices".

Ignoring those alien partitions is sort of okay, but as said, with only 512 
bytes to spare, this is all inevitable going to be a "sorta works most of 
the time perhaps" sort of solution. The best solution is a little program 
that backs things up verbatim to wherever you want (such as a floppy or USB 
stick for example).

sfdisk -d already works most of the time. Not as a verbatim tool (I actually 
semi-frequently use a "sfdisk -d /dev/hda | sfdisk" invocation as a way to 
_rewrite_ the CHS fields to other values after changing machines around on a 
disk) but something you'd backup on the FS level should, in my opinion, need 
to be less fragile than would be possible with just 512 bytes available.

Rene.

Re: [RFH] Partition table recovery

[Al Boldi]

Theodore Tso wrote:
> On Sat, Jul 21, 2007 at 07:54:14PM +0200, Rene Herman wrote:
> > sfdisk -d already works most of the time. Not as a verbatim tool (I
> > actually semi-frequently use a "sfdisk -d /dev/hda | sfdisk" invocation
> > as a way to _rewrite_ the CHS fields to other values after changing
> > machines around on a disk) but something you'd backup on the FS level
> > should, in my opinion, need to be less fragile than would be possible
> > with just 512 bytes available.
>
> *IF* you remember to store the sfdisk -d somewhere useful.  In my "How
> To Recover From Hard Drive Catastrophies" classes, I tell them to
> print out a copy of "sfdisk -l /dev/hda ; sfdisk -d /dev/hda" and tape
> it to the side of the computer.  I also tell them do regular backups.
> What to make a guess how many them actually follow this good advice?
> Far fewer than I would like, I suspect...
>
> What I'm suggesting is the equivalent of sfdisk -d, except we'd be
> doing it automatically without requiring the user to take any kind of
> explicit action.  Is it perfect?  No, although the edge conditions are
> quite rare these days and generally involve users using legacy systems
> and/or doing Weird Shit such that They Really Should Know To Do Their
> Own Explicit Backups.  But for the novice users, it should work Just
> Fine.

Sounds great, but it may be advisable to hook this into the partition 
modification routines instead of mkfs/fsck.  Which would mean that the 
partition manager could ask the kernel to instruct its fs subsystem to 
update the backup partition table for each known fs-type that supports such 
a feature.


Thanks!

--
Al

Re: [RFH] Partition table recovery

[Theodore Tso]

On Sun, Jul 22, 2007 at 07:10:31AM +0300, Al Boldi wrote:
> Sounds great, but it may be advisable to hook this into the partition 
> modification routines instead of mkfs/fsck.  Which would mean that the 
> partition manager could ask the kernel to instruct its fs subsystem to 
> update the backup partition table for each known fs-type that supports such 
> a feature.

Well, let's think about this a bit.  What are the requirements?

1) The partition manager should be able explicitly request that a new
backup of the partition tables be stashed in each filesystem that has
room for such a backup.  That way, when the user affirmatively makes a
partition table change, it can get backed up in all of the right
places automatically.

2) The fsck program should *only* stash a backup of the partition
table if there currently isn't one in the filesystem.  It may be that
the partition table has been corrupted, and so merely doing an fsck
should not transfer a current copy of the partition table to the
filesystem-secpfic backup area.  It could be that the partition table
was only partially recovered, and we don't want to overwrite the
previously existing backups except on an explicit request from the
system administrator.

3) The mkfs program should automatically create a backup of the
current partition table layout.  That way we get a backup in the newly
created filesystem as soon as it is created.

4) The exact location of the backup may vary from filesystem to
filesystem.  For ext2/3/4, bytes 512-1023 are always unused, and don't
interfere with the boot sector at bytes 0-511, so that's the obvious
location.  Other filesystems may have that location in use, and some
other location might be a better place to store it.  Ideally it will
be a well-known location, that isn't dependent on finding an inode
table, or some such, but that may not be possible for all filesystems.

OK, so how about this as a solution that meets the above requirements?

/sbin/partbackup <device> [<fspart>]

	Will scan <device> (i.e., /dev/hda, /dev/sdb, etc.) and create
	a 512 byte partition backup, using the format I've previously
	described.  If <fspart> is specified on the command line, it
	will use the blkid library to determine the filesystem type of
	<fspart>, and then attempt to execute
	/dev/partbackupfs.<fstype> to write the partition backup to
	<fspart>.  If <fspart> is '-', then it will write the 512 byte
	partition table to stdout.  If <fspart> is not specified on
	the command line, /sbin/partbackup will iterate over all
	partitions in <device>, use the blkid library to attempt to
	determine the correct filesystem type, and then execute 
	/sbin/partbackupfs.<fstype> if such a backup program exists.

/sbin/partbackupfs.<fstype> <fspart>

	... is a filesystem specific program for filesystem type
	<fstype>.  It will assure that <fspart> (i.e., /dev/hda1,
	/dev/sdb3) is of an appropriate filesystem type, and then read
	512 bytes from stdin and write it out to <fspart> to an
	appropriate place for that filesystem.

Partition managers will be encouraged to check to see if
/sbin/partbackup exists, and if so, after the partition table is
written, will check to see if /sbin/partbackup exists, and if so, to
call it with just one argument (i.e., /sbin/partbackup /dev/hdb).
They SHOULD provide an option for the user to suppress the backup from
happening, but the backup should be the default behavior.

An /etc/mkfs.<fstype> program is encouraged to run /sbin/partbackup
with two arguments (i.e., /sbin/partbackup /dev/hdb /dev/hdb3) when
creating a filesystem.

An /etc/fsck.<fstype> program is encouraged to check to see if a
partition backup exists (assuming the filesystem supports it), and if
not, call /sbin/partbackup with two arguments.

A filesystem utility package for a particular filesystem type is
encouraged to make the above changes to its mkfs and fsck programs, as
well as provide an /sbin/partbackupfs.<fstype> program.

I would do this all in userspace, though.  Is there any reason to get
the kernel involved?  I don't think so.

						- Ted

Re: [RFH] Partition table recovery

[Al Boldi]

Theodore Tso wrote:
> On Sun, Jul 22, 2007 at 07:10:31AM +0300, Al Boldi wrote:
> > Sounds great, but it may be advisable to hook this into the partition
> > modification routines instead of mkfs/fsck.  Which would mean that the
> > partition manager could ask the kernel to instruct its fs subsystem to
> > update the backup partition table for each known fs-type that supports
> > such a feature.
>
> Well, let's think about this a bit.  What are the requirements?
>
> 1) The partition manager should be able explicitly request that a new
> backup of the partition tables be stashed in each filesystem that has
> room for such a backup.  That way, when the user affirmatively makes a
> partition table change, it can get backed up in all of the right
> places automatically.
>
> 2) The fsck program should *only* stash a backup of the partition
> table if there currently isn't one in the filesystem.  It may be that
> the partition table has been corrupted, and so merely doing an fsck
> should not transfer a current copy of the partition table to the
> filesystem-secpfic backup area.  It could be that the partition table
> was only partially recovered, and we don't want to overwrite the
> previously existing backups except on an explicit request from the
> system administrator.
>
> 3) The mkfs program should automatically create a backup of the
> current partition table layout.  That way we get a backup in the newly
> created filesystem as soon as it is created.
>
> 4) The exact location of the backup may vary from filesystem to
> filesystem.  For ext2/3/4, bytes 512-1023 are always unused, and don't
> interfere with the boot sector at bytes 0-511, so that's the obvious
> location.  Other filesystems may have that location in use, and some
> other location might be a better place to store it.  Ideally it will
> be a well-known location, that isn't dependent on finding an inode
> table, or some such, but that may not be possible for all filesystems.
>
> OK, so how about this as a solution that meets the above requirements?
>
> /sbin/partbackup <device> [<fspart>]
>
> 	Will scan <device> (i.e., /dev/hda, /dev/sdb, etc.) and create
> 	a 512 byte partition backup, using the format I've previously
> 	described.  If <fspart> is specified on the command line, it
> 	will use the blkid library to determine the filesystem type of
> 	<fspart>, and then attempt to execute
> 	/dev/partbackupfs.<fstype> to write the partition backup to
> 	<fspart>.  If <fspart> is '-', then it will write the 512 byte
> 	partition table to stdout.  If <fspart> is not specified on
> 	the command line, /sbin/partbackup will iterate over all
> 	partitions in <device>, use the blkid library to attempt to
> 	determine the correct filesystem type, and then execute
> 	/sbin/partbackupfs.<fstype> if such a backup program exists.
>
> /sbin/partbackupfs.<fstype> <fspart>
>
> 	... is a filesystem specific program for filesystem type
> 	<fstype>.  It will assure that <fspart> (i.e., /dev/hda1,
> 	/dev/sdb3) is of an appropriate filesystem type, and then read
> 	512 bytes from stdin and write it out to <fspart> to an
> 	appropriate place for that filesystem.
>
> Partition managers will be encouraged to check to see if
> /sbin/partbackup exists, and if so, after the partition table is
> written, will check to see if /sbin/partbackup exists, and if so, to
> call it with just one argument (i.e., /sbin/partbackup /dev/hdb).
> They SHOULD provide an option for the user to suppress the backup from
> happening, but the backup should be the default behavior.
>
> An /etc/mkfs.<fstype> program is encouraged to run /sbin/partbackup
> with two arguments (i.e., /sbin/partbackup /dev/hdb /dev/hdb3) when
> creating a filesystem.
>
> An /etc/fsck.<fstype> program is encouraged to check to see if a
> partition backup exists (assuming the filesystem supports it), and if
> not, call /sbin/partbackup with two arguments.
>
> A filesystem utility package for a particular filesystem type is
> encouraged to make the above changes to its mkfs and fsck programs, as
> well as provide an /sbin/partbackupfs.<fstype> program.

Great!

> I would do this all in userspace, though.  Is there any reason to get
> the kernel involved?  I don't think so.

Yes, doing things in userspace, when possible, is much better.  But, a change 
in the partition table has to be relayed to the kernel, and when that change 
happens to be on a mounted disk, then the partition manager complains of not 
being able to update the kernel's view.  So how can this be addressed?


Thanks!

--
Al

Re: [RFH] Partition table recovery

[Indan Zupancic]

On Sun, July 22, 2007 18:28, Theodore Tso wrote:
> On Sun, Jul 22, 2007 at 07:10:31AM +0300, Al Boldi wrote:
>> Sounds great, but it may be advisable to hook this into the partition
>> modification routines instead of mkfs/fsck.  Which would mean that the
>> partition manager could ask the kernel to instruct its fs subsystem to
>> update the backup partition table for each known fs-type that supports such
>> a feature.
>
> Well, let's think about this a bit.  What are the requirements?
>
> 1) The partition manager should be able explicitly request that a new
> backup of the partition tables be stashed in each filesystem that has
> room for such a backup.  That way, when the user affirmatively makes a
> partition table change, it can get backed up in all of the right
> places automatically.
>
> 2) The fsck program should *only* stash a backup of the partition
> table if there currently isn't one in the filesystem.  It may be that
> the partition table has been corrupted, and so merely doing an fsck
> should not transfer a current copy of the partition table to the
> filesystem-secpfic backup area.  It could be that the partition table
> was only partially recovered, and we don't want to overwrite the
> previously existing backups except on an explicit request from the
> system administrator.
>
> 3) The mkfs program should automatically create a backup of the
> current partition table layout.  That way we get a backup in the newly
> created filesystem as soon as it is created.
>
> 4) The exact location of the backup may vary from filesystem to
> filesystem.  For ext2/3/4, bytes 512-1023 are always unused, and don't
> interfere with the boot sector at bytes 0-511, so that's the obvious
> location.  Other filesystems may have that location in use, and some
> other location might be a better place to store it.  Ideally it will
> be a well-known location, that isn't dependent on finding an inode
> table, or some such, but that may not be possible for all filesystems.

To be on the safe side, maybe also add a checksum, timestamp and
something identifying the disk the filesystem was created on.

Regards,

Indan

Re: [RFH] Partition table recovery

[Rene Herman]

[-- Attachment #1: Type: text/plain, Size: 4314 bytes --]

On 07/22/2007 06:28 PM, Theodore Tso wrote:

[ Al -- don't drop CCs please ]

> Well, let's think about this a bit.  What are the requirements?
> 
> 1) The partition manager should be able explicitly request that a new
> backup of the partition tables be stashed in each filesystem that has
> room for such a backup.  That way, when the user affirmatively makes a
> partition table change, it can get backed up in all of the right
> places automatically.

D-Bus? ;-)

> 2) The fsck program should *only* stash a backup of the partition
> table if there currently isn't one in the filesystem.  It may be that
> the partition table has been corrupted, and so merely doing an fsck
> should not transfer a current copy of the partition table to the
> filesystem-secpfic backup area.  It could be that the partition table
> was only partially recovered, and we don't want to overwrite the
> previously existing backups except on an explicit request from the
> system administrator.
> 
> 3) The mkfs program should automatically create a backup of the
> current partition table layout.  That way we get a backup in the newly
> created filesystem as soon as it is created.

On an integrated system like this, do you consider it acceptable to only do 
the MS-DOS partitions and not the other types that may be present _inside_ 
those partitions? (MINIX subpartitions, BSD slices, ...). I believe those 
should really also be done, but this would require keeping more information 
again.

> 4) The exact location of the backup may vary from filesystem to
> filesystem.  For ext2/3/4, bytes 512-1023 are always unused, and don't
> interfere with the boot sector at bytes 0-511, so that's the obvious
> location.  Other filesystems may have that location in use, and some
> other location might be a better place to store it.  Ideally it will
> be a well-known location, that isn't dependent on finding an inode
> table, or some such, but that may not be possible for all filesystems.
> 
> OK, so how about this as a solution that meets the above requirements?
> 
> /sbin/partbackup <device> [<fspart>]
> 
> 	Will scan <device> (i.e., /dev/hda, /dev/sdb, etc.) and create
> 	a 512 byte partition backup, using the format I've previously
> 	described.  If <fspart> is specified on the command line, it
> 	will use the blkid library to determine the filesystem type of
> 	<fspart>, and then attempt to execute
> 	/dev/partbackupfs.<fstype> to write the partition backup to
> 	<fspart>.  If <fspart> is '-', then it will write the 512 byte
> 	partition table to stdout.  If <fspart> is not specified on
> 	the command line, /sbin/partbackup will iterate over all
> 	partitions in <device>, use the blkid library to attempt to
> 	determine the correct filesystem type, and then execute 
> 	/sbin/partbackupfs.<fstype> if such a backup program exists.

I've cleaned up what I posted yesterday a bit and made it into the type of 
standalone-by-design program you suggest here (the version from yesterday 
required a -DTEST to be so). Not with the blkid bits though. Just dumps the 
sector to stdout (and a textual version to stderr if compiled with -DDEBUG).

I (very) briefly looked at blkid but unless I'm mistaken blkid needs device 
names? The documentation seems to be missing. When scanning the device for 
the partition table, we've built a list of partitions with offsets into the 
device and it would be nice if we could hand the fd and the offset off to 
something directly. If the program has to construct device names itself 
there's another truckload of pitfalls right there.

It wouldn't be hard to log minors as such, but you'd also need to be very 
sure you'd always do this in the same order as the kernel so that what I 
consider to be "/dev/sda2" is the same the kernel considers it to be. This 
is again rather fragile.

It might in fact make sense to just ask the kernel for the partitions on a 
device and not bother with scanning anything ourselves. Ie, just walk sysfs. 
Would you agree? This siginificantly reduces the risk of things getting out 
of sync, both scanning order and implementation.

The kernel doesn't currently store/export everything you'd want to store in 
a backup (as far as I'm aware, that is) but that could conceivably change. 
It would make things significantly less fragile.

Rene.

[-- Attachment #2: partbackup.c --]
[-- Type: text/plain, Size: 4583 bytes --]

/*
 * Public Domain 2007, Rene Herman
 */

#define _LARGEFILE64_SOURCE

#include <stdlib.h>
#include <stdio.h>
#include <stdint.h>
#include <string.h>

#include <sys/types.h>
#include <sys/stat.h>
#include <sys/ioctl.h>
#include <unistd.h>
#include <fcntl.h>

enum {
	DOS_EXTENDED   = 0x05,
	WIN98_EXTENDED = 0x0f,
	LINUX_EXTENDED = 0x85,
};

struct partition {
        uint8_t boot_ind;
	uint8_t head;
	uint8_t sector;
	uint8_t cyl;
        uint8_t sys_ind;
	uint8_t end_head;
	uint8_t end_sector;
	uint8_t end_cyl;
        uint32_t start;
        uint32_t size;
} __attribute__((packed));

struct entry {
	uint8_t flags;
	uint8_t type;
	uint16_t __1;
	uint64_t start;
	uint32_t size;
} __attribute__((packed));

enum {
	ENTRY_FLAG_PRIMARY  = 0x01,
	ENTRY_FLAG_BOOTABLE = 0x80,
};

struct backup {
	uint8_t signature[8];
	uint16_t type;
	uint8_t heads;
	uint8_t sectors;
	uint8_t count;
	uint8_t __1[3];
	struct entry table[31];
} __attribute__((packed));

#define BACKUP_SIGNATURE "PARTBAK1"

enum {
	BACKUP_TYPE_MBR = 1,
};

struct backup backup = {
	.signature = BACKUP_SIGNATURE,
	.type	   = BACKUP_TYPE_MBR,
};

#define ARRAY_SIZE(arr) (sizeof arr / sizeof arr[0])

int is_extended(struct partition *partition)
{
	int ret = 0;

	switch (partition->sys_ind) {
	case DOS_EXTENDED:
	case WIN98_EXTENDED:
	case LINUX_EXTENDED:
		ret = 1;
	}
	return ret;
}

unsigned char *get_sector(int fd, uint64_t n)
{
	unsigned char *sector;

	if (lseek64(fd, n << 9, SEEK_SET) < 0) {
		perror("lseek64");
		return NULL;
	}
	sector = malloc(512);
	if (!sector) {
		fprintf(stderr, "malloc: out of memory\n");
		return NULL;
	}
	if (read(fd, sector, 512) != 512) {
		perror("read");
		free(sector);
		return NULL;
	}
	return sector;
}

void put_sector(unsigned char *sector)
{
	free(sector);
}

#define TABLE_OFFSET (512 - 2 - 4 * sizeof(struct partition))

inline struct partition *table(unsigned char *sector)
{
	return (struct partition *)(sector + TABLE_OFFSET);
}

int do_sector(int fd, uint32_t offset, uint32_t start)
{
	unsigned char *sector;
	struct partition *cur;
	struct partition *ext = NULL;
	int ret = 0;

	sector = get_sector(fd, offset + start);
	if (!sector)
		return -1;

	if (sector[510] != 0x55 || sector[511] != 0xaa) {
		ret = -1;
		goto out;
	}
	for (cur = table(sector); cur < table(sector) + 4; cur++) {
		struct entry *entry;

		if (!cur->size)
			continue;

		cur->end_head += 1;
		if (backup.heads < cur->end_head)
			backup.heads = cur->end_head;

		cur->end_sector &= 0x3f;
		if (backup.sectors < cur->end_sector) 
			backup.sectors = cur->end_sector;

		if (is_extended(cur)) {
			if (!offset) {
				ret = do_sector(fd, cur->start, 0);
				if (ret < 0)
					goto out;
			} else if (!ext)
				ext = cur;
			continue;
		}
		if (backup.count == ARRAY_SIZE(backup.table)) {
			fprintf(stderr, "do_sector: out of space!\n");
			ret = -1;
			goto out;
		}
		entry = backup.table + backup.count++;

		entry->flags = cur->boot_ind;
		if (!offset)
			entry->flags |= ENTRY_FLAG_PRIMARY;

		entry->type  = cur->sys_ind;
		entry->start = cur->start + start;
		entry->size  = cur->size;
	}
	if (ext)
		ret = do_sector(fd, offset, ext->start);
  out:
	put_sector(sector);
	return ret;
}

void show_backup(void)
{
#ifdef DEBUG

	int i;

	fprintf(stderr, "signature: ");
	for (i = 0; i < 8; i++)
		fprintf(stderr, "%c", backup.signature[i]);
	fprintf(stderr, "\n");

	fprintf(stderr, "     type: %d\n", backup.type);
	fprintf(stderr, "    heads: %d\n", backup.heads);
	fprintf(stderr, "  sectors: %d\n", backup.sectors);
	fprintf(stderr, "    count: %d\n", backup.count);

	for (i = 0; i < backup.count; i++) {
		fprintf(stderr, "\n");
		fprintf(stderr, "%2d: flags: %02x\n", i, backup.table[i].flags);
		fprintf(stderr, "%2d:  type: %02x\n", i, backup.table[i].type);
		fprintf(stderr, "%2d: start: %llu\n", i, backup.table[i].start);
		fprintf(stderr, "%2d:  size: %u\n", i, backup.table[i].size);
	}
#endif
}

int main(int argc, char *argv[])
{
	int fd;
	struct stat buf;

	if (argc != 2) {
		printf("%s <blkdev>\n", argv[0]);
		return EXIT_FAILURE;
	}
	fd = open(argv[1], O_RDONLY);
	if (fd < 0) {
		perror("open");
		return EXIT_FAILURE;
	}
	if (fstat(fd, &buf) < 0) {
		perror("stat");
		return EXIT_FAILURE;
	}
	if (!S_ISBLK(buf.st_mode))
		fprintf(stderr, "warning: not a block device\n");
	
	if (do_sector(fd, 0, 0) < 0)
		return EXIT_FAILURE;

	if (close(fd) < 0) {
		perror("close");
		return EXIT_FAILURE;
	}
	if (write(STDOUT_FILENO, &backup, sizeof backup) != sizeof backup) {
		perror("write");
		return EXIT_FAILURE;
	}

	show_backup();

	return EXIT_SUCCESS;
}

Re: [RFH] Partition table recovery

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 5916 bytes --]

On Mon, 2007-07-23 10:15:21 +0200, Rene Herman <rene.herman@gmail.com> wrote:
> /*
>  * Public Domain 2007, Rene Herman
>  */
> 
> #define _LARGEFILE64_SOURCE
> 
> #include <stdlib.h>
> #include <stdio.h>
> #include <stdint.h>
> #include <string.h>
> 
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <sys/ioctl.h>
> #include <unistd.h>
> #include <fcntl.h>
> 
> enum {
> 	DOS_EXTENDED   = 0x05,
> 	WIN98_EXTENDED = 0x0f,
> 	LINUX_EXTENDED = 0x85,
> };
> 
> struct partition {
>         uint8_t boot_ind;
> 	uint8_t head;

Different indention.

> 	uint8_t sector;
> 	uint8_t cyl;
>         uint8_t sys_ind;
> 	uint8_t end_head;
> 	uint8_t end_sector;
> 	uint8_t end_cyl;
>         uint32_t start;
>         uint32_t size;

As multibyte on-disk variables, these will need LE/BE conversion.

> } __attribute__((packed));
> 
> struct entry {
> 	uint8_t flags;
> 	uint8_t type;
> 	uint16_t __1;
> 	uint64_t start;
> 	uint32_t size;

Dito.

> } __attribute__((packed));
> 
> enum {
> 	ENTRY_FLAG_PRIMARY  = 0x01,
> 	ENTRY_FLAG_BOOTABLE = 0x80,
> };
> 
> struct backup {
> 	uint8_t signature[8];
> 	uint16_t type;
> 	uint8_t heads;
> 	uint8_t sectors;
> 	uint8_t count;
> 	uint8_t __1[3];
> 	struct entry table[31];
> } __attribute__((packed));
> 
> #define BACKUP_SIGNATURE "PARTBAK1"
> 
> enum {
> 	BACKUP_TYPE_MBR = 1,
> };
> 
> struct backup backup = {
> 	.signature = BACKUP_SIGNATURE,
> 	.type	   = BACKUP_TYPE_MBR,
> };
> 
> #define ARRAY_SIZE(arr) (sizeof arr / sizeof arr[0])
> 
> int is_extended(struct partition *partition)
> {
> 	int ret = 0;
> 
> 	switch (partition->sys_ind) {
> 	case DOS_EXTENDED:
> 	case WIN98_EXTENDED:
> 	case LINUX_EXTENDED:
> 		ret = 1;
> 	}
> 	return ret;
> }
> 
> unsigned char *get_sector(int fd, uint64_t n)
> {
> 	unsigned char *sector;
> 
> 	if (lseek64(fd, n << 9, SEEK_SET) < 0) {
> 		perror("lseek64");
> 		return NULL;
> 	}
> 	sector = malloc(512);
> 	if (!sector) {
> 		fprintf(stderr, "malloc: out of memory\n");
> 		return NULL;
> 	}
> 	if (read(fd, sector, 512) != 512) {
> 		perror("read");
> 		free(sector);
> 		return NULL;
> 	}
> 	return sector;
> }
> 
> void put_sector(unsigned char *sector)
> {
> 	free(sector);
> }
> 
> #define TABLE_OFFSET (512 - 2 - 4 * sizeof(struct partition))
> 
> inline struct partition *table(unsigned char *sector)
> {
> 	return (struct partition *)(sector + TABLE_OFFSET);
> }
> 
> int do_sector(int fd, uint32_t offset, uint32_t start)
> {
> 	unsigned char *sector;
> 	struct partition *cur;
> 	struct partition *ext = NULL;
> 	int ret = 0;
> 
> 	sector = get_sector(fd, offset + start);
> 	if (!sector)
> 		return -1;
> 
> 	if (sector[510] != 0x55 || sector[511] != 0xaa) {
> 		ret = -1;
> 		goto out;
> 	}
> 	for (cur = table(sector); cur < table(sector) + 4; cur++) {
> 		struct entry *entry;
> 
> 		if (!cur->size)
> 			continue;
> 
> 		cur->end_head += 1;
> 		if (backup.heads < cur->end_head)
> 			backup.heads = cur->end_head;
> 
> 		cur->end_sector &= 0x3f;
> 		if (backup.sectors < cur->end_sector) 
> 			backup.sectors = cur->end_sector;
> 
> 		if (is_extended(cur)) {
> 			if (!offset) {
> 				ret = do_sector(fd, cur->start, 0);
> 				if (ret < 0)
> 					goto out;
> 			} else if (!ext)
> 				ext = cur;
> 			continue;
> 		}
> 		if (backup.count == ARRAY_SIZE(backup.table)) {
> 			fprintf(stderr, "do_sector: out of space!\n");
> 			ret = -1;
> 			goto out;
> 		}
> 		entry = backup.table + backup.count++;
> 
> 		entry->flags = cur->boot_ind;
> 		if (!offset)
> 			entry->flags |= ENTRY_FLAG_PRIMARY;
> 
> 		entry->type  = cur->sys_ind;
> 		entry->start = cur->start + start;
> 		entry->size  = cur->size;

LE/BE issues here...

> 	}
> 	if (ext)
> 		ret = do_sector(fd, offset, ext->start);
>   out:
> 	put_sector(sector);
> 	return ret;
> }
> 
> void show_backup(void)
> {
> #ifdef DEBUG
> 
> 	int i;
> 
> 	fprintf(stderr, "signature: ");
> 	for (i = 0; i < 8; i++)
> 		fprintf(stderr, "%c", backup.signature[i]);
> 	fprintf(stderr, "\n");
> 
> 	fprintf(stderr, "     type: %d\n", backup.type);
> 	fprintf(stderr, "    heads: %d\n", backup.heads);
> 	fprintf(stderr, "  sectors: %d\n", backup.sectors);
> 	fprintf(stderr, "    count: %d\n", backup.count);
> 
> 	for (i = 0; i < backup.count; i++) {
> 		fprintf(stderr, "\n");
> 		fprintf(stderr, "%2d: flags: %02x\n", i, backup.table[i].flags);
> 		fprintf(stderr, "%2d:  type: %02x\n", i, backup.table[i].type);
> 		fprintf(stderr, "%2d: start: %llu\n", i, backup.table[i].start);
> 		fprintf(stderr, "%2d:  size: %u\n", i, backup.table[i].size);

You'll output wrong values here, depending on your host system.

> 	}
> #endif
> }
> 
> int main(int argc, char *argv[])
> {
> 	int fd;
> 	struct stat buf;
> 
> 	if (argc != 2) {
> 		printf("%s <blkdev>\n", argv[0]);
> 		return EXIT_FAILURE;
> 	}
> 	fd = open(argv[1], O_RDONLY);
> 	if (fd < 0) {
> 		perror("open");
> 		return EXIT_FAILURE;
> 	}
> 	if (fstat(fd, &buf) < 0) {
> 		perror("stat");
> 		return EXIT_FAILURE;
> 	}
> 	if (!S_ISBLK(buf.st_mode))
> 		fprintf(stderr, "warning: not a block device\n");
> 	
> 	if (do_sector(fd, 0, 0) < 0)
> 		return EXIT_FAILURE;
> 
> 	if (close(fd) < 0) {
> 		perror("close");
> 		return EXIT_FAILURE;
> 	}
> 	if (write(STDOUT_FILENO, &backup, sizeof backup) != sizeof backup) {
> 		perror("write");
> 		return EXIT_FAILURE;
> 	}
> 
> 	show_backup();
> 
> 	return EXIT_SUCCESS;
> }

Looks like a useful program, but you'd definively fix the LE/BE
issues. If you do that, it'll be able to even run on BE machines, too.

MfG, JBG

-- 
      Jan-Benedict Glaw      jbglaw@lug-owl.de              +49-172-7608481
Signature of:               Träume nicht von Dein Leben: Lebe Deinen Traum!
the second  :

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFH] Partition table recovery

[Rene Herman]

[-- Attachment #1: Type: text/plain, Size: 1206 bytes --]

On 07/23/2007 10:41 AM, Jan-Benedict Glaw wrote:

> As multibyte on-disk variables, these will need LE/BE conversion.

Indeed, thanks -- has been updated in the version that is attached.

Also fixes a bug that snuck in (failed to add offset to entry->start).

>> struct entry {
>> 	uint8_t flags;
>> 	uint8_t type;
>> 	uint16_t __1;
>> 	uint64_t start;
>> 	uint32_t size;
> 
> Dito.

This can stay for now. The partition table backup would indeed need some 
defined byte-order but it might be "whatever order the filesystem it's 
backed up onto uses". Since it's not directly written to any filesystem for 
now, host order will do currently.

> Looks like a useful program, but you'd definively fix the LE/BE issues.
> If you do that, it'll be able to even run on BE machines, too.

I might disagree with the usefulness. I believe that preferably a backup 
should be made with help from the kernel (if only by walking sysfs) to avoid 
things getting out of sync between kernel and backup program.

(this program largely does the same as the kernel does but even now there's 
already a difference in so far that I didn't bother to de-garbage the 3rd 
and 4th entries in the second level extendeds).

Rene.


[-- Attachment #2: partbackup.c --]
[-- Type: text/plain, Size: 4748 bytes --]

/*
 * Public Domain 2007, Rene Herman
 */

#define _LARGEFILE64_SOURCE

#include <stdlib.h>
#include <stdio.h>
#include <stdint.h>
#include <string.h>

#include <sys/types.h>
#include <sys/stat.h>
#include <sys/ioctl.h>
#include <unistd.h>
#include <fcntl.h>

#include <endian.h>
#include <byteswap.h>

static inline uint32_t le_32(uint32_t n)
{
#ifdef __LITTLE_ENDIAN
	return n;
#else
	return bswap_32(n);
#endif
}

enum {
	DOS_EXTENDED   = 0x05,
	WIN98_EXTENDED = 0x0f,
	LINUX_EXTENDED = 0x85,
};

struct partition {
	uint8_t boot_ind;
	uint8_t head;
	uint8_t sector;
	uint8_t cyl;
	uint8_t sys_ind;
	uint8_t end_head;
	uint8_t end_sector;
	uint8_t end_cyl;
	uint32_t start;
	uint32_t size;
} __attribute__((packed));

struct entry {
	uint8_t flags;
	uint8_t type;
	uint16_t __1;
	uint64_t start;
	uint32_t size;
} __attribute__((packed));

enum {
	ENTRY_FLAG_PRIMARY  = 0x01,
	ENTRY_FLAG_BOOTABLE = 0x80,
};

struct backup {
	uint8_t signature[8];
	uint16_t type;
	uint8_t heads;
	uint8_t sectors;
	uint8_t count;
	uint8_t __1[3];
	struct entry table[31];
} __attribute__((packed));

#define BACKUP_SIGNATURE "PARTBAK1"

enum {
	BACKUP_TYPE_MBR = 1,
};

struct backup backup = {
	.signature = BACKUP_SIGNATURE,
	.type	   = BACKUP_TYPE_MBR,
};

#define ARRAY_SIZE(arr) (sizeof arr / sizeof arr[0])

int is_extended(struct partition *partition)
{
	int ret = 0;

	switch (partition->sys_ind) {
	case DOS_EXTENDED:
	case WIN98_EXTENDED:
	case LINUX_EXTENDED:
		ret = 1;
	}
	return ret;
}

unsigned char *get_sector(int fd, uint64_t n)
{
	unsigned char *sector;

	if (lseek64(fd, n << 9, SEEK_SET) < 0) {
		perror("lseek64");
		return NULL;
	}
	sector = malloc(512);
	if (!sector) {
		fprintf(stderr, "malloc: out of memory\n");
		return NULL;
	}
	if (read(fd, sector, 512) != 512) {
		perror("read");
		free(sector);
		return NULL;
	}
	return sector;
}

void put_sector(unsigned char *sector)
{
	free(sector);
}

#define TABLE_OFFSET (512 - 2 - 4 * sizeof(struct partition))

inline struct partition *table(unsigned char *sector)
{
	return (struct partition *)(sector + TABLE_OFFSET);
}

int do_sector(int fd, uint32_t offset, uint32_t start)
{
	unsigned char *sector;
	struct partition *cur;
	struct partition *ext = NULL;
	int ret = 0;

	sector = get_sector(fd, offset + start);
	if (!sector)
		return -1;

	if (sector[510] != 0x55 || sector[511] != 0xaa) {
		ret = -1;
		goto out;
	}
	for (cur = table(sector); cur < table(sector) + 4; cur++) {
		struct entry *entry;

		if (!cur->size)
			continue;

		cur->end_head += 1;
		if (backup.heads < cur->end_head)
			backup.heads = cur->end_head;

		cur->end_sector &= 0x3f;
		if (backup.sectors < cur->end_sector) 
			backup.sectors = cur->end_sector;

		if (is_extended(cur)) {
			if (!offset) {
				ret = do_sector(fd, le_32(cur->start), 0);
				if (ret < 0)
					goto out;
			} else if (!ext)
				ext = cur;
			continue;
		}
		if (backup.count == ARRAY_SIZE(backup.table)) {
			fprintf(stderr, "do_sector: out of space!\n");
			ret = -1;
			goto out;
		}
		entry = backup.table + backup.count++;

		entry->flags = cur->boot_ind;
		if (!offset)
			entry->flags |= ENTRY_FLAG_PRIMARY;

		entry->type  = cur->sys_ind;
		entry->start = le_32(cur->start) + offset + start;
		entry->size  = le_32(cur->size);
	}
	if (ext)
		ret = do_sector(fd, offset, le_32(ext->start));
  out:
	put_sector(sector);
	return ret;
}

void show_backup(void)
{
#ifdef DEBUG
	int i;

	fprintf(stderr, "signature: ");
	for (i = 0; i < 8; i++)
		fprintf(stderr, "%c", backup.signature[i]);
	fprintf(stderr, "\n");

	fprintf(stderr, "     type: %d\n", backup.type);
	fprintf(stderr, "    heads: %d\n", backup.heads);
	fprintf(stderr, "  sectors: %d\n", backup.sectors);
	fprintf(stderr, "    count: %d\n", backup.count);

	for (i = 0; i < backup.count; i++) {
		fprintf(stderr, "\n");
		fprintf(stderr, "%2d: flags: %02x\n", i, backup.table[i].flags);
		fprintf(stderr, "%2d:  type: %02x\n", i, backup.table[i].type);
		fprintf(stderr, "%2d: start: %llu\n", i, backup.table[i].start);
		fprintf(stderr, "%2d:  size: %u\n", i, backup.table[i].size);
	}
#endif
}

int main(int argc, char *argv[])
{
	int fd;
	struct stat buf;

	if (argc != 2) {
		printf("%s <blkdev>\n", argv[0]);
		return EXIT_FAILURE;
	}
	fd = open(argv[1], O_RDONLY);
	if (fd < 0) {
		perror("open");
		return EXIT_FAILURE;
	}
	if (fstat(fd, &buf) < 0) {
		perror("stat");
		return EXIT_FAILURE;
	}
	if (!S_ISBLK(buf.st_mode))
		fprintf(stderr, "warning: not a block device\n");
	
	if (do_sector(fd, 0, 0) < 0)
		return EXIT_FAILURE;

	if (close(fd) < 0) {
		perror("close");
		return EXIT_FAILURE;
	}
	if (write(STDOUT_FILENO, &backup, sizeof backup) != sizeof backup) {
		perror("write");
		return EXIT_FAILURE;
	}

	show_backup();

	return EXIT_SUCCESS;
}

Re: [RFH] Partition table recovery

[Rene Herman]

On 07/23/2007 12:54 PM, Rene Herman wrote:

> static inline uint32_t le_32(uint32_t n)
> {
> #ifdef __LITTLE_ENDIAN
> 	return n;
> #else
> 	return bswap_32(n);
> #endif
> }

#if __BYTE_ORDER == __LITTLE_ENDIAN, that is. sigh.

Rene.

Re: [RFH] Partition table recovery

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 564 bytes --]

On Mon, 2007-07-23 14:39:11 +0200, Rene Herman <rene.herman@gmail.com> wrote:
> On 07/23/2007 12:54 PM, Rene Herman wrote:
> 
> > static inline uint32_t le_32(uint32_t n)
> > {
> > #ifdef __LITTLE_ENDIAN
> >	return n;
> > #else
> > 	return bswap_32(n);
> > #endif
> > }
> 
> #if __BYTE_ORDER == __LITTLE_ENDIAN, that is. sigh.

Don't forget PDP11 byteorder :-)

MfG, JBG

-- 
      Jan-Benedict Glaw      jbglaw@lug-owl.de              +49-172-7608481
  Signature of:                           Wenn ich wach bin, träume ich.
  the second  :

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: [RFH] Partition table recovery

[Rene Herman]

On 07/23/2007 03:15 PM, Jan-Benedict Glaw wrote:

>>> static inline uint32_t le_32(uint32_t n)
>>> {
>>> #ifdef __LITTLE_ENDIAN
>>> 	return n;
>>> #else
>>> 	return bswap_32(n);
>>> #endif
>>> }
>> #if __BYTE_ORDER == __LITTLE_ENDIAN, that is. sigh.
> 
> Don't forget PDP11 byteorder :-)

How could I? It cracks me up every time...

Rene.

Re: [RFH] Partition table recovery

[Bill Davidsen]

Rene Herman wrote:
> On 07/23/2007 10:41 AM, Jan-Benedict Glaw wrote:
>
>> As multibyte on-disk variables, these will need LE/BE conversion.
>
> Indeed, thanks -- has been updated in the version that is attached.
>
> Also fixes a bug that snuck in (failed to add offset to entry->start).
>
>>> struct entry {
>>>     uint8_t flags;
>>>     uint8_t type;
>>>     uint16_t __1;
>>>     uint64_t start;
>>>     uint32_t size;
>>
>> Dito.
>
> This can stay for now. The partition table backup would indeed need 
> some defined byte-order but it might be "whatever order the filesystem 
> it's backed up onto uses". Since it's not directly written to any 
> filesystem for now, host order will do currently.
>
>> Looks like a useful program, but you'd definively fix the LE/BE issues.
>> If you do that, it'll be able to even run on BE machines, too.
>
> I might disagree with the usefulness. I believe that preferably a 
> backup should be made with help from the kernel (if only by walking 
> sysfs) to avoid things getting out of sync between kernel and backup 
> program.
>
> (this program largely does the same as the kernel does but even now 
> there's already a difference in so far that I didn't bother to 
> de-garbage the 3rd and 4th entries in the second level extendeds).
>
How can I politely say this code really needs comments?
To quote the late R. W. Benway, "If it was hard to write it should be 
hard to understand."
(regarding code in FORTRAN II on punched cards, ca 1965)
> Rene.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: [RFH] Partition table recovery

[Theodore Tso]

On Mon, Jul 23, 2007 at 10:15:21AM +0200, Rene Herman wrote:
> On an integrated system like this, do you consider it acceptable to only do 
> the MS-DOS partitions and not the other types that may be present _inside_ 
> those partitions? (MINIX subpartitions, BSD slices, ...). I believe those 
> should really also be done, but this would require keeping more information 
> again.

Well, I'm considering this to be a MBR backup scheme, so Minix and BSD
slices are legacy systems which are out of scope.  If they are busted
in the same way as MBR in terms of not having redundant backups of
critical data, when they have a lot fewer excuses that MBR, and they
can address that issue in their own way.  The number of Linux users
that also have Minix and BSD partitions are a vanishingly small number
in any case.

> I (very) briefly looked at blkid but unless I'm mistaken blkid needs device 
> names? The documentation seems to be missing. When scanning the device for 
> the partition table, we've built a list of partitions with offsets into the 
> device and it would be nice if we could hand the fd and the offset off to 
> something directly. If the program has to construct device names itself 
> there's another truckload of pitfalls right there.

Yeah, good point, I'd have to add that support into blkid.  It's been
on my todo list, but I just haven't gotten around to it yet.

> It might in fact make sense to just ask the kernel for the partitions on a 
> device and not bother with scanning anything ourselves. Ie, just walk 
> sysfs. Would you agree? This siginificantly reduces the risk of things 
> getting out of sync, both scanning order and implementation.

My concern of sysfs is that #1, it won't work on older kernels since
you would need to add new fields to backup what we want, and #2, I'm
still fundamentally distrustful of sysfs because there isn't a bright
line between what is an exported interface that will never change, and
something which is considered an "internal implementation detail" that
can change whenever some kernel hacker feels like it.  (Or when some
kernel hacker is careless...)  So as far as I'm concerned sysfs is a
terrible, TERRIBLE way to export a published interface where we
promise stability to userspace.

So I'd just as soon do this in userspace; after all, the entire
partition manager (and there are multiple ones; fdisk, sfdisk, gpart,
etc.) all in userspace, and that needs to be in synch with the kernel
partition reading code anyway.  So one more userspace implementation
is in my mind much cleaner than trying to push the needed
functionality into sysfs, and then hoping against hope that it doesn't
accidentally change in the future.

						- Ted

Re: [RFH] Partition table recovery

[Rene Herman]

On 07/23/2007 03:58 PM, Theodore Tso wrote:

> Well, I'm considering this to be a MBR backup scheme, so Minix and BSD 
> slices are legacy systems which are out of scope.  If they are busted in
> the same way as MBR in terms of not having redundant backups of critical
> data, when they have a lot fewer excuses that MBR, and they can address
> that issue in their own way.  The number of Linux users that also have
> Minix and BSD partitions are a vanishingly small number in any case.

I'd in fact expect quite a few people to have a FreeBSD partition around. 
And MINIX if they are in university and in an operating systems course...

But "they should take whatever precautions they want themselves" is a valid 
argument.

[ blkid ]

> Yeah, good point, I'd have to add that support into blkid.  It's been
> on my todo list, but I just haven't gotten around to it yet.

I'll for now stop updating the partbackup thingy as posted. Given that Linux 
  only follows the first extended in the list of extendeds (which sort of 
destroys the nice recursion anyway) it might want to be iterative instead of 
recursive if the thing has a future -- not very important though.

> My concern of sysfs is that #1, it won't work on older kernels since
> you would need to add new fields to backup what we want,

I'd be okay with that.

> and #2, I'm still fundamentally distrustful of sysfs because there isn't
> a bright line between what is an exported interface that will never
> change, and something which is considered an "internal implementation
> detail" that can change whenever some kernel hacker feels like it.  (Or
> when some kernel hacker is careless...)  So as far as I'm concerned sysfs
> is a terrible, TERRIBLE way to export a published interface where we 
> promise stability to userspace.

Oh come on, that's going overboard a bit, it's not all _that_ bad! Finding 
say "sda" will be possible without breaking too many times. Admittedly, the 
kernel's partittion scanning order is also not likely to change as it would 
certainly break userspace, but code duplication, with the possiblity of bugs 
slipping in at least userspace-ways (considering the kernel the reference no 
matter what it does) is a concern. Somewhat. A little.

> So I'd just as soon do this in userspace; after all, the entire partition
> manager (and there are multiple ones; fdisk, sfdisk, gpart, etc.) all in
> userspace, and that needs to be in synch with the kernel partition
> reading code anyway.  So one more userspace implementation is in my mind
> much cleaner than trying to push the needed functionality into sysfs, and
> then hoping against hope that it doesn't accidentally change in the
> future.

* rene envisions /lib/libpart.so...

Not to mention my Grand Visions of a totally new Linux native partitioning 
scheme probably modelled after BSD slices (as also mentioned in a previous 
message just now). Or perhaps LVM already fills that role comfortably. It's 
certainly what I hear everyone talking about these days.

Rene.

Re: [RFH] Partition table recovery

[Rene Herman]

[-- Attachment #1: Type: text/plain, Size: 3936 bytes --]

On 07/22/2007 03:11 AM, Theodore Tso wrote:

>> This is a problem. Today the CHS fields in the partition entries don't
>> mean much of anything anymore and Linux happily ignores them but DOS
>> and (hence) Windows 9x do not. From time to time I still have the
>> Windows 98 install that's sitting in a corner of my disk throw a fit
>> just by having set the BIOS from LBA to Large (meaning the geometry the
>> BIOS pretends the disk has changes) for example. Old DOS installs that
>> I keep around for the purpose of hardware testing with the originally
>> supplied drivers make for even more of a "don't touch, don't touch!"
>> thing -- various version of DOS throw fits for various reasons.
> 
> This is true, but that's due to the fundamentally broken nature of CHS.
> You need them to boot, and that's about it.  I will say up front that I
> don't particularly care about legacy operating system such as DOS,
> Windows 98, or Minix 3.  So the idea of simply having the number of heads
> and sectors in the partition header is that we can reconstruct CHS fields
> such that it is likely with modern hardware you will get it right.

Well, I still don't believe this all to be a great idea but it was sort of 
fun so the attached does largely what you want -- build a list of all data 
partitions.

The heads/sectors fields it for now just gets from the HDIO_GETGEO call. A 
better source would be guessing the values from the partition table itself 
but that _also_ doesn't make too much sense. If you're reconstructing a 
sanitized version of the table anyway, it makes better sense to reconstruct 
it with the values HDIO_GETGEO returns at restoration time.

I kept your suggested format, but in fact, the 64-bit "start" value seems 
not very useful if we're getting the value from a 32-bit field in the old 
partition tables anyway. With that shrunk down to 32-bit again, there would 
be enough room for the complete partition table entry...

> For ancient systems that do all sorts of weird things such as ECHS,
> etc., yeah, you're pretty much doomed, and the bigger danger comes
> from futzing with BIOS settings, et. al.  But it's 2007, gosh darn it!
> "Here's a quarter, kid, buy yourself a real computer".  :-)

Thanks, but real computers won't host my ISA cards...

> Yes, I'm very aware of the extended partitioning scheme mess.  What I
> was proposing to back up here is only the real partitions, not the
> fake extended partitions.  The idea is to store *just* enough
> information so that a partition table manager can recover the
> partition tables in such a way that the original filesystem
> information can be recovered.

This should do I guess. It enters all data partitions into the list, in the 
order in which they are encountered and sets a flag to signify that a 
partition was a logical rather than primary. Another option would be to just 
reserve the first 4 entries for the primaries and the rest for the logicals 
but this saves entries if there are fewer than 4 primaries and was in fact 
easier...

The program enters partitions in what should be the same order as Linux 
itself does. Primaries from slot 0 to 3 as normal (but not backed up to 
entry 0 to 3 as said -- the LOGICAL flag indentifies them), extended 
partitions in the MBR in the order as encountered, with the logicals in the 
second-level table as encountered, and following only the first extented in 
the second-level table.

Made it into a generic C program -- didn't look at e2fsprogs sources yet.

Need to be off now and haven't yet stared at this as long as I'd like so 
don't slap me if I've left a few bugs in (although it seems to work nicely). 
The program dumps the backup sector to stdout -- it's ofcourse easy to 
change it to print the entries out so they're easy to compare against, say, 
"fdisk -l -us".

Oh, and once you've looked at it, please throw it away. As said, I still 
don't think it's a great idea ;-)

Rene.


[-- Attachment #2: backup.c --]
[-- Type: text/plain, Size: 3535 bytes --]

/*
 * Public Domain 2007, Rene Herman
 *
 * gcc -W -Wall -DTEST -D_LARGEFILE64_SOURCE -o backup backup.c
 *
 */

#include <stdlib.h>

enum {
	DOS_EXTENDED   = 0x05,
	WIN98_EXTENDED = 0x0f,
	LINUX_EXTENDED = 0x85,
};

struct partition {
        unsigned char boot_ind;
	unsigned char __1[3];
        unsigned char sys_ind;
	unsigned char __2[3];
        unsigned int start;
        unsigned int size;
} __attribute__((packed));

struct entry {
	unsigned char flags;
	unsigned char type;
	unsigned short __1;
	unsigned long long start;
	unsigned int size;
} __attribute__((packed));

enum {
	ENTRY_FLAG_LOGICAL  = 0x01,
	ENTRY_FLAG_BOOTABLE = 0x80,
};

struct backup {
	unsigned char signature[8];
	unsigned short type;
	unsigned char heads;
	unsigned char sectors;
	unsigned char count;
	unsigned char __1[3];
	struct entry table[31];
} __attribute__((packed));

#define BACKUP_SIGNATURE "PARTBAK1"

enum {
	BACKUP_TYPE_MBR = 1,
};

struct backup backup = {
	.signature = BACKUP_SIGNATURE,
	.type	   = BACKUP_TYPE_MBR,
};

int is_extended(struct partition *entry)
{
	int ret = 0;

	switch (entry->sys_ind) {
	case DOS_EXTENDED:
	case WIN98_EXTENDED:
	case LINUX_EXTENDED:
		ret = 1;
	}
	return ret;
}

unsigned char *get_sector(unsigned int n);
void put_sector(unsigned char *sector);

int do_sector(unsigned int offset, unsigned int start)
{
	unsigned char *sector;
	struct partition *partition;
	struct partition *extended = NULL;

	start += offset;
	sector = get_sector(start);
	if (!sector) 
		return -1;

	if (sector[510] != 0x55 || sector[511] != 0xaa) {
		put_sector(sector);
		return -1;
	}

	partition = (struct partition *)(sector + 510 - 4 * sizeof *partition);
	do {
		struct entry *entry;

		if (!partition->size)
			continue;

		if (is_extended(partition)) {
			if (!offset)
				do_sector(partition->start, 0);
			else if (!extended)
				extended = partition;
			continue;
		}
		entry = backup.table + backup.count++;

		entry->flags = partition->boot_ind;
		if (offset)
			entry->flags |= ENTRY_FLAG_LOGICAL;
		entry->type  = partition->sys_ind;
		entry->start = start + partition->start;
		entry->size  = partition->size;

	} while (++partition < (struct partition *)(sector + 510));

	if (extended)
		do_sector(offset, extended->start);

	put_sector(sector);
	return 0;
}

#ifdef TEST
#include <stdio.h>

#include <sys/types.h>
#include <sys/stat.h>
#include <sys/ioctl.h>
#include <unistd.h>
#include <fcntl.h>

#include <linux/hdreg.h>

int fd;

unsigned char *get_sector(unsigned int n)
{
	unsigned char *sector = malloc(512);

	if (!sector)
		return NULL;

	if (lseek64(fd, (off64_t)n << 9, SEEK_SET) < 0)
		goto free;

	if (read(fd, sector, 512) != 512) {
  free:		
		free(sector);
		sector = NULL;
	}
	return sector;
}

void put_sector(unsigned char *sector)
{
	free(sector);
}

int main(int argc, char *argv[])
{
	struct stat buf;
	struct hd_geometry geometry;

	if (argc != 2) {
		printf("%s <blkdev>\n", argv[0]);
		return EXIT_FAILURE;
	}
	fd = open(argv[1], O_RDONLY);
	if (fd < 0) {
		perror("open");
		return EXIT_FAILURE;
	}
	if (fstat(fd, &buf) < 0 || !S_ISBLK(buf.st_mode)) {
		perror("stat");
		return EXIT_FAILURE;
	}
	if (ioctl(fd, HDIO_GETGEO, &geometry) == 0) {
		backup.heads = geometry.heads;
		backup.sectors = geometry.sectors;
	}
	if (do_sector(0, 0) < 0)
		return EXIT_FAILURE;

	if (close(fd) < 0) {
		perror("close");
		return EXIT_FAILURE;
	}
	if (write(STDOUT_FILENO, &backup, sizeof backup) != sizeof backup) {
		perror("write");
		return EXIT_FAILURE;
	}
	return EXIT_SUCCESS;
}
#endif

Re: [RFH] Partition table recovery

[Theodore Tso]

On Mon, Jul 23, 2007 at 09:34:25AM +0200, Rene Herman wrote:
> 
> The most profound issue is _what_ to save. I for example don't cylinder 
> align my partitions (I hate wasting disk just to appease broken software) 
> meaning that not all my end_head/sector values are consistent even at the 
> best of times. Admittedly, I'm terminally analy retentive.
> 
> Exactly. This is why I earlier said that really the only functional thing 
> to do is not try to be smart and just grab things verbatim. This ofcourse 
> does hinge on one's view of "good enough"...

Heh.  Well, my definition of "good enough" is enough so that the C/H/S
fields are sane so that the BIOS can boot the system.  So I don't care
about grabbing things verbatim since my higher priority is stuffing as
much data for as many partitions as possible into 512 bytes.  Things
that require grabbing the C/H/S fields verbatim to avoid breakage I
would classify as "broken software".  But, your mileage may vary.  :-)

> That's not quite correct. Logicals have a start field relative to the 
> encompassing extended (ie, for me always 1, for others often always 63) and 
> the encompassing extended are relative not to the previous extended but to 
> the level 0 extended (the one in the MBR). 

This assumes that the extended partition is at the beginning of the
disk, yes?  Why would anyone do that?  I normally have /dev/hda1 at
the beginning of the disk, and I normally make /dev/hda4 my extended,
and place it *after* partitions at /dev/hda2, /dev/hda3, etc.

> So at most you get can 32-bit + 32-bit which could, yeah, in principle 
> overflow into the 32nd bit -- it normally won't ofcourse since the start 
> field, being relative, will be small, and I'd expect quite a few bits of 
> software to break on this condition.

It would be interesting to see how badly modern Windows systems breaks
on this.  If Windows 2000 and above works, and Linux works, then if
other things break it might be quite sufficient to consider them
"broken software" that we don't need to worry about.

> With 32-bit values (and 512-byte sectors) you can service 2TB -- anything 
> above that requires something better than MS-DOS partition tables. 

Well, in about 2-3 years or so we'll seeing having singleton disks
bigger 2TB.  I'm not terribly sanguine about BIOS vendors and OS
providers migrating to something better by then, alas.  Life is sure
going to be interesting.  :-)

						- Ted

Re: [RFH] Partition table recovery

[Rene Herman]

On 07/23/2007 03:48 PM, Theodore Tso wrote:

> On Mon, Jul 23, 2007 at 09:34:25AM +0200, Rene Herman wrote:

>> That's not quite correct. Logicals have a start field relative to the 
>> encompassing extended (ie, for me always 1, for others often always 63) and 
>> the encompassing extended are relative not to the previous extended but to 
>> the level 0 extended (the one in the MBR). 
> 
> This assumes that the extended partition is at the beginning of the
> disk, yes?

Err, well, no, that's not what I meant. The "start" field for the extended 
partition that sits in the primary partitition table (the one in the MBR) is 
absolute, or "relative to the start of the disk", but the "start" field for 
the empty extended partitions that together form the logical partition list 
are relative not to the previous one in the list, but all to this outermost 
extended partition.

> Why would anyone do that?  I normally have /dev/hda1 at the beginning of
> the disk, and I normally make /dev/hda4 my extended, and place it *after*
> partitions at /dev/hda2, /dev/hda3, etc.

... but having said that, I do actually have an extended partition as my 
/dev/hda1 at the beginning of the disk. This is the current layout on my 
main system:

    Device Boot    Start       End   #sectors  Id  System
/dev/sda1             1 231733119  231733119  85  Linux extended
/dev/sda2   * 231733120 240121727    8388608   c  W95 FAT32 (LBA)
/dev/sda3             0         -          0   0  Empty
/dev/sda4             0         -          0   0  Empty
/dev/sda5             2   2097153    2097152  82  Linux swap
/dev/sda6       2097155  18874370   16777216  83  Linux
/dev/sda7      18874372  35651587   16777216  83  Linux
/dev/sda8      35651589 231733119  196081531  83  Linux

As you can see, everything neatly non-cylinder-aligned, with not a single 
sector wasted ;-) Table sectors at 0 (MBR), 1 (outer extended), 2097154, 
18874371 and 35651588 (list-extendeds).

/dev/sda2 used to be a FreeBSD install (partition type 0xa5), /dev/sda3 a 
MINIX install (type 0x81) and /dev/sda4 the still present FAT32 Windows 98 
partition at the very end of the disk. I removed FreeBSD and MINIX due to 
space shortage...

The reason that I use the first entry for an extended is that I view the 
type "Linux Extended" simply as "Linux": That is, I see 0x85 simply as the 
one and only Linux type with all my Linux data partitions on the logicals 
inside -- very much like 0xa5 is the one FreeBSD type with all its data 
partitions on the slices inside, and 0x81 the one MINIX partition with its 
data partitions on the subpartitions inside.

That is, I've been using a "Linux native partitioning scheme" where the 
Linux native layout just happens to coincide with a DOS/Windows native layout.

My Linux partition is at the start of the disk since it's the system I use. 
The others are/were there just to boot perhaps a few times a year to check 
some things -- and the start of the disk is the fastest bit, so I certainly 
want my main system to use that.

Anyone find my "Native Linux Partitioning Scheme" interesting? Designing and 
using a better way than regular logicals to carve up the space inside (such 
as something designed after BSD slices) would work for me as well ;-)

> It would be interesting to see how badly modern Windows systems breaks
> on this.  If Windows 2000 and above works, and Linux works, then if
> other things break it might be quite sufficient to consider them
> "broken software" that we don't need to worry about.

Googling for it, the 2TB limit of DOS partitioning is widely known and there 
would be no point worrying even about the single-bit overflow possibly of 
the list of extendeds...

>> With 32-bit values (and 512-byte sectors) you can service 2TB -- anything 
>> above that requires something better than MS-DOS partition tables. 
> 
> Well, in about 2-3 years or so we'll seeing having singleton disks
> bigger 2TB.  I'm not terribly sanguine about BIOS vendors and OS
> providers migrating to something better by then, alas.  Life is sure
> going to be interesting.  :-)

And sectors probably larger than 512 bytes. I hope they'll not do _that_ 
until plain old partitions are truly abandoned since before you know it 
someone going to view it as an excuse to keep using this fragile mess ;-)

Rene.

Re: [RFH] Partion table recovery

[Jeffrey V. Merkey]

Al Boldi wrote:

>As always, a good friend of mine managed to scratch my partion table by 
>cat'ing /dev/full into /dev/sda.  I was able to push him out of the way, but 
>at least the first 100MB are gone.  I can probably live without the first 
>partion, but there are many partitions after that, which I hope should 
>easily be recoverable.
>
>I tried parted, but it's not working out for me.  Does anybody know of a 
>simple partition recovery tool, that would just scan the disk for lost 
>partions?
>
>
>Thanks!
>
>--
>Al
>
>-
>To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>the body of a message to majordomo@vger.kernel.org
>More majordomo info at  http://vger.kernel.org/majordomo-info.html
>Please read the FAQ at  http://www.tux.org/lkml/
>
>  
>
One thing NetWare always did was to stamp a copy of the partition table 
at the time a partition was created as the second logical sector (offset 
1) from the start of a newly created partition. This allowed the disk to 
be scanned for the original (or last) partition table copy.

Jeff

Re: [RFH] Partition table recovery

[Al Boldi]

Jeffrey V. Merkey wrote:
> Al Boldi wrote:
> >As always, a good friend of mine managed to scratch my partion table by
> >cat'ing /dev/full into /dev/sda.  I was able to push him out of the way,
> > but at least the first 100MB are gone.  I can probably live without the
> > first partion, but there are many partitions after that, which I hope
> > should easily be recoverable.
> >
> >I tried parted, but it's not working out for me.  Does anybody know of a
> >simple partition recovery tool, that would just scan the disk for lost
> >partions?
>
> One thing NetWare always did was to stamp a copy of the partition table
> at the time a partition was created as the second logical sector (offset
> 1) from the start of a newly created partition. This allowed the disk to
> be scanned for the original (or last) partition table copy.

This is really a good idea, as this would save you the trouble of 
reconstructing the table due to older overlapping entries.

Can linux do something like that?


Thanks!

--
Al

Re: [RFH] Partion table recovery

[Anton Altaparmakov]

On 20 Jul 2007, at 06:13, Al Boldi wrote:
> As always, a good friend of mine managed to scratch my partion  
> table by
> cat'ing /dev/full into /dev/sda.  I was able to push him out of the  
> way, but
> at least the first 100MB are gone.  I can probably live without the  
> first
> partion, but there are many partitions after that, which I hope should
> easily be recoverable.
>
> I tried parted, but it's not working out for me.  Does anybody know  
> of a
> simple partition recovery tool, that would just scan the disk for lost
> partions?

parted and its derivatives are pile of crap...  They cause corruption  
to totally healthy systems at the best of times.  Don't go near them.

Use TestDisk (http://www.cgsecurity.org/wiki/TestDisk) and be happy.   
(-:

Best regards,

	Anton
-- 
Anton Altaparmakov <aia21 at cam.ac.uk> (replace at with @)
Unix Support, Computing Service, University of Cambridge, CB2 3QH, UK
Linux NTFS maintainer, http://www.linux-ntfs.org/

Re: [RFH] Partion table recovery

[Willy Tarreau]

On Fri, Jul 20, 2007 at 08:35:45AM +0100, Anton Altaparmakov wrote:
> On 20 Jul 2007, at 06:13, Al Boldi wrote:
> >As always, a good friend of mine managed to scratch my partion  
> >table by
> >cat'ing /dev/full into /dev/sda.  I was able to push him out of the  
> >way, but
> >at least the first 100MB are gone.  I can probably live without the  
> >first
> >partion, but there are many partitions after that, which I hope should
> >easily be recoverable.
> >
> >I tried parted, but it's not working out for me.  Does anybody know  
> >of a
> >simple partition recovery tool, that would just scan the disk for lost
> >partions?
> 
> parted and its derivatives are pile of crap...  They cause corruption  
> to totally healthy systems at the best of times.  Don't go near them.
> 
> Use TestDisk (http://www.cgsecurity.org/wiki/TestDisk) and be happy.   
> (-:

Thanks for this link Anton, it looks awesome and I'll add it to my
collection :-)

I mostly use vche (virtual console hex editor) to look for strings, find
offsets and/or fix data by hand. This should make me save some time !

Regards,
Willy

Re: [RFH] Partion table recovery

[Bill Davidsen]

Al Boldi wrote:
> As always, a good friend of mine managed to scratch my partion table by 
> cat'ing /dev/full into /dev/sda.  I was able to push him out of the way, but 
> at least the first 100MB are gone.  I can probably live without the first 
> partion, but there are many partitions after that, which I hope should 
> easily be recoverable.
>
> I tried parted, but it's not working out for me.  Does anybody know of a 
> simple partition recovery tool, that would just scan the disk for lost 
> partions?
>   
You have gotten a bunch of thoughts on this, I will just say that plain 
old "fdisk -l" saved somewhere safe is probably all you need, in human 
readable format. Doesn't do you any good now, but all the complicated 
schemes discussed don't thrill me, I want to be able to see this, and 
recovery by partition table manual rebuild is so rare I would rather do 
it by hand than trust some software I rarely use.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: [RFH] Partion table recovery

[Rene Herman]

On 07/23/2007 10:08 PM, Bill Davidsen wrote:

> Al Boldi wrote:

>> As always, a good friend of mine managed to scratch my partion table 
>> by cat'ing /dev/full into /dev/sda.  I was able to push him out of the 
>> way, but at least the first 100MB are gone.  I can probably live 
>> without the first partion, but there are many partitions after that, 
>> which I hope should easily be recoverable.
>>
>> I tried parted, but it's not working out for me.  Does anybody know of 
>> a simple partition recovery tool, that would just scan the disk for 
>> lost partions?
>>   
> You have gotten a bunch of thoughts on this, I will just say that plain 
> old "fdisk -l" saved somewhere safe is probably all you need, in human 
> readable format. Doesn't do you any good now, but all the complicated 
> schemes discussed don't thrill me, I want to be able to see this, and 
> recovery by partition table manual rebuild is so rare I would rather do 
> it by hand than trust some software I rarely use.

ACK. Or NNAK (Non-NAK) at least...

Rene.