accessing loopback filesystem+partitions on a file

accessing loopback filesystem+partitions on a file

[Luke Kenneth Casson Leighton]

[if you are happy to reply at all, please reply cc'd thank you.]

hi,

i'm really sorry to be bothering people on this list but i genuinely
don't what phrases to google for what i am looking for without getting
swamped by useless pages, which you will understand why when you see
the question, below.

background:

	i'm sort-of helping test a xen install project where a
	block device is presented as the DRIVE - not, i repeat
	not, the partitions on the drive which is the quotes
	normal quotes way of doing xen installes

	(yes there are good reasons for doing this).

	the thing is that the install is failing, and we'd duh
	like to analyse what's going on (access the log files -
	no, sshd hasn't been installed yet) after terminating
	the xen session [because afaik in xen guest sessions
	there's no way i know of to access virtual consoles 2-6]

the question is, therefore:

	* how the hell do you loopback mount (or lvm mount
	  or _anything_! something!)  partitions that have
	  been created in a loopback'd file!!!!

	  [aside from booting up a second pre-installed xen
	  guest domain and making the filesystem-in-a-file
	  available as /dev/hdb of course.]

answers of the form "work out where the partitions are, then use
hexedit to remove the first few blocks" will win no prizes here.

l.

-- 
--
<a href="http://lkcl.net">http://lkcl.net</a>
--

Re: accessing loopback filesystem+partitions on a file

[Jason Lunz]

lkcl@lkcl.net said:
> 	* how the hell do you loopback mount (or lvm mount
> 	  or _anything_! something!)  partitions that have
> 	  been created in a loopback'd file!!!!

use the offset option when mounting.

Jason

Re: accessing loopback filesystem+partitions on a file

[Grzegorz Kulewski]

On Wed, 29 Jun 2005, Luke Kenneth Casson Leighton wrote:

> [if you are happy to reply at all, please reply cc'd thank you.]
>
> hi,
>
> i'm really sorry to be bothering people on this list but i genuinely
> don't what phrases to google for what i am looking for without getting
> swamped by useless pages, which you will understand why when you see
> the question, below.
>
> the question is, therefore:
>
> 	* how the hell do you loopback mount (or lvm mount
> 	  or _anything_! something!)  partitions that have
> 	  been created in a loopback'd file!!!!
>
> 	  [aside from booting up a second pre-installed xen
> 	  guest domain and making the filesystem-in-a-file
> 	  available as /dev/hdb of course.]
>
> answers of the form "work out where the partitions are, then use
> hexedit to remove the first few blocks" will win no prizes here.

The bad news: it was impossible (or at least very hard to do).

The good news: it is possible now. The anwser is:
- figure where the partitions are (possibly using some simple script),
- use device-mapper to create block devices covering partitions,
- mount them.

I do not know if this anwser will win your price but it is IMHO far better 
than hexedit... :-) And probably this is the only anwser.

(IIRC if you have one partition you can skip partition table with offset 
option to losetup. But this will only work in this special case...)


Grzegorz Kulewski

Re: accessing loopback filesystem+partitions on a file

[Luke Kenneth Casson Leighton]

oh, bugger :)

xen guest OSes manage it fine - the xen layer provides a means to
present any block device as a "disk".

that loopback filesystems cannot be presented as block devices
by the linux kernel (with no involvement of xen) seems to be
a curious omission.

... loopbackblock.ko, anyone?

btw the [as yet formally-unannounced] project is at
http://hands.com/d-i.  try the xen0 install and then follow
the instructions for creating a guest domain [by booting debian
installer in a guest domain, as opposed to running debootstrap,
the "normal" xen recommended method].

fortunately, phil has fixed the xen-guest-install, such that it
completes successfully.

that _still_ leaves a hard-disk-with-its-partitions in an actual LVM
partition which is totally inaccessible.

okay - i _say_ inaccessible: there is always the possibility of finding
a spare hard drive, and then doing "dd if=/dev/vg/xen0hda of=/dev/hdd".

this being 2005 last time i checked, it does seem to me to
be a rather large waste of a) an entire hard drive b) time
spent copying.

*sigh*.

sadly, answers on back of envelope ideally need to involve a
tool or procedure of some sort that can be run by dummies such
as myself on a regular basis without fear of major-cock-up,
rather than something [ending oh say in .pl] that stands a good
chance of being exorcised [or exercised] as voodoo witchcraft.

unmitigated steaming "perl" advocates need not respond.  please.

l.

On Wed, Jun 29, 2005 at 02:35:25AM +0200, Grzegorz Kulewski wrote:

> On Wed, 29 Jun 2005, Luke Kenneth Casson Leighton wrote:
> 
> >[if you are happy to reply at all, please reply cc'd thank you.]
> >
> >hi,
> >
> >i'm really sorry to be bothering people on this list but i genuinely
> >don't what phrases to google for what i am looking for without getting
> >swamped by useless pages, which you will understand why when you see
> >the question, below.
> >
> >the question is, therefore:
> >
> >	* how the hell do you loopback mount (or lvm mount
> >	  or _anything_! something!)  partitions that have
> >	  been created in a loopback'd file!!!!
> >
> >	  [aside from booting up a second pre-installed xen
> >	  guest domain and making the filesystem-in-a-file
> >	  available as /dev/hdb of course.]
> >
> >answers of the form "work out where the partitions are, then use
> >hexedit to remove the first few blocks" will win no prizes here.
> 
> The bad news: it was impossible (or at least very hard to do).
> 
> The good news: it is possible now. The anwser is:
> - figure where the partitions are (possibly using some simple script),
> - use device-mapper to create block devices covering partitions,
> - mount them.
> 
> I do not know if this anwser will win your price but it is IMHO far better 
> than hexedit... :-) And probably this is the only anwser.
> 
> (IIRC if you have one partition you can skip partition table with offset 
> option to losetup. But this will only work in this special case...)
> 
> 
> Grzegorz Kulewski
> 

-- 
--
<a href="http://lkcl.net">http://lkcl.net</a>
--

Re: accessing loopback filesystem+partitions on a file

[Peter Chubb]

>>>>> "Luke" == Luke Kenneth Casson Leighton <lkcl@lkcl.net> writes:


Luke> xen guest OSes manage it fine - the xen layer provides a means
Luke> to present any block device as a "disk".

You can do this witht he loop device, but you need to get the
partition info out first.

Use parted or fdisk to get the partition info.
Then use losetup with the appropriate offset.

Luke> that loopback filesystems cannot be presented as block devices
Luke> by the linux kernel (with no involvement of xen) seems to be a
Luke> curious omission.

But they can!  But a loopback device can't be partitioned.  So do it
one partition at a time.

You'll probably only have a few real filesystems on the disk image
anyway.



-- 
Dr Peter Chubb  http://www.gelato.unsw.edu.au  peterc AT gelato.unsw.edu.au
The technical we do immediately,  the political takes *forever*

Re: accessing loopback filesystem+partitions on a file

[Luke Kenneth Casson Leighton]

On Wed, Jun 29, 2005 at 11:48:40AM +1000, Peter Chubb wrote:

> Luke> that loopback filesystems cannot be presented as block devices
> Luke> by the linux kernel (with no involvement of xen) seems to be a
> Luke> curious omission.
> 
> But they can!  But a loopback device can't be partitioned.  So do it
> one partition at a time.

 ian and mark kindly responded pointing out some programs which
 make that possible.

 i liked the LVM one best.

> You'll probably only have a few real filesystems on the disk image
> anyway.
 
 hi peter, thank you for responding.

 see http://hands.com/d-i, searching for "xen0".

 the key is to be able to test-run debian installations in
 a xen guest domain (with absolute minimal changes to the
 packages or the debian boot installs).

 and of course a debian install expects to see a hard drive, which it
 expects to be able to partition - so it is given one.

 l.

Re: accessing loopback filesystem+partitions on a file

[Luke Kenneth Casson Leighton]

okay - i should be clearer.

the thing that is missing, that only xen provides, is the presentation
of another block device as a hard drive.

e.g. /dev/volumegroup/volumename --> /dev/loopblocka
or   /dev/loop0 -> /dev/loopblocka

such that it is possible to then subsequently do this:

fdisk /dev/loopblocka and
mkfs.ext2 /dev/loopblocka1
mount /dev/loopblocka1 -t ext2 /mnt/somewhere

you get the gist.

basically, the thing that is missing (or i can't find it)
from linux is a driver with the ability to present [any] block
devices with their major+minor numbers as a [fsck-]recogniseable
block device with its own major number, with the implicit
ability to create minor numbers within it.

lvm is in its own way a sort of mad-cap over-extended version
of the above, if you think about it carefully and can understand
the sentence.

l.


On Wed, Jun 29, 2005 at 02:35:25AM +0200, Grzegorz Kulewski wrote:
> On Wed, 29 Jun 2005, Luke Kenneth Casson Leighton wrote:
> 
> >[if you are happy to reply at all, please reply cc'd thank you.]
> >
> >hi,
> >
> >i'm really sorry to be bothering people on this list but i genuinely
> >don't what phrases to google for what i am looking for without getting
> >swamped by useless pages, which you will understand why when you see
> >the question, below.
> >
> >the question is, therefore:
> >
> >	* how the hell do you loopback mount (or lvm mount
> >	  or _anything_! something!)  partitions that have
> >	  been created in a loopback'd file!!!!
> >
> >	  [aside from booting up a second pre-installed xen
> >	  guest domain and making the filesystem-in-a-file
> >	  available as /dev/hdb of course.]
> >
> >answers of the form "work out where the partitions are, then use
> >hexedit to remove the first few blocks" will win no prizes here.
> 
> The bad news: it was impossible (or at least very hard to do).
> 
> The good news: it is possible now. The anwser is:
> - figure where the partitions are (possibly using some simple script),
> - use device-mapper to create block devices covering partitions,
> - mount them.
> 
> I do not know if this anwser will win your price but it is IMHO far better 
> than hexedit... :-) And probably this is the only anwser.
> 
> (IIRC if you have one partition you can skip partition table with offset 
> option to losetup. But this will only work in this special case...)
> 
> 
> Grzegorz Kulewski
> 

-- 
--
<a href="http://lkcl.net">http://lkcl.net</a>
--

Re: accessing loopback filesystem+partitions on a file

[Jari Ruusu]

[-- Attachment #1: Type: text/plain, Size: 973 bytes --]

Luke Kenneth Casson Leighton wrote:
> such that it is possible to then subsequently do this:
> 
> fdisk /dev/loopblocka and
> mkfs.ext2 /dev/loopblocka1
> mount /dev/loopblocka1 -t ext2 /mnt/somewhere

Using attached image-mount script that is possible. Like this:

  ln -s image-mount image-losetup
  fdisk filename
       (remember to set correct cyl/head/sect before creating partitions)
  ./image-losetup /dev/loop7 filename 2
  mkfs.ext2 /dev/loop7
  mount /dev/loop7 -t ext2 /mnt/somewhere
  umount /mnt/somewhere
  losetup -d /dev/loop7

or

  ./image-mount filename 2 /mnt/somewhere
  umount /mnt/somewhere

Good news is that image-mount and image-losetup scripts get both size and
offset of the partition right, so mkfs creates correct size file system. Bad
news is that it only works with losetup and mount programs from loop-AES
package. Google for "loop-AES" if you can't find it.

-- 
Jari Ruusu  1024R/3A220F51 5B 4B F9 BB D3 3F 52 E9  DB 1D EB E3 24 0E A9 DD

[-- Attachment #2: image-mount.bz2 --]
[-- Type: application/octet-stream, Size: 572 bytes --]

Re: accessing loopback filesystem+partitions on a file

[Luke Kenneth Casson Leighton]

jari, thank you.

there are _loads_ of people with mad and bad solutions - why
are none of them easily accessible via oh say debian packages???

l.


On Wed, Jun 29, 2005 at 04:27:49PM +0300, Jari Ruusu wrote:
> Luke Kenneth Casson Leighton wrote:
> > such that it is possible to then subsequently do this:
> > 
> > fdisk /dev/loopblocka and
> > mkfs.ext2 /dev/loopblocka1
> > mount /dev/loopblocka1 -t ext2 /mnt/somewhere
> 
> Using attached image-mount script that is possible. Like this:
> 
>   ln -s image-mount image-losetup
>   fdisk filename
>        (remember to set correct cyl/head/sect before creating partitions)
>   ./image-losetup /dev/loop7 filename 2
>   mkfs.ext2 /dev/loop7
>   mount /dev/loop7 -t ext2 /mnt/somewhere
>   umount /mnt/somewhere
>   losetup -d /dev/loop7
> 
> or
> 
>   ./image-mount filename 2 /mnt/somewhere
>   umount /mnt/somewhere
> 
> Good news is that image-mount and image-losetup scripts get both size and
> offset of the partition right, so mkfs creates correct size file system. Bad
> news is that it only works with losetup and mount programs from loop-AES
> package. Google for "loop-AES" if you can't find it.
> 
> -- 
> Jari Ruusu  1024R/3A220F51 5B 4B F9 BB D3 3F 52 E9  DB 1D EB E3 24 0E A9 DD


-- 
--
<a href="http://lkcl.net">http://lkcl.net</a>
--

Re: accessing loopback filesystem+partitions on a file

[P]

Luke Kenneth Casson Leighton wrote:
> 
> 	* how the hell do you loopback mount (or lvm mount
> 	  or _anything_! something!)  partitions that have
> 	  been created in a loopback'd file!!!!

Have a look at http://www.pixelbeat.org/scripts/lomount.sh

Pádraig.

RE: accessing loopback filesystem+partitions on a file

[Ian Pratt]

> basically, the thing that is missing (or i can't find it) 
> from linux is a driver with the ability to present [any] 
> block devices with their major+minor numbers as a 
> [fsck-]recogniseable block device with its own major number, 
> with the implicit ability to create minor numbers within it.

The easiest thing to use is the 'lomount' utility that comes with qemu.

http://www.dad-answers.com/qemu/utilities/QEMU-HD-Mounter/lomount/lomoun
t.c

There's also kpartx which is part of device mapper, but its rather
trickier to build.

I think we should pull lomount into our tree until kpartx becomes
ubiquitous.

Ian