kpartx for raisin hvm tests

kpartx for raisin hvm tests

[Géza Gémes]

Hi,

I've tried to run the raisin test suite, while pv tests pass the hvm 
tests fail. I've identified a number of problems, starting with two 
small disk size to formating the whole disk and then being unable to 
install grub to the boot sector. I've trace down these problems into the 
incorrect invocation of the _create_loop_device function in 
scripts/lopartsetup.

My question: Will it be acceptable if I would replace this part of the 
code with a kpartx call? Or introducing kpartx is a too big change in 
the list of dependencies?

Cheers,

Geza



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: kpartx for raisin hvm tests

[Stefano Stabellini]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 847 bytes --]

On Tue, 21 Feb 2017, Géza Gémes wrote:
> Hi,
> 
> I've tried to run the raisin test suite, while pv tests pass the hvm tests
> fail. I've identified a number of problems, starting with two small disk size
> to formating the whole disk and then being unable to install grub to the boot
> sector. I've trace down these problems into the incorrect invocation of the
> _create_loop_device function in scripts/lopartsetup.
> 
> My question: Will it be acceptable if I would replace this part of the code
> with a kpartx call? Or introducing kpartx is a too big change in the list of
> dependencies?
 
I understand that kpartx makes things much easier, but before
introducing it as a dependency, I would like to understand this problem
a bit better.

Why is _create_loop_device invoked incorrectly? Is it index or offset
that is calculate incorrectly?

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: kpartx for raisin hvm tests

[Géza Gémes]

On 2017-02-21 23:10, Stefano Stabellini wrote:
> On Tue, 21 Feb 2017, Géza Gémes wrote:
>> Hi,
>>
>> I've tried to run the raisin test suite, while pv tests pass the hvm tests
>> fail. I've identified a number of problems, starting with two small disk size
>> to formating the whole disk and then being unable to install grub to the boot
>> sector. I've trace down these problems into the incorrect invocation of the
>> _create_loop_device function in scripts/lopartsetup.
>>
>> My question: Will it be acceptable if I would replace this part of the code
>> with a kpartx call? Or introducing kpartx is a too big change in the list of
>> dependencies?
>   
> I understand that kpartx makes things much easier, but before
> introducing it as a dependency, I would like to understand this problem
> a bit better.
>
> Why is _create_loop_device invoked incorrectly? Is it index or offset
> that is calculate incorrectly?

Hi Stefano,

In scripts/lopartsetup:56 unit="`fdisk -lu $filename 2>/dev/null | grep 
-e "^Units = " | cut -d " " -f 9`" . Using ubuntu 16.04 (fdisk coming 
from util-linux-2.27.1-6ubuntu3.2) this yields to an empty variable, as:

$sudo fdisk -lu /tmp/tmp.x9UN6uxaG2/busybox-vm-disk 2>/dev/null

Disk /tmp/tmp.x9UN6uxaG2/busybox-vm-disk: 60 MiB, 62914560 bytes, 122880 
sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes

Because of this both unit and offset are wrong ( offset=`echo $i | tr -s 
" " | cut -d " " -f 2`, where i=fdisk -lu $filename 2>/dev/null | grep 
-e "^$filename")

As I think that different versions of fdisk will produce different 
results, we either introduce an additional logic for the fdisk version, 
either change this part completely.

Cheers,

Geza



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: kpartx for raisin hvm tests

[Stefano Stabellini]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 2418 bytes --]

On Wed, 22 Feb 2017, Géza Gémes wrote:
> On 2017-02-21 23:10, Stefano Stabellini wrote:
> > On Tue, 21 Feb 2017, Géza Gémes wrote:
> > > Hi,
> > > 
> > > I've tried to run the raisin test suite, while pv tests pass the hvm tests
> > > fail. I've identified a number of problems, starting with two small disk
> > > size
> > > to formating the whole disk and then being unable to install grub to the
> > > boot
> > > sector. I've trace down these problems into the incorrect invocation of
> > > the
> > > _create_loop_device function in scripts/lopartsetup.
> > > 
> > > My question: Will it be acceptable if I would replace this part of the
> > > code
> > > with a kpartx call? Or introducing kpartx is a too big change in the list
> > > of
> > > dependencies?
> >   I understand that kpartx makes things much easier, but before
> > introducing it as a dependency, I would like to understand this problem
> > a bit better.
> > 
> > Why is _create_loop_device invoked incorrectly? Is it index or offset
> > that is calculate incorrectly?
> 
> Hi Stefano,
> 
> In scripts/lopartsetup:56 unit="`fdisk -lu $filename 2>/dev/null | grep -e
> "^Units = " | cut -d " " -f 9`" . Using ubuntu 16.04 (fdisk coming from
> util-linux-2.27.1-6ubuntu3.2) this yields to an empty variable, as:
> 
> $sudo fdisk -lu /tmp/tmp.x9UN6uxaG2/busybox-vm-disk 2>/dev/null
> 
> Disk /tmp/tmp.x9UN6uxaG2/busybox-vm-disk: 60 MiB, 62914560 bytes, 122880
> sectors
> Units: sectors of 1 * 512 = 512 bytes
> Sector size (logical/physical): 512 bytes / 512 bytes
> I/O size (minimum/optimal): 512 bytes / 512 bytes
> 
> Because of this both unit and offset are wrong ( offset=`echo $i | tr -s " " |
> cut -d " " -f 2`, where i=fdisk -lu $filename 2>/dev/null | grep -e
> "^$filename")
> 
> As I think that different versions of fdisk will produce different results, we
> either introduce an additional logic for the fdisk version, either change this
> part completely.

It doesn't look like fdisk changed output in this case. It looks like
the disk doesn't have any partitions in it. Am I right?

It would be easy to add support to lopartsetup to detect disks without
partitions, and deal with them correctly, without bringing in kpartx.
However, this scenario shouldn't occur, because lopartsetup is only
called by create_one_partition, right after creating a partition on the
disk.

Do you know why create_one_partition doesn't work as expected?

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: kpartx for raisin hvm tests

[Gémes Géza]

2017-02-27 23:52 keltezéssel, Stefano Stabellini írta:
> On Wed, 22 Feb 2017, Géza Gémes wrote:
>> On 2017-02-21 23:10, Stefano Stabellini wrote:
>>> On Tue, 21 Feb 2017, Géza Gémes wrote:
>>>> Hi,
>>>>
>>>> I've tried to run the raisin test suite, while pv tests pass the hvm tests
>>>> fail. I've identified a number of problems, starting with two small disk
>>>> size
>>>> to formating the whole disk and then being unable to install grub to the
>>>> boot
>>>> sector. I've trace down these problems into the incorrect invocation of
>>>> the
>>>> _create_loop_device function in scripts/lopartsetup.
>>>>
>>>> My question: Will it be acceptable if I would replace this part of the
>>>> code
>>>> with a kpartx call? Or introducing kpartx is a too big change in the list
>>>> of
>>>> dependencies?
>>>    I understand that kpartx makes things much easier, but before
>>> introducing it as a dependency, I would like to understand this problem
>>> a bit better.
>>>
>>> Why is _create_loop_device invoked incorrectly? Is it index or offset
>>> that is calculate incorrectly?
>> Hi Stefano,
>>
>> In scripts/lopartsetup:56 unit="`fdisk -lu $filename 2>/dev/null | grep -e
>> "^Units = " | cut -d " " -f 9`" . Using ubuntu 16.04 (fdisk coming from
>> util-linux-2.27.1-6ubuntu3.2) this yields to an empty variable, as:
>>
>> $sudo fdisk -lu /tmp/tmp.x9UN6uxaG2/busybox-vm-disk 2>/dev/null
>>
>> Disk /tmp/tmp.x9UN6uxaG2/busybox-vm-disk: 60 MiB, 62914560 bytes, 122880
>> sectors
>> Units: sectors of 1 * 512 = 512 bytes
>> Sector size (logical/physical): 512 bytes / 512 bytes
>> I/O size (minimum/optimal): 512 bytes / 512 bytes
>>
>> Because of this both unit and offset are wrong ( offset=`echo $i | tr -s " " |
>> cut -d " " -f 2`, where i=fdisk -lu $filename 2>/dev/null | grep -e
>> "^$filename")
>>
>> As I think that different versions of fdisk will produce different results, we
>> either introduce an additional logic for the fdisk version, either change this
>> part completely.
> It doesn't look like fdisk changed output in this case. It looks like
> the disk doesn't have any partitions in it. Am I right?
>
> It would be easy to add support to lopartsetup to detect disks without
> partitions, and deal with them correctly, without bringing in kpartx.
> However, this scenario shouldn't occur, because lopartsetup is only
> called by create_one_partition, right after creating a partition on the
> disk.
>
> Do you know why create_one_partition doesn't work as expected?

Hi Stefano,

Sorry for the late answer. The only change I've did on Ubuntu 16.04 was 
to increase the hvm disk size to 60 MB.

In the meanwhile I set up an ubuntu 14.04 test system and for the first 
time tried to run the tests as an ordinary user instead of root. I've 
found a set of problems. I've made a patch 
(https://github.com/geza-gemes/raisin/commit/8a1227d96697a4d8be9130fd9b16404decbe7605) 
for those. Although this fixes the problem of running the tests as 
non-root, it turned out, that even on ubuntu 14.04 the 20MB disk is not 
enough for the hvm guest. I'll try to find the suitable disk size there 
and if successful I'll move back to ubuntu 16.04.

Cheers,

Geza


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: kpartx for raisin hvm tests

[Stefano Stabellini]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3845 bytes --]

On Wed, 1 Mar 2017, Gémes Géza wrote:
> 2017-02-27 23:52 keltezéssel, Stefano Stabellini írta:
> > On Wed, 22 Feb 2017, Géza Gémes wrote:
> > > On 2017-02-21 23:10, Stefano Stabellini wrote:
> > > > On Tue, 21 Feb 2017, Géza Gémes wrote:
> > > > > Hi,
> > > > > 
> > > > > I've tried to run the raisin test suite, while pv tests pass the hvm
> > > > > tests
> > > > > fail. I've identified a number of problems, starting with two small
> > > > > disk
> > > > > size
> > > > > to formating the whole disk and then being unable to install grub to
> > > > > the
> > > > > boot
> > > > > sector. I've trace down these problems into the incorrect invocation
> > > > > of
> > > > > the
> > > > > _create_loop_device function in scripts/lopartsetup.
> > > > > 
> > > > > My question: Will it be acceptable if I would replace this part of the
> > > > > code
> > > > > with a kpartx call? Or introducing kpartx is a too big change in the
> > > > > list
> > > > > of
> > > > > dependencies?
> > > >    I understand that kpartx makes things much easier, but before
> > > > introducing it as a dependency, I would like to understand this problem
> > > > a bit better.
> > > > 
> > > > Why is _create_loop_device invoked incorrectly? Is it index or offset
> > > > that is calculate incorrectly?
> > > Hi Stefano,
> > > 
> > > In scripts/lopartsetup:56 unit="`fdisk -lu $filename 2>/dev/null | grep -e
> > > "^Units = " | cut -d " " -f 9`" . Using ubuntu 16.04 (fdisk coming from
> > > util-linux-2.27.1-6ubuntu3.2) this yields to an empty variable, as:
> > > 
> > > $sudo fdisk -lu /tmp/tmp.x9UN6uxaG2/busybox-vm-disk 2>/dev/null
> > > 
> > > Disk /tmp/tmp.x9UN6uxaG2/busybox-vm-disk: 60 MiB, 62914560 bytes, 122880
> > > sectors
> > > Units: sectors of 1 * 512 = 512 bytes
> > > Sector size (logical/physical): 512 bytes / 512 bytes
> > > I/O size (minimum/optimal): 512 bytes / 512 bytes
> > > 
> > > Because of this both unit and offset are wrong ( offset=`echo $i | tr -s "
> > > " |
> > > cut -d " " -f 2`, where i=fdisk -lu $filename 2>/dev/null | grep -e
> > > "^$filename")
> > > 
> > > As I think that different versions of fdisk will produce different
> > > results, we
> > > either introduce an additional logic for the fdisk version, either change
> > > this
> > > part completely.
> > It doesn't look like fdisk changed output in this case. It looks like
> > the disk doesn't have any partitions in it. Am I right?
> > 
> > It would be easy to add support to lopartsetup to detect disks without
> > partitions, and deal with them correctly, without bringing in kpartx.
> > However, this scenario shouldn't occur, because lopartsetup is only
> > called by create_one_partition, right after creating a partition on the
> > disk.
> > 
> > Do you know why create_one_partition doesn't work as expected?
> 
> Hi Stefano,
> 
> Sorry for the late answer. The only change I've did on Ubuntu 16.04 was to
> increase the hvm disk size to 60 MB.
> 
> In the meanwhile I set up an ubuntu 14.04 test system and for the first time
> tried to run the tests as an ordinary user instead of root. I've found a set
> of problems. I've made a patch
> (https://github.com/geza-gemes/raisin/commit/8a1227d96697a4d8be9130fd9b16404decbe7605)
> for those.

That's a good patch, thank you. Could you please submit it to xen-devel?


> Although this fixes the problem of running the tests as non-root,
> it turned out, that even on ubuntu 14.04 the 20MB disk is not enough for the
> hvm guest. I'll try to find the suitable disk size there and if successful
> I'll move back to ubuntu 16.04.

The tiny disk size was only meant to be used for busybox guests. Also, I
still don't understand what's wrong with the fdisk code. Could you
please check if the partition table has been setup correctly on the
disk? In other words, does create_one_partition work correctly?

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: kpartx for raisin hvm tests

[Gémes Géza]

2017-03-01 20:48 keltezéssel, Stefano Stabellini írta:
> On Wed, 1 Mar 2017, Gémes Géza wrote:
>> 2017-02-27 23:52 keltezéssel, Stefano Stabellini írta:
>>> On Wed, 22 Feb 2017, Géza Gémes wrote:
>>>> On 2017-02-21 23:10, Stefano Stabellini wrote:
>>>>> On Tue, 21 Feb 2017, Géza Gémes wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I've tried to run the raisin test suite, while pv tests pass the hvm
>>>>>> tests
>>>>>> fail. I've identified a number of problems, starting with two small
>>>>>> disk
>>>>>> size
>>>>>> to formating the whole disk and then being unable to install grub to
>>>>>> the
>>>>>> boot
>>>>>> sector. I've trace down these problems into the incorrect invocation
>>>>>> of
>>>>>> the
>>>>>> _create_loop_device function in scripts/lopartsetup.
>>>>>>
>>>>>> My question: Will it be acceptable if I would replace this part of the
>>>>>> code
>>>>>> with a kpartx call? Or introducing kpartx is a too big change in the
>>>>>> list
>>>>>> of
>>>>>> dependencies?
>>>>>     I understand that kpartx makes things much easier, but before
>>>>> introducing it as a dependency, I would like to understand this problem
>>>>> a bit better.
>>>>>
>>>>> Why is _create_loop_device invoked incorrectly? Is it index or offset
>>>>> that is calculate incorrectly?
>>>> Hi Stefano,
>>>>
>>>> In scripts/lopartsetup:56 unit="`fdisk -lu $filename 2>/dev/null | grep -e
>>>> "^Units = " | cut -d " " -f 9`" . Using ubuntu 16.04 (fdisk coming from
>>>> util-linux-2.27.1-6ubuntu3.2) this yields to an empty variable, as:
>>>>
>>>> $sudo fdisk -lu /tmp/tmp.x9UN6uxaG2/busybox-vm-disk 2>/dev/null
>>>>
>>>> Disk /tmp/tmp.x9UN6uxaG2/busybox-vm-disk: 60 MiB, 62914560 bytes, 122880
>>>> sectors
>>>> Units: sectors of 1 * 512 = 512 bytes
>>>> Sector size (logical/physical): 512 bytes / 512 bytes
>>>> I/O size (minimum/optimal): 512 bytes / 512 bytes
>>>>
>>>> Because of this both unit and offset are wrong ( offset=`echo $i | tr -s "
>>>> " |
>>>> cut -d " " -f 2`, where i=fdisk -lu $filename 2>/dev/null | grep -e
>>>> "^$filename")
>>>>
>>>> As I think that different versions of fdisk will produce different
>>>> results, we
>>>> either introduce an additional logic for the fdisk version, either change
>>>> this
>>>> part completely.
>>> It doesn't look like fdisk changed output in this case. It looks like
>>> the disk doesn't have any partitions in it. Am I right?
>>>
>>> It would be easy to add support to lopartsetup to detect disks without
>>> partitions, and deal with them correctly, without bringing in kpartx.
>>> However, this scenario shouldn't occur, because lopartsetup is only
>>> called by create_one_partition, right after creating a partition on the
>>> disk.
>>>
>>> Do you know why create_one_partition doesn't work as expected?
>> Hi Stefano,
>>
>> Sorry for the late answer. The only change I've did on Ubuntu 16.04 was to
>> increase the hvm disk size to 60 MB.
>>
>> In the meanwhile I set up an ubuntu 14.04 test system and for the first time
>> tried to run the tests as an ordinary user instead of root. I've found a set
>> of problems. I've made a patch
>> (https://github.com/geza-gemes/raisin/commit/8a1227d96697a4d8be9130fd9b16404decbe7605)
>> for those.
> That's a good patch, thank you. Could you please submit it to xen-devel?
>
>
>> Although this fixes the problem of running the tests as non-root,
>> it turned out, that even on ubuntu 14.04 the 20MB disk is not enough for the
>> hvm guest. I'll try to find the suitable disk size there and if successful
>> I'll move back to ubuntu 16.04.
> The tiny disk size was only meant to be used for busybox guests. Also, I
> still don't understand what's wrong with the fdisk code. Could you
> please check if the partition table has been setup correctly on the
> disk? In other words, does create_one_partition work correctly?

Hi Stefano,

I've sent the patch to xen-devel as an ordinary e-mail as I didn't 
manage to make git send-email work with my gmail account :-(.

My experience with ubuntu 14.04 and 16.04 shows, that the disk size is 
too small to accommodate the kernel and the initrd. Increasing to 60MB 
solves that problem, but the image is still not bootable (grub cannot be 
installed on 16.04 as it looks like the whole disk gets formated, and 
grub cannot load the kernel on 14.04.). I'll continue investigating 
14.04 tomorrow.

Cheers,

Geza


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: kpartx for raisin hvm tests

[Stefano Stabellini]

[-- Attachment #1: Type: TEXT/PLAIN, Size: 5147 bytes --]

On Wed, 1 Mar 2017, Gémes Géza wrote:
> 2017-03-01 20:48 keltezéssel, Stefano Stabellini írta:
> > On Wed, 1 Mar 2017, Gémes Géza wrote:
> > > 2017-02-27 23:52 keltezéssel, Stefano Stabellini írta:
> > > > On Wed, 22 Feb 2017, Géza Gémes wrote:
> > > > > On 2017-02-21 23:10, Stefano Stabellini wrote:
> > > > > > On Tue, 21 Feb 2017, Géza Gémes wrote:
> > > > > > > Hi,
> > > > > > > 
> > > > > > > I've tried to run the raisin test suite, while pv tests pass the
> > > > > > > hvm
> > > > > > > tests
> > > > > > > fail. I've identified a number of problems, starting with two
> > > > > > > small
> > > > > > > disk
> > > > > > > size
> > > > > > > to formating the whole disk and then being unable to install grub
> > > > > > > to
> > > > > > > the
> > > > > > > boot
> > > > > > > sector. I've trace down these problems into the incorrect
> > > > > > > invocation
> > > > > > > of
> > > > > > > the
> > > > > > > _create_loop_device function in scripts/lopartsetup.
> > > > > > > 
> > > > > > > My question: Will it be acceptable if I would replace this part of
> > > > > > > the
> > > > > > > code
> > > > > > > with a kpartx call? Or introducing kpartx is a too big change in
> > > > > > > the
> > > > > > > list
> > > > > > > of
> > > > > > > dependencies?
> > > > > >     I understand that kpartx makes things much easier, but before
> > > > > > introducing it as a dependency, I would like to understand this
> > > > > > problem
> > > > > > a bit better.
> > > > > > 
> > > > > > Why is _create_loop_device invoked incorrectly? Is it index or
> > > > > > offset
> > > > > > that is calculate incorrectly?
> > > > > Hi Stefano,
> > > > > 
> > > > > In scripts/lopartsetup:56 unit="`fdisk -lu $filename 2>/dev/null |
> > > > > grep -e
> > > > > "^Units = " | cut -d " " -f 9`" . Using ubuntu 16.04 (fdisk coming
> > > > > from
> > > > > util-linux-2.27.1-6ubuntu3.2) this yields to an empty variable, as:
> > > > > 
> > > > > $sudo fdisk -lu /tmp/tmp.x9UN6uxaG2/busybox-vm-disk 2>/dev/null
> > > > > 
> > > > > Disk /tmp/tmp.x9UN6uxaG2/busybox-vm-disk: 60 MiB, 62914560 bytes,
> > > > > 122880
> > > > > sectors
> > > > > Units: sectors of 1 * 512 = 512 bytes
> > > > > Sector size (logical/physical): 512 bytes / 512 bytes
> > > > > I/O size (minimum/optimal): 512 bytes / 512 bytes
> > > > > 
> > > > > Because of this both unit and offset are wrong ( offset=`echo $i | tr
> > > > > -s "
> > > > > " |
> > > > > cut -d " " -f 2`, where i=fdisk -lu $filename 2>/dev/null | grep -e
> > > > > "^$filename")
> > > > > 
> > > > > As I think that different versions of fdisk will produce different
> > > > > results, we
> > > > > either introduce an additional logic for the fdisk version, either
> > > > > change
> > > > > this
> > > > > part completely.
> > > > It doesn't look like fdisk changed output in this case. It looks like
> > > > the disk doesn't have any partitions in it. Am I right?
> > > > 
> > > > It would be easy to add support to lopartsetup to detect disks without
> > > > partitions, and deal with them correctly, without bringing in kpartx.
> > > > However, this scenario shouldn't occur, because lopartsetup is only
> > > > called by create_one_partition, right after creating a partition on the
> > > > disk.
> > > > 
> > > > Do you know why create_one_partition doesn't work as expected?
> > > Hi Stefano,
> > > 
> > > Sorry for the late answer. The only change I've did on Ubuntu 16.04 was to
> > > increase the hvm disk size to 60 MB.
> > > 
> > > In the meanwhile I set up an ubuntu 14.04 test system and for the first
> > > time
> > > tried to run the tests as an ordinary user instead of root. I've found a
> > > set
> > > of problems. I've made a patch
> > > (https://github.com/geza-gemes/raisin/commit/8a1227d96697a4d8be9130fd9b16404decbe7605)
> > > for those.
> > That's a good patch, thank you. Could you please submit it to xen-devel?
> > 
> > 
> > > Although this fixes the problem of running the tests as non-root,
> > > it turned out, that even on ubuntu 14.04 the 20MB disk is not enough for
> > > the
> > > hvm guest. I'll try to find the suitable disk size there and if successful
> > > I'll move back to ubuntu 16.04.
> > The tiny disk size was only meant to be used for busybox guests. Also, I
> > still don't understand what's wrong with the fdisk code. Could you
> > please check if the partition table has been setup correctly on the
> > disk? In other words, does create_one_partition work correctly?
> 
> Hi Stefano,
> 
> I've sent the patch to xen-devel as an ordinary e-mail as I didn't manage to
> make git send-email work with my gmail account :-(.

It takes some time to get right.


> My experience with ubuntu 14.04 and 16.04 shows, that the disk size is too
> small to accommodate the kernel and the initrd. Increasing to 60MB solves that
> problem, but the image is still not bootable (grub cannot be installed on
> 16.04 as it looks like the whole disk gets formated, and grub cannot load the
> kernel on 14.04.). I'll continue investigating 14.04 tomorrow.

All right, thanks. For debugging the fdisk problem, you might as well
increase the disk size as much as it takes.

[-- Attachment #2: Type: text/plain, Size: 127 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: kpartx for raisin hvm tests

[Gémes Géza]

2017-03-02 02:16 keltezéssel, Stefano Stabellini írta:
> On Wed, 1 Mar 2017, Gémes Géza wrote:
>> 2017-03-01 20:48 keltezéssel, Stefano Stabellini írta:
>>> On Wed, 1 Mar 2017, Gémes Géza wrote:
>>>> 2017-02-27 23:52 keltezéssel, Stefano Stabellini írta:
>>>>> On Wed, 22 Feb 2017, Géza Gémes wrote:
>>>>>> On 2017-02-21 23:10, Stefano Stabellini wrote:
>>>>>>> On Tue, 21 Feb 2017, Géza Gémes wrote:
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> I've tried to run the raisin test suite, while pv tests pass the
>>>>>>>> hvm
>>>>>>>> tests
>>>>>>>> fail. I've identified a number of problems, starting with two
>>>>>>>> small
>>>>>>>> disk
>>>>>>>> size
>>>>>>>> to formating the whole disk and then being unable to install grub
>>>>>>>> to
>>>>>>>> the
>>>>>>>> boot
>>>>>>>> sector. I've trace down these problems into the incorrect
>>>>>>>> invocation
>>>>>>>> of
>>>>>>>> the
>>>>>>>> _create_loop_device function in scripts/lopartsetup.
>>>>>>>>
>>>>>>>> My question: Will it be acceptable if I would replace this part of
>>>>>>>> the
>>>>>>>> code
>>>>>>>> with a kpartx call? Or introducing kpartx is a too big change in
>>>>>>>> the
>>>>>>>> list
>>>>>>>> of
>>>>>>>> dependencies?
>>>>>>>      I understand that kpartx makes things much easier, but before
>>>>>>> introducing it as a dependency, I would like to understand this
>>>>>>> problem
>>>>>>> a bit better.
>>>>>>>
>>>>>>> Why is _create_loop_device invoked incorrectly? Is it index or
>>>>>>> offset
>>>>>>> that is calculate incorrectly?
>>>>>> Hi Stefano,
>>>>>>
>>>>>> In scripts/lopartsetup:56 unit="`fdisk -lu $filename 2>/dev/null |
>>>>>> grep -e
>>>>>> "^Units = " | cut -d " " -f 9`" . Using ubuntu 16.04 (fdisk coming
>>>>>> from
>>>>>> util-linux-2.27.1-6ubuntu3.2) this yields to an empty variable, as:
>>>>>>
>>>>>> $sudo fdisk -lu /tmp/tmp.x9UN6uxaG2/busybox-vm-disk 2>/dev/null
>>>>>>
>>>>>> Disk /tmp/tmp.x9UN6uxaG2/busybox-vm-disk: 60 MiB, 62914560 bytes,
>>>>>> 122880
>>>>>> sectors
>>>>>> Units: sectors of 1 * 512 = 512 bytes
>>>>>> Sector size (logical/physical): 512 bytes / 512 bytes
>>>>>> I/O size (minimum/optimal): 512 bytes / 512 bytes
>>>>>>
>>>>>> Because of this both unit and offset are wrong ( offset=`echo $i | tr
>>>>>> -s "
>>>>>> " |
>>>>>> cut -d " " -f 2`, where i=fdisk -lu $filename 2>/dev/null | grep -e
>>>>>> "^$filename")
>>>>>>
>>>>>> As I think that different versions of fdisk will produce different
>>>>>> results, we
>>>>>> either introduce an additional logic for the fdisk version, either
>>>>>> change
>>>>>> this
>>>>>> part completely.
>>>>> It doesn't look like fdisk changed output in this case. It looks like
>>>>> the disk doesn't have any partitions in it. Am I right?
>>>>>
>>>>> It would be easy to add support to lopartsetup to detect disks without
>>>>> partitions, and deal with them correctly, without bringing in kpartx.
>>>>> However, this scenario shouldn't occur, because lopartsetup is only
>>>>> called by create_one_partition, right after creating a partition on the
>>>>> disk.
>>>>>
>>>>> Do you know why create_one_partition doesn't work as expected?
>>>> Hi Stefano,
>>>>
>>>> Sorry for the late answer. The only change I've did on Ubuntu 16.04 was to
>>>> increase the hvm disk size to 60 MB.
>>>>
>>>> In the meanwhile I set up an ubuntu 14.04 test system and for the first
>>>> time
>>>> tried to run the tests as an ordinary user instead of root. I've found a
>>>> set
>>>> of problems. I've made a patch
>>>> (https://github.com/geza-gemes/raisin/commit/8a1227d96697a4d8be9130fd9b16404decbe7605)
>>>> for those.
>>> That's a good patch, thank you. Could you please submit it to xen-devel?
>>>
>>>
>>>> Although this fixes the problem of running the tests as non-root,
>>>> it turned out, that even on ubuntu 14.04 the 20MB disk is not enough for
>>>> the
>>>> hvm guest. I'll try to find the suitable disk size there and if successful
>>>> I'll move back to ubuntu 16.04.
>>> The tiny disk size was only meant to be used for busybox guests. Also, I
>>> still don't understand what's wrong with the fdisk code. Could you
>>> please check if the partition table has been setup correctly on the
>>> disk? In other words, does create_one_partition work correctly?
>> Hi Stefano,
>>
>> I've sent the patch to xen-devel as an ordinary e-mail as I didn't manage to
>> make git send-email work with my gmail account :-(.
> It takes some time to get right.
>
>
>> My experience with ubuntu 14.04 and 16.04 shows, that the disk size is too
>> small to accommodate the kernel and the initrd. Increasing to 60MB solves that
>> problem, but the image is still not bootable (grub cannot be installed on
>> 16.04 as it looks like the whole disk gets formated, and grub cannot load the
>> kernel on 14.04.). I'll continue investigating 14.04 tomorrow.
> All right, thanks. For debugging the fdisk problem, you might as well
> increase the disk size as much as it takes.

Hi Stefano,

I'm still fighting to get grub install correctly. In the meanwhile I've 
set up a CentOS7 system and found some issues, for which I've started 
sending patches.

Cheers,

Geza


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel