Integrate fully partman-reiser4 into d-i

Integrate fully partman-reiser4 into d-i

[Jose R R]

Niltze, all!>
(picking up where I left off in previous thread: <
http://marc.info/?l=reiserfs-devel&m=144040412406067&w=2 >

Initiating a new thread since Debian-Installer's partman-reiser4 seems
to be partially functioning. All the previous Reiser4 issues have been
mostly resolved.

So I cloned partman-jfs udeb source into partman-reiser4 udeb (seems
reiserfs udeb has gotten old) and I modified relevant scripts.

However, I need to fine tune those modified scripts specifically for
reiser4 in d-i *and* there is where I need your insight.

Basically, option to format device media into Reiser4 is now available
during relevant d-i phase:
< https://pbs.twimg.com/media/CNfOQw1VEAEYBUk.png:large >

Nevertheless, I get this gibberish dialog when selecting and
proceeding (by pressing enter):
< https://pbs.twimg.com/media/CNfOQ4kUcAAK0Tx.png:large >

If I select continue at the "??? ???" gibberish dialog, d-i is
apparently able to go to the Partition Settings phase:
< https://pbs.twimg.com/media/CNfOQpWUYAA8tqY.png:large >

Selecting "Done setting up the partition", notwithstanding, d-i
returns to initial phase where raw virtual device has not even been
initiated (with MBR)

Cool thing is partman-reiser4 now makes the Reiser4-patched kernel
automatically modprobe reiser4-modules:
< https://pbs.twimg.com/media/CNfOQwxUwAEHh68.png:large >

I have at d-i build/localudebs:
[...]
localudebs/mkreiser4-udeb_1.0.9-2_amd64.udeb
localudebs/reiser4-modules-4.1.0-2-amd64-di_4.1.6-1_amd64.udeb
localudebs/reiser4progs-udeb_1.0.9-2_amd64.udeb
localudebs/partman-reiser4_1_all.udeb
[...]

and added at d-i build/pkg-lists/netboot/amd64.cfg:
[...]
fat-modules-${kernel:Version}
ext4-modules-${kernel:Version}
jfs-modules-${kernel:Version}
reiser4-modules-${kernel:Version}
mkreiser4-udeb
reiser4progs-udeb
partman-reiser4
[...]

Note: without partman-reiser4 specified above, d-i will not show
option to format in Reiser4 -- even if reiser4 UDEB is included under
build/localudebs/

Advice welcomed in advance to make the gibberish dialog go away and to
effectively allow d-i routine to integrate reiser4 fully so as to
complete the front-end formatting proceduere.


Best Professional Regards.


-- 
Jose R R
http://metztli.it
---------------------------------------------------------------------------------------------
Try at no charge http://b2evolution.net for http://OpenShift.com PaaS
---------------------------------------------------------------------------------------------
from our GitHub http://Nepohualtzintzin.com repository. Cloud the easy way!
---------------------------------------------------------------------------------------------

Re: Integrate fully partman-reiser4 into d-i

[Jose R R]

I have realized that although partman-reiser4 udeb enables Reiser4 to
be listed as an option during the Debian-Installer routine of a
netboot ISO image, the lack of support for reiser4 in GNU Parted
(Debian's libparted) prevents installation from the GUI interface.

Moreover, it is clear that although btrfs came *after* Reiser4, the
former had the support of the debian installer team -- which produced
a patch relatively quickly-- but the latter was scorned for whatever
reason and/or undeserved reputation.

Accordingly, although I have been able to generate *all* required
UDEBs to enable the Debian-Installer to format and install Debian into
a Reiser4 root partition, using the GUI interface which makes use of
libparte to accomplish the task is not possible.

On the other hand, using my Reiser4-enabled Debian-Installer netboot
ISO image and selecting "expert install", I successfully installed
Debian into a root partition. Using the menu driven interface, I
initially formated the root partition into JFS -- to satisfy the
installer routine. Then, prior to the "install the base system" menu
option, I opened a shell and tarred the /target/etc/fstab, umounted
the JFS root partition, reformatted: mkfs.reiser4 -o "create=reg40"
/dev/sdaX

Subsequently mounted newly created Reiser4 root partition into
/target; untarred /target/etc/fstab and used nano editor to modify
/etc/fstab -- changing jfs to reiser4 and went back to "install the
base system" interactive menu. Thus Debian routine interface does not
notice the change and the base system installation continues.

There are still some wrinkles to iron out but hopefully I will later
make available a more polished d-i derived Reiser4 Debian-Installer.

Thank you all for your help.


Best Professional Regards.



On Fri, Aug 28, 2015 at 3:56 AM, Jose R R <Jose.r.r@metztli-it.com> wrote:
> Niltze, all!>
> (picking up where I left off in previous thread: <
> http://marc.info/?l=reiserfs-devel&m=144040412406067&w=2 >
>
> Initiating a new thread since Debian-Installer's partman-reiser4 seems
> to be partially functioning. All the previous Reiser4 issues have been
> mostly resolved.
>
> So I cloned partman-jfs udeb source into partman-reiser4 udeb (seems
> reiserfs udeb has gotten old) and I modified relevant scripts.
>
> However, I need to fine tune those modified scripts specifically for
> reiser4 in d-i *and* there is where I need your insight.
>
> Basically, option to format device media into Reiser4 is now available
> during relevant d-i phase:
> < https://pbs.twimg.com/media/CNfOQw1VEAEYBUk.png:large >
>
> Nevertheless, I get this gibberish dialog when selecting and
> proceeding (by pressing enter):
> < https://pbs.twimg.com/media/CNfOQ4kUcAAK0Tx.png:large >
>
> If I select continue at the "??? ???" gibberish dialog, d-i is
> apparently able to go to the Partition Settings phase:
> < https://pbs.twimg.com/media/CNfOQpWUYAA8tqY.png:large >
>
> Selecting "Done setting up the partition", notwithstanding, d-i
> returns to initial phase where raw virtual device has not even been
> initiated (with MBR)
>
> Cool thing is partman-reiser4 now makes the Reiser4-patched kernel
> automatically modprobe reiser4-modules:
> < https://pbs.twimg.com/media/CNfOQwxUwAEHh68.png:large >
>
> I have at d-i build/localudebs:
> [...]
> localudebs/mkreiser4-udeb_1.0.9-2_amd64.udeb
> localudebs/reiser4-modules-4.1.0-2-amd64-di_4.1.6-1_amd64.udeb
> localudebs/reiser4progs-udeb_1.0.9-2_amd64.udeb
> localudebs/partman-reiser4_1_all.udeb
> [...]
>
> and added at d-i build/pkg-lists/netboot/amd64.cfg:
> [...]
> fat-modules-${kernel:Version}
> ext4-modules-${kernel:Version}
> jfs-modules-${kernel:Version}
> reiser4-modules-${kernel:Version}
> mkreiser4-udeb
> reiser4progs-udeb
> partman-reiser4
> [...]
>
> Note: without partman-reiser4 specified above, d-i will not show
> option to format in Reiser4 -- even if reiser4 UDEB is included under
> build/localudebs/
>
> Advice welcomed in advance to make the gibberish dialog go away and to
> effectively allow d-i routine to integrate reiser4 fully so as to
> complete the front-end formatting proceduere.
>
>
> Best Professional Regards.
>
>
> --
> Jose R R
> http://metztli.it
> ---------------------------------------------------------------------------------------------
> Try at no charge http://b2evolution.net for http://OpenShift.com PaaS
> ---------------------------------------------------------------------------------------------
> from our GitHub http://Nepohualtzintzin.com repository. Cloud the easy way!
> ---------------------------------------------------------------------------------------------



-- 
Jose R R
http://metztli.it
---------------------------------------------------------------------------------------------
Try at no charge http://b2evolution.net for http://OpenShift.com PaaS
---------------------------------------------------------------------------------------------
from our GitHub http://Nepohualtzintzin.com repository. Cloud the easy way!
---------------------------------------------------------------------------------------------

Re: Integrate fully partman-reiser4 into d-i

[Colin Watson]

On Sun, Sep 20, 2015 at 07:40:37AM -0700, Jose R R wrote:
> I have realized that although partman-reiser4 udeb enables Reiser4 to
> be listed as an option during the Debian-Installer routine of a
> netboot ISO image, the lack of support for reiser4 in GNU Parted
> (Debian's libparted) prevents installation from the GUI interface.

It should be pretty easy to add that to parted, since nowadays the only
filesystem support it has is probing support.  I suggest preparing a
patch against git://git.sv.gnu.org/parted.git master that adds the
ability to detect existing reiser4 partitions and sending it upstream;
it should only be on the order of a hundred lines.

I'd be happy to cherry-pick such a patch into the Debian packaging once
it has been committed upstream.

Re: Integrate fully partman-reiser4 into d-i

[Edward Shishkin]

On 09/21/2015 01:12 PM, Colin Watson wrote:
> On Sun, Sep 20, 2015 at 07:40:37AM -0700, Jose R R wrote:
>> I have realized that although partman-reiser4 udeb enables Reiser4 to
>> be listed as an option during the Debian-Installer routine of a
>> netboot ISO image, the lack of support for reiser4 in GNU Parted
>> (Debian's libparted) prevents installation from the GUI interface.
> It should be pretty easy to add that to parted, since nowadays the only
> filesystem support it has is probing support.  I suggest preparing a
> patch against git://git.sv.gnu.org/parted.git master that adds the
> ability to detect existing reiser4 partitions and sending it upstream;
> it should only be on the order of a hundred lines.


Does anybody care to prepare the patch?
I think we just need to a read a sector at REISER4_MAGIC_OFFSET
and check the magic string ("ReIsEr4"), see definition of the struct
reiser4_master_sb.

Thanks,
Edward.


> I'd be happy to cherry-pick such a patch into the Debian packaging once
> it has been committed upstream.
>

Re: Integrate fully partman-reiser4 into d-i

[Jose R R]

On Mon, Sep 21, 2015 at 7:49 AM, Edward Shishkin
<edward.shishkin@gmail.com> wrote:
> On 09/21/2015 01:12 PM, Colin Watson wrote:
>>
>> On Sun, Sep 20, 2015 at 07:40:37AM -0700, Jose R R wrote:
>>>
>>> I have realized that although partman-reiser4 udeb enables Reiser4 to
>>> be listed as an option during the Debian-Installer routine of a
>>> netboot ISO image, the lack of support for reiser4 in GNU Parted
>>> (Debian's libparted) prevents installation from the GUI interface.
>>
>> It should be pretty easy to add that to parted, since nowadays the only
>> filesystem support it has is probing support.  I suggest preparing a
>> patch against git://git.sv.gnu.org/parted.git master that adds the
>> ability to detect existing reiser4 partitions and sending it upstream;
>> it should only be on the order of a hundred lines.
>
>
>
> Does anybody care to prepare the patch?
I am working on it Ed. I have been ironing out some wrinkles from the
last Reiser4 patch proposed several years ago in the GNU Parted
mailing lists. It's components and structure were basically cloned by
the initial btrfs patch submitted to Debian. autoreconf has been my
friend ;-)

> I think we just need to a read a sector at REISER4_MAGIC_OFFSET
> and check the magic string ("ReIsEr4"), see definition of the struct
> reiser4_master_sb.
Please enlighten me with reference to its documentation -- online or
in the Reiser4 source code?

>> I'd be happy to cherry-pick such a patch into the Debian packaging once
>> it has been committed upstream.
>>
>
Best Professional Regards.


-- 
Jose R R
http://metztli.it
---------------------------------------------------------------------------------------------
Try at no charge http://b2evolution.net for http://OpenShift.com PaaS
---------------------------------------------------------------------------------------------
from our GitHub http://Nepohualtzintzin.com repository. Cloud the easy way!
---------------------------------------------------------------------------------------------

Re: Integrate fully partman-reiser4 into d-i

[Edward Shishkin]

On 11/12/2015 12:28 PM, Jose R R wrote:
> On Mon, Sep 21, 2015 at 7:49 AM, Edward Shishkin
> <edward.shishkin@gmail.com> wrote:
>> On 09/21/2015 01:12 PM, Colin Watson wrote:
>>> On Sun, Sep 20, 2015 at 07:40:37AM -0700, Jose R R wrote:
>>>> I have realized that although partman-reiser4 udeb enables Reiser4 to
>>>> be listed as an option during the Debian-Installer routine of a
>>>> netboot ISO image, the lack of support for reiser4 in GNU Parted
>>>> (Debian's libparted) prevents installation from the GUI interface.
>>> It should be pretty easy to add that to parted, since nowadays the only
>>> filesystem support it has is probing support.  I suggest preparing a
>>> patch against git://git.sv.gnu.org/parted.git master that adds the
>>> ability to detect existing reiser4 partitions and sending it upstream;
>>> it should only be on the order of a hundred lines.
>>
>>
>> Does anybody care to prepare the patch?
> I am working on it Ed. I have been ironing out some wrinkles from the
> last Reiser4 patch proposed several years ago in the GNU Parted
> mailing lists. It's components and structure were basically cloned by
> the initial btrfs patch submitted to Debian. autoreconf has been my
> friend ;-)
>
>> I think we just need to a read a sector at REISER4_MAGIC_OFFSET
>> and check the magic string ("ReIsEr4"), see definition of the struct
>> reiser4_master_sb.
> Please enlighten me with reference to its documentation -- online or
> in the Reiser4 source code?

Alas, only source code..

In reiser4 kernel sources look for REISER4_MAGIC_OFFSET
and REISER4_SUPER_MAGIC_STRING.

In reiser4progs it is called REISER4_MASTER_OFFSET and
REISER4_MASTER_MAGIC respectively.

Thanks,
Edward.

Re: Integrate fully partman-reiser4 into d-i

[Jose R R]

I hacked (literally) a preliminary version of the of GNU Parted 3.2
source with Reiser4 support.

On Thu, Nov 12, 2015 at 8:23 AM, Edward Shishkin
<edward.shishkin@gmail.com> wrote:
>
>
> On 11/12/2015 12:28 PM, Jose R R wrote:
>>
>> On Mon, Sep 21, 2015 at 7:49 AM, Edward Shishkin
>> <edward.shishkin@gmail.com> wrote:
>>>
>>> On 09/21/2015 01:12 PM, Colin Watson wrote:
>>>>
>>>> On Sun, Sep 20, 2015 at 07:40:37AM -0700, Jose R R wrote:
>>>>>
>>>>> I have realized that although partman-reiser4 udeb enables Reiser4 to
>>>>> be listed as an option during the Debian-Installer routine of a
>>>>> netboot ISO image, the lack of support for reiser4 in GNU Parted
>>>>> (Debian's libparted) prevents installation from the GUI interface.
>>>>
>>>> It should be pretty easy to add that to parted, since nowadays the only
>>>> filesystem support it has is probing support.  I suggest preparing a
>>>> patch against git://git.sv.gnu.org/parted.git master that adds the
>>>> ability to detect existing reiser4 partitions and sending it upstream;
>>>> it should only be on the order of a hundred lines.
>>>
>>>
>>>
>>> Does anybody care to prepare the patch?
>>
>> I am working on it Ed. I have been ironing out some wrinkles from the
>> last Reiser4 patch proposed several years ago in the GNU Parted
>> mailing lists. It's components and structure were basically cloned by
>> the initial btrfs patch submitted to Debian. autoreconf has been my
>> friend ;-)
>>
>>> I think we just need to a read a sector at REISER4_MAGIC_OFFSET
>>> and check the magic string ("ReIsEr4"), see definition of the struct
>>> reiser4_master_sb.
>>
>> Please enlighten me with reference to its documentation -- online or
>> in the Reiser4 source code?
>
>
> Alas, only source code..
>
> In reiser4 kernel sources look for REISER4_MAGIC_OFFSET
> and REISER4_SUPER_MAGIC_STRING.
>
> In reiser4progs it is called REISER4_MASTER_OFFSET and
> REISER4_MASTER_MAGIC respectively.
>

It can be built with the usual:

apt-get install libdevmapper-dev

./configure [ --prefix=/opt/parted-3.2/usr/local ]
make
make install

The resulting parted binary recognizes Reiser4 partitions when invoked
at your shell:
$ parted
GNU Parted 3.2
Using /dev/xda
Welcome to GNU Parted! Type 'help' to view a list of commands.
(parted) print
And it will print recognized partitions -- including reiser4, of course.

Although it was developed and tested in a local machine,
I ran PoC build on small Google Compute Engine (GCE) Cloud instance
with a couple of existing Reiser4 partitions:

Snapshot:
https://pbs.twimg.com/media/CT8U78zUwAE-ZTc.png:large

Here is GNU Parted 3.2 bz2-compressed src modified for Reiser4 (in
case I no longer have time to improve it ;-)
Source: 2.6M reiser4-parted-3.2m.tar.bz2 @
http://metztli.it/readOnlyEphemeral/reiser4-parted-3.2m.tar.bz2
SHA256SUM:
http://metztli.it/readOnlyEphemeral/reiser4-parted-3.2m.tar.bz2.SHA256SUM


Best Professional Regards.

-- 
Jose R R
http://metztli.it
---------------------------------------------------------------------------------------------
Try at no charge http://b2evolution.net for http://OpenShift.com PaaS
---------------------------------------------------------------------------------------------
from our GitHub http://Nepohualtzintzin.com repository. Cloud the easy way!
---------------------------------------------------------------------------------------------

Re: Integrate fully partman-reiser4 into d-i

[Jose R R]

Niltze, everyone!

On Mon, Nov 16, 2015 at 7:37 AM, Jose R R <Jose.r.r@metztli-it.com> wrote:
> I hacked (literally) a preliminary version of the of GNU Parted 3.2
> source with Reiser4 support.
>
> On Thu, Nov 12, 2015 at 8:23 AM, Edward Shishkin
> <edward.shishkin@gmail.com> wrote:
>>
>>
>> On 11/12/2015 12:28 PM, Jose R R wrote:
>>>
>>> On Mon, Sep 21, 2015 at 7:49 AM, Edward Shishkin
>>> <edward.shishkin@gmail.com> wrote:
>>>>
>>>> On 09/21/2015 01:12 PM, Colin Watson wrote:
>>>>>
>>>>> On Sun, Sep 20, 2015 at 07:40:37AM -0700, Jose R R wrote:
>>>>>>
>>>>>> I have realized that although partman-reiser4 udeb enables Reiser4 to
>>>>>> be listed as an option during the Debian-Installer routine of a
>>>>>> netboot ISO image, the lack of support for reiser4 in GNU Parted
>>>>>> (Debian's libparted) prevents installation from the GUI interface.
>>>>>
>>>>> It should be pretty easy to add that to parted, since nowadays the only
>>>>> filesystem support it has is probing support.  I suggest preparing a
>>>>> patch against git://git.sv.gnu.org/parted.git master that adds the
>>>>> ability to detect existing reiser4 partitions and sending it upstream;
>>>>> it should only be on the order of a hundred lines.
>>>>
>>>>
>>>>
>>>> Does anybody care to prepare the patch?
>>>
>>> I am working on it Ed. I have been ironing out some wrinkles from the
>>> last Reiser4 patch proposed several years ago in the GNU Parted
>>> mailing lists. It's components and structure were basically cloned by
>>> the initial btrfs patch submitted to Debian. autoreconf has been my
>>> friend ;-)
>>>
>>>> I think we just need to a read a sector at REISER4_MAGIC_OFFSET
>>>> and check the magic string ("ReIsEr4"), see definition of the struct
>>>> reiser4_master_sb.
>>>
>>> Please enlighten me with reference to its documentation -- online or
>>> in the Reiser4 source code?
>>
>>
>> Alas, only source code..
>>
>> In reiser4 kernel sources look for REISER4_MAGIC_OFFSET
>> and REISER4_SUPER_MAGIC_STRING.
>>
>> In reiser4progs it is called REISER4_MASTER_OFFSET and
>> REISER4_MASTER_MAGIC respectively.
>>
>
> It can be built with the usual:
>
> apt-get install libdevmapper-dev
>
> ./configure [ --prefix=/opt/parted-3.2/usr/local ]
> make
> make install
>
> The resulting parted binary recognizes Reiser4 partitions when invoked
> at your shell:
> $ parted
> GNU Parted 3.2
> Using /dev/xda
> Welcome to GNU Parted! Type 'help' to view a list of commands.
> (parted) print
> And it will print recognized partitions -- including reiser4, of course.
>
> Although it was developed and tested in a local machine,
> I ran PoC build on small Google Compute Engine (GCE) Cloud instance
> with a couple of existing Reiser4 partitions:
>
> Snapshot:
> https://pbs.twimg.com/media/CT8U78zUwAE-ZTc.png:large
>
> Here is GNU Parted 3.2 bz2-compressed src modified for Reiser4 (in
> case I no longer have time to improve it ;-)
> Source: 2.6M reiser4-parted-3.2m.tar.bz2 @
> http://metztli.it/readOnlyEphemeral/reiser4-parted-3.2m.tar.bz2
> SHA256SUM:
> http://metztli.it/readOnlyEphemeral/reiser4-parted-3.2m.tar.bz2.SHA256SUM
>
>

Just to thank all of you for your insightful input on my quest to
create a Debian-Installer (d-i) network ISO image --hacked to natively
partition, format, and install in/to Reiser4 regular and/or
flash-based media.

As mentioned before, I hacked GNU Parted 3.2 to provide Reiser4
*string probe* support only. Subsequently generated appropriate UDEBs:

libparted2-udeb_3.2-10_amd64.udeb
parted-udeb_3.2-10_amd64.udeb
libparted-fs-resize0-udeb_3.2-10_amd64.udeb

By including the above in d-i, I would not get the weird gibberish
when selecting to format root (/) partition in Reiser4 during expert
installation. Notwithstanding, I encountered subsequent error and
thus...

I modified partman-reiser4-1, (yes again) last evening -- since I was
getting something similar to: "debian-installer: Installation freezes
with "The attempt to mount a files system with type ext4 in SCSI1 (0,
0, 0), partition #1 (sda) at / failed." <
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=779922 > but, of
course, with reiser4 instead of ext4. Afterwards, I generated another
partman-reiser4_1_all.udeb

At this very moment the native Reiser4 installation is proceeding in a
VirtualBox instance. I did not have to do any unusual hacks to make
the Reiser4-based installation to proceed. Hence my quest seems over
-- as I attained my objective:

Here is an initial splash screenshot of the custom d-i installer:

< https://pbs.twimg.com/media/CU5exc0UcAAZmd4.png:large >

Needless to say, this is only a working instance and I will respect
the Debian logo by removing it from the image-generating template.

And here is the /etc/fstab that was created by the installer at the
/target/etc/fstab ephemeral installation location:

< https://pbs.twimg.com/media/CU5euw6VEAA8ZGz.png:large >

d-i partman component recognized and honored Reiser4! :D

This native Reiser4 Debian-Installer simplifies my tasks -- as in the
past I went through much pain creating a template for Google Cloud
Engine (GCE) image instances based on Reiser4.

Once again, I offer you all my sincere thanks.


Best Professional Regards.

P.S. Yes, I have to provide an alternate source and/or repository
where the Reiser4-patched Linux kernel image may be fetched during the
Debian installation but thats relatively easy.

Additionally, although I am not involved in Debian in any way --except
as a user, I would not mind contributing to Reiser4 maintenance if
mainstream Debian developers decide to include Reiser4 support in
their regular vanilla distribution. Yes, I will submit the
Reiser4-patch to GNU Parted, later on.

-- 
Jose R R
http://metztli.it
---------------------------------------------------------------------------------------------
Try at no charge http://b2evolution.net for http://OpenShift.com PaaS
---------------------------------------------------------------------------------------------
from our GitHub http://Nepohualtzintzin.com repository. Cloud the easy way!
---------------------------------------------------------------------------------------------