[PATCH] Split of the normal mode

[PATCH] Split of the normal mode

[Bean]

[-- Attachment #1: Type: text/plain, Size: 1346 bytes --]

Hi,

This patch split the function of normal mode into small modules, here
is a summary:

1, Move dynamic command loader to commands/dyncmd.c (dyncmd.mod)
2, Move automatic fs loader to fs/autofs.c (autofs.mod)
3, Split normal mode into three major parts:
parser/normal - normal mode parser (nparser.mod)
reader/normal - normal mode command line reader (nreader.mod)
menu/text - menu viewer (tmenu.mod)
4, Generic menu function move to menu/menu.c and menu/menu_viewer.c. (menu.mod)
5, Misc function move to lib/misc.c lib/completion.c (misc.mod)
6, setjmp function move to lib/i386, lib/x86_64, lib/powerpc and
lib/sparc64 (setjmp.mod)

Three new handler class:
parser
reader
menu_viewer

The kernel also split into parser and reader code.

grub-mkimage now support boot config file, which would be embed in
core.img. At startup, the kernel parser would read the config file and
execute the commands. Here is a sample boot config file that would
bring grub2 to an environment similar to the previous normal mode:

insmod dyncmd
insmod autofs
insmod nparser
handler parser normal
insmod nreader
handler reader normal
insmod tmenu
configfile /boot/grub/grub.cfg

The kernel parser is not very smart, so only use unix eol ('\n'), and
don't leave extra space.

grub-mkimage -c boot.cfg ...

This patch should apply cleanly with svn r2046.

-- 
Bean

[-- Attachment #2: split.zip --]
[-- Type: application/zip, Size: 39687 bytes --]

Re: [PATCH] Split of the normal mode

[Vesa Jääskeläinen]

Bean wrote:
> Hi,
> 
> This patch split the function of normal mode into small modules, here
> is a summary:
> 
> 1, Move dynamic command loader to commands/dyncmd.c (dyncmd.mod)
> 2, Move automatic fs loader to fs/autofs.c (autofs.mod)
> 3, Split normal mode into three major parts:
> parser/normal - normal mode parser (nparser.mod)
> reader/normal - normal mode command line reader (nreader.mod)
> menu/text - menu viewer (tmenu.mod)
> 4, Generic menu function move to menu/menu.c and menu/menu_viewer.c. (menu.mod)
> 5, Misc function move to lib/misc.c lib/completion.c (misc.mod)
> 6, setjmp function move to lib/i386, lib/x86_64, lib/powerpc and
> lib/sparc64 (setjmp.mod)

Umh... I could somewhat agree with moving text menu out of normal.mod.
But...

I feel we have passed the over engineering milestone sometime ago... and
 I am not frankly happy about that...

Changes like this just make it a) harder to maintain, b) bigger overall
size impact, c) as a plus more modular approach...

I also fear that we seem to be missing some lead for the project that
could use his authority to make decisions...

Re: [PATCH] Split of the normal mode

[Yoshinori K. Okuji]

On Sunday 29 March 2009 18:40:16 Vesa Jääskeläinen wrote:
> Bean wrote:
> > Hi,
> >
> > This patch split the function of normal mode into small modules, here
> > is a summary:
> >
> > 1, Move dynamic command loader to commands/dyncmd.c (dyncmd.mod)
> > 2, Move automatic fs loader to fs/autofs.c (autofs.mod)
> > 3, Split normal mode into three major parts:
> > parser/normal - normal mode parser (nparser.mod)
> > reader/normal - normal mode command line reader (nreader.mod)
> > menu/text - menu viewer (tmenu.mod)
> > 4, Generic menu function move to menu/menu.c and menu/menu_viewer.c.
> > (menu.mod) 5, Misc function move to lib/misc.c lib/completion.c
> > (misc.mod)
> > 6, setjmp function move to lib/i386, lib/x86_64, lib/powerpc and
> > lib/sparc64 (setjmp.mod)
>
> Umh... I could somewhat agree with moving text menu out of normal.mod.
> But...
>
> I feel we have passed the over engineering milestone sometime ago... and
>  I am not frankly happy about that...
>
> Changes like this just make it a) harder to maintain, b) bigger overall
> size impact, c) as a plus more modular approach...
>
> I also fear that we seem to be missing some lead for the project that
> could use his authority to make decisions...

Indeed. I don't understand this tendency about splitting modules at all. What 
is the motivation behind? What is the real benefit for the user?

From my point of view, it is wrong to force the user to manually load modules, 
generally. This includes writing "insmod" in config files. Look at Linux. It 
is quite rare to execute insmod or modprobe directly. Most of the time, 
modules are loaded on demand. This is the user-friendliness.

You should select compile-time "loading" (i.e. linking) whenever possible. If 
a function is loaded eventually, it should be linked at compile time.

You should select automatic loading, if you need runtime linking.

Manual loading should be considered evil, in principle.

FYI, I am planning to make a full review of recent changes (i.e. all changes 
made after I stopped reviewing every patch), and trash/revert/rewrite/blame 
everything I don't like. Changes made for no good reason must be all 
reverted.

In brief, I take back the leadership of this project for general directions. 
For some subsystems (e.g. the coreboot support), I continue leaving the 
responsibility to those who know better or are more active. Once the current 
code is reviewed and fixed (at some degree), I will make a new release.

Any objection?

Regards,
Okuji

Re: [PATCH] Split of the normal mode

[phcoder]

I'm actually quite unhappy with the grub's authority in general. Some 
people can commit their patches after a week of no replies while others 
like me have to wait that someone has time to review their patches in 
depth. I already have a collection of patches that are not commited, not 
because someone objects against them but just because nobody qualified 
enough has time to review and commit it. All this despite having already 
signed copyright assignment which because of slowness of FSF took more 
time than it should. From developer viewpoint it's very frustrating 
experience. If I wasn't so motivated as I am I would have already given 
up. IMO opinion if we want people coding for grub2 this has to be 
changed. But now you come and say you want to revert some patches just 
because they seem useless to you or rewrite some code just because you 
find it ugly or because it has a minor bug which could be easily fixed 
without rewriting. IMO this can easily drive developers to fork or leave 
project altogether. And additionally your energy would be much better 
spent in writing new stuff and making/reviewing design propositions than 
rewriting chunks of already working code.
Sorry for being somewhat rude but I really find it frustrating this 
coder-unfriendliness
> In brief, I take back the leadership of this project for general directions. 
> For some subsystems (e.g. the coreboot support), I continue leaving the 
> responsibility to those who know better or are more active. Once the current 
> code is reviewed and fixed (at some degree), I will make a new release.
> 
> Any objection?
> 
> Regards,
> Okuji


-- 

Regards
Vladimir 'phcoder' Serbinenko

Re: [PATCH] Split of the normal mode

[Vesa Jääskeläinen]

Here I must comment that in some cases it is favorable to do it this
way. I personally will only commit changes that I am happy to live with.
Even if you are developer with SVN commit rights it is in some cases
good idea to have general review of the patch... but there has been some
lack of the time from people to review them (and it does not cross
everyones interest zones, thus get attention to the patch).

phcoder wrote:
> I'm actually quite unhappy with the grub's authority in general. Some
> people can commit their patches after a week of no replies while others
> like me have to wait that someone has time to review their patches in
> depth. I already have a collection of patches that are not commited, not
> because someone objects against them but just because nobody qualified
> enough has time to review and commit it. All this despite having already
> signed copyright assignment which because of slowness of FSF took more
> time than it should. From developer viewpoint it's very frustrating
> experience. If I wasn't so motivated as I am I would have already given
> up. IMO opinion if we want people coding for grub2 this has to be
> changed. But now you come and say you want to revert some patches just
> because they seem useless to you or rewrite some code just because you
> find it ugly or because it has a minor bug which could be easily fixed
> without rewriting. IMO this can easily drive developers to fork or leave
> project altogether. And additionally your energy would be much better
> spent in writing new stuff and making/reviewing design propositions than
> rewriting chunks of already working code.
> Sorry for being somewhat rude but I really find it frustrating this
> coder-unfriendliness
>> In brief, I take back the leadership of this project for general
>> directions. For some subsystems (e.g. the coreboot support), I
>> continue leaving the responsibility to those who know better or are
>> more active. Once the current code is reviewed and fixed (at some
>> degree), I will make a new release.
>>
>> Any objection?
>>
>> Regards,
>> Okuji
> 
> 

Re: [PATCH] Split of the normal mode

[phcoder]

I agree when it's about big changes. I wouldalso not commit anything 
what isn't quite tested or may break something. But when one-line 
bugfixes stay uncommited during 2-3 weaks it's something that isn't 
quite ok. Also if noone comments a relatively small patch in 1-2 weeks I 
think we should consider that everybody is happy with the patch and it 
should be commited. However somehow on this project it's not the same 
rules for everyone.
Vesa Jääskeläinen wrote:
> Here I must comment that in some cases it is favorable to do it this
> way. I personally will only commit changes that I am happy to live with.
> Even if you are developer with SVN commit rights it is in some cases
> good idea to have general review of the patch... but there has been some
> lack of the time from people to review them (and it does not cross
> everyones interest zones, thus get attention to the patch).
> 
> phcoder wrote:
>> I'm actually quite unhappy with the grub's authority in general. Some
>> people can commit their patches after a week of no replies while others
>> like me have to wait that someone has time to review their patches in
>> depth. I already have a collection of patches that are not commited, not
>> because someone objects against them but just because nobody qualified
>> enough has time to review and commit it. All this despite having already
>> signed copyright assignment which because of slowness of FSF took more
>> time than it should. From developer viewpoint it's very frustrating
>> experience. If I wasn't so motivated as I am I would have already given
>> up. IMO opinion if we want people coding for grub2 this has to be
>> changed. But now you come and say you want to revert some patches just
>> because they seem useless to you or rewrite some code just because you
>> find it ugly or because it has a minor bug which could be easily fixed
>> without rewriting. IMO this can easily drive developers to fork or leave
>> project altogether. And additionally your energy would be much better
>> spent in writing new stuff and making/reviewing design propositions than
>> rewriting chunks of already working code.
>> Sorry for being somewhat rude but I really find it frustrating this
>> coder-unfriendliness
>>> In brief, I take back the leadership of this project for general
>>> directions. For some subsystems (e.g. the coreboot support), I
>>> continue leaving the responsibility to those who know better or are
>>> more active. Once the current code is reviewed and fixed (at some
>>> degree), I will make a new release.
>>>
>>> Any objection?
>>>
>>> Regards,
>>> Okuji
>>
> 
> 
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel


-- 

Regards
Vladimir 'phcoder' Serbinenko

Re: [PATCH] Split of the normal mode

[Yoshinori K. Okuji]

On Sunday 29 March 2009 20:33:48 phcoder wrote:
> I agree when it's about big changes. I wouldalso not commit anything
> what isn't quite tested or may break something. But when one-line
> bugfixes stay uncommited during 2-3 weaks it's something that isn't
> quite ok. Also if noone comments a relatively small patch in 1-2 weeks I
> think we should consider that everybody is happy with the patch and it
> should be commited. However somehow on this project it's not the same
> rules for everyone.

You are right.

For very small, trivial patches, I will fix the problem by giving the write 
permission to you.

For complex or big patches, I recommend that you use the bug tracker on 
Savannah. I know that not many people use it, but putting patches on the 
mailing list is not quite convenient when we have a huge backlog. The 
discussion can be still done in the mailing list, but I need an easy way to 
glance at a list of all pending patches.

Is this good enough for you?

Regards,
Okuji

Re: [PATCH] Split of the normal mode

[Bean]

Hi,

Some of my consideration about splitting of normal mode.

1, In some environment, we have size limit of the boot image.
Normal.mod is too big, and rescue mode has too little functionality.
Using the split method, we could combine modules in anyway we wanted.

2, Speaking of linux, it's actually doing the same thing. The kernel
is in vmlinuz, while the initialization script is in initrd.img. We
don't want the user to enter those commands manually, a default
boot.cfg should be used by grub-mkimage.

3. Currently, the structure of normal.mod just mix things together to
a point that make modification difficult, if not impossible. For
example, the current bash script engine is not quite suitable for gui
interaction. With the split mode, we could develop a new parser
without interfere with existing function.

-- 
Bean

Re: [PATCH] Split of the normal mode

[Yoshinori K. Okuji]

On Sunday 29 March 2009 21:09:05 Bean wrote:
> Hi,
>
> Some of my consideration about splitting of normal mode.
>
> 1, In some environment, we have size limit of the boot image.
> Normal.mod is too big, and rescue mode has too little functionality.
> Using the split method, we could combine modules in anyway we wanted.

In my opinion, you are postponing the decision to the runtime too much. If you 
have N modules, you have N! combinations, but we don't need to support that 
many in reality. I bet that you know which portions of the normal mode you 
would like to select for your own need. Surely, others might have different 
needs, but the number of modules you added looks overkill to me.

> 2, Speaking of linux, it's actually doing the same thing. The kernel
> is in vmlinuz, while the initialization script is in initrd.img. We
> don't want the user to enter those commands manually, a default
> boot.cfg should be used by grub-mkimage.

Mmh, I hardly agree on this. The purpose of initrd.img is, although it could 
be abused, to bootstrap an OS environment for further work, which is 
analogous to core.img in GRUB. For the rest of the work, ifplugd, udev, etc. 
take care of loading appropriate modules on demand.

> 3. Currently, the structure of normal.mod just mix things together to
> a point that make modification difficult, if not impossible. For
> example, the current bash script engine is not quite suitable for gui
> interaction. With the split mode, we could develop a new parser
> without interfere with existing function.

I prefer that you replace the existing code with a better implementation in 
this case. From my point of view, fancy menu support is a key feature in 
GRUB, thus if the current engine is not good enough, we need to improve it 
rather than to provide an alternative.

Regards,
Okuji

Re: [PATCH] Split of the normal mode

[Bean]

On Sun, Mar 29, 2009 at 9:10 PM, Yoshinori K. Okuji <okuji@enbug.org> wrote:
> On Sunday 29 March 2009 21:09:05 Bean wrote:
>> Hi,
>>
>> Some of my consideration about splitting of normal mode.
>>
>> 1, In some environment, we have size limit of the boot image.
>> Normal.mod is too big, and rescue mode has too little functionality.
>> Using the split method, we could combine modules in anyway we wanted.
>
> In my opinion, you are postponing the decision to the runtime too much. If you
> have N modules, you have N! combinations, but we don't need to support that
> many in reality. I bet that you know which portions of the normal mode you
> would like to select for your own need. Surely, others might have different
> needs, but the number of modules you added looks overkill to me.

Actually, there is still only one normal configuration. But others
could be useful in some situation. We could hide the details from
normal user, but they can do such configuration if required.

>
>> 2, Speaking of linux, it's actually doing the same thing. The kernel
>> is in vmlinuz, while the initialization script is in initrd.img. We
>> don't want the user to enter those commands manually, a default
>> boot.cfg should be used by grub-mkimage.
>
> Mmh, I hardly agree on this. The purpose of initrd.img is, although it could
> be abused, to bootstrap an OS environment for further work, which is
> analogous to core.img in GRUB. For the rest of the work, ifplugd, udev, etc.
> take care of loading appropriate modules on demand.
>

Sometime we need to setup the environment before it can access boot
media. For example, locating the boot partition using label/uuid,
setting the network address, etc, this is where boot.cfg can be quite
useful.

>> 3. Currently, the structure of normal.mod just mix things together to
>> a point that make modification difficult, if not impossible. For
>> example, the current bash script engine is not quite suitable for gui
>> interaction. With the split mode, we could develop a new parser
>> without interfere with existing function.
>
> I prefer that you replace the existing code with a better implementation in
> this case. From my point of view, fancy menu support is a key feature in
> GRUB, thus if the current engine is not good enough, we need to improve it
> rather than to provide an alternative.

If I were to fix this problem, I'd separate parser and reader code
anyway. In fact, colin already separate the menu viewer code. The
problem is to still keep them in a single normal.mod, or to move them
to logical independent modules. I see no problem with the later.

-- 
Bean

Re: [PATCH] Split of the normal mode

[Yoshinori K. Okuji]

On Sunday 29 March 2009 22:59:53 Bean wrote:
> On Sun, Mar 29, 2009 at 9:10 PM, Yoshinori K. Okuji <okuji@enbug.org> wrote:
> > On Sunday 29 March 2009 21:09:05 Bean wrote:
> >> Hi,
> >>
> >> Some of my consideration about splitting of normal mode.
> >>
> >> 1, In some environment, we have size limit of the boot image.
> >> Normal.mod is too big, and rescue mode has too little functionality.
> >> Using the split method, we could combine modules in anyway we wanted.
> >
> > In my opinion, you are postponing the decision to the runtime too much.
> > If you have N modules, you have N! combinations, but we don't need to
> > support that many in reality. I bet that you know which portions of the
> > normal mode you would like to select for your own need. Surely, others
> > might have different needs, but the number of modules you added looks
> > overkill to me.
>
> Actually, there is still only one normal configuration. But others
> could be useful in some situation. We could hide the details from
> normal user, but they can do such configuration if required.

My basic attitude is "making it only if it is used in pracice". If your goal 
is virtual, I don't want to accept it.

>
> >> 2, Speaking of linux, it's actually doing the same thing. The kernel
> >> is in vmlinuz, while the initialization script is in initrd.img. We
> >> don't want the user to enter those commands manually, a default
> >> boot.cfg should be used by grub-mkimage.
> >
> > Mmh, I hardly agree on this. The purpose of initrd.img is, although it
> > could be abused, to bootstrap an OS environment for further work, which
> > is analogous to core.img in GRUB. For the rest of the work, ifplugd,
> > udev, etc. take care of loading appropriate modules on demand.
>
> Sometime we need to setup the environment before it can access boot
> media. For example, locating the boot partition using label/uuid,
> setting the network address, etc, this is where boot.cfg can be quite
> useful.

If you use a label, label support should be loaded automatically.
If you use an uuid, uuid support should be loaded automatically.
If you set up a network, network support should be loaded automatically.

So I see no reason to stop automatic loading.

> >> 3. Currently, the structure of normal.mod just mix things together to
> >> a point that make modification difficult, if not impossible. For
> >> example, the current bash script engine is not quite suitable for gui
> >> interaction. With the split mode, we could develop a new parser
> >> without interfere with existing function.
> >
> > I prefer that you replace the existing code with a better implementation
> > in this case. From my point of view, fancy menu support is a key feature
> > in GRUB, thus if the current engine is not good enough, we need to
> > improve it rather than to provide an alternative.
>
> If I were to fix this problem, I'd separate parser and reader code
> anyway. In fact, colin already separate the menu viewer code. The
> problem is to still keep them in a single normal.mod, or to move them
> to logical independent modules. I see no problem with the later.

Views can be separete. I don't deny this. But the scripting engine itself 
should stay in normal mode. What is wrong with this?

Regards,
Okuji

Re: [PATCH] Split of the normal mode

[Bean]

On Sun, Mar 29, 2009 at 10:20 PM, Yoshinori K. Okuji <okuji@enbug.org> wrote:
>> >> 2, Speaking of linux, it's actually doing the same thing. The kernel
>> >> is in vmlinuz, while the initialization script is in initrd.img. We
>> >> don't want the user to enter those commands manually, a default
>> >> boot.cfg should be used by grub-mkimage.
>> >
>> > Mmh, I hardly agree on this. The purpose of initrd.img is, although it
>> > could be abused, to bootstrap an OS environment for further work, which
>> > is analogous to core.img in GRUB. For the rest of the work, ifplugd,
>> > udev, etc. take care of loading appropriate modules on demand.
>>
>> Sometime we need to setup the environment before it can access boot
>> media. For example, locating the boot partition using label/uuid,
>> setting the network address, etc, this is where boot.cfg can be quite
>> useful.
>
> If you use a label, label support should be loaded automatically.
> If you use an uuid, uuid support should be loaded automatically.
> If you set up a network, network support should be loaded automatically.
>
> So I see no reason to stop automatic loading.

But how to store the parameters ? Boot media is not available, so we
can't read file, and we can't embed them in code.

>
>> >> 3. Currently, the structure of normal.mod just mix things together to
>> >> a point that make modification difficult, if not impossible. For
>> >> example, the current bash script engine is not quite suitable for gui
>> >> interaction. With the split mode, we could develop a new parser
>> >> without interfere with existing function.
>> >
>> > I prefer that you replace the existing code with a better implementation
>> > in this case. From my point of view, fancy menu support is a key feature
>> > in GRUB, thus if the current engine is not good enough, we need to
>> > improve it rather than to provide an alternative.
>>
>> If I were to fix this problem, I'd separate parser and reader code
>> anyway. In fact, colin already separate the menu viewer code. The
>> problem is to still keep them in a single normal.mod, or to move them
>> to logical independent modules. I see no problem with the later.
>
> Views can be separete. I don't deny this. But the scripting engine itself
> should stay in normal mode. What is wrong with this?

The script engine is quite large, if we put them all in normal.mod,
it'd be messy.

-- 
Bean

Re: [PATCH] Split of the normal mode

[Yoshinori K. Okuji]

On Sunday 29 March 2009 23:30:48 Bean wrote:
> > If you use a label, label support should be loaded automatically.
> > If you use an uuid, uuid support should be loaded automatically.
> > If you set up a network, network support should be loaded automatically.
> >
> > So I see no reason to stop automatic loading.
>
> But how to store the parameters ? Boot media is not available, so we
> can't read file, and we can't embed them in code.

Could you be more specific? What case are you talking about?

> >> >> 3. Currently, the structure of normal.mod just mix things together to
> >> >> a point that make modification difficult, if not impossible. For
> >> >> example, the current bash script engine is not quite suitable for gui
> >> >> interaction. With the split mode, we could develop a new parser
> >> >> without interfere with existing function.
> >> >
> >> > I prefer that you replace the existing code with a better
> >> > implementation in this case. From my point of view, fancy menu support
> >> > is a key feature in GRUB, thus if the current engine is not good
> >> > enough, we need to improve it rather than to provide an alternative.
> >>
> >> If I were to fix this problem, I'd separate parser and reader code
> >> anyway. In fact, colin already separate the menu viewer code. The
> >> problem is to still keep them in a single normal.mod, or to move them
> >> to logical independent modules. I see no problem with the later.
> >
> > Views can be separete. I don't deny this. But the scripting engine itself
> > should stay in normal mode. What is wrong with this?
>
> The script engine is quite large, if we put them all in normal.mod,
> it'd be messy.

Do you want to use GRUB without the scripting engine?

Regards,
Okuji

Re: [PATCH] Split of the normal mode

[Bean]

On Sun, Mar 29, 2009 at 10:54 PM, Yoshinori K. Okuji <okuji@enbug.org> wrote:
> On Sunday 29 March 2009 23:30:48 Bean wrote:
>> > If you use a label, label support should be loaded automatically.
>> > If you use an uuid, uuid support should be loaded automatically.
>> > If you set up a network, network support should be loaded automatically.
>> >
>> > So I see no reason to stop automatic loading.
>>
>> But how to store the parameters ? Boot media is not available, so we
>> can't read file, and we can't embed them in code.
>
> Could you be more specific? What case are you talking about?
>

For example, something the boot modules can't detect boot partition
properly, such as some raid/lvm, etc. We might want to use uuid or
label to identify the partition. Then, there is a problem, how to
store the uuid value for the boot modules to use.

Previously, I have thought of embedding a envblk in the kernel, then
modules can read variables at startup. But this method is quite
clumsy, and it needs to reserve valuable kernel size. Using a boot
script seems much flexible to me. Besides, it doesn't require
additional kernel space. The kernel can already parse commands in
rescue shell, I just reuse this function to run the boot script.

>> >> >> 3. Currently, the structure of normal.mod just mix things together to
>> >> >> a point that make modification difficult, if not impossible. For
>> >> >> example, the current bash script engine is not quite suitable for gui
>> >> >> interaction. With the split mode, we could develop a new parser
>> >> >> without interfere with existing function.
>> >> >
>> >> > I prefer that you replace the existing code with a better
>> >> > implementation in this case. From my point of view, fancy menu support
>> >> > is a key feature in GRUB, thus if the current engine is not good
>> >> > enough, we need to improve it rather than to provide an alternative.
>> >>
>> >> If I were to fix this problem, I'd separate parser and reader code
>> >> anyway. In fact, colin already separate the menu viewer code. The
>> >> problem is to still keep them in a single normal.mod, or to move them
>> >> to logical independent modules. I see no problem with the later.
>> >
>> > Views can be separete. I don't deny this. But the scripting engine itself
>> > should stay in normal mode. What is wrong with this?
>>
>> The script engine is quite large, if we put them all in normal.mod,
>> it'd be messy.
>
> Do you want to use GRUB without the scripting engine?

The point is to support alternative script engine. Actually, I'm
thinking about integrating clisp into grub2. But some people may not
know about it, so they can still use the old bash engine.

-- 
Bean

Re: [PATCH] Split of the normal mode

[Yoshinori K. Okuji]

On Monday 30 March 2009 00:17:59 Bean wrote:
> >> But how to store the parameters ? Boot media is not available, so we
> >> can't read file, and we can't embed them in code.
> >
> > Could you be more specific? What case are you talking about?
>
> For example, something the boot modules can't detect boot partition
> properly, such as some raid/lvm, etc. We might want to use uuid or
> label to identify the partition. Then, there is a problem, how to
> store the uuid value for the boot modules to use.
>
> Previously, I have thought of embedding a envblk in the kernel, then
> modules can read variables at startup. But this method is quite
> clumsy, and it needs to reserve valuable kernel size. Using a boot
> script seems much flexible to me. Besides, it doesn't require
> additional kernel space. The kernel can already parse commands in
> rescue shell, I just reuse this function to run the boot script.

I understand. Actually, GRUB Legacy has the same feature, named "preset menu", 
with which you can embed a config file in stage2. This feature is not used 
very much, because it requires recompilation in GRUB Legacy.

> > Do you want to use GRUB without the scripting engine?
>
> The point is to support alternative script engine. Actually, I'm
> thinking about integrating clisp into grub2. But some people may not
> know about it, so they can still use the old bash engine.

Sounds nice. :)

But I still think that your patch splits into too many. If you split normal 
mode into:

- a script module (sh.mod?)
- a text menu module (menu.mod?)
- the rest (normal.mod)

then, that's fine.

Regards,
Okuji

Re: [PATCH] Split of the normal mode

[Colin D Bennett]

[-- Attachment #1: Type: text/plain, Size: 1784 bytes --]

On Sun, 29 Mar 2009 22:10:34 +0900
"Yoshinori K. Okuji" <okuji@enbug.org> wrote:

> On Sunday 29 March 2009 21:09:05 Bean wrote:
> > 3. Currently, the structure of normal.mod just mix things together to
> > a point that make modification difficult, if not impossible. For
> > example, the current bash script engine is not quite suitable for gui
> > interaction. With the split mode, we could develop a new parser
> > without interfere with existing function.
> 
> I prefer that you replace the existing code with a better implementation in 
> this case. From my point of view, fancy menu support is a key feature in 
> GRUB, thus if the current engine is not good enough, we need to improve it 
> rather than to provide an alternative.

About the fancy menu support, I hope to pursue getting it finally
committed soon; I just need to synchronize my graphical menu patch set
with the current GRUB codebase.  I have just been extremely busy until
now, but I expect to get this done in the near future.


The current GRUB scripting engine is OK for basic configuration, and no
other scripting support is really required for the fancy menu.

However, I still like the idea of additionally supporting a tiny but
powerful general-purpose scripting language such as Lua (~100 KB), 
Lisp/Scheme (? KB), or ECMAScript (? KB) to allow graphical menu
enhancements by themes.

For instance, in only a screenful of Lua code (and me never having
written Lua before) I created an animated background for the menu.
Certainly there are many other purposes as well that a general purpose
scripting language could support.

Perhaps later, after the graphical menu support is integrated into
GRUB, we can explore adding scripting language support.

Regards,
Colin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [PATCH] Split of the normal mode

[Bean]

On Tue, Mar 31, 2009 at 4:04 AM, Colin D Bennett <colin@gibibit.com> wrote:
> About the fancy menu support, I hope to pursue getting it finally
> committed soon; I just need to synchronize my graphical menu patch set
> with the current GRUB codebase.  I have just been extremely busy until
> now, but I expect to get this done in the near future.

Oh, that would be nice.

> The current GRUB scripting engine is OK for basic configuration, and no
> other scripting support is really required for the fancy menu.
>
> However, I still like the idea of additionally supporting a tiny but
> powerful general-purpose scripting language such as Lua (~100 KB),
> Lisp/Scheme (? KB), or ECMAScript (? KB) to allow graphical menu
> enhancements by themes.
>
> For instance, in only a screenful of Lua code (and me never having
> written Lua before) I created an animated background for the menu.
> Certainly there are many other purposes as well that a general purpose
> scripting language could support.
>
> Perhaps later, after the graphical menu support is integrated into
> GRUB, we can explore adding scripting language support.

You're right, LUA is compact yet still powerful, it's quite suitable
for our purpose. If it's fine with other developers, I can start
working on it. Oh btw, is the license compatible ?

-- 
Bean

Re: [PATCH] Split of the normal mode

[Colin D Bennett]

[-- Attachment #1: Type: text/plain, Size: 352 bytes --]

On Tue, 31 Mar 2009 14:11:28 +0800
Bean <bean123ch@gmail.com> wrote:

> You're right, LUA is compact yet still powerful, it's quite suitable
> for our purpose. If it's fine with other developers, I can start
> working on it. Oh btw, is the license compatible ?

Lua 5.1 is under the MIT X11 license, which is GPL-compatible.

Regards,
Colin

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: [PATCH] Split of the normal mode

[phcoder]

Yes, it's perfect
> For very small, trivial patches, I will fix the problem by giving the write 
> permission to you.
> 
> For complex or big patches, I recommend that you use the bug tracker on 
> Savannah. I know that not many people use it, but putting patches on the 
> mailing list is not quite convenient when we have a huge backlog. The 
> discussion can be still done in the mailing list, but I need an easy way to 
> glance at a list of all pending patches.
Savannah stuff is another point which needs to be fixed. E.g. we have 
todos both on savannah and on the wiki. Both are outdated
-- 

Regards
Vladimir 'phcoder' Serbinenko

Re: [PATCH] Split of the normal mode

[Yoshinori K. Okuji]

On Sunday 29 March 2009 22:00:05 phcoder wrote:
> Yes, it's perfect
>
> > For very small, trivial patches, I will fix the problem by giving the
> > write permission to you.
> >
> > For complex or big patches, I recommend that you use the bug tracker on
> > Savannah. I know that not many people use it, but putting patches on the
> > mailing list is not quite convenient when we have a huge backlog. The
> > discussion can be still done in the mailing list, but I need an easy way
> > to glance at a list of all pending patches.
>
> Savannah stuff is another point which needs to be fixed. E.g. we have
> todos both on savannah and on the wiki. Both are outdated

I wish your help on this.

Regards,
Okuji

Re: [PATCH] Split of the normal mode

[phcoder]

Yoshinori K. Okuji wrote:
  > I wish your help on this.
Ok. You have it. Just at the moment I'm quite busybut in 2 weeks I'll be 
more available
-- 

Regards
Vladimir 'phcoder' Serbinenko

Re: [PATCH] Split of the normal mode

[phcoder]

I moved todo to the svannah task track.
3 pieces aren't moved yet
-PPC / Ultrasparc
-Command list
-Feature requests by users
phcoder wrote:
> 
> Yoshinori K. Okuji wrote:
>  > I wish your help on this.
> Ok. You have it. Just at the moment I'm quite busybut in 2 weeks I'll be 
> more available


-- 

Regards
Vladimir 'phcoder' Serbinenko

Re: [PATCH] Split of the normal mode

[Yoshinori K. Okuji]

On Monday 30 March 2009 05:35:08 phcoder wrote:
> I moved todo to the svannah task track.
> 3 pieces aren't moved yet
> -PPC / Ultrasparc
> -Command list
> -Feature requests by users

Isn't it better to remove the tasks which you moved to Savannah from the wiki?

Regards,
Okuji

Re: [PATCH] Split of the normal mode

[phcoder]

Once I finish (not a lot remaining) I'll just put a notice in the head 
that this page is preserved for historical purposes only and indications 
to use savannah
Yoshinori K. Okuji wrote:
> On Monday 30 March 2009 05:35:08 phcoder wrote:
>> I moved todo to the svannah task track.
>> 3 pieces aren't moved yet
>> -PPC / Ultrasparc
>> -Command list
>> -Feature requests by users
> 
> Isn't it better to remove the tasks which you moved to Savannah from the wiki?
> 
> Regards,
> Okuji
> 
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel


-- 

Regards
Vladimir 'phcoder' Serbinenko

Re: [PATCH] Split of the normal mode

[phcoder]

Move from todo on wiki is done except UltraSparc.
David Miller: these todos seem to be very outdated by your work. If some 
  of them or new tasks are applicable could you add them to task tracker?
# Install on disk
# Grub emu
# Boot something
# Write a grub-install script to install grub and configure OF variables 
properly. This will be different from PPC because Sparc has openpromfs 
and is missing sysfs "devspec" links into the device tree.
phcoder wrote:
> Once I finish (not a lot remaining) I'll just put a notice in the head 
> that this page is preserved for historical purposes only and indications 
> to use savannah
> Yoshinori K. Okuji wrote:
>> On Monday 30 March 2009 05:35:08 phcoder wrote:
>>> I moved todo to the svannah task track.
>>> 3 pieces aren't moved yet
>>> -PPC / Ultrasparc
>>> -Command list
>>> -Feature requests by users
>>
>> Isn't it better to remove the tasks which you moved to Savannah from 
>> the wiki?
>>
>> Regards,
>> Okuji
>>
>>
>> _______________________________________________
>> Grub-devel mailing list
>> Grub-devel@gnu.org
>> http://lists.gnu.org/mailman/listinfo/grub-devel
> 
> 


-- 

Regards
Vladimir 'phcoder' Serbinenko

Re: [PATCH] Split of the normal mode

[David Miller]

From: phcoder <phcoder@gmail.com>
Date: Wed, 01 Apr 2009 09:43:01 +0200

> Move from todo on wiki is done except UltraSparc.
> David Miller: these todos seem to be very outdated by your work. If some of them or new tasks are applicable could you add them to task tracker?
> # Install on disk
> # Grub emu
> # Boot something
> # Write a grub-install script to install grub and configure OF variables properly. This will be different from PPC because Sparc has openpromfs and is missing sysfs "devspec" links into the device tree.

This should all just be deleted.  It's either going to be done by
my patches or is irrelevant.

Re: [PATCH] Split of the normal mode

[phcoder]

David Miller wrote:
> From: phcoder <phcoder@gmail.com>
> Date: Wed, 01 Apr 2009 09:43:01 +0200
> 
>> Move from todo on wiki is done except UltraSparc.
>> David Miller: these todos seem to be very outdated by your work. If some of them or new tasks are applicable could you add them to task tracker?
>> # Install on disk
>> # Grub emu
>> # Boot something
>> # Write a grub-install script to install grub and configure OF variables properly. This will be different from PPC because Sparc has openpromfs and is missing sysfs "devspec" links into the device tree.
> 
> This should all just be deleted.  It's either going to be done by
> my patches or is irrelevant.
ok, thanks. Now everything is in savannah task tracker. As for command 
list I added important missing commands to task tracker and skipped 
not-so-useful ones. As there is some subjectivity involved if you don't 
agree with my choice just say it. Similar situation is with user's 
feature requests. I moved the ones that are out of scope of grub2 or 
already done to "Invalid requests" section. As for other I see none 
which are priority right now. If you disagree again just say.
Also I'm currently adding patches which are pending for more then one week

-- 

Regards
Vladimir 'phcoder' Serbinenko

Re: [PATCH] Split of the normal mode

[Yoshinori K. Okuji]

On Wednesday 01 April 2009 18:22:46 phcoder wrote:
> ok, thanks. Now everything is in savannah task tracker. As for command
> list I added important missing commands to task tracker and skipped
> not-so-useful ones. As there is some subjectivity involved if you don't
> agree with my choice just say it. Similar situation is with user's
> feature requests. I moved the ones that are out of scope of grub2 or
> already done to "Invalid requests" section. As for other I see none
> which are priority right now. If you disagree again just say.

Thank you so much. It's now so easy to look. :)

Personally, I think "Add more loaders" and "Add more filesystems" should be 
removed, because they are not tasks but directions. They could be written at 
somewhere, but I don't think it is appropriate to put them into the "task 
manager". From my point of view, tasks must be possible to finish.

Regards,
Okuji

Re: [PATCH] Split of the normal mode

[phcoder]

Yoshinori K. Okuji wrote:
> On Wednesday 01 April 2009 18:22:46 phcoder wrote:
>> ok, thanks. Now everything is in savannah task tracker. As for command
>> list I added important missing commands to task tracker and skipped
>> not-so-useful ones. As there is some subjectivity involved if you don't
>> agree with my choice just say it. Similar situation is with user's
>> feature requests. I moved the ones that are out of scope of grub2 or
>> already done to "Invalid requests" section. As for other I see none
>> which are priority right now. If you disagree again just say.
> 
> Thank you so much. It's now so easy to look. :)
You're welcome. Some pending patches are still not in the tracker. I 
added only the ones sent after 25.01.2009 but I'll add more as time 
permits. Also I omitted my own patches and I'll just ping to my threads 
on the list.
> 
> Personally, I think "Add more loaders" and "Add more filesystems" should be 
> removed, because they are not tasks but directions. They could be written at 
> somewhere, but I don't think it is appropriate to put them into the "task 
> manager". From my point of view, tasks must be possible to finish.
> 
I agree that these aren't appropriate for task manager. However I don't 
want to create a separate document for just two lines because this 
document will be either forgotten or misused. I always have this problem 
when making order on my computer or in my room. Some stuff is also left 
which passes to no box and then I have to choose which box to use
> Regards,
> Okuji
> 
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel


-- 

Regards
Vladimir 'phcoder' Serbinenko

Re: [PATCH] Split of the normal mode

[Yoshinori K. Okuji]

On Sunday 29 March 2009 19:43:33 phcoder wrote:
> I'm actually quite unhappy with the grub's authority in general. Some
> people can commit their patches after a week of no replies while others
> like me have to wait that someone has time to review their patches in
> depth. I already have a collection of patches that are not commited, not
> because someone objects against them but just because nobody qualified
> enough has time to review and commit it. All this despite having already
> signed copyright assignment which because of slowness of FSF took more
> time than it should. From developer viewpoint it's very frustrating
> experience. If I wasn't so motivated as I am I would have already given
> up. IMO opinion if we want people coding for grub2 this has to be
> changed. But now you come and say you want to revert some patches just
> because they seem useless to you or rewrite some code just because you
> find it ugly or because it has a minor bug which could be easily fixed
> without rewriting. IMO this can easily drive developers to fork or leave
> project altogether. And additionally your energy would be much better
> spent in writing new stuff and making/reviewing design propositions than
> rewriting chunks of already working code.
> Sorry for being somewhat rude but I really find it frustrating this
> coder-unfriendliness

No problem. I am very glad to hear your honest opinion.

Since you are relatively new to this project, I would like to describe how it 
*was" when I or Marco was very active:

- Trivial changes, in particular bug fixes, were allowed to be checked in 
without any review, if the developer is trusted. Generally, I tend to trust 
all people who have the write permission. Just like in real life, trust may 
not be built instantly; it must be harvested gradually by showing what you do 
and what you don't do.

- Rather significant changes, even bug fixes when these require code 
restructuring, had to be reviewed. I think I was the only exception on this, 
because I am the designor. In reality, however, even I often posted messages 
to this mailing list before I made changes, because I appreciated others' 
ideas.

- Design-level changes had to be always discussed a lot before being accepted. 
This included myself, because even I, the original author, didn't know every 
aspect of impacts.

These rules were adopted, partly because GRUB was very much in wide use, so we 
should not break our software, including compatibility issues. In particular, 
user-visible changes must be treated very carefully, because we may not 
simply get rid of them, once they are incorporated, otherwise we will make 
millions of machines in the world unbootable.

Afterwards, these rules were somehow forgotten for a practical reason: patches 
were not reviewed quickly or even ignored for a long time, because of the 
absense of leadership. I expected that we could overcome such a situation by 
co-maintainership, but after both I and Marco got too busy with other things, 
it stopped working. As you should know, several people, like Robert, Vesa, 
Pavel, and Bean, helped greatly, but it was not still good enough for your 
demand.

So the current situation is like this:

- Many changes have already been incorporated without proper reviews, 
including bad changes. The current GRUB is in a sense partly worse than 
before, due to this.

- Many patches are pending.

- Contributors work very randomly, because nobody shows what we should do and 
what we should not.

So if nobody else can, I would like to get it straight again myself, although 
I am pretty busy (as I have a startup company, can you imagine how tough it 
is?).

So, the first thing I would like to do is to remind people of the check-in 
rules, and apply them to past changes. Since a new version is not released 
yet (after things got bad), we can still fix up the bad pieces safely. If we 
miss this occasion, we will have to struggle with badly implemented features 
for years due to compatibility. I have experienced this enough with GRUB 
Legacy. I don't want it again. Otherwise, I wouldn't have made GRUB 2.

Regards,
Okuji

Re: [PATCH] Split of the normal mode

[David Miller]

From: "Yoshinori K. Okuji" <okuji@enbug.org>
Date: Sun, 29 Mar 2009 20:29:26 +0900

> So if nobody else can, I would like to get it straight again myself,
> although I am pretty busy (as I have a startup company, can you
> imagine how tough it is?).

Everybody is too busy to give this project the attention and time it
deserves to be maintained properly.

I honestly do not think the situation will change significantly until
someone is able to devote real time as a maintainer and process all of
the patches that get submitted each day.

As top level maintainter that has to be your number one priority,
processing patches efficiently.  Otherwise developers go elsewhere and
you will eventually be working on all the features and bug fixes
yourself.

If you, on the other hand, devote the necessary time to patches
then developers will be motivated.  They will do all the work and
you can take all the credit :-)

But I do appreicate that you understand the issue of trust.  It is
the only way to operate sanely.

Re: [PATCH] Split of the normal mode

[Yoshinori K. Okuji]

On Sunday 29 March 2009 20:40:17 David Miller wrote:
> From: "Yoshinori K. Okuji" <okuji@enbug.org>
> Date: Sun, 29 Mar 2009 20:29:26 +0900
>
> > So if nobody else can, I would like to get it straight again myself,
> > although I am pretty busy (as I have a startup company, can you
> > imagine how tough it is?).
>
> Everybody is too busy to give this project the attention and time it
> deserves to be maintained properly.
>
> I honestly do not think the situation will change significantly until
> someone is able to devote real time as a maintainer and process all of
> the patches that get submitted each day.

This is ideal but not absolutely required. If you look at some popular 
projects, such as Linux and Firefox, you can find out that not all (actually, 
very few) patches are handled so quickly, but those projects are functioning 
so well.

> As top level maintainter that has to be your number one priority,
> processing patches efficiently.  Otherwise developers go elsewhere and
> you will eventually be working on all the features and bug fixes
> yourself.
>
> If you, on the other hand, devote the necessary time to patches
> then developers will be motivated.  They will do all the work and
> you can take all the credit :-)
>
> But I do appreicate that you understand the issue of trust.  It is
> the only way to operate sanely.

Thanks.

Okuji

Re: [PATCH] Split of the normal mode

[Vesa Jääskeläinen]

Yoshinori K. Okuji wrote:
> On Sunday 29 March 2009 20:40:17 David Miller wrote:
>> Everybody is too busy to give this project the attention and time it
>> deserves to be maintained properly.
>>
>> I honestly do not think the situation will change significantly until
>> someone is able to devote real time as a maintainer and process all of
>> the patches that get submitted each day.
> 
> This is ideal but not absolutely required. If you look at some popular 
> projects, such as Linux and Firefox, you can find out that not all (actually, 
> very few) patches are handled so quickly, but those projects are functioning 
> so well.

For both of those projects there are people that are paid to do that
work either directly or indirectly. How it internally affects, I don't
know.

Anyway... when people are paid to work there is certainly different
driving force behind it.

Both of those projects has divided work force dedicated to maintain and
drive enhancements to defined goals.

Now if we map this to our situation:

- We are missing what we want to do (eg. roadmap, feature plan)

- What different components should be able to do, eg. design documentation.

- Use cases what we want to support

- We don't really have defined responsibilities (expect for maintainers,
and even that can be a bit vague)

- What is philosophy what kind of work is being accepted and what we
require for patches/commits

- Systematic software functionality verification (either manual or
automated)

- If I am not mistaken no-one is being paid to maintain GRUB* or to
develop for. (not so big deal)

I have tried from time to time enhance some of those... but they seem
not to drive enough interest. Perhaps with better coordination it could
work.

So perhaps it would be best to form some kind of organization that
defines the goals and then defines responsibilities and backups for
components and tries to drive targeting those goals. Those could be like
though like internal maintainers for specific components. It could be
like bi-monthly meeting to tackle issues on horizon.

Re: [PATCH] Split of the normal mode

[Yoshinori K. Okuji]

On Sunday 29 March 2009 22:23:11 Vesa Jääskeläinen wrote:
> For both of those projects there are people that are paid to do that
> work either directly or indirectly. How it internally affects, I don't
> know.
>
> Anyway... when people are paid to work there is certainly different
> driving force behind it.

It does not matter if people are paid or not in this discussion, because:

- even if some people are paid, there are tons of volunteers
- volunteers are still contributing, although patches are not reviewed quickly

> Both of those projects has divided work force dedicated to maintain and
> drive enhancements to defined goals.
>
> Now if we map this to our situation:
>
> - We are missing what we want to do (eg. roadmap, feature plan)

This is somehow intentional, because I believe that volunteers do only what 
they want to do anyway. In fact, the TODO list in the wiki has several high 
priority items, but they have been pending for a long, long time (e.g. 
writing a manual).

> - What different components should be able to do, eg. design documentation.
>
> - Use cases what we want to support
>
> - We don't really have defined responsibilities (expect for maintainers,
> and even that can be a bit vague)
>
> - What is philosophy what kind of work is being accepted and what we
> require for patches/commits
>
> - Systematic software functionality verification (either manual or
> automated)
>
> - If I am not mistaken no-one is being paid to maintain GRUB* or to
> develop for. (not so big deal)

Only temporarily, as far as I know. For example, some students were paid in 
SoC. I was paid for the initial research by IPA (PUPA).

> I have tried from time to time enhance some of those... but they seem
> not to drive enough interest. Perhaps with better coordination it could
> work.
>
> So perhaps it would be best to form some kind of organization that
> defines the goals and then defines responsibilities and backups for
> components and tries to drive targeting those goals. Those could be like
> though like internal maintainers for specific components. It could be
> like bi-monthly meeting to tackle issues on horizon.

So you like formalism. I myself like a loose development model. If you have 
many active developers, formalism works better, because they start to 
conflict and consume most of the time for endless discussions, otherwise. 
But, people appear and disappear frequently in GRUB. Do you really think it 
works with GRUB?

Some examples...

I am involved with GRUB for 10 years, but I sometimes disappear completely for 
several months. Then, back again.

I think Pavel is also working on GRUB for nearly as long as me, but he works 
from time to time. It looks nearly random to me.

Robert seems to be relatively constant these days, but he apparently does not 
have time to comment on all patches.

So, till now, "Do whatever you want to do when you feel that you want to" has 
been the most practical, and legwork has been finished by official 
maintainers who have some formal responsibility (actually, official 
maintainers have responsibility only for endorsing the GNU philosophy, but 
they have done more than this).

Regards,
Okuji

Re: [PATCH] Split of the normal mode

[Vesa Jääskeläinen]

Yoshinori K. Okuji wrote:
> On Sunday 29 March 2009 22:23:11 Vesa Jääskeläinen wrote:
>> Both of those projects has divided work force dedicated to maintain and
>> drive enhancements to defined goals.
>>
>> Now if we map this to our situation:
>>
>> - We are missing what we want to do (eg. roadmap, feature plan)
> 
> This is somehow intentional, because I believe that volunteers do only what 
> they want to do anyway. In fact, the TODO list in the wiki has several high 
> priority items, but they have been pending for a long, long time (e.g. 
> writing a manual).

That item has been in my mind too as people complain that there ain't
one, even the developer one would be useful. But even if there are items
in TODO list, people hesitate what to do about it. Some questions that
could arise: Who do they consult? is my contribution really wanted? Do I
waste my time by doing this? I could be interested in this task, but
there is a "task reservation name" there.

>> So perhaps it would be best to form some kind of organization that
>> defines the goals and then defines responsibilities and backups for
>> components and tries to drive targeting those goals. Those could be like
>> though like internal maintainers for specific components. It could be
>> like bi-monthly meeting to tackle issues on horizon.
> 
> So you like formalism. I myself like a loose development model. If you have 
> many active developers, formalism works better, because they start to 
> conflict and consume most of the time for endless discussions, otherwise. 
> But, people appear and disappear frequently in GRUB. Do you really think it 
> works with GRUB?

Here is another example:

Someone writes a large pile (or even a small change) of code that he/she
would like to get into GRUB. He/she just sends a patch to mailing list
(or to tracker) and feels happy that I have done something now. Even if
there are several developers that could understand the code, they do not
even look at that as it is not within their close interests, or they
might not have time. Or they feel that there is someone else with more
knowledge that should handle this. So no-one really makes any kind of
response as there is no "pressure". Thus time the person has spent doing
something is wasted. This again generates negative publicity...

Another related issue is copyright assignment to FSF that seems to
hinder also some people. But for that we can't really do much.

> Some examples...
> 
> I am involved with GRUB for 10 years, but I sometimes disappear completely for 
> several months. Then, back again.
> 
> I think Pavel is also working on GRUB for nearly as long as me, but he works 
> from time to time. It looks nearly random to me.

Those scenarios also map to myself.

> Robert seems to be relatively constant these days, but he apparently does not 
> have time to comment on all patches.
> 
> So, till now, "Do whatever you want to do when you feel that you want to" has 
> been the most practical, and legwork has been finished by official 
> maintainers who have some formal responsibility (actually, official 
> maintainers have responsibility only for endorsing the GNU philosophy, but 
> they have done more than this).

Right. But if there is no other responsibility for anyone, you cannot
really ask people to follow any kind of design philosophy if there is
no-one watching the progress. If you give free permission to do what
ever you want, then you can pretty much expect that you get what ever
gets there.

I have from time to time though about projects that I have followed,
what has failed those or what could be the problem why there is no real
progress. In my opinion most of those have failed because there is no
real guidance on what to do next. People just do and assume that other
people will do something until they get enough frustrated and make the
necessary change themselves. If this happens, then it certainly does not
feel like fun.

Projects with active leadership/management seem to flourish much better
than on projects that where open sourcing means that here is the code,
do what ever you like with it.

So... to answer your question about formal development, yes, I feel that
it could help here. A bit relaxed model perhaps would be best but there
should always be tasks in horizon that needs to be done. Even some kind
of dependency graph for those could be a good idea to concentrate task
that needs to be done first. If tasks are somewhat defined then it is
easy just to pick one and do that, it also helps on keeping track of the
progress. It needs management but for that I would say that bi-monthly
irc/skype/or whatever meeting would be useful for. Also if there are
some design issues that might need multiple people to decide what to do
(or they might even need a vote or alternative authority override).

Re: [PATCH] Split of the normal mode

[David Miller]

From: Vesa Jääskeläinen <chaac@nic.fi>
Date: Sun, 29 Mar 2009 16:23:11 +0300

> Yoshinori K. Okuji wrote:
> > On Sunday 29 March 2009 20:40:17 David Miller wrote:
> >> Everybody is too busy to give this project the attention and time it
> >> deserves to be maintained properly.
> >>
> >> I honestly do not think the situation will change significantly until
> >> someone is able to devote real time as a maintainer and process all of
> >> the patches that get submitted each day.
> > 
> > This is ideal but not absolutely required. If you look at some popular 
> > projects, such as Linux and Firefox, you can find out that not all (actually, 
> > very few) patches are handled so quickly, but those projects are functioning 
> > so well.
> 
> For both of those projects there are people that are paid to do that
> work either directly or indirectly. How it internally affects, I don't
> know.

I'm just as active applying Linux networking patches now as a paid
employee of a Linux company as when I was just someone doing it for
fun.

It's not such a difference, really.

Re: [PATCH] Split of the normal mode

[David Miller]

From: "Yoshinori K. Okuji" <okuji@enbug.org>
Date: Sun, 29 Mar 2009 21:55:34 +0900

> On Sunday 29 March 2009 20:40:17 David Miller wrote:
> > From: "Yoshinori K. Okuji" <okuji@enbug.org>
> > Date: Sun, 29 Mar 2009 20:29:26 +0900
> >
> > > So if nobody else can, I would like to get it straight again myself,
> > > although I am pretty busy (as I have a startup company, can you
> > > imagine how tough it is?).
> >
> > Everybody is too busy to give this project the attention and time it
> > deserves to be maintained properly.
> >
> > I honestly do not think the situation will change significantly until
> > someone is able to devote real time as a maintainer and process all of
> > the patches that get submitted each day.
> 
> This is ideal but not absolutely required. If you look at some popular 
> projects, such as Linux and Firefox, you can find out that not all (actually, 
> very few) patches are handled so quickly, but those projects are functioning 
> so well.

As a top-level maintainer in one of those projects, I beg to differ.

The only reason I have so many people underneath me actively working
on the Linux networking stack is the simple fact that I process all
new patches processed each and every day.  I sometimes apply 100 or
more patches in a day, and that's critical for keeping the project
healthy.

And I do it using email because that is what's easiest for developers.
It is so easy to track patches automatically, using email instead of
some bug system, using a facility such as patchwork.

Otherwise I'd have the situation you guys have here, frustrated
developers and slow progress.

You can keep denying that this matters, but my practical experience
of over 15 years disagrees with your's.

Re: [PATCH] Split of the normal mode

[phcoder]

I agree. Developers should feel that their work is appreciated. 
Otherwise they turn to something else. THis is especially true for 
opensource where the only holding factor is the motivation

David Miller wrote:
> From: "Yoshinori K. Okuji" <okuji@enbug.org>
> Date: Sun, 29 Mar 2009 21:55:34 +0900
> 
>> On Sunday 29 March 2009 20:40:17 David Miller wrote:
>>> From: "Yoshinori K. Okuji" <okuji@enbug.org>
>>> Date: Sun, 29 Mar 2009 20:29:26 +0900
>>>
>>>> So if nobody else can, I would like to get it straight again myself,
>>>> although I am pretty busy (as I have a startup company, can you
>>>> imagine how tough it is?).
>>> Everybody is too busy to give this project the attention and time it
>>> deserves to be maintained properly.
>>>
>>> I honestly do not think the situation will change significantly until
>>> someone is able to devote real time as a maintainer and process all of
>>> the patches that get submitted each day.
>> This is ideal but not absolutely required. If you look at some popular 
>> projects, such as Linux and Firefox, you can find out that not all (actually, 
>> very few) patches are handled so quickly, but those projects are functioning 
>> so well.
> 
> As a top-level maintainer in one of those projects, I beg to differ.
> 
> The only reason I have so many people underneath me actively working
> on the Linux networking stack is the simple fact that I process all
> new patches processed each and every day.  I sometimes apply 100 or
> more patches in a day, and that's critical for keeping the project
> healthy.
> 
> And I do it using email because that is what's easiest for developers.
> It is so easy to track patches automatically, using email instead of
> some bug system, using a facility such as patchwork.
> 
> Otherwise I'd have the situation you guys have here, frustrated
> developers and slow progress.
> 
> You can keep denying that this matters, but my practical experience
> of over 15 years disagrees with your's.
> 
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel


-- 

Regards
Vladimir 'phcoder' Serbinenko

Re: [PATCH] Split of the normal mode

[Bean]

Hi,

I see the main objection here is about too many new modules, and the
use of script to configure boot process. But how about the new
parser/reader/menu_viewer model ? If I put them back in one
normal.mod, but keep the parts separated in source, would that be
acceptable ?

Also, some thoughts about grub2 authority. I do understand the needs
to keep source clean and safe for a boot loader, but sometimes the
process just seems too slow. For example, Colin's graphic menu patch
has been pending for almost half a year. IMO, if a patch implement a
generally useful feature, have no serious design faults, and don't
conflict with other parts of grub2, then it's safe to commit. For
small issues, we could fix it later.

Perhaps we can add a testing branch, in which developers have more
freedom to commit their work. The maintainer can review them
periodically and merge selected parts with the main branch. Many open
source project have more than one branches, perhaps it'd work with
grub2 as well.

-- 
Bean

Re: [PATCH] Split of the normal mode

[Yoshinori K. Okuji]

On Tuesday 31 March 2009 00:43:01 Bean wrote:
> Hi,
>
> I see the main objection here is about too many new modules, and the
> use of script to configure boot process.

I don't object to the latter any longer. Did you read my message?

> But how about the new 
> parser/reader/menu_viewer model ?

Nothing wrong. I have already proposed a similar idea for long:

https://savannah.gnu.org/task/?7668

> If I put them back in one 
> normal.mod, but keep the parts separated in source, would that be
> acceptable ?
>
> Also, some thoughts about grub2 authority. I do understand the needs
> to keep source clean and safe for a boot loader, but sometimes the
> process just seems too slow. For example, Colin's graphic menu patch
> has been pending for almost half a year. IMO, if a patch implement a
> generally useful feature, have no serious design faults, and don't
> conflict with other parts of grub2, then it's safe to commit. For
> small issues, we could fix it later.
>
> Perhaps we can add a testing branch, in which developers have more
> freedom to commit their work. The maintainer can review them
> periodically and merge selected parts with the main branch. Many open
> source project have more than one branches, perhaps it'd work with
> grub2 as well.

I just feel headache when you say "open source" in our project. Please stop.

About Colin's patch, I have *no idea*. I am sorry but I have a limitation. I 
can't do every thing suddenly.

For branching, you can try. I had tried it in GRUB Legacy, but I concluded 
that our resources were not sufficient to have multiple lines. This might 
work if branches are used only for experimental purposes.

Regards,
Okuji

Re: [PATCH] Split of the normal mode

[Bean]

On Tue, Mar 31, 2009 at 12:22 AM, Yoshinori K. Okuji <okuji@enbug.org> wrote:
> On Tuesday 31 March 2009 00:43:01 Bean wrote:
>> Hi,
>>
>> I see the main objection here is about too many new modules, and the
>> use of script to configure boot process.
>
> I don't object to the latter any longer. Did you read my message?

Hi,

Oh sorry, I just saw your reply. It works for me, but perhaps we could
split it a little bit more:

normal.mod
sh.mod
textmenu.mod

menu.mod
misc.mod
setjmp.mod

menu.mod is for generic menu function, like menu_viewer registration, etc.

misc.mod for misc function. For example, the function to read a line
from file can be useful in other places, completion is used by menu
viewers, etc. Also combine with old crc and hexdump in lib directory.

setjmp.mod is for setjmp function. Now normal.mod don't need to use
setjmp anymore. setjmp is platform dependent. By moving it out, we can
include normal.mod in common.rmk.

-- 
Bean

Re: [PATCH] Split of the normal mode

[Vesa Jääskeläinen]

Bean wrote:
> Also, some thoughts about grub2 authority. I do understand the needs
> to keep source clean and safe for a boot loader, but sometimes the
> process just seems too slow. For example, Colin's graphic menu patch
> has been pending for almost half a year. IMO, if a patch implement a
> generally useful feature, have no serious design faults, and don't
> conflict with other parts of grub2, then it's safe to commit. For
> small issues, we could fix it later.

Most of the patches has been already approved with minor changes. Colin
has his commit rights to the repository. If he do not feel like
committing the code in (no time to cleanup or what ever the reason is),
who is anyone else having any say to commit them in?

Of course there is really old version that needs some rewriting after
GSoC there, but other than that it is in my opinion really Colin's
responsibility to contribute/commit the code in.

General design (was Re: [PATCH] Split of the normal mode)

[phcoder]

If you're here to fix things then it's okay with me. We could start with 
design discussions and have a document describing rules and roadmap. Of 
course it won't be an absolute reference but any step away from roadmap 
is to be discussed thoroughfully
> - Trivial changes, in particular bug fixes, were allowed to be checked in 
> without any review, if the developer is trusted. 
I agree with this way
> - Rather significant changes, even bug fixes when these require code 
> restructuring, had to be reviewed. I think I was the only exception on this, 
> because I am the designor. In reality, however, even I often posted messages 
> to this mailing list before I made changes, because I appreciated others' 
> ideas.
> 
IMO now even you would have to let people discuss your changes if they 
aren't trivial. This is because grub2 supports many platforms and 
restructuring may break some platforms in subtle way
> - Design-level changes had to be always discussed a lot before being accepted. 
> This included myself, because even I, the original author, didn't know every 
> aspect of impacts.
This is especially true with all current ports and some people may want 
to delay design changes if some code is pending
> Afterwards, these rules were somehow forgotten for a practical reason: patches 
> were not reviewed quickly or even ignored for a long time, because of the 
> absense of leadership. I expected that we could overcome such a situation by 
> co-maintainership, but after both I and Marco got too busy with other things, 
> it stopped working. As you should know, several people, like Robert, Vesa, 
> Pavel, and Bean, helped greatly, but it was not still good enough for your 
> demand.
Get me right, I have nothing against these people. Actually I'm very 
grateful to Robert Millan that my patches were incorporated at all. 
However I find current organisation problematic. Perhaps we need another 
collaboration model to avoid such problems in future
> 
> So the current situation is like this:
> 
> - Many changes have already been incorporated without proper reviews, 
> including bad changes. The current GRUB is in a sense partly worse than 
> before, due to this.

Every of these changes has to be discussed separately. Some of them may 
lead to design discussions.
> 
> - Many patches are pending.
> 
This is my main complaint.
Also some design disscussion stay without any activity.
> So, the first thing I would like to do is to remind people of the check-in 
> rules, and apply them to past changes. Since a new version is not released 
> yet (after things got bad), we can still fix up the bad pieces safely. If we 
> miss this occasion, we will have to struggle with badly implemented features 
> for years due to compatibility. I have experienced this enough with GRUB 
> Legacy. I don't want it again. Otherwise, I wouldn't have made GRUB 2.
I think it would be a good idea to have a document describing those 
rules rather than playing "Mao" game.
> 
> Regards,
> Okuji
> 
> 
> _______________________________________________
> Grub-devel mailing list
> Grub-devel@gnu.org
> http://lists.gnu.org/mailman/listinfo/grub-devel


-- 

Regards
Vladimir 'phcoder' Serbinenko

Re: [PATCH] Split of the normal mode

[Vesa Jääskeläinen]

Yoshinori K. Okuji wrote:
> You should select compile-time "loading" (i.e. linking) whenever possible. If 
> a function is loaded eventually, it should be linked at compile time.
> 
> You should select automatic loading, if you need runtime linking.
> 
> Manual loading should be considered evil, in principle.

I agree.

> FYI, I am planning to make a full review of recent changes (i.e. all changes 
> made after I stopped reviewing every patch), and trash/revert/rewrite/blame 
> everything I don't like. Changes made for no good reason must be all 
> reverted.
> 
> In brief, I take back the leadership of this project for general directions. 
> For some subsystems (e.g. the coreboot support), I continue leaving the 
> responsibility to those who know better or are more active. Once the current 
> code is reviewed and fixed (at some degree), I will make a new release.
> 
> Any objection?

Not really. I will make some grief, but it might be a good thing in overall.

I would thou propose that you discuss the issue on mailing list so we
can probably fine tune principles and objectives of GRUB 2 at the same time.

Re: [PATCH] Split of the normal mode

[Yoshinori K. Okuji]

On Sunday 29 March 2009 19:48:45 Vesa Jääskeläinen wrote:
> Yoshinori K. Okuji wrote:
> > You should select compile-time "loading" (i.e. linking) whenever
> > possible. If a function is loaded eventually, it should be linked at
> > compile time.
> >
> > You should select automatic loading, if you need runtime linking.
> >
> > Manual loading should be considered evil, in principle.
>
> I agree.
>
> > FYI, I am planning to make a full review of recent changes (i.e. all
> > changes made after I stopped reviewing every patch), and
> > trash/revert/rewrite/blame everything I don't like. Changes made for no
> > good reason must be all reverted.
> >
> > In brief, I take back the leadership of this project for general
> > directions. For some subsystems (e.g. the coreboot support), I continue
> > leaving the responsibility to those who know better or are more active.
> > Once the current code is reviewed and fixed (at some degree), I will make
> > a new release.
> >
> > Any objection?
>
> Not really. I will make some grief, but it might be a good thing in
> overall.
>
> I would thou propose that you discuss the issue on mailing list so we
> can probably fine tune principles and objectives of GRUB 2 at the same
> time.

I will.

Thanks,
Okuji

Re: [PATCH] Split of the normal mode

[Vesa Jääskeläinen]

Vesa Jääskeläinen wrote:
> Yoshinori K. Okuji wrote:
>> Any objection?
> 
> Not really. I will make some grief, but it might be a good thing in overall.

Just to make it clear:

It will... ;)

Re: [PATCH] Split of the normal mode

[Robert Millan]

On Sun, Mar 29, 2009 at 07:09:56PM +0900, Yoshinori K. Okuji wrote:
> 
> Indeed. I don't understand this tendency about splitting modules at all. What 
> is the motivation behind? What is the real benefit for the user?
> 
> From my point of view, it is wrong to force the user to manually load modules, 
> generally. This includes writing "insmod" in config files. Look at Linux. It 
> is quite rare to execute insmod or modprobe directly. Most of the time, 
> modules are loaded on demand. This is the user-friendliness.

I agree with your point.  It should be noted, though, that in some of these
situations grub-mkconfig compensates that by generating "insmod xxx" statements
on-demand (a good example would be that "insmod lvm" can be generated by
prepare_grub_to_access_device).

> FYI, I am planning to make a full review of recent changes (i.e. all changes 
> made after I stopped reviewing every patch), and trash/revert/rewrite/blame 
> everything I don't like. Changes made for no good reason must be all 
> reverted.
> 
> In brief, I take back the leadership of this project for general directions. 
> For some subsystems (e.g. the coreboot support), I continue leaving the 
> responsibility to those who know better or are more active. Once the current 
> code is reviewed and fixed (at some degree), I will make a new release.
> 
> Any objection?

GRUB is in dire need of an active project lead.  I'm happy to see what you
plan to take back on that role.  Over the last few months, I tried to cover
up for some of the work you and Marco weren't doing, like processing the
patches nobody wanted to look at, or requesting copyright assignments.  TBH,
it wasn't a pleasant task, and if you'll be doing it from now on, it's a
relief for me.

As for reverting changes, I acknowledge you have the authority to do that,
but it would be very harmful to the project if all sort of things started
being reverted with no proper discussion.  When I say harmful, I mean that
some people might end up sticking with their own private trees, which later
can't easily be merged.  It's very important to build consensus on such
things.

So my suggestion is that you bring up discussion on which things you plan
to revert.  If we can't reach consensus, you have the authority to impose
your own view, but please try to find consensus, and be open to arguments
that might convince your POV.

To summarize, I approve of your decision, but I'd be very disappointed if
all this just happens so you can revert a bunch of stuff and inmediately
afterwards we're left with no active maintainer again.

-- 
Robert Millan

  The DRM opt-in fallacy: "Your data belongs to us. We will decide when (and
  how) you may access your data; but nobody's threatening your freedom: we
  still allow you to remove your data and not access it at all."

Re: [PATCH] Split of the normal mode

[Yoshinori K. Okuji]

On Wednesday 01 April 2009 22:19:00 Robert Millan wrote:
> GRUB is in dire need of an active project lead.  I'm happy to see what you
> plan to take back on that role.  Over the last few months, I tried to cover
> up for some of the work you and Marco weren't doing, like processing the
> patches nobody wanted to look at, or requesting copyright assignments. 
> TBH, it wasn't a pleasant task, and if you'll be doing it from now on, it's
> a relief for me.

Well, I don't like that so much, either. So I appreciate if you can continue 
to do that. :p

> As for reverting changes, I acknowledge you have the authority to do that,
> but it would be very harmful to the project if all sort of things started
> being reverted with no proper discussion.  When I say harmful, I mean that
> some people might end up sticking with their own private trees, which later
> can't easily be merged.  It's very important to build consensus on such
> things.
>
> So my suggestion is that you bring up discussion on which things you plan
> to revert.  If we can't reach consensus, you have the authority to impose
> your own view, but please try to find consensus, and be open to arguments
> that might convince your POV.
>
> To summarize, I approve of your decision, but I'd be very disappointed if
> all this just happens so you can revert a bunch of stuff and inmediately
> afterwards we're left with no active maintainer again.

Thank you for your comment. I fully understand what you mean. I will take 
care.

Regards,
Okuji