grub2 accessibility: beeps

grub2 accessibility: beeps

[Hollis Blanchard]

One topic that I've heard a lot about in the past few weeks is
accessibility in grub2. Blind users, who otherwise have various devices
like screen readers and brail TTYs, need to be able to interact with the
bootloader.

This can be as simple as using the PC speaker to beep when GRUB starts,
and also to beep differently (e.g. different pitch, or different number
of beeps) when highlighting different menu entries.

Thanks to Vincent, we already have a PC speaker driver (cool!). However,
we lack the ability to trigger it when switching menu entries (without
selecting one). The problem boils down to this:
        menuentry "foo" {
        	linux foo
        }

The commands inside the menuentry block are only executed when the entry
is selected, not highlighted. Perhaps the menuentry command could be
extended to take an optional beep argument:
        menuentry "foo" <pitch> <duration> {
        	linux foo
        }

But wouldn't it be better to have something more general? Something like
        menuentry {
        	title "foo"
        	beep pitch duration
        }
        {
        	linux foo
        }

Thoughts?

-Hollis

Re: grub2 accessibility: beeps

[Marco Gerards]

Hollis Blanchard <hollis@penguinppc.org> writes:

> One topic that I've heard a lot about in the past few weeks is
> accessibility in grub2. Blind users, who otherwise have various devices
> like screen readers and brail TTYs, need to be able to interact with the
> bootloader.

Agreed.

> This can be as simple as using the PC speaker to beep when GRUB starts,
> and also to beep differently (e.g. different pitch, or different number
> of beeps) when highlighting different menu entries.

Right.

> Thanks to Vincent, we already have a PC speaker driver (cool!). However,
> we lack the ability to trigger it when switching menu entries (without
> selecting one). The problem boils down to this:
>         menuentry "foo" {
>         	linux foo
>         }
>
> The commands inside the menuentry block are only executed when the entry
> is selected, not highlighted. Perhaps the menuentry command could be
> extended to take an optional beep argument:
>         menuentry "foo" <pitch> <duration> {
>         	linux foo
>         }

I don't like this.  One reason is that this is hard to implement.  The
other is it is very specific and I am afraid "menuentry" becomes very
bloated.

> But wouldn't it be better to have something more general? Something like
>         menuentry {
>         	title "foo"
>         	beep pitch duration
>         }
>         {
>         	linux foo
>         }

Perhaps, I was thinking about something like:

menuentry "foo" --beep=pitch,duration

In that case it looks like a command.

Actually, what I would prefer is something hooks.  It would be nice if
the C code in GRUB and perhaps also scripts can be hooked anywhere.
We have to be very careful about this, so perhaps limit the sets of
commands that can be used this way.  But I think it will make GRUB
more flexible while it is not too much effort.

I am thinking of the following functions:

One to install a hook:
grub_err_t grub_hook_register (const char *name, (*hook) (...));

To trigger the hook (for example, when the user types something,
selects another menu entry, etc.
void grub_hook_call (const char *name, ...);

How to pass arguments to functions and to scripts is something that we
have to think about.  For example, scripting functions have to get
strings, while in other cases we just want to deal with regular
function calls.  Perhaps a printf-like mechanism can help us with
this.

In that case your problem can be solved by using:

function handlemenu {
  beep depending on menuentry selected, the index is passed to this function
}

hook --install --hook=select-menu-entry --function=handlemenu

...

--
Marco

Re: user-defined hooks

[Hollis Blanchard]

On Thu, 2006-11-30 at 20:50 +0100, Marco Gerards wrote:
> Actually, what I would prefer is something hooks.  It would be nice if
> the C code in GRUB and perhaps also scripts can be hooked anywhere.
> We have to be very careful about this, so perhaps limit the sets of
> commands that can be used this way.  But I think it will make GRUB
> more flexible while it is not too much effort.

I like this idea.

> I am thinking of the following functions:
> 
> One to install a hook:
> grub_err_t grub_hook_register (const char *name, (*hook) (...));
> 
> To trigger the hook (for example, when the user types something,
> selects another menu entry, etc.
> void grub_hook_call (const char *name, ...);
> 
> How to pass arguments to functions and to scripts is something that we
> have to think about.  For example, scripting functions have to get
> strings, while in other cases we just want to deal with regular
> function calls.  Perhaps a printf-like mechanism can help us with
> this.

I do not think it's important that we be able to hook C functions
directly. In other words, the argument passed to --function (below)
should be a script command (or script function), and not a C function
name.

> In that case your problem can be solved by using:
> 
> function handlemenu {
>   beep depending on menuentry selected, the index is passed to this function
> }
> 
> hook --install --hook=select-menu-entry --function=handlemenu

Defining the set of useful hooks, the arguments passed to each, and
documenting them will be critical. Also, just like with commands, once
we implement a hook we can't remove or change it without breaking config
files.

-Hollis

Re: user-defined hooks

[Thomas Schwinge]

[-- Attachment #1: Type: text/plain, Size: 469 bytes --]

Hello!

On Thu, Nov 30, 2006 at 01:59:18PM -0600, Hollis Blanchard wrote:
> Defining the set of useful hooks, the arguments passed to each, and
> documenting them will be critical. Also, just like with commands, once
> we implement a hook we can't remove or change it without breaking config
> files.

True, unless you apply some sort of versioning to the config files.  I'm
not directly advocating to do that for GRUB, though.  But it'd be an
idea.


Regards,
 Thomas

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: grub2 accessibility: beeps

[Yoshinori K. Okuji]

On Thursday 30 November 2006 20:50, Marco Gerards wrote:
> Perhaps, I was thinking about something like:
>
> menuentry "foo" --beep=pitch,duration
>
> In that case it looks like a command.

I don't like this, either. My preference is to use a style sheet. How the menu 
interface appears should be defined by a style sheet instead of commands. I 
wanted to implement such support, but I haven't done yet.

Okuji

Re: grub2 accessibility: beeps

[Marco Gerards]

"Yoshinori K. Okuji" <okuji@enbug.org> writes:

> On Thursday 30 November 2006 20:50, Marco Gerards wrote:
>> Perhaps, I was thinking about something like:
>>
>> menuentry "foo" --beep=pitch,duration
>>
>> In that case it looks like a command.
>
> I don't like this, either. My preference is to use a style sheet. How the menu 
> interface appears should be defined by a style sheet instead of commands. I 
> wanted to implement such support, but I haven't done yet.

Well, the point that I am trying to make is that I would like to use
hooks for this in some way.  So you are not agreeing with the first
alternative I suggested.  But you ignored the second alternative,
which was the most important part in my email.

Can you explain what the style sheet would look like?  In that case we
can discuss some things so we are all agree so we can work towards the
same goal.

--
Marco

Re: grub2 accessibility: beeps

[Yoshinori K. Okuji]

On Monday 04 December 2006 21:40, Marco Gerards wrote:
> Well, the point that I am trying to make is that I would like to use
> hooks for this in some way.  So you are not agreeing with the first
> alternative I suggested.  But you ignored the second alternative,
> which was the most important part in my email.

You look so offensive to me. Did I ignore it? No! I only didn't mention any 
opinion about that.

> Can you explain what the style sheet would look like?  In that case we
> can discuss some things so we are all agree so we can work towards the
> same goal.

I think you remember that I wanted to use a kind of style sheet to specify a 
fancy menu interface. This fits to the beeping issue. For now, I really don't 
care how it looks like, as long as it is easy even for non-programmers. I use 
a CSS-like syntax as an example:

grub.css:

# The default.
menuentry {
  beep: 1 2 # Whatever these integers mean
}

# An usual entry.
menuentry.usual {
  beep: 3 4
}

# A rescue entry.
menuentry.rescue {
  beep: 5 6
}

grub.cfg

stylesheet grub.css

menuentry --class=usual "Usual OS" {
  ...
}

menuentry --class=rescue "Rescue OS" {
  ...
}


I didn't mention anything about your idea on hooks, because I only want to 
separate visual or sound effects from the config file itself. I don't care so 
much about the underlying implementation, whether it uses hooks or not. 
That's because I bet that how to boot operating systems is independent of how 
the menu looks.

Okuji

Re: grub2 accessibility: beeps

[Marco Gerards]

"Yoshinori K. Okuji" <okuji@enbug.org> writes:

> On Monday 04 December 2006 21:40, Marco Gerards wrote:
>> Well, the point that I am trying to make is that I would like to use
>> hooks for this in some way.  So you are not agreeing with the first
>> alternative I suggested.  But you ignored the second alternative,
>> which was the most important part in my email.
>
> You look so offensive to me. Did I ignore it? No! I only didn't mention any 
> opinion about that.

I am not offensive at all, why would I be?  I am merely saying that
your reply is on a part of the email that I do not consider relevant,
while the relevant part is ignored, at least in your reply.  I just
hope you will react on the other part as well, when you have the time.

>> Can you explain what the style sheet would look like?  In that case we
>> can discuss some things so we are all agree so we can work towards the
>> same goal.
>
> I think you remember that I wanted to use a kind of style sheet to specify a 
> fancy menu interface. This fits to the beeping issue. For now, I really don't 
> care how it looks like, as long as it is easy even for non-programmers. I use 
> a CSS-like syntax as an example:

[...]

> I didn't mention anything about your idea on hooks, because I only want to 
> separate visual or sound effects from the config file itself. I don't care so 
> much about the underlying implementation, whether it uses hooks or not. 
> That's because I bet that how to boot operating systems is independent of how 
> the menu looks.

Well, what I proposed is using hooks so you can program the behavior
instead of describing it in some kind of syntax.  A scripting language
is always more expressive than syntax (like arguments as Hollis
proposed, or your style sheet).

But I do think your style sheet proposal is useful, for example for
defining the visual layout of the menu.  Perhaps it can also be used
someway to communicate with the scripts behind the hooks.  But I do
not think it is useful to describe behavior in the style sheet because
this is what scripts are for.

--
Marco

Re: grub2 accessibility: beeps

[Yoshinori K. Okuji]

On Wednesday 13 December 2006 13:45, Marco Gerards wrote:
> I am not offensive at all, why would I be?  I am merely saying that
> your reply is on a part of the email that I do not consider relevant,
> while the relevant part is ignored, at least in your reply.  I just
> hope you will react on the other part as well, when you have the time.

Then please just say that. I am quite nervous at the word "ignore". ;)

> Well, what I proposed is using hooks so you can program the behavior
> instead of describing it in some kind of syntax.  A scripting language
> is always more expressive than syntax (like arguments as Hollis
> proposed, or your style sheet).
>
> But I do think your style sheet proposal is useful, for example for
> defining the visual layout of the menu.  Perhaps it can also be used
> someway to communicate with the scripts behind the hooks.  But I do
> not think it is useful to describe behavior in the style sheet because
> this is what scripts are for.

Can you describe some examples besides "beep"? I find it difficult to 
recognize what you mean by "behavior".

I continue by guessing. First, it is better to define my own definition of 
"behavior". This is complicated, because it is possible to say that 
everything is "behavior" in a sense. For example, when you push a down key, 
the selected entry goes down. This is a behavior. When the selection changes, 
a previous entry stops showing a highlight color (behavior), and a current 
entry starts showing a highlight color (behavior).

The ultimate way is to implement all the "behaviors" with the scripting 
engine. But I feel that this is overkill, and I guess you agree with this. I 
guese we do not want to do everything in scripting.

For me, things classified into "behaviors" are what the boot loader executes 
for entries in order to transfer control to an operating system at a point. 
In this definition, highlighting entries is not a behavior, because it has 
nothing with booting or executing something. Beeping is not a behavior, 
either, because beeping itself does not perform anything to boot. Actually, 
beeping when an entry is chosen is quite analogous to highlighting. This was 
my point in the previous message. I like to call this "appearance" rather 
than "behavior".

When the user tries to express a certain "appearance", her ability is limited 
to within functionality exported from GRUB. This is not different from hooks, 
because GRUB must export, for example, commands to allow her to express it.

Now let's see how HTML works. When you want to make a change dynamically with 
corresponding to a click, you write a script expression with, for example, 
onSelect. Then, the expression changes a style sheet or an attribute to a tag 
or a class or something, if you want to change the appearance.

We have at least two choices in GRUB. One way is to mimic HTML+CSS+JavaScript. 
In this case, we provide a hook which is executed by a user action, and we 
write some script to control the style sheet. This should work definitely.

But I have another way, that is, specifying what happens only in the style 
sheet, like a:hover in HTML.

Personally, I prefer the latter to the former. The latter is simple. All the 
user should take care of is the style sheet. The user does not have to wander 
around.

Then, why does HTML offer this chimera technique? The reason is partially 
historical. Originally HTML was extremely static, and they added more and 
more features later, so it gets to the current situation. But another reason 
is that scripting can do more than changing appearance. For example, loading 
new data from a server, and insert new rows into a table. This kind of 
"behavior" is very hard to express with a style sheet. Thus it makes sense to 
use scripting for this purpose.

So the question is how much we want to make GRUB dynamic. If you just want to 
execute arbitrary commands when executing an entry, you can write commands in 
the entry itself. No problem. But, for example, if you want to insert new 
menu entries when only selecting an entry (selecting means just moving a 
cursor on an entry here), you cannot do this only with a style sheet.

I myself don't need this amount of dynamism for my own use. I prefer that the 
content of a menu is static. But others might think of different 
characteristics for something that I cannot imagine at the moment.

So, what is your idea about this, Marco?

Okuji

Re: grub2 accessibility: beeps

[tgingold]

Quoting Hollis Blanchard <hollis@penguinppc.org>:

> One topic that I've heard a lot about in the past few weeks is
> accessibility in grub2. Blind users, who otherwise have various devices
> like screen readers and brail TTYs, need to be able to interact with the
> bootloader.
Yes and yes.
[...]
> But wouldn't it be better to have something more general? Something like
>         menuentry {
>         	title "foo"
>         	beep pitch duration
>         }
>         {
>         	linux foo
>         }
I like this one.
I also plan to add a logo/picture attribute.

Tristan.