Re: module-init-tools/udev and module auto-loading

Re: module-init-tools/udev and module auto-loading

["Andrey Borzenkov"]

>> Hi
>> 
>> A quick question on module-init-tools/udev and module auto-loading ...
>> lets say I have a module called 'foo', that I want the kernel to
>> auto-load.
>
> Wait, stop right there.  When do you want the module autoloaded?

for legacy hardware that cannot generate any hotplug event when
connected.

Parallel port Jaz that I have. I usually have it switched off
or simply disconnected (let's leave the question of how safe is
to plug in parallel port cable aside). When I turn it on there is
no hotplug event available. Meaning

- either I have to load ppa (given current implementation)
- or initiate rescan of ppa scsi bus (if it is ever changed to new
  model)

I guess other parallel port devices share the same issue.

so there are cases when "action on access" makes sense.

regards

-andrey

Re: module-init-tools/udev and module auto-loading

[Greg KH]

On Mon, Feb 02, 2004 at 10:20:15AM +0300, "Andrey Borzenkov"  wrote:
> 
> so there are cases when "action on access" makes sense.

Yes, there are cases like this where that might make sense.  However
there's no real way for udev itself to solve those cases.  You will have
to rely on some other method to do this (script to load module, making
the /dev node yourself, script to make dev node and then access it which
causes kmod to load the module, etc.)

thanks,

greg k-h

Re: module-init-tools/udev and module auto-loading

["Andrey Borzenkov"]

You already have the implementation , it is called devfs. Why
reinvent the wheel?

> I guess there will be cries of murder if 'somebody' suggested that if
> a node in /dev is opened, but not there, the kernel can call
> 'modprobe -q /dev/foo' to load whatever alias there might have been?

this is exactly what was described as "unsolvable races in devfs
code". The problem is:

- lookup is run under directory i_sem. If you spawn anything 
  synchronously (waiting for it to finish) and it tries (intentionally
  or not) access the same directory you get deadlock.

- calling it asynchronously does not buy you much because it still
  means you must return -ENOENT first time.

I hope to have fixed the first type of races meaning that either
devfs (after some - significant - cleanup) may be used for that
or another file system written from scratch.

main problems in devfs are associated with the fact that contents
may change asynchronously wrt to upper layer. Removing everything
related to name registration from kernel will give you ligh weight
implementation capable of do what you want.

-andrey

module-init-tools/udev and module auto-loading

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 951 bytes --]

Hi

A quick question on module-init-tools/udev and module auto-loading ...
lets say I have a module called 'foo', that I want the kernel to
auto-load.

If I now have in modprobe.conf:

--
alias char-major-<foo_major>-* foo
alias /dev/foo foo
--

and /dev/foo exists, it works just fine.  If I delete /dev/foo
however, it does not.  Say the module _do_ support sysfs (meaning
udev will create /dev/foo on loading, this do not really affect
things, as without /dev/foo it do not work anyhow.

This a known issue (sure I know I can add it to local initscript to
load, but this is not always the preferred 'fix')?  Any ideas on
how to 'fix' this?

Then a distant related issue - anybody thought about dynamic major
numbers of 2.7/2.8 (?) and the 'alias char-major-<whatever>-* whatever'
type modprobe rules (as the whole fact of them being dynamic, will make
that alias type worthless ...)?


Thanks,

-- 
Martin Schlemmer

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: module-init-tools/udev and module auto-loading

[Rusty Russell]

In message <1075674718.27454.17.camel@nosferatu.lan> you write:
> A quick question on module-init-tools/udev and module auto-loading ...
> lets say I have a module called 'foo', that I want the kernel to
> auto-load.

The *idea* of udev et al is that the kernel finds the devices,
/sbin/hotplug loads the driver etc.

This does not cover the class of things which are entirely created by
the driver (eg. dummy devices, socket families), so cannot be
"detected".  Many of these (eg. socket families) can be handled by
explicit request_module() in the core and MODULE_ALIAS in the driver.
Some of them cannot at the moment: the first the kernel knows of them
is an attempt to open the device.  Some variant of devfs would solve
this.

> Then a distant related issue - anybody thought about dynamic major
> numbers of 2.7/2.8 (?) and the 'alias char-major-<whatever>-* whatever'
> type modprobe rules (as the whole fact of them being dynamic, will make
> that alias type worthless ...)?

Yes.  This could be changed to probe by device name, not number
though.  And most names can't be dynamic: /dev/null has certain, fixed
semantics.

The "I found this hardware, who will drive it?" mechanism of udev, and
the "User asked for this, who will supply it?" mechanism of kmod have
some overlap, but I think both will end up being required.

Cheers,
Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Re: module-init-tools/udev and module auto-loading

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 2380 bytes --]

On Mon, 2004-02-02 at 02:10, Rusty Russell wrote:
> In message <1075674718.27454.17.camel@nosferatu.lan> you write:
> > A quick question on module-init-tools/udev and module auto-loading ...
> > lets say I have a module called 'foo', that I want the kernel to
> > auto-load.
> 
> The *idea* of udev et al is that the kernel finds the devices,
> /sbin/hotplug loads the driver etc.
> 

Right.

> This does not cover the class of things which are entirely created by
> the driver (eg. dummy devices, socket families), so cannot be
> "detected".  Many of these (eg. socket families) can be handled by
> explicit request_module() in the core and MODULE_ALIAS in the driver.
> Some of them cannot at the moment: the first the kernel knows of them
> is an attempt to open the device.  Some variant of devfs would solve
> this.
> 

I guess there will be cries of murder if 'somebody' suggested that if
a node in /dev is opened, but not there, the kernel can call
'modprobe -q /dev/foo' to load whatever alias there might have been?

Understand me correctly, I do not want devfs back.  I am just checking
if we could do something for similar behaviour.  I know it has not
_really_ hit the lists, but I have already had complaints about this
not being supported anymore.  I think the complaints will start even
more if raminitfs eventually hits functionality (as /dev will then be
then consisting only of loaded drivers (if I am not mistaken).

Once again I do not say we should give in to it, but it _does_ make
things easier.  So if there is a lightweight way to do this, then
great - and this is what I was trying to get discussion directed to.

> > Then a distant related issue - anybody thought about dynamic major
> > numbers of 2.7/2.8 (?) and the 'alias char-major-<whatever>-* whatever'
> > type modprobe rules (as the whole fact of them being dynamic, will make
> > that alias type worthless ...)?
> 
> Yes.  This could be changed to probe by device name, not number
> though.  And most names can't be dynamic: /dev/null has certain, fixed
> semantics.
> 
> The "I found this hardware, who will drive it?" mechanism of udev, and
> the "User asked for this, who will supply it?" mechanism of kmod have
> some overlap, but I think both will end up being required.
> 

Ok, was just wondering.


Thanks,

-- 
Martin Schlemmer

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: module-init-tools/udev and module auto-loading

[Rusty Russell]

In message <1075748550.6931.10.camel@nosferatu.lan> you write:
> > This does not cover the class of things which are entirely created by
> > the driver (eg. dummy devices, socket families), so cannot be
> > "detected".  Many of these (eg. socket families) can be handled by
> > explicit request_module() in the core and MODULE_ALIAS in the driver.
> > Some of them cannot at the moment: the first the kernel knows of them
> > is an attempt to open the device.  Some variant of devfs would solve
> > this.
> > 
> 
> I guess there will be cries of murder if 'somebody' suggested that if
> a node in /dev is opened, but not there, the kernel can call
> 'modprobe -q /dev/foo' to load whatever alias there might have been?

I think it's an excellent idea, although ideally we would have this
mechanism in userspace as much as possible.  Anything from some
special hack to block -ENOENT on directory lookups and notify an fd,
to some exotic overlay filesystem.

I'm sure Al Viro has an opinion on this...

Cheers,
Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Re: module-init-tools/udev and module auto-loading

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 1262 bytes --]

On Tue, 2004-02-03 at 02:55, Rusty Russell wrote:
> In message <1075748550.6931.10.camel@nosferatu.lan> you write:
> > > This does not cover the class of things which are entirely created by
> > > the driver (eg. dummy devices, socket families), so cannot be
> > > "detected".  Many of these (eg. socket families) can be handled by
> > > explicit request_module() in the core and MODULE_ALIAS in the driver.
> > > Some of them cannot at the moment: the first the kernel knows of them
> > > is an attempt to open the device.  Some variant of devfs would solve
> > > this.
> > > 
> > 
> > I guess there will be cries of murder if 'somebody' suggested that if
> > a node in /dev is opened, but not there, the kernel can call
> > 'modprobe -q /dev/foo' to load whatever alias there might have been?
> 
> I think it's an excellent idea, although ideally we would have this
> mechanism in userspace as much as possible.  Anything from some
> special hack to block -ENOENT on directory lookups and notify an fd,
> to some exotic overlay filesystem.
> 

I was thinking of just delaying the failure or such, depending on the
modprobe outcome.  Should not be much if any overhead, although I have
not looked at the code.


-- 
Martin Schlemmer

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: module-init-tools/udev and module auto-loading

[Martin Schlemmer]

[-- Attachment #1.1: Type: text/plain, Size: 2338 bytes --]

On Tue, 2004-02-03 at 02:55, Rusty Russell wrote:
> In message <1075748550.6931.10.camel@nosferatu.lan> you write:
> > > This does not cover the class of things which are entirely created by
> > > the driver (eg. dummy devices, socket families), so cannot be
> > > "detected".  Many of these (eg. socket families) can be handled by
> > > explicit request_module() in the core and MODULE_ALIAS in the driver.
> > > Some of them cannot at the moment: the first the kernel knows of them
> > > is an attempt to open the device.  Some variant of devfs would solve
> > > this.
> > > 
> > 
> > I guess there will be cries of murder if 'somebody' suggested that if
> > a node in /dev is opened, but not there, the kernel can call
> > 'modprobe -q /dev/foo' to load whatever alias there might have been?
> 
> I think it's an excellent idea, although ideally we would have this
> mechanism in userspace as much as possible.  Anything from some
> special hack to block -ENOENT on directory lookups and notify an fd,
> to some exotic overlay filesystem.
> 

Something like attached.  Besides me not knowing if there is a better
place for it, it have the following issues:

1) Its a bit racy - iow, I get this (but only for urandom):

  request_module: runaway loop modprobe /dev/urandom

I am not sure if its the kthread patches with the one for
kthread use in modules that contributes to this (have not
checked yet).

2) Although it does work (load the right module), it seems to do it
too late.  I have removed the sleep in udev, so it cannot be this.
I do not know if this might be another kthread contribution (if I
understand the module/kthread patches correctly).  It does seem from
the request_module code though that it 'should' wait for modprobe to
complete ...  Maybe add some sort of locking (this be efficient?)?

3) Might be nice to keep track of "/dev/foo" 'aliases' that modprobe
fails for, and not try them again.

4) It only works if the whatever opened's name start with "/dev/", but
I do not think that is an issue, as it should (?) be how most things
access nodes, and should minimise complexity.

I know its very basic, I just thought to get something out of the
door for discussion and to give direction (or what direction not to
take ...).


Thanks,

-- 
Martin Schlemmer

[-- Attachment #1.2: mod-autoload.patch --]
[-- Type: text/x-patch, Size: 1499 bytes --]

--- 1/fs/namei.c	2004-02-03 14:35:16.000000000 +0200
+++ 2/fs/namei.c	2004-02-03 17:20:21.000000000 +0200
@@ -1240,7 +1240,7 @@ int open_namei(const char * pathname, in
 	int acc_mode, error = 0;
 	struct dentry *dentry;
 	struct dentry *dir;
-	int count = 0;
+	int count = 0, retry = 0;
 
 	acc_mode = ACC_MODE(flag);
 
@@ -1253,13 +1253,14 @@ int open_namei(const char * pathname, in
 	nd->intent.open.flags = flag;
 	nd->intent.open.create_mode = mode;
 
+retry_open:
 	/*
 	 * The simplest case - just a plain lookup.
 	 */
 	if (!(flag & O_CREAT)) {
 		error = path_lookup(pathname, lookup_flags(flag)|LOOKUP_OPEN, nd);
 		if (error)
-			return error;
+			goto error;
 		dentry = nd->dentry;
 		goto ok;
 	}
@@ -1269,7 +1270,7 @@ int open_namei(const char * pathname, in
 	 */
 	error = path_lookup(pathname, LOOKUP_PARENT|LOOKUP_OPEN|LOOKUP_CREATE, nd);
 	if (error)
-		return error;
+		goto error;
 
 	/*
 	 * We have the parent and last component. First of all, check
@@ -1343,8 +1344,22 @@ ok:
 exit_dput:
 	dput(dentry);
 exit:
+	if (error != 0 && strstr(pathname, "/dev/") == pathname && !retry) {
+		if (request_module(pathname) == 0) {
+			retry = 1;
+			goto retry_open;
+		}
+	}
 	path_release(nd);
 	return error;
+error:
+	if (error != 0 && strstr(pathname, "/dev/") == pathname && !retry) {
+		if (request_module(pathname) == 0) {
+			retry = 1;
+			goto retry_open;
+		}
+	}
+	return error;
 
 do_link:
 	error = -ELOOP;

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: module-init-tools/udev and module auto-loading

[viro]

On Tue, Feb 03, 2004 at 07:48:06PM +0200, Martin Schlemmer wrote:
> > > I guess there will be cries of murder if 'somebody' suggested that if
> > > a node in /dev is opened, but not there, the kernel can call
> > > 'modprobe -q /dev/foo' to load whatever alias there might have been?

Vetoed.  _Especially_ when you are checking that on "pathname prefix"
level - namei.c is not a place for such special-casing.

Re: module-init-tools/udev and module auto-loading

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 790 bytes --]

On Tue, 2004-02-03 at 21:33, viro@parcelfarce.linux.theplanet.co.uk
wrote:
> On Tue, Feb 03, 2004 at 07:48:06PM +0200, Martin Schlemmer wrote:
> > > > I guess there will be cries of murder if 'somebody' suggested that if
> > > > a node in /dev is opened, but not there, the kernel can call
> > > > 'modprobe -q /dev/foo' to load whatever alias there might have been?
> 
> Vetoed.  _Especially_ when you are checking that on "pathname prefix"
> level - namei.c is not a place for such special-casing.
> 

Well, I do not scratch around in there in general.  I guess the question
is:

1)  This this idea is Ok to make it (not patch or where it is, but the
idea in general.

2)  If yes to 1), any suggested place I should have a look at?


Thanks,

-- 
Martin Schlemmer

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: module-init-tools/udev and module auto-loading

[viro]

On Tue, Feb 03, 2004 at 10:47:13PM +0200, Martin Schlemmer wrote:
> On Tue, 2004-02-03 at 21:33, viro@parcelfarce.linux.theplanet.co.uk
> wrote:
> > On Tue, Feb 03, 2004 at 07:48:06PM +0200, Martin Schlemmer wrote:
> > > > > I guess there will be cries of murder if 'somebody' suggested that if
> > > > > a node in /dev is opened, but not there, the kernel can call
> > > > > 'modprobe -q /dev/foo' to load whatever alias there might have been?
> > 
> > Vetoed.  _Especially_ when you are checking that on "pathname prefix"
> > level - namei.c is not a place for such special-casing.
> > 
> 
> Well, I do not scratch around in there in general.  I guess the question
> is:
> 
> 1)  This this idea is Ok to make it (not patch or where it is, but the
> idea in general.

It is not.  Consider the effect of cd /dev followed by lookups.  Do you
really want a different behaviour in that case?  Ditto for use of
symlinks, yadda, yadda.

Re: module-init-tools/udev and module auto-loading

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 1159 bytes --]

On Tue, 2004-02-03 at 22:53, viro@parcelfarce.linux.theplanet.co.uk
wrote:
> On Tue, Feb 03, 2004 at 10:47:13PM +0200, Martin Schlemmer wrote:
> > On Tue, 2004-02-03 at 21:33, viro@parcelfarce.linux.theplanet.co.uk
> > wrote:
> > > On Tue, Feb 03, 2004 at 07:48:06PM +0200, Martin Schlemmer wrote:
> > > > > > I guess there will be cries of murder if 'somebody' suggested that if
> > > > > > a node in /dev is opened, but not there, the kernel can call
> > > > > > 'modprobe -q /dev/foo' to load whatever alias there might have been?
> > > 
> > > Vetoed.  _Especially_ when you are checking that on "pathname prefix"
> > > level - namei.c is not a place for such special-casing.
> > > 
> > 
> > Well, I do not scratch around in there in general.  I guess the question
> > is:
> > 
> > 1)  This this idea is Ok to make it (not patch or where it is, but the
> > idea in general.
> 
> It is not.  Consider the effect of cd /dev followed by lookups.  Do you
> really want a different behaviour in that case?  Ditto for use of
> symlinks, yadda, yadda.

Any other ideas for the problem in general? =)


Thanks,

-- 
Martin Schlemmer

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: module-init-tools/udev and module auto-loading

[Rusty Russell]

In message <1075830486.7473.32.camel@nosferatu.lan> you write:
> > I think it's an excellent idea, although ideally we would have this
> > mechanism in userspace as much as possible.  Anything from some
> > special hack to block -ENOENT on directory lookups and notify an fd,
> > to some exotic overlay filesystem.
> 
> Something like attached.  Besides me not knowing if there is a better
> place for it, it have the following issues:

Dude, you're brave.  I mean, really, really brave.

However, it strikes me that the automounter does similar tricks, and
so a similar setup should be possible for /dev.

Al?  Suggestions appreciated?

Thanks,
Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Re: module-init-tools/udev and module auto-loading

[viro]

On Wed, Feb 04, 2004 at 12:22:31PM +1100, Rusty Russell wrote:
> In message <1075830486.7473.32.camel@nosferatu.lan> you write:
> > > I think it's an excellent idea, although ideally we would have this
> > > mechanism in userspace as much as possible.  Anything from some
> > > special hack to block -ENOENT on directory lookups and notify an fd,
> > > to some exotic overlay filesystem.
> > 
> > Something like attached.  Besides me not knowing if there is a better
> > place for it, it have the following issues:
> 
> Dude, you're brave.  I mean, really, really brave.
> 
> However, it strikes me that the automounter does similar tricks, and
> so a similar setup should be possible for /dev.
> 
> Al?  Suggestions appreciated?

<shrug> If you want to mount autofs on /dev, more power to you.  However,
it got to be explicit "I mount <this> at /dev" - _anything_ based on
"if pathname starts with /dev, it's special" is FUBAR.

Re: module-init-tools/udev and module auto-loading

[Rusty Russell]

In message <20040204020425.GA21151@parcelfarce.linux.theplanet.co.uk> you write:
> <shrug> If you want to mount autofs on /dev, more power to you.  However,
> it got to be explicit "I mount <this> at /dev" - _anything_ based on
> "if pathname starts with /dev, it's special" is FUBAR.

Having a process say "tell me when something non-existent under this
dir gets accessed" seems to be what is desired.  And (from my
v. limited understanding) autofs comes close already, yes?

That at least gives Martin somewhere to start...
Rusty.
--
  Anyone who quotes me in their sig is an idiot. -- Rusty Russell.

Re: module-init-tools/udev and module auto-loading

[Greg KH]

On Mon, Feb 02, 2004 at 12:31:58AM +0200, Martin Schlemmer wrote:
> Hi
> 
> A quick question on module-init-tools/udev and module auto-loading ...
> lets say I have a module called 'foo', that I want the kernel to
> auto-load.

Wait, stop right there.  When do you want the module autoloaded?

If you want it loaded when the device is plugged in, then great, the
hotplug scripts will do that.

If you want the module loaded when you try to access the /dev node, then
see the FAQ about udev for that :)

> Then a distant related issue - anybody thought about dynamic major
> numbers of 2.7/2.8 (?) and the 'alias char-major-<whatever>-* whatever'
> type modprobe rules (as the whole fact of them being dynamic, will make
> that alias type worthless ...)?

Yes, it will make the char-major-* stuff worthless, however the distro
I use has not used that style of alias for years, why would yours?  :)

Rusty had it correct in that you need to try to load for the type of
module:
	alias eth1 tulip
	alias usb-controller usb-ohci
and so on.  That's the much better way.

thanks,

greg (I hate kmod) k-h

Re: module-init-tools/udev and module auto-loading

[Jamie Lokier]

Greg KH wrote:
> Rusty had it correct in that you need to try to load for the type of
> module:
> 	alias eth1 tulip
> 	alias usb-controller usb-ohci
> and so on.  That's the much better way.

Shouldn't these things be autodetected?

"alias usb-controller" is a poor example.  Shouldn't the kernel
automatically load drivers for all USB controllers that are found?
(On my system, there's more than one type, which makes "alias
usb-controller" even uglier).

"alias eth1" may be necessary to bind discovered NICs with the right
interface names, but it's unfortunate.  One system where I had "alias
eth0 ..."  failed to boot after a kernel upgrade when the named driver
no longer supported that network card (a different driver took over
support).

-- Jamie

Re: module-init-tools/udev and module auto-loading

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 460 bytes --]

On Mon, 2004-02-02 at 20:12, Jamie Lokier wrote:
> Greg KH wrote:
> > Rusty had it correct in that you need to try to load for the type of
> > module:
> > 	alias eth1 tulip
> > 	alias usb-controller usb-ohci
> > and so on.  That's the much better way.
> 
> Shouldn't these things be autodetected?
> 

My general experience is that it does work fine in most cases, except
maybe sound and net which you need your aliases.


-- 
Martin Schlemmer

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: module-init-tools/udev and module auto-loading

[Martin Schlemmer]

[-- Attachment #1: Type: text/plain, Size: 1775 bytes --]

On Mon, 2004-02-02 at 07:21, Greg KH wrote:
> On Mon, Feb 02, 2004 at 12:31:58AM +0200, Martin Schlemmer wrote:
> > Hi
> > 
> > A quick question on module-init-tools/udev and module auto-loading ...
> > lets say I have a module called 'foo', that I want the kernel to
> > auto-load.
> 
> Wait, stop right there.  When do you want the module autoloaded?
> 
> If you want it loaded when the device is plugged in, then great, the
> hotplug scripts will do that.
> 
> If you want the module loaded when you try to access the /dev node, then
> see the FAQ about udev for that :)
> 

Whoa, stop the horses :)  I do not want to start another devfs/udev
argument 8)

As I said - this feature _does_ work of autoloading the module _if_
the device node and alias exists.  If the node is deleted however it
does not.  So this tells me there functionality is half way there,
and I was wondering if it could be completed without too much extra
overhead.

You can check the mail to Rusty for more what I mean (hopefully it
comes through correctly).

> > Then a distant related issue - anybody thought about dynamic major
> > numbers of 2.7/2.8 (?) and the 'alias char-major-<whatever>-* whatever'
> > type modprobe rules (as the whole fact of them being dynamic, will make
> > that alias type worthless ...)?
> 
> Yes, it will make the char-major-* stuff worthless, however the distro
> I use has not used that style of alias for years, why would yours?  :)
> 
> Rusty had it correct in that you need to try to load for the type of
> module:
> 	alias eth1 tulip
> 	alias usb-controller usb-ohci
> and so on.  That's the much better way.
> 

Stupid question - is there a complete list of all these aliases?


Thanks,

-- 
Martin Schlemmer

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 189 bytes --]