Cold plug starting services too soon

Cold plug starting services too soon

[Greg KH]

<dragging in the Debian and SuSE developers responsible for this>

Hi Kay and Marco.

Gentoo has run into the problem that if we load modules for some types
of devices (like network devices for example), we can generate hotplug
events that want to run things on disks that are not mounted yet, as we
havn't even done mounting of the non-root filesystem yet (caused by udev
doing the coldplug pretty much the first thing at boot time.)

How has Debian and SuSE kept from having this issue?  I think for SuSE
we rely on the fact that the initrd/initramfs has loaded the proper
modules already for us, right?

thanks,

greg k-h


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x103432&bid#0486&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Cold plug starting services too soon

[Greg KH]

On Tue, Jan 17, 2006 at 12:32:44PM -0800, Greg KH wrote:
> <dragging in the Debian and SuSE developers responsible for this>
> 
> Hi Kay and Marco.
> 
> Gentoo has run into the problem that if we load modules for some types
> of devices (like network devices for example), we can generate hotplug
> events that want to run things on disks that are not mounted yet, as we
> havn't even done mounting of the non-root filesystem yet (caused by udev
> doing the coldplug pretty much the first thing at boot time.)
> 
> How has Debian and SuSE kept from having this issue?  I think for SuSE
> we rely on the fact that the initrd/initramfs has loaded the proper
> modules already for us, right?

Oh, and if you both feel this is a Gentoo specific issue, feel free to
tell us to go away and deal with it ourselves :)

thanks,

greg k-h


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x103432&bid#0486&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Cold plug starting services too soon

[Kay Sievers]

On Tue, Jan 17, 2006 at 12:32:44PM -0800, Greg KH wrote:
> <dragging in the Debian and SuSE developers responsible for this>
> 
> Hi Kay and Marco.
> 
> Gentoo has run into the problem that if we load modules for some types
> of devices (like network devices for example), we can generate hotplug
> events that want to run things on disks that are not mounted yet, as we
> havn't even done mounting of the non-root filesystem yet (caused by udev
> doing the coldplug pretty much the first thing at boot time.)
> 
> How has Debian and SuSE kept from having this issue?  I think for SuSE
> we rely on the fact that the initrd/initramfs has loaded the proper
> modules already for us, right?

All running events are exported in /dev/.udev/queue/, events that fail
will be moved to /dev/.udev/failed/. Make sure the script return non-zero
if it can't run and retry all failed events from /dev/.udev/failed/
after "localfs" is done. If the same event then runs successfully, it
will go away from /dev/.udev/failed/. On SUSE I have a second udev boot
script that is called /etc/init.d/boot.udev_retry.

Kay


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x103432&bid#0486&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Cold plug starting services too soon

[Marco d'Itri]

[-- Attachment #1: Type: text/plain, Size: 914 bytes --]

On Jan 17, Greg KH <greg@kroah.com> wrote:

> Gentoo has run into the problem that if we load modules for some types
> of devices (like network devices for example), we can generate hotplug
> events that want to run things on disks that are not mounted yet, as we
> havn't even done mounting of the non-root filesystem yet (caused by udev
> doing the coldplug pretty much the first thing at boot time.)
> 
> How has Debian and SuSE kept from having this issue?  I think for SuSE
On Debian, packages which install RUN rules must either not care if
events happening before /usr is mounted are lost or if appropriate
use scripts which wait until whatever is needed is available (e.g. the
SUBSYSTEM=="net" agent waits for the lo interface to be up).
I am not sure if I want to replay failed events, because we don't know
why they failed and so far it's not clear if this is needed.

-- 
ciao,
Marco

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Cold plug starting services too soon

[Greg KH]

On Tue, Jan 17, 2006 at 09:43:03PM +0100, Kay Sievers wrote:
> On Tue, Jan 17, 2006 at 12:32:44PM -0800, Greg KH wrote:
> > <dragging in the Debian and SuSE developers responsible for this>
> > 
> > Hi Kay and Marco.
> > 
> > Gentoo has run into the problem that if we load modules for some types
> > of devices (like network devices for example), we can generate hotplug
> > events that want to run things on disks that are not mounted yet, as we
> > havn't even done mounting of the non-root filesystem yet (caused by udev
> > doing the coldplug pretty much the first thing at boot time.)
> > 
> > How has Debian and SuSE kept from having this issue?  I think for SuSE
> > we rely on the fact that the initrd/initramfs has loaded the proper
> > modules already for us, right?
> 
> All running events are exported in /dev/.udev/queue/, events that fail
> will be moved to /dev/.udev/failed/. Make sure the script return non-zero
> if it can't run and retry all failed events from /dev/.udev/failed/
> after "localfs" is done. If the same event then runs successfully, it
> will go away from /dev/.udev/failed/. On SUSE I have a second udev boot
> script that is called /etc/init.d/boot.udev_retry.

Cool, I'll try doing that for Gentoo and see how it works out.  Thanks
for adding this to udev already.

Roy, do you think this will work out properly?

greg k-h


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x103432&bid#0486&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Cold plug starting services too soon

[Roy Marples]

[-- Attachment #1: Type: text/plain, Size: 1974 bytes --]

On Wednesday 18 January 2006 00:27, Greg KH wrote:
> On Tue, Jan 17, 2006 at 09:43:03PM +0100, Kay Sievers wrote:
> > All running events are exported in /dev/.udev/queue/, events that fail
> > will be moved to /dev/.udev/failed/. Make sure the script return non-zero
> > if it can't run and retry all failed events from /dev/.udev/failed/
> > after "localfs" is done. If the same event then runs successfully, it
> > will go away from /dev/.udev/failed/. On SUSE I have a second udev boot
> > script that is called /etc/init.d/boot.udev_retry.
>
> Cool, I'll try doing that for Gentoo and see how it works out.  Thanks
> for adding this to udev already.
>
> Roy, do you think this will work out properly?

Yes, this can work out nicely. However, I don't see /dev/.udev/failed when we 
exit with code 1. Is there something I'm missing?

Currently I have this fairly near the top of runscript.sh

# Ensure that we have completed sysinit
if [[ ${IN_HOTPLUG} == 1 ]] ; then
    for pid in $(pidof bash) ; do
        if [[ "$(cat "/proc/${pid}/cmdline" 2>/dev/null)" \
            == "/bin/bash/sbin/rcsysinit" ]] ; then
            eerror "ERROR:  cannot hotplug ${myservice} until sysinit has 
run."
            exit 1
        fi
    done
    unset pid
fi

Now thats very hacky, but the only way I can think of to find out if our rc 
system has completed its sysinit. A better solution would be this

# Ensure that we have completed sysinit
if [[ ${RC_SYSINIT} == 1 ]] ; then
   eerror "ERROR:  cannot run ${myservice} until sysinit has completed."
   exit 1
fi

Obviously this only works if rc can configure udev to export shell variables 
before triggerring hotplugable events and can change/unset/add it after 
sysinit. Something like this maybe.

udev_set_env "RC_SYSINIT=1"

do stuff to complete sysinit

udev_sev_env "RC_SYSINIT="

Can we do this Greg?

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo Linux Developer

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Cold plug starting services too soon

[Greg KH]

On Wed, Jan 18, 2006 at 09:28:31AM +0000, Roy Marples wrote:
> On Wednesday 18 January 2006 00:27, Greg KH wrote:
> > On Tue, Jan 17, 2006 at 09:43:03PM +0100, Kay Sievers wrote:
> > > All running events are exported in /dev/.udev/queue/, events that fail
> > > will be moved to /dev/.udev/failed/. Make sure the script return non-zero
> > > if it can't run and retry all failed events from /dev/.udev/failed/
> > > after "localfs" is done. If the same event then runs successfully, it
> > > will go away from /dev/.udev/failed/. On SUSE I have a second udev boot
> > > script that is called /etc/init.d/boot.udev_retry.
> >
> > Cool, I'll try doing that for Gentoo and see how it works out.  Thanks
> > for adding this to udev already.
> >
> > Roy, do you think this will work out properly?
> 
> Yes, this can work out nicely. However, I don't see /dev/.udev/failed when we 
> exit with code 1. Is there something I'm missing?
> 
> Currently I have this fairly near the top of runscript.sh
> 
> # Ensure that we have completed sysinit
> if [[ ${IN_HOTPLUG} = 1 ]] ; then
>     for pid in $(pidof bash) ; do
>         if [[ "$(cat "/proc/${pid}/cmdline" 2>/dev/null)" \
>             = "/bin/bash/sbin/rcsysinit" ]] ; then
>             eerror "ERROR:  cannot hotplug ${myservice} until sysinit has 
> run."
>             exit 1
>         fi
>     done
>     unset pid
> fi
> 
> Now thats very hacky, but the only way I can think of to find out if our rc 
> system has completed its sysinit. A better solution would be this
> 
> # Ensure that we have completed sysinit
> if [[ ${RC_SYSINIT} = 1 ]] ; then
>    eerror "ERROR:  cannot run ${myservice} until sysinit has completed."
>    exit 1
> fi
> 
> Obviously this only works if rc can configure udev to export shell variables 
> before triggerring hotplugable events and can change/unset/add it after 
> sysinit. Something like this maybe.
> 
> udev_set_env "RC_SYSINIT=1"
> 
> do stuff to complete sysinit
> 
> udev_sev_env "RC_SYSINIT="
> 
> Can we do this Greg?

Hm, we might be able to mess around with this some, but why not just
create a temporary file when we start udev, which the rc scripts can
check?  When the initial udev startup is done (or whenever you want it),
the file can be removed.  That should be a lot simpler, right?

thanks,

greg k-h


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x103432&bid#0486&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Cold plug starting services too soon

[Roy Marples]

On Thursday 19 January 2006 03:45, Greg KH wrote:
> Hm, we might be able to mess around with this some, but why not just
> create a temporary file when we start udev, which the rc scripts can
> check?  When the initial udev startup is done (or whenever you want it),
> the file can be removed.  That should be a lot simpler, right?

Yes it would, but all disks are mounted read only at this point - the checking 
and mounting happens during sysinit. At this point we can create temporary 
files, but udev is already active and doing its thing.

Roy


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x103432&bid#0486&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Cold plug starting services too soon

[Greg KH]

On Thu, Jan 19, 2006 at 05:54:18PM +0000, Roy Marples wrote:
> On Thursday 19 January 2006 03:45, Greg KH wrote:
> > Hm, we might be able to mess around with this some, but why not just
> > create a temporary file when we start udev, which the rc scripts can
> > check?  When the initial udev startup is done (or whenever you want it),
> > the file can be removed.  That should be a lot simpler, right?
> 
> Yes it would, but all disks are mounted read only at this point - the checking 
> and mounting happens during sysinit. At this point we can create temporary 
> files, but udev is already active and doing its thing.

In our ramfs/tmpfs?  Why can't we use that, as we know we can write
there :)

thanks,

greg k-h


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x103432&bid#0486&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Cold plug starting services too soon

[Roy Marples]

[-- Attachment #1: Type: text/plain, Size: 992 bytes --]

On Thursday 19 January 2006 18:04, Greg KH wrote:
> On Thu, Jan 19, 2006 at 05:54:18PM +0000, Roy Marples wrote:
> > On Thursday 19 January 2006 03:45, Greg KH wrote:
> > > Hm, we might be able to mess around with this some, but why not just
> > > create a temporary file when we start udev, which the rc scripts can
> > > check?  When the initial udev startup is done (or whenever you want
> > > it), the file can be removed.  That should be a lot simpler, right?
> >
> > Yes it would, but all disks are mounted read only at this point - the
> > checking and mounting happens during sysinit. At this point we can create
> > temporary files, but udev is already active and doing its thing.
>
> In our ramfs/tmpfs?  Why can't we use that, as we know we can write
> there :)

Is there any guarantee that user has ramfs/tmpfs compiled into kernel?
If so, where would we mount? /lib/rcscripts/sysinit ?

Thanks

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo Linux Developer

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Cold plug starting services too soon

[Greg KH]

On Thu, Jan 19, 2006 at 06:11:55PM +0000, Roy Marples wrote:
> On Thursday 19 January 2006 18:04, Greg KH wrote:
> > On Thu, Jan 19, 2006 at 05:54:18PM +0000, Roy Marples wrote:
> > > On Thursday 19 January 2006 03:45, Greg KH wrote:
> > > > Hm, we might be able to mess around with this some, but why not just
> > > > create a temporary file when we start udev, which the rc scripts can
> > > > check?  When the initial udev startup is done (or whenever you want
> > > > it), the file can be removed.  That should be a lot simpler, right?
> > >
> > > Yes it would, but all disks are mounted read only at this point - the
> > > checking and mounting happens during sysinit. At this point we can create
> > > temporary files, but udev is already active and doing its thing.
> >
> > In our ramfs/tmpfs?  Why can't we use that, as we know we can write
> > there :)
> 
> Is there any guarantee that user has ramfs/tmpfs compiled into kernel?
> If so, where would we mount? /lib/rcscripts/sysinit ?

No, I mean the ramfs/tmpfs that udev uses to hold it's /dev tree and
database.  I can provide a function that the rc scripts can call to see
if udev is still initializing or not to keep them from having to worry
about the actual mechanics.

And yes, ramfs is always compiled into the kernel these days.

thanks,

greg k-h


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x103432&bid#0486&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel

Re: Cold plug starting services too soon

[Roy Marples]

[-- Attachment #1: Type: text/plain, Size: 679 bytes --]

On Thursday 19 January 2006 18:19, Greg KH wrote:
> No, I mean the ramfs/tmpfs that udev uses to hold it's /dev tree and
> database.  I can provide a function that the rc scripts can call to see
> if udev is still initializing or not to keep them from having to worry
> about the actual mechanics.

So I could touch /dev/.rcsysinit and remove that after sysinit completes?
That works for me :)

> And yes, ramfs is always compiled into the kernel these days.

Really?
I see CONFIG_RAMFS=y and CONFIG_TMPFS=y in my 2.6.15 kernel config so I was 
under the impression that they're optional.

Thanks

-- 
Roy Marples <uberlord@gentoo.org>
Gentoo Linux Developer

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Cold plug starting services too soon

[Greg KH]

On Thu, Jan 19, 2006 at 06:35:53PM +0000, Roy Marples wrote:
> On Thursday 19 January 2006 18:19, Greg KH wrote:
> > No, I mean the ramfs/tmpfs that udev uses to hold it's /dev tree and
> > database.  I can provide a function that the rc scripts can call to see
> > if udev is still initializing or not to keep them from having to worry
> > about the actual mechanics.
> 
> So I could touch /dev/.rcsysinit and remove that after sysinit completes?
> That works for me :)

Yup.

> > And yes, ramfs is always compiled into the kernel these days.
> 
> Really?
> I see CONFIG_RAMFS=y and CONFIG_TMPFS=y in my 2.6.15 kernel config so I was 
> under the impression that they're optional.

Hm, I thought that the build system always forced ramfs on, due to boot
issues, but in looking at the Kconfig system, I can't see that
dependancy anymore.  So you are right.

But udev on Gentoo will not work without that, so we don't really need
to worry about this :)

thanks,

greg k-h


-------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc. Do you grep through log files
for problems?  Stop!  Download the new AJAX search engine that makes
searching your log files as easy as surfing the  web.  DOWNLOAD SPLUNK!
http://sel.as-us.falkag.net/sel?cmd=lnk&kid\x103432&bid#0486&dat\x121642
_______________________________________________
Linux-hotplug-devel mailing list  http://linux-hotplug.sourceforge.net
Linux-hotplug-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/linux-hotplug-devel