Race condition? /tmp/net.ifaces and pre-pivot

Race condition? /tmp/net.ifaces and pre-pivot

[Warren Togami]

I am running into an odd issue that is supposedto be impossible.

pre-pivot/20write-ifcfg.sh is often being run but fails to write 
net.*.ifcfg files because /tmp/net.ifaces does not exist at that moment. 
  The mount was otherwise successful, and rdbreak before switch_root 
sees that /tmp/net.ifaces exists.

With plain "root=dhcp" with a NFS rootfs, it seems to never happen. 
However with "root=dhcp bridge", perhaps 25-75% of the time it is 
failing to write ifcfg files.

Some kind of race going on?  An inspection of the code seems to me that 
/tmp/net.ifaces should have already been created prior to pre-pivot?

Warren Togami
wtogami-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
--
To unsubscribe from this list: send the line "unsubscribe initramfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Race condition? /tmp/net.ifaces and pre-pivot

[Warren Togami]

On 07/23/2009 05:37 PM, Warren Togami wrote:
> I am running into an odd issue that is supposedto be impossible.
>
> pre-pivot/20write-ifcfg.sh is often being run but fails to write
> net.*.ifcfg files because /tmp/net.ifaces does not exist at that moment.
> The mount was otherwise successful, and rdbreak before switch_root sees
> that /tmp/net.ifaces exists.
>
> With plain "root=dhcp" with a NFS rootfs, it seems to never happen.
> However with "root=dhcp bridge", perhaps 25-75% of the time it is
> failing to write ifcfg files.
>
> Some kind of race going on? An inspection of the code seems to me that
> /tmp/net.ifaces should have already been created prior to pre-pivot?
>

<dillow> warren: if you don't have any ip= lines on the command line, 
/tmp/net.ifaces gets created by netroot after the handler successfully 
completes
<dillow> warren: I think that races with the check loop in init
<dillow> warren: single processor box?
<dillow> warren: it looks like you could get scheduled away from netroot 
and the init command loop could notice that root is mounted and make it 
to the pre-pivot hook during a single quantum
<dillow> warren: as a test, you could move the line '[ ! -f 
/tmp/net.ifaces ] && echo $netif > /tmp/net.ifaces' in netroot in front 
of the handler call
<dillow> warren: that should work around your issue. I don't think it is 
a long term fix, though -- the multiple NIC case would need some work to 
be clean
<dillow> warren: this would only affect NFS, as the other devices would 
create /dev/root and exit that way to the mount loop
<dillow> so they wouldn't sail through the mount loop due to the 
existence of $NEWROOT/proc
<warren> dillow: single processor qemu VM
<warren> dillow: so this race was happening before, only we didn't 
notice it since we weren't relying on anything
<warren> dillow: any suggestions of a substitution that wont break dash?
<dillow> warren: I think it got introduced with the initqueue stuff or a 
combination of that and changing the loops

Harald, any ideas?

Warren Togami
wtogami-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
--
To unsubscribe from this list: send the line "unsubscribe initramfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Race condition? /tmp/net.ifaces and pre-pivot

[Harald Hoyer]

On 07/24/2009 05:22 AM, Warren Togami wrote:
> On 07/23/2009 05:37 PM, Warren Togami wrote:
>> I am running into an odd issue that is supposedto be impossible.
>>
>> pre-pivot/20write-ifcfg.sh is often being run but fails to write
>> net.*.ifcfg files because /tmp/net.ifaces does not exist at that moment.
>> The mount was otherwise successful, and rdbreak before switch_root sees
>> that /tmp/net.ifaces exists.
>>
>> With plain "root=dhcp" with a NFS rootfs, it seems to never happen.
>> However with "root=dhcp bridge", perhaps 25-75% of the time it is
>> failing to write ifcfg files.
>>
>> Some kind of race going on? An inspection of the code seems to me that
>> /tmp/net.ifaces should have already been created prior to pre-pivot?
>>
>
> <dillow> warren: if you don't have any ip= lines on the command line,
> /tmp/net.ifaces gets created by netroot after the handler successfully
> completes
> <dillow> warren: I think that races with the check loop in init
> <dillow> warren: single processor box?
> <dillow> warren: it looks like you could get scheduled away from netroot
> and the init command loop could notice that root is mounted and make it
> to the pre-pivot hook during a single quantum
> <dillow> warren: as a test, you could move the line '[ ! -f
> /tmp/net.ifaces ] && echo $netif > /tmp/net.ifaces' in netroot in front
> of the handler call
> <dillow> warren: that should work around your issue. I don't think it is
> a long term fix, though -- the multiple NIC case would need some work to
> be clean
> <dillow> warren: this would only affect NFS, as the other devices would
> create /dev/root and exit that way to the mount loop
> <dillow> so they wouldn't sail through the mount loop due to the
> existence of $NEWROOT/proc
> <warren> dillow: single processor qemu VM
> <warren> dillow: so this race was happening before, only we didn't
> notice it since we weren't relying on anything
> <warren> dillow: any suggestions of a substitution that wont break dash?
> <dillow> warren: I think it got introduced with the initqueue stuff or a
> combination of that and changing the loops
>
> Harald, any ideas?

if you rely on a udev event to complete the solution is to place a

udevadm settle

before doing anything
--
To unsubscribe from this list: send the line "unsubscribe initramfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Race condition? /tmp/net.ifaces and pre-pivot

[Seewer Philippe]

Warren Togami wrote:
> I am running into an odd issue that is supposedto be impossible.
> 
> pre-pivot/20write-ifcfg.sh is often being run but fails to write 
> net.*.ifcfg files because /tmp/net.ifaces does not exist at that moment. 
>  The mount was otherwise successful, and rdbreak before switch_root sees 
> that /tmp/net.ifaces exists.
> 
> With plain "root=dhcp" with a NFS rootfs, it seems to never happen. 
> However with "root=dhcp bridge", perhaps 25-75% of the time it is 
> failing to write ifcfg files.
> 
> Some kind of race going on?  An inspection of the code seems to me that 
> /tmp/net.ifaces should have already been created prior to pre-pivot?

This is not entirely unexpected due to the asynchronousness of 
udev-events and the new initqueue/mount-loop.

The correct solution would be to move the whole call of netroot to the 
initqueue, which we'll have to do anyway when we start handling 
interface timeouts with arp.

> 
> Warren Togami
> wtogami-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org
> -- 
> To unsubscribe from this list: send the line "unsubscribe initramfs" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe initramfs" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html