do_clean failure on NFS drive

do_clean failure on NFS drive

[Michael Fainstein]

Recently I moved my working environment from local drive to NFS and since then do_clean task is failing all the time with error:
ERROR: Error executing a python function in ......:
OSError: [Errno 39] Directory not empty: '....../temp'

However, when I look at temp directory it is empty!

I put the following watcher on this directory:
SNAP0="xx";while [ 1 ]; do SNAP=`ls -a tmp/work/ppce500v2-fsl-linux-gnuspe/elfutils-0.125-r4/temp/`;if [ x"${SNAP}" != x"${SNAP0}" ];then echo "`date`";echo "${SNAP}";fi;SNAP0="${SNAP}";done

And got the following:
Tue Apr 23 08:28:38 IDT 2013
./
../
Tue Apr 23 08:28:51 IDT 2013
./
../
log.do_clean@
log.do_clean.572
Tue Apr 23 08:28:51 IDT 2013
./
../
log.do_clean.572
run.do_clean.572
Tue Apr 23 08:28:51 IDT 2013
./
../
.nfs000000000210f27700000eb6
Tue Apr 23 08:28:51 IDT 2013
./
../


It looks like do_clean have a log file open when it tries to remove temp directory. It doesn't interfere with removing directory on local drive, however on NFS it does. If you remove open file on NFS, the file stays in the directory with .nfsXXXXXX name till it is closed and only then it is removed (i.e. when do_clean exits). This file triggers the exception "Directory not empty".
Any suggestions how to solve this? Is this solved in latest version?
I am using Freescale's SDK 1.3 QorIQ-SDK-V1.3-20121114-yocto that is based on Yocto version 1.2.1

Thanks,
Michael



This e-mail message is intended for the recipient only and contains information which is CONFIDENTIAL and which may be proprietary to ECI Telecom. If you have received this transmission in error, please inform us by e-mail, phone or fax, and then delete the original and all copies thereof.

Re: do_clean failure on NFS drive

[Paul Eggleton]

On Tuesday 23 April 2013 06:10:05 Michael Fainstein wrote:
> Recently I moved my working environment from local drive to NFS and since
> then do_clean task is failing all the time with error: ERROR: Error
> executing a python function in ......:
> OSError: [Errno 39] Directory not empty: '....../temp'
> 
> However, when I look at temp directory it is empty!
> 
> I put the following watcher on this directory:
> SNAP0="xx";while [ 1 ]; do SNAP=`ls -a
> tmp/work/ppce500v2-fsl-linux-gnuspe/elfutils-0.125-r4/temp/`;if [
> x"${SNAP}" != x"${SNAP0}" ];then echo "`date`";echo
> "${SNAP}";fi;SNAP0="${SNAP}";done
> 
> And got the following:
> Tue Apr 23 08:28:38 IDT 2013
> ./
> ../
> Tue Apr 23 08:28:51 IDT 2013
> ./
> ../
> log.do_clean@
> log.do_clean.572
> Tue Apr 23 08:28:51 IDT 2013
> ./
> ../
> log.do_clean.572
> run.do_clean.572
> Tue Apr 23 08:28:51 IDT 2013
> ./
> ../
> .nfs000000000210f27700000eb6
> Tue Apr 23 08:28:51 IDT 2013
> ./
> ../
> 
> 
> It looks like do_clean have a log file open when it tries to remove temp
> directory. It doesn't interfere with removing directory on local drive,
> however on NFS it does. If you remove open file on NFS, the file stays in
> the directory with .nfsXXXXXX name till it is closed and only then it is
> removed (i.e. when do_clean exits). This file triggers the exception
> "Directory not empty". Any suggestions how to solve this? Is this solved in
> latest version? I am using Freescale's SDK 1.3
> QorIQ-SDK-V1.3-20121114-yocto that is based on Yocto version 1.2.1

I have to say I don't think we support having TMPDIR on NFS, particularly 
because of cases like this where NFS does not behave in the same way as a 
standard filesystem. Aside from our own code, we'd have to take care of any 
subtle issues in the build systems of every piece of upstream software being 
built, and that's a bit too much to support.

However, if you or someone else can figure out how to fix this specific problem 
in a reasonable manner and send a patch, it would probably be accepted; but 
AFAIK we still recommend not using NFS for this.

Cheers,
Paul

-- 

Paul Eggleton
Intel Open Source Technology Centre

Re: do_clean failure on NFS drive

[Michael Fainstein]

> -----Original Message-----
> From: Paul Eggleton [mailto:paul.eggleton@linux.intel.com]
> Sent: יום ד 24 אפריל 2013 13:13
> To: Michael Fainstein
> Cc: yocto@yoctoproject.org
> Subject: Re: [yocto] do_clean failure on NFS drive
>
> On Tuesday 23 April 2013 06:10:05 Michael Fainstein wrote:
> > Recently I moved my working environment from local drive to NFS and since
> > then do_clean task is failing all the time with error: ERROR: Error
> > executing a python function in ......:
> > OSError: [Errno 39] Directory not empty: '....../temp'
> >
> > However, when I look at temp directory it is empty!
> >
> > I put the following watcher on this directory:
> > SNAP0="xx";while [ 1 ]; do SNAP=`ls -a
> > tmp/work/ppce500v2-fsl-linux-gnuspe/elfutils-0.125-r4/temp/`;if [
> > x"${SNAP}" != x"${SNAP0}" ];then echo "`date`";echo
> > "${SNAP}";fi;SNAP0="${SNAP}";done
> >
> > And got the following:
> > Tue Apr 23 08:28:38 IDT 2013
> > ./
> > ../
> > Tue Apr 23 08:28:51 IDT 2013
> > ./
> > ../
> > log.do_clean@
> > log.do_clean.572
> > Tue Apr 23 08:28:51 IDT 2013
> > ./
> > ../
> > log.do_clean.572
> > run.do_clean.572
> > Tue Apr 23 08:28:51 IDT 2013
> > ./
> > ../
> > .nfs000000000210f27700000eb6
> > Tue Apr 23 08:28:51 IDT 2013
> > ./
> > ../
> >
> >
> > It looks like do_clean have a log file open when it tries to remove temp
> > directory. It doesn't interfere with removing directory on local drive,
> > however on NFS it does. If you remove open file on NFS, the file stays in
> > the directory with .nfsXXXXXX name till it is closed and only then it is
> > removed (i.e. when do_clean exits). This file triggers the exception
> > "Directory not empty". Any suggestions how to solve this? Is this solved
> in
> > latest version? I am using Freescale's SDK 1.3
> > QorIQ-SDK-V1.3-20121114-yocto that is based on Yocto version 1.2.1
>
> I have to say I don't think we support having TMPDIR on NFS, particularly
> because of cases like this where NFS does not behave in the same way as a
> standard filesystem. Aside from our own code, we'd have to take care of any
> subtle issues in the build systems of every piece of upstream software
> being
> built, and that's a bit too much to support.
>
> However, if you or someone else can figure out how to fix this specific
> problem
> in a reasonable manner and send a patch, it would probably be accepted; but
> AFAIK we still recommend not using NFS for this.
>
> Cheers,
> Paul
>

It looks like this issue was already solved by Richard Purdie in file meta/classes/utility-tasks.bbclass in change "utility-tasks: Set T to alternate location during do_clean".
If do_clean's log is in alternate location then nothing interferes with oe.path.remove of temp directory under ${WORKDIR}.
This fix is probably much better than my own fix: I replaced in /utility-tasks.bbclass  function do_clean "oe.path.remove(dir)" for ${WORKDIR} by "os.system("rm -rf %s" % (dir))".

Cheers,
Michael



This e-mail message is intended for the recipient only and contains information which is CONFIDENTIAL and which may be proprietary to ECI Telecom. If you have received this transmission in error, please inform us by e-mail, phone or fax, and then delete the original and all copies thereof.