build failure on ubuntu 64bits development system

build failure on ubuntu 64bits development system

[James Abernathy]

[-- Attachment #1: Type: text/plain, Size: 991 bytes --]

I just built a new development pc and installed Ubuntu 11.10 x64.  I wonder
if there are any new requirements to building Yocto in that environment?  I
got an error right off, but then it complete the first 63 task and stopped
successfully.  error below:

jim@ubuntu:~/poky/build-cdv$ bitbake core-image-sato
Pseudo is not present but is required, building this first before the main
build
Parsing recipes: 100% |#################################################|
Time: 00:00:25
Parsing of 797 .bb files complete (0 cached, 797 parsed). 1037 targets, 22
skipped, 0 masked, 0 errors.
ERROR: Execution of event handler 'run_buildstats' failed
Traceback (most recent call last):
  File "run_buildstats(e)", line 18, in
run_buildstats(e=<bb.event.BuildStarted object at 0x4c338d0>)
  File "buildstats.bbclass", line 21, in
set_device(e=<bb.event.BuildStarted object at 0x4c338d0>)
UnboundLocalError: local variable 'rdev' referenced before assignment


Any ideas?

JIm A

[-- Attachment #2: Type: text/html, Size: 1073 bytes --]

Re: build failure on ubuntu 64bits development system

[James Abernathy]

[-- Attachment #1: Type: text/plain, Size: 1304 bytes --]

On Wed, Jan 18, 2012 at 7:55 AM, James Abernathy <jfabernathy@gmail.com>wrote:

> I just built a new development pc and installed Ubuntu 11.10 x64.  I
> wonder if there are any new requirements to building Yocto in that
> environment?  I got an error right off, but then it complete the first 63
> task and stopped successfully.  error below:
>
> jim@ubuntu:~/poky/build-cdv$ bitbake core-image-sato
> Pseudo is not present but is required, building this first before the main
> build
> Parsing recipes: 100% |#################################################|
> Time: 00:00:25
> Parsing of 797 .bb files complete (0 cached, 797 parsed). 1037 targets, 22
> skipped, 0 masked, 0 errors.
> ERROR: Execution of event handler 'run_buildstats' failed
> Traceback (most recent call last):
>   File "run_buildstats(e)", line 18, in
> run_buildstats(e=<bb.event.BuildStarted object at 0x4c338d0>)
>   File "buildstats.bbclass", line 21, in
> set_device(e=<bb.event.BuildStarted object at 0x4c338d0>)
> UnboundLocalError: local variable 'rdev' referenced before assignment
>
>
> Any ideas?
>
> JIm A
>

I went back and tried using the tarballs for poky edison and cedartrail bsp
and the errors don't occur.  So I'm guessing the issue isn't related to
Ubuntu 32 vs. 64 bit.

Jim A

[-- Attachment #2: Type: text/html, Size: 1602 bytes --]

Re: build failure on ubuntu 64bits development system

[James Abernathy]

[-- Attachment #1: Type: text/plain, Size: 2895 bytes --]

On Wed, Jan 18, 2012 at 9:30 AM, James Abernathy <jfabernathy@gmail.com>wrote:

>
>
> On Wed, Jan 18, 2012 at 7:55 AM, James Abernathy <jfabernathy@gmail.com>wrote:
>
>> I just built a new development pc and installed Ubuntu 11.10 x64.  I
>> wonder if there are any new requirements to building Yocto in that
>> environment?  I got an error right off, but then it complete the first 63
>> task and stopped successfully.  error below:
>>
>> jim@ubuntu:~/poky/build-cdv$ bitbake core-image-sato
>> Pseudo is not present but is required, building this first before the
>> main build
>> Parsing recipes: 100% |#################################################|
>> Time: 00:00:25
>> Parsing of 797 .bb files complete (0 cached, 797 parsed). 1037 targets,
>> 22 skipped, 0 masked, 0 errors.
>> ERROR: Execution of event handler 'run_buildstats' failed
>> Traceback (most recent call last):
>>   File "run_buildstats(e)", line 18, in
>> run_buildstats(e=<bb.event.BuildStarted object at 0x4c338d0>)
>>   File "buildstats.bbclass", line 21, in
>> set_device(e=<bb.event.BuildStarted object at 0x4c338d0>)
>> UnboundLocalError: local variable 'rdev' referenced before assignment
>>
>>
>> Any ideas?
>>
>> JIm A
>>
>
> I went back and tried using the tarballs for poky edison and cedartrail
> bsp and the errors don't occur.  So I'm guessing the issue isn't related to
> Ubuntu 32 vs. 64 bit.
>
>
I spoke too soon. Same error in edison tarballs.  I looked at the code and
I can see an place were rdev could go un assigned. If you fell out of the
for loop without passing any of the if conditions, rdev would be
unassigned.  That must be what is happening in Ubuntu 11.10 x64

Anybody building with Ubuntu 11.10 x64?  This doesn't happen on x32

Jim A


def set_device(e):
    tmpdir = bb.data.getVar('TMPDIR', e.data, True)
    try:
        os.remove(bb.data.getVar('DEVFILE', e.data, True))
    except:
        pass

############################################################################
    # We look for the volume TMPDIR lives on. To do all disks would make
little
    # sense and not give us any particularly useful data. In theory we
could do
    # something like stick DL_DIR on a different partition and this would
    # throw stats gathering off. The same goes with SSTATE_DIR. However,
let's
    # get the basics in here and work on the cornercases later.

############################################################################
    device=os.stat(tmpdir)
    majordev=os.major(device.st_dev)
    minordev=os.minor(device.st_dev)
    for line in open("/proc/diskstats", "r"):
        if majordev == int(line.split()[0]) and minordev ==
int(line.split()[1]):
           rdev=line.split()[2]
    file = open(bb.data.getVar('DEVFILE', e.data, True), "w")
    file.write(rdev)
    file.close()


> Jim A
>
>

[-- Attachment #2: Type: text/html, Size: 3736 bytes --]

Re: build failure on ubuntu 64bits development system

[Gary Thomas]

On 2012-01-18 07:42, James Abernathy wrote:
>
>
> On Wed, Jan 18, 2012 at 9:30 AM, James Abernathy <jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>> wrote:
>
>
>
>     On Wed, Jan 18, 2012 at 7:55 AM, James Abernathy <jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>> wrote:
>
>         I just built a new development pc and installed Ubuntu 11.10 x64.  I wonder if there are any new requirements to building Yocto in that environment?  I got an error right
>         off, but then it complete the first 63 task and stopped successfully.  error below:
>
>         jim@ubuntu:~/poky/build-cdv$ bitbake core-image-sato
>         Pseudo is not present but is required, building this first before the main build
>         Parsing recipes: 100% |#################################################| Time: 00:00:25
>         Parsing of 797 .bb files complete (0 cached, 797 parsed). 1037 targets, 22 skipped, 0 masked, 0 errors.
>         ERROR: Execution of event handler 'run_buildstats' failed
>         Traceback (most recent call last):
>            File "run_buildstats(e)", line 18, in run_buildstats(e=<bb.event.BuildStarted object at 0x4c338d0>)
>            File "buildstats.bbclass", line 21, in set_device(e=<bb.event.BuildStarted object at 0x4c338d0>)
>         UnboundLocalError: local variable 'rdev' referenced before assignment
>
>
>         Any ideas?
>
>         JIm A
>
>
>     I went back and tried using the tarballs for poky edison and cedartrail bsp and the errors don't occur.  So I'm guessing the issue isn't related to Ubuntu 32 vs. 64 bit.
>
>
> I spoke too soon. Same error in edison tarballs.  I looked at the code and I can see an place were rdev could go un assigned. If you fell out of the for loop without passing any of
> the if conditions, rdev would be unassigned.  That must be what is happening in Ubuntu 11.10 x64
>
> Anybody building with Ubuntu 11.10 x64?  This doesn't happen on x32
>
> Jim A
>
>
> def set_device(e):
>      tmpdir = bb.data.getVar('TMPDIR', e.data, True)
>      try:
>          os.remove(bb.data.getVar('DEVFILE', e.data, True))
>      except:
>          pass
>      ############################################################################
>      # We look for the volume TMPDIR lives on. To do all disks would make little
>      # sense and not give us any particularly useful data. In theory we could do
>      # something like stick DL_DIR on a different partition and this would
>      # throw stats gathering off. The same goes with SSTATE_DIR. However, let's
>      # get the basics in here and work on the cornercases later.
>      ############################################################################
>      device=os.stat(tmpdir)
>      majordev=os.major(device.st_dev)
>      minordev=os.minor(device.st_dev)
>      for line in open("/proc/diskstats", "r"):
>          if majordev == int(line.split()[0]) and minordev == int(line.split()[1]):
>             rdev=line.split()[2]
>      file = open(bb.data.getVar('DEVFILE', e.data, True), "w")
>      file.write(rdev)
>      file.close()

Can you show what the differences are between /proc/diskstats
on the two systems?  That's obviously what's causing the error.

-- 
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Re: build failure on ubuntu 64bits development system

[William Mills]

On 01/18/2012 09:51 AM, Gary Thomas wrote:
> On 2012-01-18 07:42, James Abernathy wrote:
>>
>>
>> On Wed, Jan 18, 2012 at 9:30 AM, James Abernathy 
>> <jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>> wrote:
>>
>>
>>
>>     On Wed, Jan 18, 2012 at 7:55 AM, James Abernathy 
>> <jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>> wrote:
>>
>>         I just built a new development pc and installed Ubuntu 11.10 
>> x64.  I wonder if there are any new requirements to building Yocto in 
>> that environment?  I got an error right
>>         off, but then it complete the first 63 task and stopped 
>> successfully.  error below:
>>
>>         jim@ubuntu:~/poky/build-cdv$ bitbake core-image-sato
>>         Pseudo is not present but is required, building this first 
>> before the main build
>>         Parsing recipes: 100% 
>> |#################################################| Time: 00:00:25
>>         Parsing of 797 .bb files complete (0 cached, 797 parsed). 
>> 1037 targets, 22 skipped, 0 masked, 0 errors.
>>         ERROR: Execution of event handler 'run_buildstats' failed
>>         Traceback (most recent call last):
>>            File "run_buildstats(e)", line 18, in 
>> run_buildstats(e=<bb.event.BuildStarted object at 0x4c338d0>)
>>            File "buildstats.bbclass", line 21, in 
>> set_device(e=<bb.event.BuildStarted object at 0x4c338d0>)
>>         UnboundLocalError: local variable 'rdev' referenced before 
>> assignment
>>
>>
>>         Any ideas?
>>
>>         JIm A
>>
>>
>>     I went back and tried using the tarballs for poky edison and 
>> cedartrail bsp and the errors don't occur.  So I'm guessing the issue 
>> isn't related to Ubuntu 32 vs. 64 bit.
>>
>>
>> I spoke too soon. Same error in edison tarballs.  I looked at the 
>> code and I can see an place were rdev could go un assigned. If you 
>> fell out of the for loop without passing any of
>> the if conditions, rdev would be unassigned.  That must be what is 
>> happening in Ubuntu 11.10 x64
>>
>> Anybody building with Ubuntu 11.10 x64?  This doesn't happen on x32
>>
>> Jim A
>>
>>
>> def set_device(e):
>>      tmpdir = bb.data.getVar('TMPDIR', e.data, True)
>>      try:
>>          os.remove(bb.data.getVar('DEVFILE', e.data, True))
>>      except:
>>          pass
>>      
>> ############################################################################
>>      # We look for the volume TMPDIR lives on. To do all disks would 
>> make little
>>      # sense and not give us any particularly useful data. In theory 
>> we could do
>>      # something like stick DL_DIR on a different partition and this 
>> would
>>      # throw stats gathering off. The same goes with SSTATE_DIR. 
>> However, let's
>>      # get the basics in here and work on the cornercases later.
>>      
>> ############################################################################
>>      device=os.stat(tmpdir)
>>      majordev=os.major(device.st_dev)
>>      minordev=os.minor(device.st_dev)
>>      for line in open("/proc/diskstats", "r"):
>>          if majordev == int(line.split()[0]) and minordev == 
>> int(line.split()[1]):
>>             rdev=line.split()[2]
>>      file = open(bb.data.getVar('DEVFILE', e.data, True), "w")
>>      file.write(rdev)
>>      file.close()
>
> Can you show what the differences are between /proc/diskstats
> on the two systems?  That's obviously what's causing the error.
>

If your build dir is encyptfs or a fuse device or anything that is not a 
direct block device you will get this error.  This is to be fixed in 
1.1.1 but encyptfs will still have other problems.

build failure on ubuntu 64bits development system

[James Abernathy]

[-- Attachment #1: Type: text/plain, Size: 4313 bytes --]

---------- Forwarded message ----------
From: William Mills <wmills@ti.com>
Date: Wed, Jan 18, 2012 at 9:57 AM
Subject: Re: [yocto] build failure on ubuntu 64bits development system
To: Gary Thomas <gary@mlbassoc.com>
Cc: yocto@yoctoproject.org




On 01/18/2012 09:51 AM, Gary Thomas wrote:

> On 2012-01-18 07:42, James Abernathy wrote:
>
>>
>>
>> On Wed, Jan 18, 2012 at 9:30 AM, James Abernathy <jfabernathy@gmail.com<mailto:
>> jfabernathy@gmail.com>**> wrote:
>>
>>
>>
>>    On Wed, Jan 18, 2012 at 7:55 AM, James Abernathy <
>> jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>**> wrote:
>>
>>        I just built a new development pc and installed Ubuntu 11.10 x64.
>>  I wonder if there are any new requirements to building Yocto in that
>> environment?  I got an error right
>>        off, but then it complete the first 63 task and stopped
>> successfully.  error below:
>>
>>        jim@ubuntu:~/poky/build-cdv$ bitbake core-image-sato
>>        Pseudo is not present but is required, building this first before
>> the main build
>>        Parsing recipes: 100% |#############################**####################|
>> Time: 00:00:25
>>        Parsing of 797 .bb files complete (0 cached, 797 parsed). 1037
>> targets, 22 skipped, 0 masked, 0 errors.
>>        ERROR: Execution of event handler 'run_buildstats' failed
>>        Traceback (most recent call last):
>>           File "run_buildstats(e)", line 18, in
>> run_buildstats(e=<bb.event.**BuildStarted object at 0x4c338d0>)
>>           File "buildstats.bbclass", line 21, in set_device(e=<bb.event.*
>> *BuildStarted object at 0x4c338d0>)
>>        UnboundLocalError: local variable 'rdev' referenced before
>> assignment
>>
>>
>>        Any ideas?
>>
>>        JIm A
>>
>>
>>    I went back and tried using the tarballs for poky edison and
>> cedartrail bsp and the errors don't occur.  So I'm guessing the issue isn't
>> related to Ubuntu 32 vs. 64 bit.
>>
>>
>> I spoke too soon. Same error in edison tarballs.  I looked at the code
>> and I can see an place were rdev could go un assigned. If you fell out of
>> the for loop without passing any of
>> the if conditions, rdev would be unassigned.  That must be what is
>> happening in Ubuntu 11.10 x64
>>
>> Anybody building with Ubuntu 11.10 x64?  This doesn't happen on x32
>>
>> Jim A
>>
>>
>> def set_device(e):
>>     tmpdir = bb.data.getVar('TMPDIR', e.data, True)
>>     try:
>>         os.remove(bb.data.getVar('**DEVFILE', e.data, True))
>>     except:
>>         pass
>>     ##############################**##############################**
>> ################
>>     # We look for the volume TMPDIR lives on. To do all disks would make
>> little
>>     # sense and not give us any particularly useful data. In theory we
>> could do
>>     # something like stick DL_DIR on a different partition and this would
>>     # throw stats gathering off. The same goes with SSTATE_DIR. However,
>> let's
>>     # get the basics in here and work on the cornercases later.
>>     ##############################**##############################**
>> ################
>>     device=os.stat(tmpdir)
>>     majordev=os.major(device.st_**dev)
>>     minordev=os.minor(device.st_**dev)
>>     for line in open("/proc/diskstats", "r"):
>>         if majordev == int(line.split()[0]) and minordev ==
>> int(line.split()[1]):
>>            rdev=line.split()[2]
>>     file = open(bb.data.getVar('DEVFILE', e.data, True), "w")
>>     file.write(rdev)
>>     file.close()
>>
>
> Can you show what the differences are between /proc/diskstats
> on the two systems?  That's obviously what's causing the error.
>
>
If your build dir is encyptfs or a fuse device or anything that is not a
direct block device you will get this error.  This is to be fixed in 1.1.1
but encyptfs will still have other problems.

I build the Ubuntu 11.10 x64 system with 2 drives setup as Soft RAID 0. I
picked btrfs as the file system for no particular reason.  Should I go back
to ext4 or is RAID 0 the problem?

JIm A

______________________________**_________________
yocto mailing list
yocto@yoctoproject.org
https://lists.yoctoproject.**org/listinfo/yocto<https://lists.yoctoproject.org/listinfo/yocto>

[-- Attachment #2: Type: text/html, Size: 5589 bytes --]

Re: build failure on ubuntu 64bits development system

[William Mills]

On 01/18/2012 10:04 AM, James Abernathy wrote:
>
>
> ---------- Forwarded message ----------
> From: *William Mills* <wmills@ti.com <mailto:wmills@ti.com>>
> Date: Wed, Jan 18, 2012 at 9:57 AM
> Subject: Re: [yocto] build failure on ubuntu 64bits development system
> To: Gary Thomas <gary@mlbassoc.com <mailto:gary@mlbassoc.com>>
> Cc: yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
>
>
>
>
> On 01/18/2012 09:51 AM, Gary Thomas wrote:
>
>     On 2012-01-18 07 <tel:2012-01-18%2007>:42, James Abernathy wrote:
>
>
>
>         On Wed, Jan 18, 2012 at 9:30 AM, James Abernathy
>         <jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>         <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>>__>
>         wrote:
>
>
>
>         On Wed, Jan 18, 2012 at 7:55 AM, James Abernathy
>         <jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>         <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>>__>
>         wrote:
>
>         I just built a new development pc and installed Ubuntu 11.10
>         x64. I wonder if there are any new requirements to building
>         Yocto in that environment? I got an error right
>         off, but then it complete the first 63 task and stopped
>         successfully. error below:
>
>         jim@ubuntu:~/poky/build-cdv$ bitbake core-image-sato
>         Pseudo is not present but is required, building this first
>         before the main build
>         Parsing recipes: 100%
>         |#############################__####################| Time: 00:00:25
>         Parsing of 797 .bb files complete (0 cached, 797 parsed). 1037
>         targets, 22 skipped, 0 masked, 0 errors.
>         ERROR: Execution of event handler 'run_buildstats' failed
>         Traceback (most recent call last):
>         File "run_buildstats(e)", line 18, in
>         run_buildstats(e=<bb.event.__BuildStarted object at 0x4c338d0>)
>         File "buildstats.bbclass", line 21, in
>         set_device(e=<bb.event.__BuildStarted object at 0x4c338d0>)
>         UnboundLocalError: local variable 'rdev' referenced before
>         assignment
>
>
>         Any ideas?
>
>         JIm A
>
>
>         I went back and tried using the tarballs for poky edison and
>         cedartrail bsp and the errors don't occur. So I'm guessing the
>         issue isn't related to Ubuntu 32 vs. 64 bit.
>
>
>         I spoke too soon. Same error in edison tarballs. I looked at the
>         code and I can see an place were rdev could go un assigned. If
>         you fell out of the for loop without passing any of
>         the if conditions, rdev would be unassigned. That must be what
>         is happening in Ubuntu 11.10 x64
>
>         Anybody building with Ubuntu 11.10 x64? This doesn't happen on x32
>
>         Jim A
>
>
>         def set_device(e):
>         tmpdir = bb.data.getVar('TMPDIR', e.data, True)
>         try:
>         os.remove(bb.data.getVar('__DEVFILE', e.data, True))
>         except:
>         pass
>         ##############################__##############################__################
>         # We look for the volume TMPDIR lives on. To do all disks would
>         make little
>         # sense and not give us any particularly useful data. In theory
>         we could do
>         # something like stick DL_DIR on a different partition and this
>         would
>         # throw stats gathering off. The same goes with SSTATE_DIR.
>         However, let's
>         # get the basics in here and work on the cornercases later.
>         ##############################__##############################__################
>         device=os.stat(tmpdir)
>         majordev=os.major(device.st___dev)
>         minordev=os.minor(device.st___dev)
>         for line in open("/proc/diskstats", "r"):
>         if majordev == int(line.split()[0]) and minordev ==
>         int(line.split()[1]):
>         rdev=line.split()[2]
>         file = open(bb.data.getVar('DEVFILE', e.data, True), "w")
>         file.write(rdev)
>         file.close()
>
>
>     Can you show what the differences are between /proc/diskstats
>     on the two systems? That's obviously what's causing the error.
>
>
> If your build dir is encyptfs or a fuse device or anything that is not a
> direct block device you will get this error. This is to be fixed in
> 1.1.1 but encyptfs will still have other problems.
>
> I build the Ubuntu 11.10 x64 system with 2 drives setup as Soft RAID 0.
> I picked btrfs as the file system for no particular reason. Should I go
> back to ext4 or is RAID 0 the problem?

No, I would not do that yet.  I would think software RAID would present 
a block device so would not trigger this error.

 >  9       0 md0 133691 0 2218832 0 67133 0 5629616 0 0 0 0
 >   9       1 md1 235 0 1880 0 0 0 0 0 0 0 0

Your build dir is in md0 or md1 (wrt your other post)

>
> JIm A
>
> _________________________________________________
> yocto mailing list
> yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
> https://lists.yoctoproject.__org/listinfo/yocto
> <https://lists.yoctoproject.org/listinfo/yocto>
>
>
>
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto

Re: build failure on ubuntu 64bits development system

[James Abernathy]

[-- Attachment #1: Type: text/plain, Size: 6083 bytes --]

On Wed, Jan 18, 2012 at 10:15 AM, William Mills <wmills@ti.com> wrote:

>
>
> On 01/18/2012 10:04 AM, James Abernathy wrote:
>
>>
>>
>> ---------- Forwarded message ----------
>> From: *William Mills* <wmills@ti.com <mailto:wmills@ti.com>>
>> Date: Wed, Jan 18, 2012 at 9:57 AM
>> Subject: Re: [yocto] build failure on ubuntu 64bits development system
>> To: Gary Thomas <gary@mlbassoc.com <mailto:gary@mlbassoc.com>>
>> Cc: yocto@yoctoproject.org <mailto:yocto@yoctoproject.org**>
>>
>>
>>
>>
>> On 01/18/2012 09:51 AM, Gary Thomas wrote:
>>
>>    On 2012-01-18 07 <tel:2012-01-18%2007>:42, James Abernathy wrote:
>>
>>
>>
>>        On Wed, Jan 18, 2012 at 9:30 AM, James Abernathy
>>        <jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>>        <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>**>__>
>>
>>        wrote:
>>
>>
>>
>>        On Wed, Jan 18, 2012 at 7:55 AM, James Abernathy
>>        <jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>>        <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>**>__>
>>
>>        wrote:
>>
>>        I just built a new development pc and installed Ubuntu 11.10
>>        x64. I wonder if there are any new requirements to building
>>        Yocto in that environment? I got an error right
>>        off, but then it complete the first 63 task and stopped
>>        successfully. error below:
>>
>>        jim@ubuntu:~/poky/build-cdv$ bitbake core-image-sato
>>        Pseudo is not present but is required, building this first
>>        before the main build
>>        Parsing recipes: 100%
>>        |#############################**__####################| Time:
>> 00:00:25
>>
>>        Parsing of 797 .bb files complete (0 cached, 797 parsed). 1037
>>        targets, 22 skipped, 0 masked, 0 errors.
>>        ERROR: Execution of event handler 'run_buildstats' failed
>>        Traceback (most recent call last):
>>        File "run_buildstats(e)", line 18, in
>>        run_buildstats(e=<bb.event.__**BuildStarted object at 0x4c338d0>)
>>
>>        File "buildstats.bbclass", line 21, in
>>        set_device(e=<bb.event.__**BuildStarted object at 0x4c338d0>)
>>
>>        UnboundLocalError: local variable 'rdev' referenced before
>>        assignment
>>
>>
>>        Any ideas?
>>
>>        JIm A
>>
>>
>>        I went back and tried using the tarballs for poky edison and
>>        cedartrail bsp and the errors don't occur. So I'm guessing the
>>        issue isn't related to Ubuntu 32 vs. 64 bit.
>>
>>
>>        I spoke too soon. Same error in edison tarballs. I looked at the
>>        code and I can see an place were rdev could go un assigned. If
>>        you fell out of the for loop without passing any of
>>        the if conditions, rdev would be unassigned. That must be what
>>        is happening in Ubuntu 11.10 x64
>>
>>        Anybody building with Ubuntu 11.10 x64? This doesn't happen on x32
>>
>>        Jim A
>>
>>
>>        def set_device(e):
>>        tmpdir = bb.data.getVar('TMPDIR', e.data, True)
>>        try:
>>        os.remove(bb.data.getVar('__**DEVFILE', e.data, True))
>>        except:
>>        pass
>>        ##############################**__############################**
>> ##__################
>>
>>        # We look for the volume TMPDIR lives on. To do all disks would
>>        make little
>>        # sense and not give us any particularly useful data. In theory
>>        we could do
>>        # something like stick DL_DIR on a different partition and this
>>        would
>>        # throw stats gathering off. The same goes with SSTATE_DIR.
>>        However, let's
>>        # get the basics in here and work on the cornercases later.
>>        ##############################**__############################**
>> ##__################
>>        device=os.stat(tmpdir)
>>        majordev=os.major(device.st___**dev)
>>        minordev=os.minor(device.st___**dev)
>>
>>        for line in open("/proc/diskstats", "r"):
>>        if majordev == int(line.split()[0]) and minordev ==
>>        int(line.split()[1]):
>>        rdev=line.split()[2]
>>        file = open(bb.data.getVar('DEVFILE', e.data, True), "w")
>>        file.write(rdev)
>>        file.close()
>>
>>
>>    Can you show what the differences are between /proc/diskstats
>>    on the two systems? That's obviously what's causing the error.
>>
>>
>> If your build dir is encyptfs or a fuse device or anything that is not a
>> direct block device you will get this error. This is to be fixed in
>> 1.1.1 but encyptfs will still have other problems.
>>
>> I build the Ubuntu 11.10 x64 system with 2 drives setup as Soft RAID 0.
>> I picked btrfs as the file system for no particular reason. Should I go
>> back to ext4 or is RAID 0 the problem?
>>
>
> No, I would not do that yet.  I would think software RAID would present a
> block device so would not trigger this error.
>
> I was hoping to use RAID 0 for speed. I have a I7 2700K on a DZ68DB with 2
6Gb/s ports matched to 2 6Gb/s 7200 hard drives.  Since the builds take so
long, I was looking for an edge.

So are there any recommendations at this point?  I'm assuming that the
default ext4 directly on the SATA drive is a fall back position.

Advice?

Jim A


> >  9       0 md0 133691 0 2218832 0 67133 0 5629616 0 0 0 0
> >   9       1 md1 235 0 1880 0 0 0 0 0 0 0 0
>
> Your build dir is in md0 or md1 (wrt your other post)
>
>
>> JIm A
>>
>> ______________________________**___________________
>> yocto mailing list
>> yocto@yoctoproject.org <mailto:yocto@yoctoproject.org**>
>> https://lists.yoctoproject.__**org/listinfo/yocto
>> <https://lists.yoctoproject.**org/listinfo/yocto<https://lists.yoctoproject.org/listinfo/yocto>
>> >
>>
>>
>>
>>
>> ______________________________**_________________
>> yocto mailing list
>> yocto@yoctoproject.org
>> https://lists.yoctoproject.**org/listinfo/yocto<https://lists.yoctoproject.org/listinfo/yocto>
>>
>

[-- Attachment #2: Type: text/html, Size: 8715 bytes --]

Re: build failure on ubuntu 64bits development system

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 6805 bytes --]

On Wed, Jan 18, 2012 at 10:25:18AM -0500, James Abernathy wrote:
> On Wed, Jan 18, 2012 at 10:15 AM, William Mills <wmills@ti.com> wrote:
> 
> >
> >
> > On 01/18/2012 10:04 AM, James Abernathy wrote:
> >
> >>
> >>
> >> ---------- Forwarded message ----------
> >> From: *William Mills* <wmills@ti.com <mailto:wmills@ti.com>>
> >> Date: Wed, Jan 18, 2012 at 9:57 AM
> >> Subject: Re: [yocto] build failure on ubuntu 64bits development system
> >> To: Gary Thomas <gary@mlbassoc.com <mailto:gary@mlbassoc.com>>
> >> Cc: yocto@yoctoproject.org <mailto:yocto@yoctoproject.org**>
> >>
> >>
> >>
> >>
> >> On 01/18/2012 09:51 AM, Gary Thomas wrote:
> >>
> >>    On 2012-01-18 07 <tel:2012-01-18%2007>:42, James Abernathy wrote:
> >>
> >>
> >>
> >>        On Wed, Jan 18, 2012 at 9:30 AM, James Abernathy
> >>        <jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
> >>        <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>**>__>
> >>
> >>        wrote:
> >>
> >>
> >>
> >>        On Wed, Jan 18, 2012 at 7:55 AM, James Abernathy
> >>        <jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
> >>        <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>**>__>
> >>
> >>        wrote:
> >>
> >>        I just built a new development pc and installed Ubuntu 11.10
> >>        x64. I wonder if there are any new requirements to building
> >>        Yocto in that environment? I got an error right
> >>        off, but then it complete the first 63 task and stopped
> >>        successfully. error below:
> >>
> >>        jim@ubuntu:~/poky/build-cdv$ bitbake core-image-sato
> >>        Pseudo is not present but is required, building this first
> >>        before the main build
> >>        Parsing recipes: 100%
> >>        |#############################**__####################| Time:
> >> 00:00:25
> >>
> >>        Parsing of 797 .bb files complete (0 cached, 797 parsed). 1037
> >>        targets, 22 skipped, 0 masked, 0 errors.
> >>        ERROR: Execution of event handler 'run_buildstats' failed
> >>        Traceback (most recent call last):
> >>        File "run_buildstats(e)", line 18, in
> >>        run_buildstats(e=<bb.event.__**BuildStarted object at 0x4c338d0>)
> >>
> >>        File "buildstats.bbclass", line 21, in
> >>        set_device(e=<bb.event.__**BuildStarted object at 0x4c338d0>)
> >>
> >>        UnboundLocalError: local variable 'rdev' referenced before
> >>        assignment
> >>
> >>
> >>        Any ideas?
> >>
> >>        JIm A
> >>
> >>
> >>        I went back and tried using the tarballs for poky edison and
> >>        cedartrail bsp and the errors don't occur. So I'm guessing the
> >>        issue isn't related to Ubuntu 32 vs. 64 bit.
> >>
> >>
> >>        I spoke too soon. Same error in edison tarballs. I looked at the
> >>        code and I can see an place were rdev could go un assigned. If
> >>        you fell out of the for loop without passing any of
> >>        the if conditions, rdev would be unassigned. That must be what
> >>        is happening in Ubuntu 11.10 x64
> >>
> >>        Anybody building with Ubuntu 11.10 x64? This doesn't happen on x32
> >>
> >>        Jim A
> >>
> >>
> >>        def set_device(e):
> >>        tmpdir = bb.data.getVar('TMPDIR', e.data, True)
> >>        try:
> >>        os.remove(bb.data.getVar('__**DEVFILE', e.data, True))
> >>        except:
> >>        pass
> >>        ##############################**__############################**
> >> ##__################
> >>
> >>        # We look for the volume TMPDIR lives on. To do all disks would
> >>        make little
> >>        # sense and not give us any particularly useful data. In theory
> >>        we could do
> >>        # something like stick DL_DIR on a different partition and this
> >>        would
> >>        # throw stats gathering off. The same goes with SSTATE_DIR.
> >>        However, let's
> >>        # get the basics in here and work on the cornercases later.
> >>        ##############################**__############################**
> >> ##__################
> >>        device=os.stat(tmpdir)
> >>        majordev=os.major(device.st___**dev)
> >>        minordev=os.minor(device.st___**dev)
> >>
> >>        for line in open("/proc/diskstats", "r"):
> >>        if majordev == int(line.split()[0]) and minordev ==
> >>        int(line.split()[1]):
> >>        rdev=line.split()[2]
> >>        file = open(bb.data.getVar('DEVFILE', e.data, True), "w")
> >>        file.write(rdev)
> >>        file.close()
> >>
> >>
> >>    Can you show what the differences are between /proc/diskstats
> >>    on the two systems? That's obviously what's causing the error.
> >>
> >>
> >> If your build dir is encyptfs or a fuse device or anything that is not a
> >> direct block device you will get this error. This is to be fixed in
> >> 1.1.1 but encyptfs will still have other problems.
> >>
> >> I build the Ubuntu 11.10 x64 system with 2 drives setup as Soft RAID 0.
> >> I picked btrfs as the file system for no particular reason. Should I go
> >> back to ext4 or is RAID 0 the problem?
> >>
> >
> > No, I would not do that yet.  I would think software RAID would present a
> > block device so would not trigger this error.
> >
> > I was hoping to use RAID 0 for speed. I have a I7 2700K on a DZ68DB with 2
> 6Gb/s ports matched to 2 6Gb/s 7200 hard drives.  Since the builds take so
> long, I was looking for an edge.
> 
> So are there any recommendations at this point?  I'm assuming that the
> default ext4 directly on the SATA drive is a fall back position.

FWIW: I'm using sw (mdadm) RAID0 (3 SATA2 drivers) for WORKDIR without any issues.

Cheers,

> 
> Advice?
> 
> Jim A
> 
> 
> > >  9       0 md0 133691 0 2218832 0 67133 0 5629616 0 0 0 0
> > >   9       1 md1 235 0 1880 0 0 0 0 0 0 0 0
> >
> > Your build dir is in md0 or md1 (wrt your other post)
> >
> >
> >> JIm A
> >>
> >> ______________________________**___________________
> >> yocto mailing list
> >> yocto@yoctoproject.org <mailto:yocto@yoctoproject.org**>
> >> https://lists.yoctoproject.__**org/listinfo/yocto
> >> <https://lists.yoctoproject.**org/listinfo/yocto<https://lists.yoctoproject.org/listinfo/yocto>
> >> >
> >>
> >>
> >>
> >>
> >> ______________________________**_________________
> >> yocto mailing list
> >> yocto@yoctoproject.org
> >> https://lists.yoctoproject.**org/listinfo/yocto<https://lists.yoctoproject.org/listinfo/yocto>
> >>
> >

> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto


-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: build failure on ubuntu 64bits development system

[William Mills]

On 01/18/2012 10:25 AM, James Abernathy wrote:
>
>
> On Wed, Jan 18, 2012 at 10:15 AM, William Mills <wmills@ti.com
> <mailto:wmills@ti.com>> wrote:
>
>
>
>     On 01/18/2012 10:04 AM, James Abernathy wrote:
>
>
>
>         ---------- Forwarded message ----------
>         From: *William Mills* <wmills@ti.com <mailto:wmills@ti.com>
>         <mailto:wmills@ti.com <mailto:wmills@ti.com>>>
>         Date: Wed, Jan 18, 2012 at 9:57 AM
>         Subject: Re: [yocto] build failure on ubuntu 64bits development
>         system
>         To: Gary Thomas <gary@mlbassoc.com <mailto:gary@mlbassoc.com>
>         <mailto:gary@mlbassoc.com <mailto:gary@mlbassoc.com>>>
>         Cc: yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
>         <mailto:yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>__>
>
>
>
>
>         On 01/18/2012 09:51 AM, Gary Thomas wrote:
>
>         On 2012-01-18 07 <tel:2012-01-18%2007> <tel:2012-01-18%2007>:42,
>         James Abernathy wrote:
>
>
>
>         On Wed, Jan 18, 2012 at 9:30 AM, James Abernathy
>         <jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>         <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>>
>         <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>         <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>>__>__>
>
>         wrote:
>
>
>
>         On Wed, Jan 18, 2012 at 7:55 AM, James Abernathy
>         <jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>         <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>>
>         <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>         <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>>__>__>
>
>         wrote:
>
>         I just built a new development pc and installed Ubuntu 11.10
>         x64. I wonder if there are any new requirements to building
>         Yocto in that environment? I got an error right
>         off, but then it complete the first 63 task and stopped
>         successfully. error below:
>
>         jim@ubuntu:~/poky/build-cdv$ bitbake core-image-sato
>         Pseudo is not present but is required, building this first
>         before the main build
>         Parsing recipes: 100%
>         |#############################____####################| Time:
>         00:00:25
>
>         Parsing of 797 .bb files complete (0 cached, 797 parsed). 1037
>         targets, 22 skipped, 0 masked, 0 errors.
>         ERROR: Execution of event handler 'run_buildstats' failed
>         Traceback (most recent call last):
>         File "run_buildstats(e)", line 18, in
>         run_buildstats(e=<bb.event.____BuildStarted object at 0x4c338d0>)
>
>         File "buildstats.bbclass", line 21, in
>         set_device(e=<bb.event.____BuildStarted object at 0x4c338d0>)
>
>         UnboundLocalError: local variable 'rdev' referenced before
>         assignment
>
>
>         Any ideas?
>
>         JIm A
>
>
>         I went back and tried using the tarballs for poky edison and
>         cedartrail bsp and the errors don't occur. So I'm guessing the
>         issue isn't related to Ubuntu 32 vs. 64 bit.
>
>
>         I spoke too soon. Same error in edison tarballs. I looked at the
>         code and I can see an place were rdev could go un assigned. If
>         you fell out of the for loop without passing any of
>         the if conditions, rdev would be unassigned. That must be what
>         is happening in Ubuntu 11.10 x64
>
>         Anybody building with Ubuntu 11.10 x64? This doesn't happen on x32
>
>         Jim A
>
>
>         def set_device(e):
>         tmpdir = bb.data.getVar('TMPDIR', e.data, True)
>         try:
>         os.remove(bb.data.getVar('____DEVFILE', e.data, True))
>         except:
>         pass
>         ##############################____############################__##__################
>
>         # We look for the volume TMPDIR lives on. To do all disks would
>         make little
>         # sense and not give us any particularly useful data. In theory
>         we could do
>         # something like stick DL_DIR on a different partition and this
>         would
>         # throw stats gathering off. The same goes with SSTATE_DIR.
>         However, let's
>         # get the basics in here and work on the cornercases later.
>         ##############################____############################__##__################
>         device=os.stat(tmpdir)
>         majordev=os.major(device.st_____dev)
>         minordev=os.minor(device.st_____dev)
>
>         for line in open("/proc/diskstats", "r"):
>         if majordev == int(line.split()[0]) and minordev ==
>         int(line.split()[1]):
>         rdev=line.split()[2]
>         file = open(bb.data.getVar('DEVFILE', e.data, True), "w")
>         file.write(rdev)
>         file.close()
>
>
>         Can you show what the differences are between /proc/diskstats
>         on the two systems? That's obviously what's causing the error.
>
>
>         If your build dir is encyptfs or a fuse device or anything that
>         is not a
>         direct block device you will get this error. This is to be fixed in
>         1.1.1 but encyptfs will still have other problems.
>
>         I build the Ubuntu 11.10 x64 system with 2 drives setup as Soft
>         RAID 0.
>         I picked btrfs as the file system for no particular reason.
>         Should I go
>         back to ext4 or is RAID 0 the problem?
>
>
>     No, I would not do that yet. I would think software RAID would
>     present a block device so would not trigger this error.
>
> I was hoping to use RAID 0 for speed. I have a I7 2700K on a DZ68DB with
> 2 6Gb/s ports matched to 2 6Gb/s 7200 hard drives. Since the builds take
> so long, I was looking for an edge.
>
> So are there any recommendations at this point? I'm assuming that the
> default ext4 directly on the SATA drive is a fall back position.
>
> Advice?

If it were me, I would instrument (hack) that code above to see what 
part is failing.  Are you getting the right dev major/ minor from the 
stat code or is the /proc/diskstats search code failing.

Alternatively you could try the 1.1.1 branch to see if that fixes it.


>
> Jim A
>
>
>      > 9 0 md0 133691 0 2218832 0 67133 0 5629616 0 0 0 0
>      > 9 1 md1 235 0 1880 0 0 0 0 0 0 0 0
>
>     Your build dir is in md0 or md1 (wrt your other post)
>
>
>         JIm A
>
>         ___________________________________________________
>         yocto mailing list
>         yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
>         <mailto:yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>__>
>         https://lists.yoctoproject.____org/listinfo/yocto
>         <https://lists.yoctoproject.__org/listinfo/yocto
>         <https://lists.yoctoproject.org/listinfo/yocto>>
>
>
>
>
>         _________________________________________________
>         yocto mailing list
>         yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
>         https://lists.yoctoproject.__org/listinfo/yocto
>         <https://lists.yoctoproject.org/listinfo/yocto>
>
>

Re: build failure on ubuntu 64bits development system

[Jim Abernathy]

On 01/18/2012 11:06 AM, William Mills wrote:
>
>
> On 01/18/2012 10:25 AM, James Abernathy wrote:
>>
>>
>> On Wed, Jan 18, 2012 at 10:15 AM, William Mills <wmills@ti.com
>> <mailto:wmills@ti.com>> wrote:
>>
>>
>>
>>     On 01/18/2012 10:04 AM, James Abernathy wrote:
>>
>>
>>
>>         ---------- Forwarded message ----------
>>         From: *William Mills* <wmills@ti.com <mailto:wmills@ti.com>
>> <mailto:wmills@ti.com <mailto:wmills@ti.com>>>
>>         Date: Wed, Jan 18, 2012 at 9:57 AM
>>         Subject: Re: [yocto] build failure on ubuntu 64bits development
>>         system
>>         To: Gary Thomas <gary@mlbassoc.com <mailto:gary@mlbassoc.com>
>> <mailto:gary@mlbassoc.com <mailto:gary@mlbassoc.com>>>
>>         Cc: yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
>> <mailto:yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>__>
>>
>>
>>
>>
>>         On 01/18/2012 09:51 AM, Gary Thomas wrote:
>>
>>         On 2012-01-18 07 <tel:2012-01-18%2007> <tel:2012-01-18%2007>:42,
>>         James Abernathy wrote:
>>
>>
>>
>>         On Wed, Jan 18, 2012 at 9:30 AM, James Abernathy
>> <jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>> <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>>
>> <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>> <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>>__>__>
>>
>>         wrote:
>>
>>
>>
>>         On Wed, Jan 18, 2012 at 7:55 AM, James Abernathy
>> <jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>> <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>>
>> <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>> <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>>__>__>
>>
>>         wrote:
>>
>>         I just built a new development pc and installed Ubuntu 11.10
>>         x64. I wonder if there are any new requirements to building
>>         Yocto in that environment? I got an error right
>>         off, but then it complete the first 63 task and stopped
>>         successfully. error below:
>>
>>         jim@ubuntu:~/poky/build-cdv$ bitbake core-image-sato
>>         Pseudo is not present but is required, building this first
>>         before the main build
>>         Parsing recipes: 100%
>>         |#############################____####################| Time:
>>         00:00:25
>>
>>         Parsing of 797 .bb files complete (0 cached, 797 parsed). 1037
>>         targets, 22 skipped, 0 masked, 0 errors.
>>         ERROR: Execution of event handler 'run_buildstats' failed
>>         Traceback (most recent call last):
>>         File "run_buildstats(e)", line 18, in
>>         run_buildstats(e=<bb.event.____BuildStarted object at 
>> 0x4c338d0>)
>>
>>         File "buildstats.bbclass", line 21, in
>>         set_device(e=<bb.event.____BuildStarted object at 0x4c338d0>)
>>
>>         UnboundLocalError: local variable 'rdev' referenced before
>>         assignment
>>
>>
>>         Any ideas?
>>
>>         JIm A
>>
>>
>>         I went back and tried using the tarballs for poky edison and
>>         cedartrail bsp and the errors don't occur. So I'm guessing the
>>         issue isn't related to Ubuntu 32 vs. 64 bit.
>>
>>
>>         I spoke too soon. Same error in edison tarballs. I looked at the
>>         code and I can see an place were rdev could go un assigned. If
>>         you fell out of the for loop without passing any of
>>         the if conditions, rdev would be unassigned. That must be what
>>         is happening in Ubuntu 11.10 x64
>>
>>         Anybody building with Ubuntu 11.10 x64? This doesn't happen 
>> on x32
>>
>>         Jim A
>>
>>
>>         def set_device(e):
>>         tmpdir = bb.data.getVar('TMPDIR', e.data, True)
>>         try:
>>         os.remove(bb.data.getVar('____DEVFILE', e.data, True))
>>         except:
>>         pass
>>         
>> ##############################____############################__##__################
>>
>>         # We look for the volume TMPDIR lives on. To do all disks would
>>         make little
>>         # sense and not give us any particularly useful data. In theory
>>         we could do
>>         # something like stick DL_DIR on a different partition and this
>>         would
>>         # throw stats gathering off. The same goes with SSTATE_DIR.
>>         However, let's
>>         # get the basics in here and work on the cornercases later.
>>         
>> ##############################____############################__##__################
>>         device=os.stat(tmpdir)
>>         majordev=os.major(device.st_____dev)
>>         minordev=os.minor(device.st_____dev)
>>
>>         for line in open("/proc/diskstats", "r"):
>>         if majordev == int(line.split()[0]) and minordev ==
>>         int(line.split()[1]):
>>         rdev=line.split()[2]
>>         file = open(bb.data.getVar('DEVFILE', e.data, True), "w")
>>         file.write(rdev)
>>         file.close()
>>
>>
>>         Can you show what the differences are between /proc/diskstats
>>         on the two systems? That's obviously what's causing the error.
>>
>>
>>         If your build dir is encyptfs or a fuse device or anything that
>>         is not a
>>         direct block device you will get this error. This is to be 
>> fixed in
>>         1.1.1 but encyptfs will still have other problems.
>>
>>         I build the Ubuntu 11.10 x64 system with 2 drives setup as Soft
>>         RAID 0.
>>         I picked btrfs as the file system for no particular reason.
>>         Should I go
>>         back to ext4 or is RAID 0 the problem?
>>
>>
>>     No, I would not do that yet. I would think software RAID would
>>     present a block device so would not trigger this error.
>>
>> I was hoping to use RAID 0 for speed. I have a I7 2700K on a DZ68DB with
>> 2 6Gb/s ports matched to 2 6Gb/s 7200 hard drives. Since the builds take
>> so long, I was looking for an edge.
>>
>> So are there any recommendations at this point? I'm assuming that the
>> default ext4 directly on the SATA drive is a fall back position.
>>
>> Advice?
>
> If it were me, I would instrument (hack) that code above to see what 
> part is failing.  Are you getting the right dev major/ minor from the 
> stat code or is the /proc/diskstats search code failing.
>
> Alternatively you could try the 1.1.1 branch to see if that fixes it.
>
I'm not sure I'm the right guys to be debugging this :-)
In the interest of my schedule, and since this is a brand new 
workstation, I'm just going to try EXT4 on Soft RAID 0 and see if that 
works.  If not, I'll look at 1.1.1.  I Assume that would be the M4 
release at this time.

Jim A
>
>>
>> Jim A
>>
>>
>> > 9 0 md0 133691 0 2218832 0 67133 0 5629616 0 0 0 0
>> > 9 1 md1 235 0 1880 0 0 0 0 0 0 0 0
>>
>>     Your build dir is in md0 or md1 (wrt your other post)
>>
>>
>>         JIm A
>>
>>         ___________________________________________________
>>         yocto mailing list
>>         yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
>> <mailto:yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>__>
>>         https://lists.yoctoproject.____org/listinfo/yocto
>> <https://lists.yoctoproject.__org/listinfo/yocto
>> <https://lists.yoctoproject.org/listinfo/yocto>>
>>
>>
>>
>>
>>         _________________________________________________
>>         yocto mailing list
>>         yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
>>         https://lists.yoctoproject.__org/listinfo/yocto
>> <https://lists.yoctoproject.org/listinfo/yocto>
>>
>>

Re: build failure on ubuntu 64bits development system

[Saul Wold]

On 01/18/2012 08:17 AM, Jim Abernathy wrote:
> On 01/18/2012 11:06 AM, William Mills wrote:
>>
>>
>> On 01/18/2012 10:25 AM, James Abernathy wrote:
>>>
>>>
>>> On Wed, Jan 18, 2012 at 10:15 AM, William Mills <wmills@ti.com
>>> <mailto:wmills@ti.com>> wrote:
>>>
>>>
>>>
>>> On 01/18/2012 10:04 AM, James Abernathy wrote:
>>>
>>>
>>>
>>> ---------- Forwarded message ----------
>>> From: *William Mills* <wmills@ti.com <mailto:wmills@ti.com>
>>> <mailto:wmills@ti.com <mailto:wmills@ti.com>>>
>>> Date: Wed, Jan 18, 2012 at 9:57 AM
>>> Subject: Re: [yocto] build failure on ubuntu 64bits development
>>> system
>>> To: Gary Thomas <gary@mlbassoc.com <mailto:gary@mlbassoc.com>
>>> <mailto:gary@mlbassoc.com <mailto:gary@mlbassoc.com>>>
>>> Cc: yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
>>> <mailto:yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>__>
>>>
>>>
>>>
>>>
>>> On 01/18/2012 09:51 AM, Gary Thomas wrote:
>>>
>>> On 2012-01-18 07 <tel:2012-01-18%2007> <tel:2012-01-18%2007>:42,
>>> James Abernathy wrote:
>>>
>>>
>>>
>>> On Wed, Jan 18, 2012 at 9:30 AM, James Abernathy
>>> <jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>>> <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>>
>>> <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>>> <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>>__>__>
>>>
>>> wrote:
>>>
>>>
>>>
>>> On Wed, Jan 18, 2012 at 7:55 AM, James Abernathy
>>> <jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>>> <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>>
>>> <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>>> <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>>__>__>
>>>
>>> wrote:
>>>
>>> I just built a new development pc and installed Ubuntu 11.10
>>> x64. I wonder if there are any new requirements to building
>>> Yocto in that environment? I got an error right
>>> off, but then it complete the first 63 task and stopped
>>> successfully. error below:
>>>
>>> jim@ubuntu:~/poky/build-cdv$ bitbake core-image-sato
>>> Pseudo is not present but is required, building this first
>>> before the main build
>>> Parsing recipes: 100%
>>> |#############################____####################| Time:
>>> 00:00:25
>>>
>>> Parsing of 797 .bb files complete (0 cached, 797 parsed). 1037
>>> targets, 22 skipped, 0 masked, 0 errors.
>>> ERROR: Execution of event handler 'run_buildstats' failed
>>> Traceback (most recent call last):
>>> File "run_buildstats(e)", line 18, in
>>> run_buildstats(e=<bb.event.____BuildStarted object at 0x4c338d0>)
>>>
>>> File "buildstats.bbclass", line 21, in
>>> set_device(e=<bb.event.____BuildStarted object at 0x4c338d0>)
>>>
>>> UnboundLocalError: local variable 'rdev' referenced before
>>> assignment
>>>
>>>
>>> Any ideas?
>>>
>>> JIm A
>>>
>>>
>>> I went back and tried using the tarballs for poky edison and
>>> cedartrail bsp and the errors don't occur. So I'm guessing the
>>> issue isn't related to Ubuntu 32 vs. 64 bit.
>>>
>>>
>>> I spoke too soon. Same error in edison tarballs. I looked at the
>>> code and I can see an place were rdev could go un assigned. If
>>> you fell out of the for loop without passing any of
>>> the if conditions, rdev would be unassigned. That must be what
>>> is happening in Ubuntu 11.10 x64
>>>
>>> Anybody building with Ubuntu 11.10 x64? This doesn't happen on x32
>>>
>>> Jim A
>>>
>>>
>>> def set_device(e):
>>> tmpdir = bb.data.getVar('TMPDIR', e.data, True)
>>> try:
>>> os.remove(bb.data.getVar('____DEVFILE', e.data, True))
>>> except:
>>> pass
>>> ##############################____############################__##__################
>>>
>>>
>>> # We look for the volume TMPDIR lives on. To do all disks would
>>> make little
>>> # sense and not give us any particularly useful data. In theory
>>> we could do
>>> # something like stick DL_DIR on a different partition and this
>>> would
>>> # throw stats gathering off. The same goes with SSTATE_DIR.
>>> However, let's
>>> # get the basics in here and work on the cornercases later.
>>> ##############################____############################__##__################
>>>
>>> device=os.stat(tmpdir)
>>> majordev=os.major(device.st_____dev)
>>> minordev=os.minor(device.st_____dev)
>>>
>>> for line in open("/proc/diskstats", "r"):
>>> if majordev == int(line.split()[0]) and minordev ==
>>> int(line.split()[1]):
>>> rdev=line.split()[2]
>>> file = open(bb.data.getVar('DEVFILE', e.data, True), "w")
>>> file.write(rdev)
>>> file.close()
>>>
>>>
>>> Can you show what the differences are between /proc/diskstats
>>> on the two systems? That's obviously what's causing the error.
>>>
>>>
>>> If your build dir is encyptfs or a fuse device or anything that
>>> is not a
>>> direct block device you will get this error. This is to be fixed in
>>> 1.1.1 but encyptfs will still have other problems.
>>>
>>> I build the Ubuntu 11.10 x64 system with 2 drives setup as Soft
>>> RAID 0.
>>> I picked btrfs as the file system for no particular reason.
>>> Should I go
>>> back to ext4 or is RAID 0 the problem?
>>>
>>>
>>> No, I would not do that yet. I would think software RAID would
>>> present a block device so would not trigger this error.
>>>
>>> I was hoping to use RAID 0 for speed. I have a I7 2700K on a DZ68DB with
>>> 2 6Gb/s ports matched to 2 6Gb/s 7200 hard drives. Since the builds take
>>> so long, I was looking for an edge.
>>>
>>> So are there any recommendations at this point? I'm assuming that the
>>> default ext4 directly on the SATA drive is a fall back position.
>>>
>>> Advice?
>>
>> If it were me, I would instrument (hack) that code above to see what
>> part is failing. Are you getting the right dev major/ minor from the
>> stat code or is the /proc/diskstats search code failing.
>>
>> Alternatively you could try the 1.1.1 branch to see if that fixes it.
>>
> I'm not sure I'm the right guys to be debugging this :-)
> In the interest of my schedule, and since this is a brand new
> workstation, I'm just going to try EXT4 on Soft RAID 0 and see if that
> works. If not, I'll look at 1.1.1. I Assume that would be the M4 release
> at this time.
>
You could try cherry-picking the following patch which is in master, I 
an not sure this change will be in 1.1.1, but I recommended to Joshua to 
add it.

f17c9d3 buildstats: tolerate absence of /proc/diskstats

Sau!

> Jim A
>>
>>>
>>> Jim A
>>>
>>>
>>> > 9 0 md0 133691 0 2218832 0 67133 0 5629616 0 0 0 0
>>> > 9 1 md1 235 0 1880 0 0 0 0 0 0 0 0
>>>
>>> Your build dir is in md0 or md1 (wrt your other post)
>>>
>>>
>>> JIm A
>>>
>>> ___________________________________________________
>>> yocto mailing list
>>> yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
>>> <mailto:yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>__>
>>> https://lists.yoctoproject.____org/listinfo/yocto
>>> <https://lists.yoctoproject.__org/listinfo/yocto
>>> <https://lists.yoctoproject.org/listinfo/yocto>>
>>>
>>>
>>>
>>>
>>> _________________________________________________
>>> yocto mailing list
>>> yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
>>> https://lists.yoctoproject.__org/listinfo/yocto
>>> <https://lists.yoctoproject.org/listinfo/yocto>
>>>
>>>
>
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto
>

Re: build failure on ubuntu 64bits development system

[Jim Abernathy]

On 01/18/2012 11:32 AM, Saul Wold wrote:
> On 01/18/2012 08:17 AM, Jim Abernathy wrote:
>> On 01/18/2012 11:06 AM, William Mills wrote:
>>>
>>>
>>> On 01/18/2012 10:25 AM, James Abernathy wrote:
>>>>
>>>>
>>>> On Wed, Jan 18, 2012 at 10:15 AM, William Mills <wmills@ti.com
>>>> <mailto:wmills@ti.com>> wrote:
>>>>
>>>>
>>>>
>>>> On 01/18/2012 10:04 AM, James Abernathy wrote:
>>>>
>>>>
>>>>
>>>> ---------- Forwarded message ----------
>>>> From: *William Mills* <wmills@ti.com <mailto:wmills@ti.com>
>>>> <mailto:wmills@ti.com <mailto:wmills@ti.com>>>
>>>> Date: Wed, Jan 18, 2012 at 9:57 AM
>>>> Subject: Re: [yocto] build failure on ubuntu 64bits development
>>>> system
>>>> To: Gary Thomas <gary@mlbassoc.com <mailto:gary@mlbassoc.com>
>>>> <mailto:gary@mlbassoc.com <mailto:gary@mlbassoc.com>>>
>>>> Cc: yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
>>>> <mailto:yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>__>
>>>>
>>>>
>>>>
>>>>
>>>> On 01/18/2012 09:51 AM, Gary Thomas wrote:
>>>>
>>>> On 2012-01-18 07 <tel:2012-01-18%2007> <tel:2012-01-18%2007>:42,
>>>> James Abernathy wrote:
>>>>
>>>>
>>>>
>>>> On Wed, Jan 18, 2012 at 9:30 AM, James Abernathy
>>>> <jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>>>> <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>>
>>>> <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>>>> <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>>__>__>
>>>>
>>>> wrote:
>>>>
>>>>
>>>>
>>>> On Wed, Jan 18, 2012 at 7:55 AM, James Abernathy
>>>> <jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>>>> <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>>
>>>> <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>>>> <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>>__>__>
>>>>
>>>> wrote:
>>>>
>>>> I just built a new development pc and installed Ubuntu 11.10
>>>> x64. I wonder if there are any new requirements to building
>>>> Yocto in that environment? I got an error right
>>>> off, but then it complete the first 63 task and stopped
>>>> successfully. error below:
>>>>
>>>> jim@ubuntu:~/poky/build-cdv$ bitbake core-image-sato
>>>> Pseudo is not present but is required, building this first
>>>> before the main build
>>>> Parsing recipes: 100%
>>>> |#############################____####################| Time:
>>>> 00:00:25
>>>>
>>>> Parsing of 797 .bb files complete (0 cached, 797 parsed). 1037
>>>> targets, 22 skipped, 0 masked, 0 errors.
>>>> ERROR: Execution of event handler 'run_buildstats' failed
>>>> Traceback (most recent call last):
>>>> File "run_buildstats(e)", line 18, in
>>>> run_buildstats(e=<bb.event.____BuildStarted object at 0x4c338d0>)
>>>>
>>>> File "buildstats.bbclass", line 21, in
>>>> set_device(e=<bb.event.____BuildStarted object at 0x4c338d0>)
>>>>
>>>> UnboundLocalError: local variable 'rdev' referenced before
>>>> assignment
>>>>
>>>>
>>>> Any ideas?
>>>>
>>>> JIm A
>>>>
>>>>
>>>> I went back and tried using the tarballs for poky edison and
>>>> cedartrail bsp and the errors don't occur. So I'm guessing the
>>>> issue isn't related to Ubuntu 32 vs. 64 bit.
>>>>
>>>>
>>>> I spoke too soon. Same error in edison tarballs. I looked at the
>>>> code and I can see an place were rdev could go un assigned. If
>>>> you fell out of the for loop without passing any of
>>>> the if conditions, rdev would be unassigned. That must be what
>>>> is happening in Ubuntu 11.10 x64
>>>>
>>>> Anybody building with Ubuntu 11.10 x64? This doesn't happen on x32
>>>>
>>>> Jim A
>>>>
>>>>
>>>> def set_device(e):
>>>> tmpdir = bb.data.getVar('TMPDIR', e.data, True)
>>>> try:
>>>> os.remove(bb.data.getVar('____DEVFILE', e.data, True))
>>>> except:
>>>> pass
>>>> ##############################____############################__##__################ 
>>>>
>>>>
>>>>
>>>> # We look for the volume TMPDIR lives on. To do all disks would
>>>> make little
>>>> # sense and not give us any particularly useful data. In theory
>>>> we could do
>>>> # something like stick DL_DIR on a different partition and this
>>>> would
>>>> # throw stats gathering off. The same goes with SSTATE_DIR.
>>>> However, let's
>>>> # get the basics in here and work on the cornercases later.
>>>> ##############################____############################__##__################ 
>>>>
>>>>
>>>> device=os.stat(tmpdir)
>>>> majordev=os.major(device.st_____dev)
>>>> minordev=os.minor(device.st_____dev)
>>>>
>>>> for line in open("/proc/diskstats", "r"):
>>>> if majordev == int(line.split()[0]) and minordev ==
>>>> int(line.split()[1]):
>>>> rdev=line.split()[2]
>>>> file = open(bb.data.getVar('DEVFILE', e.data, True), "w")
>>>> file.write(rdev)
>>>> file.close()
>>>>
>>>>
>>>> Can you show what the differences are between /proc/diskstats
>>>> on the two systems? That's obviously what's causing the error.
>>>>
>>>>
>>>> If your build dir is encyptfs or a fuse device or anything that
>>>> is not a
>>>> direct block device you will get this error. This is to be fixed in
>>>> 1.1.1 but encyptfs will still have other problems.
>>>>
>>>> I build the Ubuntu 11.10 x64 system with 2 drives setup as Soft
>>>> RAID 0.
>>>> I picked btrfs as the file system for no particular reason.
>>>> Should I go
>>>> back to ext4 or is RAID 0 the problem?
>>>>
>>>>
>>>> No, I would not do that yet. I would think software RAID would
>>>> present a block device so would not trigger this error.
>>>>
>>>> I was hoping to use RAID 0 for speed. I have a I7 2700K on a DZ68DB 
>>>> with
>>>> 2 6Gb/s ports matched to 2 6Gb/s 7200 hard drives. Since the builds 
>>>> take
>>>> so long, I was looking for an edge.
>>>>
>>>> So are there any recommendations at this point? I'm assuming that the
>>>> default ext4 directly on the SATA drive is a fall back position.
>>>>
>>>> Advice?
>>>
>>> If it were me, I would instrument (hack) that code above to see what
>>> part is failing. Are you getting the right dev major/ minor from the
>>> stat code or is the /proc/diskstats search code failing.
>>>
>>> Alternatively you could try the 1.1.1 branch to see if that fixes it.
>>>
>> I'm not sure I'm the right guys to be debugging this :-)
>> In the interest of my schedule, and since this is a brand new
>> workstation, I'm just going to try EXT4 on Soft RAID 0 and see if that
>> works. If not, I'll look at 1.1.1. I Assume that would be the M4 release
>> at this time.
>>
> You could try cherry-picking the following patch which is in master, I 
> an not sure this change will be in 1.1.1, but I recommended to Joshua 
> to add it.
>
> f17c9d3 buildstats: tolerate absence of /proc/diskstats
>
> Sau!
>
Okay, problem solved for me at least.  I went back and rebuilt the 
system with Ubuntu 11.10 Alternate 64bit install cdrom and still used 
Soft RAID 0, but file system was chosen to be EXT4. Build works fine 
with Edison branch.

FYI for those wanting to use Soft RAID, make sure you create one very 
small primary partition for GRUB2 to put the second part of the 
boot-loader in.  Can't use the old process.

Jim A

>> Jim A
>>>
>>>>
>>>> Jim A
>>>>
>>>>
>>>> > 9 0 md0 133691 0 2218832 0 67133 0 5629616 0 0 0 0
>>>> > 9 1 md1 235 0 1880 0 0 0 0 0 0 0 0
>>>>
>>>> Your build dir is in md0 or md1 (wrt your other post)
>>>>
>>>>
>>>> JIm A
>>>>
>>>> ___________________________________________________
>>>> yocto mailing list
>>>> yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
>>>> <mailto:yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>__>
>>>> https://lists.yoctoproject.____org/listinfo/yocto
>>>> <https://lists.yoctoproject.__org/listinfo/yocto
>>>> <https://lists.yoctoproject.org/listinfo/yocto>>
>>>>
>>>>
>>>>
>>>>
>>>> _________________________________________________
>>>> yocto mailing list
>>>> yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
>>>> https://lists.yoctoproject.__org/listinfo/yocto
>>>> <https://lists.yoctoproject.org/listinfo/yocto>
>>>>
>>>>
>>
>> _______________________________________________
>> yocto mailing list
>> yocto@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/yocto
>>

Re: build failure on ubuntu 64bits development system

[Darren Hart]

On 01/18/2012 09:05 AM, Jim Abernathy wrote:

> FYI for those wanting to use Soft RAID, make sure you create one very 
> small primary partition for GRUB2 to put the second part of the 
> boot-loader in.  Can't use the old process.

I strongly recommend using a separate DISK for your OS installation.
Yocto builds are hard on disks, and RAID 0 increases your risk of
failure in exchange for the added performance. I use a small SSD for my
OS disk and a large RAID0 array of spinning disks for /build and another
array for /virt (where my VM images live - easily recreated).

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: build failure on ubuntu 64bits development system

[Jim Abernathy]

[-- Attachment #1: Type: text/plain, Size: 1384 bytes --]

On 01/18/2012 04:34 PM, Darren Hart wrote:
>
> On 01/18/2012 09:05 AM, Jim Abernathy wrote:
>
>> FYI for those wanting to use Soft RAID, make sure you create one very
>> small primary partition for GRUB2 to put the second part of the
>> boot-loader in.  Can't use the old process.
> I strongly recommend using a separate DISK for your OS installation.
> Yocto builds are hard on disks, and RAID 0 increases your risk of
> failure in exchange for the added performance. I use a small SSD for my
> OS disk and a large RAID0 array of spinning disks for /build and another
> array for /virt (where my VM images live - easily recreated).
>
Learned a few things in this process. I appreciate all the help and advice.

 1. So we know that at least with Edison, btrfs does not work with bitbake.
 2. When I rebuilt the system, this time I put the Linux root directory
    on an 80GB SSD.  That is where I also have my clone of Linux-Yocto
    repository, poky, and download directory , DL_DIR.
 3. I have create /build with EXT4 format on a Software RAID 0 (striped)
    partition, using 2 separate hard drives,  to use as the working
    build directory for bitbake. I have a striped swap file on the same
    two drives.  But with 8GB or RAM, I shouldn't be using that much.

My build times for some of the basic meta-intel BSPs is around 103 minutes.

Jim A



[-- Attachment #2: Type: text/html, Size: 1950 bytes --]

Re: build failure on ubuntu 64bits development system

[William Mills]

On 01/19/2012 08:55 AM, Jim Abernathy wrote:
> On 01/18/2012 04:34 PM, Darren Hart wrote:
>>
>> On 01/18/2012 09:05 AM, Jim Abernathy wrote:
>>
>>> FYI for those wanting to use Soft RAID, make sure you create one very
>>> small primary partition for GRUB2 to put the second part of the
>>> boot-loader in.  Can't use the old process.
>> I strongly recommend using a separate DISK for your OS installation.
>> Yocto builds are hard on disks, and RAID 0 increases your risk of
>> failure in exchange for the added performance. I use a small SSD for my
>> OS disk and a large RAID0 array of spinning disks for /build and another
>> array for /virt (where my VM images live - easily recreated).
>>
> Learned a few things in this process. I appreciate all the help and advice.
>
>    1. So we know that at least with Edison, btrfs does not work with
>       bitbake.

This looks pretty important to me.  Can you create a bug report for this 
Jim?  It may be decided that it is a dup of the one Darren already 
pointed to but I would like to see someone prove that.  It is not clear 
to me why the blockstats feature could not be supported on btrfs so 
could justify seperate tracking anyway.  (The resolution of the one 
Darren pointed to will be to just disable blockstats for filesystem that 
don't have block devices.)

>    2. When I rebuilt the system, this time I put the Linux root
>       directory on an 80GB SSD. That is where I also have my clone of
>       Linux-Yocto repository, poky, and download directory , DL_DIR.
>    3. I have create /build with EXT4 format on a Software RAID 0
>       (striped) partition, using 2 separate hard drives, to use as the
>       working build directory for bitbake. I have a striped swap file on
>       the same two drives. But with 8GB or RAM, I shouldn't be using
>       that much.
>
> My build times for some of the basic meta-intel BSPs is around 103 minutes.
>
> Jim A
>
>
>
>
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto

Re: build failure on ubuntu 64bits development system

[Darren Hart]

On 01/19/2012 05:55 AM, Jim Abernathy wrote:
> On 01/18/2012 04:34 PM, Darren Hart wrote:
>>
>> On 01/18/2012 09:05 AM, Jim Abernathy wrote:
>>
>>> FYI for those wanting to use Soft RAID, make sure you create one very 
>>> small primary partition for GRUB2 to put the second part of the 
>>> boot-loader in.  Can't use the old process.
>> I strongly recommend using a separate DISK for your OS installation.
>> Yocto builds are hard on disks, and RAID 0 increases your risk of
>> failure in exchange for the added performance. I use a small SSD for my
>> OS disk and a large RAID0 array of spinning disks for /build and another
>> array for /virt (where my VM images live - easily recreated).
>>
> Learned a few things in this process. I appreciate all the help and advice.
> 
>  1. So we know that at least with Edison, btrfs does not work with bitbake.
>  2. When I rebuilt the system, this time I put the Linux root directory
>     on an 80GB SSD.  That is where I also have my clone of Linux-Yocto
>     repository, poky, and download directory , DL_DIR.
>  3. I have create /build with EXT4 format on a Software RAID 0 (striped)
>     partition, using 2 separate hard drives,  to use as the working
>     build directory for bitbake. I have a striped swap file on the same
>     two drives.  But with 8GB or RAM, I shouldn't be using that much.
> 
> My build times for some of the basic meta-intel BSPs is around 103 minutes.

You may be able to improve upon that with the following in /etc/fstab:

/dev/md0  /build  ext4  noauto,noatime,nodiratime,commit=6000  0  2

This reduces the number of writes due to updated access time and
increases the commit interval so it doesn't stall while writing out
every 5 minutes per default.

NOTE: THIS INCREASES YOUR RISK OF DATA LOSS

If your machine goes down during a build, you should plan on formatting
that drive. If you only keep builds on it, they easily recreateable and
you may find the performance boost is worth the risk.

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: build failure on ubuntu 64bits development system

[Jim Abernathy]

On 01/19/2012 12:44 PM, Darren Hart wrote:
>
> On 01/19/2012 05:55 AM, Jim Abernathy wrote:
>> On 01/18/2012 04:34 PM, Darren Hart wrote:
>>> On 01/18/2012 09:05 AM, Jim Abernathy wrote:
>>>
>>>> FYI for those wanting to use Soft RAID, make sure you create one very
>>>> small primary partition for GRUB2 to put the second part of the
>>>> boot-loader in.  Can't use the old process.
>>> I strongly recommend using a separate DISK for your OS installation.
>>> Yocto builds are hard on disks, and RAID 0 increases your risk of
>>> failure in exchange for the added performance. I use a small SSD for my
>>> OS disk and a large RAID0 array of spinning disks for /build and another
>>> array for /virt (where my VM images live - easily recreated).
>>>
>> Learned a few things in this process. I appreciate all the help and advice.
>>
>>   1. So we know that at least with Edison, btrfs does not work with bitbake.
>>   2. When I rebuilt the system, this time I put the Linux root directory
>>      on an 80GB SSD.  That is where I also have my clone of Linux-Yocto
>>      repository, poky, and download directory , DL_DIR.
>>   3. I have create /build with EXT4 format on a Software RAID 0 (striped)
>>      partition, using 2 separate hard drives,  to use as the working
>>      build directory for bitbake. I have a striped swap file on the same
>>      two drives.  But with 8GB or RAM, I shouldn't be using that much.
>>
>> My build times for some of the basic meta-intel BSPs is around 103 minutes.
> You may be able to improve upon that with the following in /etc/fstab:
>
> /dev/md0  /build  ext4  noauto,noatime,nodiratime,commit=6000  0  2
>
> This reduces the number of writes due to updated access time and
> increases the commit interval so it doesn't stall while writing out
> every 5 minutes per default.
>
> NOTE: THIS INCREASES YOUR RISK OF DATA LOSS
>
> If your machine goes down during a build, you should plan on formatting
> that drive. If you only keep builds on it, they easily recreateable and
> you may find the performance boost is worth the risk.
>
I'll test this out and see.  I'm plan on keeping this system on a UPS 
anyway.

Jim S

Re: build failure on ubuntu 64bits development system

[Darren Hart]

On 01/18/2012 07:25 AM, James Abernathy wrote:
> 
> 
> On Wed, Jan 18, 2012 at 10:15 AM, William Mills <wmills@ti.com
> <mailto:wmills@ti.com>> wrote:
> 
> 
> 
>     On 01/18/2012 10:04 AM, James Abernathy wrote:
> 
> 
> 
>         ---------- Forwarded message ----------
>         From: *William Mills* <wmills@ti.com <mailto:wmills@ti.com>
>         <mailto:wmills@ti.com <mailto:wmills@ti.com>>>
>         Date: Wed, Jan 18, 2012 at 9:57 AM
>         Subject: Re: [yocto] build failure on ubuntu 64bits development
>         system
>         To: Gary Thomas <gary@mlbassoc.com <mailto:gary@mlbassoc.com>
>         <mailto:gary@mlbassoc.com <mailto:gary@mlbassoc.com>>>
>         Cc: yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
>         <mailto:yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>__>
> 
> 
> 
> 
>         On 01/18/2012 09:51 AM, Gary Thomas wrote:
> 
>            On 2012-01-18 07 <tel:2012-01-18%2007>
>         <tel:2012-01-18%2007>:42, James Abernathy wrote:
> 
> 
> 
>                On Wed, Jan 18, 2012 at 9:30 AM, James Abernathy
>                <jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>         <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>>
>                <mailto:jfabernathy@gmail.com
>         <mailto:jfabernathy@gmail.com> <mailto:jfabernathy@gmail.com
>         <mailto:jfabernathy@gmail.com>>__>__>
> 
>                wrote:
> 
> 
> 
>                On Wed, Jan 18, 2012 at 7:55 AM, James Abernathy
>                <jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>
>         <mailto:jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>>
>                <mailto:jfabernathy@gmail.com
>         <mailto:jfabernathy@gmail.com> <mailto:jfabernathy@gmail.com
>         <mailto:jfabernathy@gmail.com>>__>__>
> 
>                wrote:
> 
>                I just built a new development pc and installed Ubuntu 11.10
>                x64. I wonder if there are any new requirements to building
>                Yocto in that environment? I got an error right
>                off, but then it complete the first 63 task and stopped
>                successfully. error below:
> 
>                jim@ubuntu:~/poky/build-cdv$ bitbake core-image-sato
>                Pseudo is not present but is required, building this first
>                before the main build
>                Parsing recipes: 100%
>                |#############################____####################|
>         Time: 00:00:25
> 
>                Parsing of 797 .bb files complete (0 cached, 797 parsed).
>         1037
>                targets, 22 skipped, 0 masked, 0 errors.
>                ERROR: Execution of event handler 'run_buildstats' failed
>                Traceback (most recent call last):
>                File "run_buildstats(e)", line 18, in
>                run_buildstats(e=<bb.event.____BuildStarted object at
>         0x4c338d0>)
> 
>                File "buildstats.bbclass", line 21, in
>                set_device(e=<bb.event.____BuildStarted object at 0x4c338d0>)
> 
>                UnboundLocalError: local variable 'rdev' referenced before
>                assignment
> 
> 
>                Any ideas?
> 
>                JIm A
> 
> 
>                I went back and tried using the tarballs for poky edison and
>                cedartrail bsp and the errors don't occur. So I'm
>         guessing the
>                issue isn't related to Ubuntu 32 vs. 64 bit.
> 
> 
>                I spoke too soon. Same error in edison tarballs. I looked
>         at the
>                code and I can see an place were rdev could go un
>         assigned. If
>                you fell out of the for loop without passing any of
>                the if conditions, rdev would be unassigned. That must be
>         what
>                is happening in Ubuntu 11.10 x64
> 
>                Anybody building with Ubuntu 11.10 x64? This doesn't
>         happen on x32
> 
>                Jim A
> 
> 
>                def set_device(e):
>                tmpdir = bb.data.getVar('TMPDIR', e.data, True)
>                try:
>                os.remove(bb.data.getVar('____DEVFILE', e.data, True))
>                except:
>                pass
>              
>          ##############################____############################__##__################
> 
>                # We look for the volume TMPDIR lives on. To do all disks
>         would
>                make little
>                # sense and not give us any particularly useful data. In
>         theory
>                we could do
>                # something like stick DL_DIR on a different partition
>         and this
>                would
>                # throw stats gathering off. The same goes with SSTATE_DIR.
>                However, let's
>                # get the basics in here and work on the cornercases later.
>              
>          ##############################____############################__##__################
>                device=os.stat(tmpdir)
>                majordev=os.major(device.st_____dev)
>                minordev=os.minor(device.st_____dev)
> 
>                for line in open("/proc/diskstats", "r"):
>                if majordev == int(line.split()[0]) and minordev ==
>                int(line.split()[1]):
>                rdev=line.split()[2]
>                file = open(bb.data.getVar('DEVFILE', e.data, True), "w")
>                file.write(rdev)
>                file.close()
> 
> 
>            Can you show what the differences are between /proc/diskstats
>            on the two systems? That's obviously what's causing the error.
> 
> 
>         If your build dir is encyptfs or a fuse device or anything that
>         is not a
>         direct block device you will get this error. This is to be fixed in
>         1.1.1 but encyptfs will still have other problems.
> 
>         I build the Ubuntu 11.10 x64 system with 2 drives setup as Soft
>         RAID 0.
>         I picked btrfs as the file system for no particular reason.
>         Should I go
>         back to ext4 or is RAID 0 the problem?
> 
> 
>     No, I would not do that yet.  I would think software RAID would
>     present a block device so would not trigger this error.
> 
> I was hoping to use RAID 0 for speed. I have a I7 2700K on a DZ68DB with
> 2 6Gb/s ports matched to 2 6Gb/s 7200 hard drives.  Since the builds
> take so long, I was looking for an edge.
> 
> So are there any recommendations at this point?  I'm assuming that the
> default ext4 directly on the SATA drive is a fall back position.

I use software RAID0 with ext4 on Ubuntu 11.10 amd64 for my primary
build system. I have never tried btrfs.

--
Darren

> 
> Advice?
> 
> Jim A
> 
> 
>     >  9       0 md0 133691 0 2218832 0 67133 0 5629616 0 0 0 0
>     >   9       1 md1 235 0 1880 0 0 0 0 0 0 0 0
> 
>     Your build dir is in md0 or md1 (wrt your other post)
> 
> 
>         JIm A
> 
>         ___________________________________________________
>         yocto mailing list
>         yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
>         <mailto:yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>__>
>         https://lists.yoctoproject.____org/listinfo/yocto
>         <https://lists.yoctoproject.__org/listinfo/yocto
>         <https://lists.yoctoproject.org/listinfo/yocto>>
> 
> 
> 
> 
>         _________________________________________________
>         yocto mailing list
>         yocto@yoctoproject.org <mailto:yocto@yoctoproject.org>
>         https://lists.yoctoproject.__org/listinfo/yocto
>         <https://lists.yoctoproject.org/listinfo/yocto>
> 
> 
> 
> 
> _______________________________________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/yocto

-- 
Darren Hart
Intel Open Source Technology Center
Yocto Project - Linux Kernel

Re: build failure on ubuntu 64bits development system

[James Abernathy]

[-- Attachment #1: Type: text/plain, Size: 7477 bytes --]

On Wed, Jan 18, 2012 at 9:51 AM, Gary Thomas <gary@mlbassoc.com> wrote:

> On 2012-01-18 07:42, James Abernathy wrote:
>
>>
>>
>> On Wed, Jan 18, 2012 at 9:30 AM, James Abernathy <jfabernathy@gmail.com<mailto:
>> jfabernathy@gmail.com>**> wrote:
>>
>>
>>
>>
>>    On Wed, Jan 18, 2012 at 7:55 AM, James Abernathy <
>> jfabernathy@gmail.com <mailto:jfabernathy@gmail.com>**> wrote:
>>
>>        I just built a new development pc and installed Ubuntu 11.10 x64.
>>  I wonder if there are any new requirements to building Yocto in that
>> environment?  I got an error right
>>        off, but then it complete the first 63 task and stopped
>> successfully.  error below:
>>
>>        jim@ubuntu:~/poky/build-cdv$ bitbake core-image-sato
>>        Pseudo is not present but is required, building this first before
>> the main build
>>        Parsing recipes: 100% |#############################**####################|
>> Time: 00:00:25
>>        Parsing of 797 .bb files complete (0 cached, 797 parsed). 1037
>> targets, 22 skipped, 0 masked, 0 errors.
>>        ERROR: Execution of event handler 'run_buildstats' failed
>>        Traceback (most recent call last):
>>           File "run_buildstats(e)", line 18, in
>> run_buildstats(e=<bb.event.**BuildStarted object at 0x4c338d0>)
>>           File "buildstats.bbclass", line 21, in set_device(e=<bb.event.*
>> *BuildStarted object at 0x4c338d0>)
>>        UnboundLocalError: local variable 'rdev' referenced before
>> assignment
>>
>>
>>        Any ideas?
>>
>>        JIm A
>>
>>
>>    I went back and tried using the tarballs for poky edison and
>> cedartrail bsp and the errors don't occur.  So I'm guessing the issue isn't
>> related to Ubuntu 32 vs. 64 bit.
>>
>>
>> I spoke too soon. Same error in edison tarballs.  I looked at the code
>> and I can see an place were rdev could go un assigned. If you fell out of
>> the for loop without passing any of
>> the if conditions, rdev would be unassigned.  That must be what is
>> happening in Ubuntu 11.10 x64
>>
>> Anybody building with Ubuntu 11.10 x64?  This doesn't happen on x32
>>
>> Jim A
>>
>>
>> def set_device(e):
>>     tmpdir = bb.data.getVar('TMPDIR', e.data, True)
>>     try:
>>         os.remove(bb.data.getVar('**DEVFILE', e.data, True))
>>     except:
>>         pass
>>     ##############################**##############################**
>> ################
>>     # We look for the volume TMPDIR lives on. To do all disks would make
>> little
>>     # sense and not give us any particularly useful data. In theory we
>> could do
>>     # something like stick DL_DIR on a different partition and this would
>>     # throw stats gathering off. The same goes with SSTATE_DIR. However,
>> let's
>>     # get the basics in here and work on the cornercases later.
>>     ##############################**##############################**
>> ################
>>     device=os.stat(tmpdir)
>>     majordev=os.major(device.st_**dev)
>>     minordev=os.minor(device.st_**dev)
>>     for line in open("/proc/diskstats", "r"):
>>         if majordev == int(line.split()[0]) and minordev ==
>> int(line.split()[1]):
>>            rdev=line.split()[2]
>>     file = open(bb.data.getVar('DEVFILE', e.data, True), "w")
>>     file.write(rdev)
>>     file.close()
>>
>
> Can you show what the differences are between /proc/diskstats
> on the two systems?  That's obviously what's causing the error.
>
> --

on the x64 system it's:
jim@ubuntu:~/poky-edison-6.0/build$ cat /proc/diskstats
   1       0 ram0 0 0 0 0 0 0 0 0 0 0 0
   1       1 ram1 0 0 0 0 0 0 0 0 0 0 0
   1       2 ram2 0 0 0 0 0 0 0 0 0 0 0
   1       3 ram3 0 0 0 0 0 0 0 0 0 0 0
   1       4 ram4 0 0 0 0 0 0 0 0 0 0 0
   1       5 ram5 0 0 0 0 0 0 0 0 0 0 0
   1       6 ram6 0 0 0 0 0 0 0 0 0 0 0
   1       7 ram7 0 0 0 0 0 0 0 0 0 0 0
   1       8 ram8 0 0 0 0 0 0 0 0 0 0 0
   1       9 ram9 0 0 0 0 0 0 0 0 0 0 0
   1      10 ram10 0 0 0 0 0 0 0 0 0 0 0
   1      11 ram11 0 0 0 0 0 0 0 0 0 0 0
   1      12 ram12 0 0 0 0 0 0 0 0 0 0 0
   1      13 ram13 0 0 0 0 0 0 0 0 0 0 0
   1      14 ram14 0 0 0 0 0 0 0 0 0 0 0
   1      15 ram15 0 0 0 0 0 0 0 0 0 0 0
   7       0 loop0 0 0 0 0 0 0 0 0 0 0 0
   7       1 loop1 0 0 0 0 0 0 0 0 0 0 0
   7       2 loop2 0 0 0 0 0 0 0 0 0 0 0
   7       3 loop3 0 0 0 0 0 0 0 0 0 0 0
   7       4 loop4 0 0 0 0 0 0 0 0 0 0 0
   7       5 loop5 0 0 0 0 0 0 0 0 0 0 0
   7       6 loop6 0 0 0 0 0 0 0 0 0 0 0
   7       7 loop7 0 0 0 0 0 0 0 0 0 0 0
   8       0 sda 33076 23382 1115462 1829656 31802 9209 2849176 624296 0
190664 2454576
   8       1 sda1 32289 23295 1108604 1827592 27505 9200 2848960 560828 0
131548 2388540
   8       2 sda2 429 24 3618 1224 18 9 216 64 0 1260 1288
   8       3 sda3 2 0 20 108 0 0 0 0 0 108 108
   8       5 sda5 191 63 1900 460 0 0 0 0 0 436 456
   8      16 sdb 32063 23153 1112192 1760580 25981 8723 2780656 549424 0
134176 2310272
   8      17 sdb1 31842 23153 1110554 1759984 21702 8723 2780656 501512 0
88724 2261696
   8      18 sdb2 2 0 12 52 0 0 0 0 0 52 52
   8      21 sdb5 47 0 250 228 0 0 0 0 0 228 228
  11       0 sr0 0 0 0 0 0 0 0 0 0 0 0
   9       0 md0 133691 0 2218832 0 67133 0 5629616 0 0 0 0
   9       1 md1 235 0 1880 0 0 0 0 0 0 0 0

On the x32 system it's:
  im@ubuntu-workstation:~$ cat /proc/diskstats
  1       0 ram0 0 0 0 0 0 0 0 0 0 0 0
  1       1 ram1 0 0 0 0 0 0 0 0 0 0 0
  1       2 ram2 0 0 0 0 0 0 0 0 0 0 0
  1       3 ram3 0 0 0 0 0 0 0 0 0 0 0
  1       4 ram4 0 0 0 0 0 0 0 0 0 0 0
  1       5 ram5 0 0 0 0 0 0 0 0 0 0 0
  1       6 ram6 0 0 0 0 0 0 0 0 0 0 0
  1       7 ram7 0 0 0 0 0 0 0 0 0 0 0
  1       8 ram8 0 0 0 0 0 0 0 0 0 0 0
  1       9 ram9 0 0 0 0 0 0 0 0 0 0 0
  1      10 ram10 0 0 0 0 0 0 0 0 0 0 0
  1      11 ram11 0 0 0 0 0 0 0 0 0 0 0
  1      12 ram12 0 0 0 0 0 0 0 0 0 0 0
  1      13 ram13 0 0 0 0 0 0 0 0 0 0 0
  1      14 ram14 0 0 0 0 0 0 0 0 0 0 0
  1      15 ram15 0 0 0 0 0 0 0 0 0 0 0
  7       0 loop0 0 0 0 0 0 0 0 0 0 0 0
  7       1 loop1 0 0 0 0 0 0 0 0 0 0 0
  7       2 loop2 0 0 0 0 0 0 0 0 0 0 0
  7       3 loop3 0 0 0 0 0 0 0 0 0 0 0
  7       4 loop4 0 0 0 0 0 0 0 0 0 0 0
  7       5 loop5 0 0 0 0 0 0 0 0 0 0 0
  7       6 loop6 0 0 0 0 0 0 0 0 0 0 0
  7       7 loop7 0 0 0 0 0 0 0 0 0 0 0
  8       0 sda 335 3 2704 284 0 0 0 0 0 284 284
  8       1 sda1 167 0 1336 152 0 0 0 0 0 152 152
  8      16 sdb 360 3 2904 480 0 0 0 0 0 376 480
  8      17 sdb1 165 0 1320 120 0 0 0 0 0 120 120
  8      18 sdb2 29 0 232 212 0 0 0 0 0 212 212
  8      32 sdc 5605 15794 503606 680328 578 1698 19480 35292 0 20740 715608
  8      33 sdc1 5274 15763 500722 678628 565 1698 19480 34748 0 20068
713364
  8      34 sdc2 2 0 4 0 0 0 0 0 0 0 0
  8      37 sdc5 162 31 1544 892 0 0 0 0 0 884 892
 11       0 sr0 0 0 0 0 0 0 0 0 0 0 0
jim@ubuntu-workstation:~$

The file system is BTRFS  on Software RAID 0 with 2 disks.

JIm A


> ------------------------------**------------------------------
> Gary Thomas                 |  Consulting for the
> MLB Associates              |    Embedded world
> ------------------------------**------------------------------
> ______________________________**_________________
> yocto mailing list
> yocto@yoctoproject.org
> https://lists.yoctoproject.**org/listinfo/yocto<https://lists.yoctoproject.org/listinfo/yocto>
>

[-- Attachment #2: Type: text/html, Size: 9070 bytes --]