[PATCH 0/4] Add extensible SDK update test

[PATCH 0/4] Add extensible SDK update test

[Aníbal Limón]

The following changes since commit 8c0ba8d9aff6dec6e458739aafb79b185e951afd:

  bitbake: toaster: toastergui Fix invalid char test and implementation (2016-02-19 15:38:49 +0000)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib alimon/esdk_update
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=alimon/esdk_update

Aníbal Limón (4):
  classes/testsdk: Move code for avoid PATHs to oeqa.utils
  classes/testsdk: Move the removal of bitbake PATH to eSDK context only
  classes/testsdk: Pass tcname to SDK and SDKExt contexts
  oeqa/sdkext: Add sdk_update.SDKUpdateTest class.

 meta/classes/testsdk.bbclass       | 29 +++++++++-------------------
 meta/lib/oeqa/oetest.py            | 29 ++++++++++++++++++++++++----
 meta/lib/oeqa/sdkext/sdk_update.py | 39 ++++++++++++++++++++++++++++++++++++++
 meta/lib/oeqa/utils/__init__.py    | 17 +++++++++++++++++
 4 files changed, 90 insertions(+), 24 deletions(-)
 create mode 100644 meta/lib/oeqa/sdkext/sdk_update.py

-- 
2.1.4

[PATCH 1/4] classes/testsdk: Move code for avoid PATHs to oeqa.utils

[Aníbal Limón]

From: Aníbal Limón <limon.anibal@gmail.com>

Due to the neeed to use in other modules.

Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
---
 meta/classes/testsdk.bbclass    | 18 ++++--------------
 meta/lib/oeqa/utils/__init__.py | 17 +++++++++++++++++
 2 files changed, 21 insertions(+), 14 deletions(-)

diff --git a/meta/classes/testsdk.bbclass b/meta/classes/testsdk.bbclass
index a56ad5e..7e245e9 100644
--- a/meta/classes/testsdk.bbclass
+++ b/meta/classes/testsdk.bbclass
@@ -88,30 +88,20 @@ def testsdkext_main(d):
     import os
     import oeqa.sdkext
     import subprocess
-    from oeqa.oetest import SDKTestContext, SDKExtTestContext
     from bb.utils import export_proxies
+    from oeqa.oetest import SDKTestContext, SDKExtTestContext
+    from oeqa.utils import avoid_paths_in_environ
+
 
     # extensible sdk use network
     export_proxies(d)
 
     # extensible sdk shows a warning if found bitbake in the path
     # because can cause problems so clean it
-    new_path = ''
     paths_to_avoid = ['bitbake/bin', 'poky/scripts',
                        d.getVar('STAGING_DIR', True),
                        d.getVar('BASE_WORKDIR', True)]
-    for p in os.environ['PATH'].split(':'):
-       avoid = False
-       for pa in paths_to_avoid:
-           if pa in p:
-              avoid = True
-              break
-       if avoid:
-           continue
-
-       new_path = new_path + p + ':'
-    new_path = new_path[:-1]
-    os.environ['PATH'] = new_path
+    avoid_paths_in_environ(paths_to_avoid)
 
     pn = d.getVar("PN", True)
     bb.utils.mkdirhier(d.getVar("TEST_LOG_SDKEXT_DIR", True))
diff --git a/meta/lib/oeqa/utils/__init__.py b/meta/lib/oeqa/utils/__init__.py
index 2260046..b246c69 100644
--- a/meta/lib/oeqa/utils/__init__.py
+++ b/meta/lib/oeqa/utils/__init__.py
@@ -13,3 +13,20 @@ class CommandError(Exception):
     def __str__(self):
         return "Command '%s' returned non-zero exit status %d with output: %s" % (self.cmd, self.retcode, self.output)
 
+def avoid_paths_in_environ(paths_to_avoid):
+    import os
+
+    new_path = ''
+    for p in os.environ['PATH'].split(':'):
+       avoid = False
+       for pa in paths_to_avoid:
+           if pa in p:
+              avoid = True
+              break
+       if avoid:
+           continue
+
+       new_path = new_path + p + ':'
+
+    new_path = new_path[:-1]
+    os.environ['PATH'] = new_path
-- 
2.1.4

[PATCH 2/4] classes/testsdk: Move the removal of bitbake PATH to eSDK context only

[Aníbal Limón]

From: Aníbal Limón <limon.anibal@gmail.com>

The removal of bitbake and scripts PATH is only needed by eSDK tests
so move to eSDK context only.

This also it's a support for eSDK update test because it needs to
execute oe-publish-sdk from scripts.

Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
---
 meta/classes/testsdk.bbclass |  9 ++++-----
 meta/lib/oeqa/oetest.py      | 22 +++++++++++++++++++++-
 2 files changed, 25 insertions(+), 6 deletions(-)

diff --git a/meta/classes/testsdk.bbclass b/meta/classes/testsdk.bbclass
index 7e245e9..01d37c4 100644
--- a/meta/classes/testsdk.bbclass
+++ b/meta/classes/testsdk.bbclass
@@ -96,11 +96,10 @@ def testsdkext_main(d):
     # extensible sdk use network
     export_proxies(d)
 
-    # extensible sdk shows a warning if found bitbake in the path
-    # because can cause problems so clean it
-    paths_to_avoid = ['bitbake/bin', 'poky/scripts',
-                       d.getVar('STAGING_DIR', True),
-                       d.getVar('BASE_WORKDIR', True)]
+    # extensible sdk can be contaminated if native programs are
+    # in PATH, i.e. use perl-native instead of eSDK one.
+    paths_to_avoid = [d.getVar('STAGING_DIR', True),
+                      d.getVar('BASE_WORKDIR', True)]
     avoid_paths_in_environ(paths_to_avoid)
 
     pn = d.getVar("PN", True)
diff --git a/meta/lib/oeqa/oetest.py b/meta/lib/oeqa/oetest.py
index 3e2ea0f..cd1e7e0 100644
--- a/meta/lib/oeqa/oetest.py
+++ b/meta/lib/oeqa/oetest.py
@@ -21,6 +21,7 @@ import logging
 import oeqa.runtime
 import oeqa.sdkext
 from oeqa.utils.decorators import LogResults, gettag, getResults
+from oeqa.utils import avoid_paths_in_environ
 
 logger = logging.getLogger("BitBake")
 
@@ -128,7 +129,26 @@ class oeSDKTest(oeTest):
         return subprocess.check_output(". %s > /dev/null; %s;" % (self.tc.sdkenv, cmd), shell=True)
 
 class oeSDKExtTest(oeSDKTest):
-    pass
+    def _run(self, cmd):
+        output = None
+
+        paths = os.environ['PATH']
+
+        # extensible sdk shows a warning if found bitbake in the path
+        # because can cause contamination, i.e. use devtool from
+        # poky/scripts instead of eSDK one.
+        paths_to_avoid = ['bitbake/bin', 'poky/scripts']
+        avoid_paths_in_environ(paths_to_avoid)
+
+        try:
+            output = subprocess.check_output(". %s > /dev/null; %s;" % \
+                (self.tc.sdkenv, cmd), shell=True)
+        except:
+            os.environ['PATH'] = paths
+            raise
+
+        os.environ['PATH'] = paths
+        return output
 
 def getmodule(pos=2):
     # stack returns a list of tuples containg frame information
-- 
2.1.4

Re: [PATCH 2/4] classes/testsdk: Move the removal of bitbake PATH to eSDK context only

[Randy Witt]

On 02/22/2016 07:03 AM, Aníbal Limón wrote:
> From: Aníbal Limón <limon.anibal@gmail.com>
>
> The removal of bitbake and scripts PATH is only needed by eSDK tests
> so move to eSDK context only.
>
> This also it's a support for eSDK update test because it needs to
> execute oe-publish-sdk from scripts.
>
> Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
> ---
>   meta/classes/testsdk.bbclass |  9 ++++-----
>   meta/lib/oeqa/oetest.py      | 22 +++++++++++++++++++++-
>   2 files changed, 25 insertions(+), 6 deletions(-)
>
> diff --git a/meta/classes/testsdk.bbclass b/meta/classes/testsdk.bbclass
> index 7e245e9..01d37c4 100644
> --- a/meta/classes/testsdk.bbclass
> +++ b/meta/classes/testsdk.bbclass
> @@ -96,11 +96,10 @@ def testsdkext_main(d):
>       # extensible sdk use network
>       export_proxies(d)
>
> -    # extensible sdk shows a warning if found bitbake in the path
> -    # because can cause problems so clean it
> -    paths_to_avoid = ['bitbake/bin', 'poky/scripts',
> -                       d.getVar('STAGING_DIR', True),
> -                       d.getVar('BASE_WORKDIR', True)]
> +    # extensible sdk can be contaminated if native programs are
> +    # in PATH, i.e. use perl-native instead of eSDK one.
> +    paths_to_avoid = [d.getVar('STAGING_DIR', True),
> +                      d.getVar('BASE_WORKDIR', True)]
>       avoid_paths_in_environ(paths_to_avoid)
>
>       pn = d.getVar("PN", True)
> diff --git a/meta/lib/oeqa/oetest.py b/meta/lib/oeqa/oetest.py
> index 3e2ea0f..cd1e7e0 100644
> --- a/meta/lib/oeqa/oetest.py
> +++ b/meta/lib/oeqa/oetest.py
> @@ -21,6 +21,7 @@ import logging
>   import oeqa.runtime
>   import oeqa.sdkext
>   from oeqa.utils.decorators import LogResults, gettag, getResults
> +from oeqa.utils import avoid_paths_in_environ
>
>   logger = logging.getLogger("BitBake")
>
> @@ -128,7 +129,26 @@ class oeSDKTest(oeTest):
>           return subprocess.check_output(". %s > /dev/null; %s;" % (self.tc.sdkenv, cmd), shell=True)
>
>   class oeSDKExtTest(oeSDKTest):
> -    pass
> +    def _run(self, cmd):
> +        output = None
> +
> +        paths = os.environ['PATH']
> +
> +        # extensible sdk shows a warning if found bitbake in the path
> +        # because can cause contamination, i.e. use devtool from
> +        # poky/scripts instead of eSDK one.
> +        paths_to_avoid = ['bitbake/bin', 'poky/scripts']
> +        avoid_paths_in_environ(paths_to_avoid)
> +
> +        try:
> +            output = subprocess.check_output(". %s > /dev/null; %s;" % \
> +                (self.tc.sdkenv, cmd), shell=True)

Just pass env to the check_output call, then you won't have to worry about 
resetting the parent process' env back to the original value.

> +        except:
> +            os.environ['PATH'] = paths
> +            raise
> +
> +        os.environ['PATH'] = paths
> +        return output
>
>   def getmodule(pos=2):
>       # stack returns a list of tuples containg frame information
>

Re: [PATCH 2/4] classes/testsdk: Move the removal of bitbake PATH to eSDK context only

[Aníbal Limón]

[-- Attachment #1: Type: text/plain, Size: 3316 bytes --]



On 02/22/2016 09:54 AM, Randy Witt wrote:
> On 02/22/2016 07:03 AM, Aníbal Limón wrote:
>> From: Aníbal Limón <limon.anibal@gmail.com>
>>
>> The removal of bitbake and scripts PATH is only needed by eSDK tests
>> so move to eSDK context only.
>>
>> This also it's a support for eSDK update test because it needs to
>> execute oe-publish-sdk from scripts.
>>
>> Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
>> ---
>>   meta/classes/testsdk.bbclass |  9 ++++-----
>>   meta/lib/oeqa/oetest.py      | 22 +++++++++++++++++++++-
>>   2 files changed, 25 insertions(+), 6 deletions(-)
>>
>> diff --git a/meta/classes/testsdk.bbclass b/meta/classes/testsdk.bbclass
>> index 7e245e9..01d37c4 100644
>> --- a/meta/classes/testsdk.bbclass
>> +++ b/meta/classes/testsdk.bbclass
>> @@ -96,11 +96,10 @@ def testsdkext_main(d):
>>       # extensible sdk use network
>>       export_proxies(d)
>>
>> -    # extensible sdk shows a warning if found bitbake in the path
>> -    # because can cause problems so clean it
>> -    paths_to_avoid = ['bitbake/bin', 'poky/scripts',
>> -                       d.getVar('STAGING_DIR', True),
>> -                       d.getVar('BASE_WORKDIR', True)]
>> +    # extensible sdk can be contaminated if native programs are
>> +    # in PATH, i.e. use perl-native instead of eSDK one.
>> +    paths_to_avoid = [d.getVar('STAGING_DIR', True),
>> +                      d.getVar('BASE_WORKDIR', True)]
>>       avoid_paths_in_environ(paths_to_avoid)
>>
>>       pn = d.getVar("PN", True)
>> diff --git a/meta/lib/oeqa/oetest.py b/meta/lib/oeqa/oetest.py
>> index 3e2ea0f..cd1e7e0 100644
>> --- a/meta/lib/oeqa/oetest.py
>> +++ b/meta/lib/oeqa/oetest.py
>> @@ -21,6 +21,7 @@ import logging
>>   import oeqa.runtime
>>   import oeqa.sdkext
>>   from oeqa.utils.decorators import LogResults, gettag, getResults
>> +from oeqa.utils import avoid_paths_in_environ
>>
>>   logger = logging.getLogger("BitBake")
>>
>> @@ -128,7 +129,26 @@ class oeSDKTest(oeTest):
>>           return subprocess.check_output(". %s > /dev/null; %s;" %
>> (self.tc.sdkenv, cmd), shell=True)
>>
>>   class oeSDKExtTest(oeSDKTest):
>> -    pass
>> +    def _run(self, cmd):
>> +        output = None
>> +
>> +        paths = os.environ['PATH']
>> +
>> +        # extensible sdk shows a warning if found bitbake in the path
>> +        # because can cause contamination, i.e. use devtool from
>> +        # poky/scripts instead of eSDK one.
>> +        paths_to_avoid = ['bitbake/bin', 'poky/scripts']
>> +        avoid_paths_in_environ(paths_to_avoid)
>> +
>> +        try:
>> +            output = subprocess.check_output(". %s > /dev/null; %s;" % \
>> +                (self.tc.sdkenv, cmd), shell=True)
> 
> Just pass env to the check_output call, then you won't have to worry
> about resetting the parent process' env back to the original value.

Could be but then we need the code for generate the env line so for
practically the result is the same.


> 
>> +        except:
>> +            os.environ['PATH'] = paths
>> +            raise
>> +
>> +        os.environ['PATH'] = paths
>> +        return output
>>
>>   def getmodule(pos=2):
>>       # stack returns a list of tuples containg frame information
>>
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH 2/4] classes/testsdk: Move the removal of bitbake PATH to eSDK context only

[Randy Witt]

On 02/22/2016 08:09 AM, Aníbal Limón wrote:
>
>
> On 02/22/2016 09:54 AM, Randy Witt wrote:
>> On 02/22/2016 07:03 AM, Aníbal Limón wrote:
>>> From: Aníbal Limón <limon.anibal@gmail.com>
>>>
>>> The removal of bitbake and scripts PATH is only needed by eSDK tests
>>> so move to eSDK context only.
>>>
>>> This also it's a support for eSDK update test because it needs to
>>> execute oe-publish-sdk from scripts.
>>>
>>> Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
>>> ---
>>>    meta/classes/testsdk.bbclass |  9 ++++-----
>>>    meta/lib/oeqa/oetest.py      | 22 +++++++++++++++++++++-
>>>    2 files changed, 25 insertions(+), 6 deletions(-)
>>>
>>> diff --git a/meta/classes/testsdk.bbclass b/meta/classes/testsdk.bbclass
>>> index 7e245e9..01d37c4 100644
>>> --- a/meta/classes/testsdk.bbclass
>>> +++ b/meta/classes/testsdk.bbclass
>>> @@ -96,11 +96,10 @@ def testsdkext_main(d):
>>>        # extensible sdk use network
>>>        export_proxies(d)
>>>
>>> -    # extensible sdk shows a warning if found bitbake in the path
>>> -    # because can cause problems so clean it
>>> -    paths_to_avoid = ['bitbake/bin', 'poky/scripts',
>>> -                       d.getVar('STAGING_DIR', True),
>>> -                       d.getVar('BASE_WORKDIR', True)]
>>> +    # extensible sdk can be contaminated if native programs are
>>> +    # in PATH, i.e. use perl-native instead of eSDK one.
>>> +    paths_to_avoid = [d.getVar('STAGING_DIR', True),
>>> +                      d.getVar('BASE_WORKDIR', True)]
>>>        avoid_paths_in_environ(paths_to_avoid)
>>>
>>>        pn = d.getVar("PN", True)
>>> diff --git a/meta/lib/oeqa/oetest.py b/meta/lib/oeqa/oetest.py
>>> index 3e2ea0f..cd1e7e0 100644
>>> --- a/meta/lib/oeqa/oetest.py
>>> +++ b/meta/lib/oeqa/oetest.py
>>> @@ -21,6 +21,7 @@ import logging
>>>    import oeqa.runtime
>>>    import oeqa.sdkext
>>>    from oeqa.utils.decorators import LogResults, gettag, getResults
>>> +from oeqa.utils import avoid_paths_in_environ
>>>
>>>    logger = logging.getLogger("BitBake")
>>>
>>> @@ -128,7 +129,26 @@ class oeSDKTest(oeTest):
>>>            return subprocess.check_output(". %s > /dev/null; %s;" %
>>> (self.tc.sdkenv, cmd), shell=True)
>>>
>>>    class oeSDKExtTest(oeSDKTest):
>>> -    pass
>>> +    def _run(self, cmd):
>>> +        output = None
>>> +
>>> +        paths = os.environ['PATH']
>>> +
>>> +        # extensible sdk shows a warning if found bitbake in the path
>>> +        # because can cause contamination, i.e. use devtool from
>>> +        # poky/scripts instead of eSDK one.
>>> +        paths_to_avoid = ['bitbake/bin', 'poky/scripts']
>>> +        avoid_paths_in_environ(paths_to_avoid)
>>> +
>>> +        try:
>>> +            output = subprocess.check_output(". %s > /dev/null; %s;" % \
>>> +                (self.tc.sdkenv, cmd), shell=True)
>>
>> Just pass env to the check_output call, then you won't have to worry
>> about resetting the parent process' env back to the original value.
>
> Could be but then we need the code for generate the env line so for
> practically the result is the same.

Modify avoid_paths_in_environ() to instead return the environment rather than 
modify it as a side effect. Then the code reads more easily and prevents future 
bugs where someone uses avoid_paths_in_environ, but forgets to reset the os.environ.

>
>>
>>> +        except:
>>> +            os.environ['PATH'] = paths
>>> +            raise
>>> +
>>> +        os.environ['PATH'] = paths
>>> +        return output
>>>
>>>    def getmodule(pos=2):
>>>        # stack returns a list of tuples containg frame information
>>>
>>
>

Re: [PATCH 2/4] classes/testsdk: Move the removal of bitbake PATH to eSDK context only

[Aníbal Limón]

[-- Attachment #1: Type: text/plain, Size: 4132 bytes --]



On 02/22/2016 10:23 AM, Randy Witt wrote:
> On 02/22/2016 08:09 AM, Aníbal Limón wrote:
>>
>>
>> On 02/22/2016 09:54 AM, Randy Witt wrote:
>>> On 02/22/2016 07:03 AM, Aníbal Limón wrote:
>>>> From: Aníbal Limón <limon.anibal@gmail.com>
>>>>
>>>> The removal of bitbake and scripts PATH is only needed by eSDK tests
>>>> so move to eSDK context only.
>>>>
>>>> This also it's a support for eSDK update test because it needs to
>>>> execute oe-publish-sdk from scripts.
>>>>
>>>> Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
>>>> ---
>>>>    meta/classes/testsdk.bbclass |  9 ++++-----
>>>>    meta/lib/oeqa/oetest.py      | 22 +++++++++++++++++++++-
>>>>    2 files changed, 25 insertions(+), 6 deletions(-)
>>>>
>>>> diff --git a/meta/classes/testsdk.bbclass
>>>> b/meta/classes/testsdk.bbclass
>>>> index 7e245e9..01d37c4 100644
>>>> --- a/meta/classes/testsdk.bbclass
>>>> +++ b/meta/classes/testsdk.bbclass
>>>> @@ -96,11 +96,10 @@ def testsdkext_main(d):
>>>>        # extensible sdk use network
>>>>        export_proxies(d)
>>>>
>>>> -    # extensible sdk shows a warning if found bitbake in the path
>>>> -    # because can cause problems so clean it
>>>> -    paths_to_avoid = ['bitbake/bin', 'poky/scripts',
>>>> -                       d.getVar('STAGING_DIR', True),
>>>> -                       d.getVar('BASE_WORKDIR', True)]
>>>> +    # extensible sdk can be contaminated if native programs are
>>>> +    # in PATH, i.e. use perl-native instead of eSDK one.
>>>> +    paths_to_avoid = [d.getVar('STAGING_DIR', True),
>>>> +                      d.getVar('BASE_WORKDIR', True)]
>>>>        avoid_paths_in_environ(paths_to_avoid)
>>>>
>>>>        pn = d.getVar("PN", True)
>>>> diff --git a/meta/lib/oeqa/oetest.py b/meta/lib/oeqa/oetest.py
>>>> index 3e2ea0f..cd1e7e0 100644
>>>> --- a/meta/lib/oeqa/oetest.py
>>>> +++ b/meta/lib/oeqa/oetest.py
>>>> @@ -21,6 +21,7 @@ import logging
>>>>    import oeqa.runtime
>>>>    import oeqa.sdkext
>>>>    from oeqa.utils.decorators import LogResults, gettag, getResults
>>>> +from oeqa.utils import avoid_paths_in_environ
>>>>
>>>>    logger = logging.getLogger("BitBake")
>>>>
>>>> @@ -128,7 +129,26 @@ class oeSDKTest(oeTest):
>>>>            return subprocess.check_output(". %s > /dev/null; %s;" %
>>>> (self.tc.sdkenv, cmd), shell=True)
>>>>
>>>>    class oeSDKExtTest(oeSDKTest):
>>>> -    pass
>>>> +    def _run(self, cmd):
>>>> +        output = None
>>>> +
>>>> +        paths = os.environ['PATH']
>>>> +
>>>> +        # extensible sdk shows a warning if found bitbake in the path
>>>> +        # because can cause contamination, i.e. use devtool from
>>>> +        # poky/scripts instead of eSDK one.
>>>> +        paths_to_avoid = ['bitbake/bin', 'poky/scripts']
>>>> +        avoid_paths_in_environ(paths_to_avoid)
>>>> +
>>>> +        try:
>>>> +            output = subprocess.check_output(". %s > /dev/null;
>>>> %s;" % \
>>>> +                (self.tc.sdkenv, cmd), shell=True)
>>>
>>> Just pass env to the check_output call, then you won't have to worry
>>> about resetting the parent process' env back to the original value.
>>
>> Could be but then we need the code for generate the env line so for
>> practically the result is the same.
> 
> Modify avoid_paths_in_environ() to instead return the environment rather
> than modify it as a side effect. Then the code reads more easily and
> prevents future bugs where someone uses avoid_paths_in_environ, but
> forgets to reset the os.environ.

I agree with you to modify avoid_paths_in_environ for return the new
PATH variable is better than only modify it internally but for
simplicity i will maintain the os.environ['PATH'] set/restore instead of
generate the environment line.


> 
>>
>>>
>>>> +        except:
>>>> +            os.environ['PATH'] = paths
>>>> +            raise
>>>> +
>>>> +        os.environ['PATH'] = paths
>>>> +        return output
>>>>
>>>>    def getmodule(pos=2):
>>>>        # stack returns a list of tuples containg frame information
>>>>
>>>
>>
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH 2/4] classes/testsdk: Move the removal of bitbake PATH to eSDK context only

[Burton, Ross]

[-- Attachment #1: Type: text/plain, Size: 1001 bytes --]

On 22 February 2016 at 16:37, Aníbal Limón <anibal.limon@linux.intel.com>
wrote:

> I agree with you to modify avoid_paths_in_environ for return the new
> PATH variable is better than only modify it internally but for
> simplicity i will maintain the os.environ['PATH'] set/restore instead of
> generate the environment line.
>

Totally agree with Randy here for what it's worth.  The environment-munging
code in avoid_paths... should return the strings instead of manipulating
the current environment so the caller has the choice whether to modify the
current environment or pass a new environment to subprocess.  And in
general I'd say that passing modified environments to subprocess is a
cleaner solution as it means that there's no way cleanup can fail to
happen.  Whilst that's just a try/except now, the code could get copied and
extended and end up with codepaths that don't hit the right cleanup.  By
having an explicit environment passed in, this isn't possible.

Ross

[-- Attachment #2: Type: text/html, Size: 1440 bytes --]

Re: [PATCH 2/4] classes/testsdk: Move the removal of bitbake PATH to eSDK context only

[Aníbal Limón]

[-- Attachment #1: Type: text/plain, Size: 1274 bytes --]



On 02/22/2016 10:48 AM, Burton, Ross wrote:
> On 22 February 2016 at 16:37, Aníbal Limón <anibal.limon@linux.intel.com>
> wrote:
> 
>> I agree with you to modify avoid_paths_in_environ for return the new
>> PATH variable is better than only modify it internally but for
>> simplicity i will maintain the os.environ['PATH'] set/restore instead of
>> generate the environment line.
>>
> 
> Totally agree with Randy here for what it's worth.  The environment-munging
> code in avoid_paths... should return the strings instead of manipulating
> the current environment so the caller has the choice whether to modify the
> current environment or pass a new environment to subprocess.  And in
> general I'd say that passing modified environments to subprocess is a
> cleaner solution as it means that there's no way cleanup can fail to
> happen.  Whilst that's just a try/except now, the code could get copied and
> extended and end up with codepaths that don't hit the right cleanup.  By
> having an explicit environment passed in, this isn't possible.

Agree, now modified at,

http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=alimon/esdk_update_v2&id=3143bf09130c52cd71e3f2f9795208e17152005d

Cheers,
	alimon

> 
> Ross
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH 2/4] classes/testsdk: Move the removal of bitbake PATH to eSDK context only

[Randy Witt]

On 02/22/2016 09:05 AM, Aníbal Limón wrote:
>
>
> On 02/22/2016 10:48 AM, Burton, Ross wrote:
>> On 22 February 2016 at 16:37, Aníbal Limón <anibal.limon@linux.intel.com>
>> wrote:
>>
>>> I agree with you to modify avoid_paths_in_environ for return the new
>>> PATH variable is better than only modify it internally but for
>>> simplicity i will maintain the os.environ['PATH'] set/restore instead of
>>> generate the environment line.
>>>
>>
>> Totally agree with Randy here for what it's worth.  The environment-munging
>> code in avoid_paths... should return the strings instead of manipulating
>> the current environment so the caller has the choice whether to modify the
>> current environment or pass a new environment to subprocess.  And in
>> general I'd say that passing modified environments to subprocess is a
>> cleaner solution as it means that there's no way cleanup can fail to
>> happen.  Whilst that's just a try/except now, the code could get copied and
>> extended and end up with codepaths that don't hit the right cleanup.  By
>> having an explicit environment passed in, this isn't possible.
>
> Agree, now modified at,
>
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=alimon/esdk_update_v2&id=3143bf09130c52cd71e3f2f9795208e17152005d

If you either convert path to a dict or have avoid_paths_in_environ() return a 
dict you can do:

+            output = subprocess.check_output(". %s > /dev/null; %s;" % \
+                (self.tc.sdkenv, cmd), env=path, shell=True)

It's not quite obvious in the docs that you can pass that in, they refer you to 
the Popen docs.

> Cheers,
> 	alimon
>
>>
>> Ross
>>
>

Re: [PATCH 2/4] classes/testsdk: Move the removal of bitbake PATH to eSDK context only

[Aníbal Limón]

[-- Attachment #1: Type: text/plain, Size: 2206 bytes --]



On 02/22/2016 11:27 AM, Randy Witt wrote:
> On 02/22/2016 09:05 AM, Aníbal Limón wrote:
>>
>>
>> On 02/22/2016 10:48 AM, Burton, Ross wrote:
>>> On 22 February 2016 at 16:37, Aníbal Limón
>>> <anibal.limon@linux.intel.com>
>>> wrote:
>>>
>>>> I agree with you to modify avoid_paths_in_environ for return the new
>>>> PATH variable is better than only modify it internally but for
>>>> simplicity i will maintain the os.environ['PATH'] set/restore
>>>> instead of
>>>> generate the environment line.
>>>>
>>>
>>> Totally agree with Randy here for what it's worth.  The
>>> environment-munging
>>> code in avoid_paths... should return the strings instead of manipulating
>>> the current environment so the caller has the choice whether to
>>> modify the
>>> current environment or pass a new environment to subprocess.  And in
>>> general I'd say that passing modified environments to subprocess is a
>>> cleaner solution as it means that there's no way cleanup can fail to
>>> happen.  Whilst that's just a try/except now, the code could get
>>> copied and
>>> extended and end up with codepaths that don't hit the right cleanup.  By
>>> having an explicit environment passed in, this isn't possible.
>>
>> Agree, now modified at,
>>
>> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=alimon/esdk_update_v2&id=3143bf09130c52cd71e3f2f9795208e17152005d
>>
> 
> If you either convert path to a dict or have avoid_paths_in_environ()
> return a dict you can do:
> 
> +            output = subprocess.check_output(". %s > /dev/null; %s;" % \
> +                (self.tc.sdkenv, cmd), env=path, shell=True)
> 
> It's not quite obvious in the docs that you can pass that in, they refer
> you to the Popen docs.


I know that you could pass env in kwargs but since it's using shell=True
is easy to set PATH inline also it causes a little overhead for copy the
os.environ.

But if you want the another way that's ok, see [1].

[1]
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=alimon/esdk_update_v2&id=ab84d325c5cbb751d7b18e861964b757d9682e0f

	alimon

> 
>> Cheers,
>>     alimon
>>
>>>
>>> Ross
>>>
>>
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH 2/4] classes/testsdk: Move the removal of bitbake PATH to eSDK context only

[Aníbal Limón]

[-- Attachment #1: Type: text/plain, Size: 2522 bytes --]



On 02/22/2016 11:47 AM, Aníbal Limón wrote:
> 
> 
> On 02/22/2016 11:27 AM, Randy Witt wrote:
>> On 02/22/2016 09:05 AM, Aníbal Limón wrote:
>>>
>>>
>>> On 02/22/2016 10:48 AM, Burton, Ross wrote:
>>>> On 22 February 2016 at 16:37, Aníbal Limón
>>>> <anibal.limon@linux.intel.com>
>>>> wrote:
>>>>
>>>>> I agree with you to modify avoid_paths_in_environ for return the new
>>>>> PATH variable is better than only modify it internally but for
>>>>> simplicity i will maintain the os.environ['PATH'] set/restore
>>>>> instead of
>>>>> generate the environment line.
>>>>>
>>>>
>>>> Totally agree with Randy here for what it's worth.  The
>>>> environment-munging
>>>> code in avoid_paths... should return the strings instead of manipulating
>>>> the current environment so the caller has the choice whether to
>>>> modify the
>>>> current environment or pass a new environment to subprocess.  And in
>>>> general I'd say that passing modified environments to subprocess is a
>>>> cleaner solution as it means that there's no way cleanup can fail to
>>>> happen.  Whilst that's just a try/except now, the code could get
>>>> copied and
>>>> extended and end up with codepaths that don't hit the right cleanup.  By
>>>> having an explicit environment passed in, this isn't possible.
>>>
>>> Agree, now modified at,
>>>
>>> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=alimon/esdk_update_v2&id=3143bf09130c52cd71e3f2f9795208e17152005d
>>>
>>
>> If you either convert path to a dict or have avoid_paths_in_environ()
>> return a dict you can do:
>>
>> +            output = subprocess.check_output(". %s > /dev/null; %s;" % \
>> +                (self.tc.sdkenv, cmd), env=path, shell=True)
>>
>> It's not quite obvious in the docs that you can pass that in, they refer
>> you to the Popen docs.
> 
> 
> I know that you could pass env in kwargs but since it's using shell=True
> is easy to set PATH inline also it causes a little overhead for copy the
> os.environ.
> 
> But if you want the another way that's ok, see [1].
> 
> [1]
> http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=alimon/esdk_update_v2&id=ab84d325c5cbb751d7b18e861964b757d9682e0f

Bad rev this is the one that works [1].

[1]
http://git.yoctoproject.org/cgit/cgit.cgi/poky-contrib/commit/?h=alimon/esdk_update_v2&id=228e8e506f946a1245798b0d0e818f63aca221ce
> 
> 	alimon
> 
>>
>>> Cheers,
>>>     alimon
>>>
>>>>
>>>> Ross
>>>>
>>>
>>
> 
> 
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

[PATCH 3/4] classes/testsdk: Pass tcname to SDK and SDKExt contexts

[Aníbal Limón]

From: Aníbal Limón <limon.anibal@gmail.com>

tcname is needed for eSDK update testcase will be used for
 publish it and then try to update

Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
---
 meta/classes/testsdk.bbclass | 2 +-
 meta/lib/oeqa/oetest.py      | 7 ++++---
 2 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/meta/classes/testsdk.bbclass b/meta/classes/testsdk.bbclass
index 01d37c4..bcb506d 100644
--- a/meta/classes/testsdk.bbclass
+++ b/meta/classes/testsdk.bbclass
@@ -22,7 +22,7 @@ def run_test_context(CTestContext, d, testdir, tcname, pn, *args):
     targets = glob.glob(d.expand(testdir + "/tc/environment-setup-*"))
     for sdkenv in targets:
         bb.plain("Testing %s" % sdkenv)
-        tc = CTestContext(d, testdir, sdkenv, args)
+        tc = CTestContext(d, testdir, sdkenv, tcname, args)
 
         # this is a dummy load of tests
         # we are doing that to find compile errors in the tests themselves
diff --git a/meta/lib/oeqa/oetest.py b/meta/lib/oeqa/oetest.py
index cd1e7e0..6bfd6e3 100644
--- a/meta/lib/oeqa/oetest.py
+++ b/meta/lib/oeqa/oetest.py
@@ -396,11 +396,12 @@ class ImageTestContext(TestContext):
         setattr(oeRuntimeTest, "pscmd", "ps -ef" if oeTest.hasPackage("procps") else "ps")
 
 class SDKTestContext(TestContext):
-    def __init__(self, d, sdktestdir, sdkenv, *args):
+    def __init__(self, d, sdktestdir, sdkenv, tcname, *args):
         super(SDKTestContext, self).__init__(d)
 
         self.sdktestdir = sdktestdir
         self.sdkenv = sdkenv
+        self.tcname = tcname
 
         if not hasattr(self, 'target_manifest'):
             self.target_manifest = d.getVar("SDK_TARGET_MANIFEST", True)
@@ -429,7 +430,7 @@ class SDKTestContext(TestContext):
                 "auto").split() if t != "auto"]
 
 class SDKExtTestContext(SDKTestContext):
-    def __init__(self, d, sdktestdir, sdkenv, *args):
+    def __init__(self, d, sdktestdir, sdkenv, tcname, *args):
         self.target_manifest = d.getVar("SDK_EXT_TARGET_MANIFEST", True)
         self.host_manifest = d.getVar("SDK_EXT_HOST_MANIFEST", True)
         if args:
@@ -437,7 +438,7 @@ class SDKExtTestContext(SDKTestContext):
         else:
             self.cm = False
 
-        super(SDKExtTestContext, self).__init__(d, sdktestdir, sdkenv)
+        super(SDKExtTestContext, self).__init__(d, sdktestdir, sdkenv, tcname)
 
         self.sdkextfilesdir = os.path.join(os.path.dirname(os.path.abspath(
             oeqa.sdkext.__file__)), "files")
-- 
2.1.4

[PATCH 4/4] oeqa/sdkext: Add sdk_update.SDKUpdateTest class.

[Aníbal Limón]

From: Aníbal Limón <limon.anibal@gmail.com>

The SDKUpdateTest class test devtool sdk-update mechanism inside
eSDK.

The SDKUpdateTest class search for new sdk if not found uses
the main one then it publish the eSDK into known folder
inside work and it starts a web server for serve the eSDK.

Finally it executes sdk-update over http, the local test is
commented due to bug [1].

[1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=9043

[YOCTO #9089]

Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
---
 meta/lib/oeqa/sdkext/sdk_update.py | 39 ++++++++++++++++++++++++++++++++++++++
 1 file changed, 39 insertions(+)
 create mode 100644 meta/lib/oeqa/sdkext/sdk_update.py

diff --git a/meta/lib/oeqa/sdkext/sdk_update.py b/meta/lib/oeqa/sdkext/sdk_update.py
new file mode 100644
index 0000000..16f5b10
--- /dev/null
+++ b/meta/lib/oeqa/sdkext/sdk_update.py
@@ -0,0 +1,39 @@
+import os
+import shutil
+import subprocess
+
+from oeqa.oetest import oeSDKExtTest
+from oeqa.utils.httpserver import HTTPService
+
+class SdkUpdateTest(oeSDKExtTest):
+
+    @classmethod
+    def setUpClass(self):
+        self.publish_dir = os.path.join(self.tc.sdktestdir, 'esdk_publish')
+        if os.path.exists(self.publish_dir):
+            shutil.rmtree(self.publish_dir)
+        os.mkdir(self.publish_dir)
+
+        tcname_new = self.tc.d.expand(
+            "${SDK_DEPLOY}/${TOOLCHAINEXT_OUTPUTNAME}-new.sh")
+        if not os.path.exists(tcname_new):
+            tcname_new = self.tc.tcname
+
+        cmd = 'oe-publish-sdk %s %s' % (tcname_new, self.publish_dir)
+        subprocess.check_output(cmd, shell=True)
+
+        self.http_service = HTTPService(self.publish_dir)
+        self.http_service.start()
+
+        self.http_url = "http://127.0.0.1:%d" % self.http_service.port
+
+    def test_sdk_update_http(self):
+        output = self._run("devtool sdk-update \"%s\"" % self.http_url)
+
+#    def test_sdk_update_local(self):
+#        output = self._run("devtool sdk-update \"%s\"" % self.publish_dir)
+
+    @classmethod
+    def tearDownClass(self):
+        self.http_service.stop()
+        shutil.rmtree(self.publish_dir)
-- 
2.1.4

Re: [PATCH 4/4] oeqa/sdkext: Add sdk_update.SDKUpdateTest class.

[Randy Witt]

On 02/22/2016 07:03 AM, Aníbal Limón wrote:
> From: Aníbal Limón <limon.anibal@gmail.com>
>
> The SDKUpdateTest class test devtool sdk-update mechanism inside
> eSDK.
>
> The SDKUpdateTest class search for new sdk if not found uses
> the main one then it publish the eSDK into known folder
> inside work and it starts a web server for serve the eSDK.
>
> Finally it executes sdk-update over http, the local test is
> commented due to bug [1].
>
> [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=9043
>
> [YOCTO #9089]
>
> Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
> ---
>   meta/lib/oeqa/sdkext/sdk_update.py | 39 ++++++++++++++++++++++++++++++++++++++
>   1 file changed, 39 insertions(+)
>   create mode 100644 meta/lib/oeqa/sdkext/sdk_update.py
>
> diff --git a/meta/lib/oeqa/sdkext/sdk_update.py b/meta/lib/oeqa/sdkext/sdk_update.py
> new file mode 100644
> index 0000000..16f5b10
> --- /dev/null
> +++ b/meta/lib/oeqa/sdkext/sdk_update.py
> @@ -0,0 +1,39 @@
> +import os
> +import shutil
> +import subprocess
> +
> +from oeqa.oetest import oeSDKExtTest
> +from oeqa.utils.httpserver import HTTPService
> +
> +class SdkUpdateTest(oeSDKExtTest):
> +
> +    @classmethod
> +    def setUpClass(self):
> +        self.publish_dir = os.path.join(self.tc.sdktestdir, 'esdk_publish')
> +        if os.path.exists(self.publish_dir):
> +            shutil.rmtree(self.publish_dir)
> +        os.mkdir(self.publish_dir)
> +
> +        tcname_new = self.tc.d.expand(
> +            "${SDK_DEPLOY}/${TOOLCHAINEXT_OUTPUTNAME}-new.sh")
> +        if not os.path.exists(tcname_new):
> +            tcname_new = self.tc.tcname
> +
> +        cmd = 'oe-publish-sdk %s %s' % (tcname_new, self.publish_dir)
> +        subprocess.check_output(cmd, shell=True)
> +
> +        self.http_service = HTTPService(self.publish_dir)
> +        self.http_service.start()

I think Paul and I briefly mentioned it, but SimpleHTTPServer fails as an sstate 
mirror if enough fetchers run. We think it was because of a limit on the number 
of simultaneous connections. So I would expect this to fail for instance if all 
packages were updated.

But really we shouldn't care too much about the sstate updating part, and more 
about the layer updating etc. Ideally this test would check to make sure the 
local sdk now matches the remote.

> +        self.http_url = "http://127.0.0.1:%d" % self.http_service.port
> +
> +    def test_sdk_update_http(self):
> +        output = self._run("devtool sdk-update \"%s\"" % self.http_url)
> +
> +#    def test_sdk_update_local(self):
> +#        output = self._run("devtool sdk-update \"%s\"" % self.publish_dir)
> +
> +    @classmethod
> +    def tearDownClass(self):
> +        self.http_service.stop()
> +        shutil.rmtree(self.publish_dir)
>

Re: [PATCH 4/4] oeqa/sdkext: Add sdk_update.SDKUpdateTest class.

[Aníbal Limón]

[-- Attachment #1: Type: text/plain, Size: 3271 bytes --]



On 02/22/2016 10:18 AM, Randy Witt wrote:
> On 02/22/2016 07:03 AM, Aníbal Limón wrote:
>> From: Aníbal Limón <limon.anibal@gmail.com>
>>
>> The SDKUpdateTest class test devtool sdk-update mechanism inside
>> eSDK.
>>
>> The SDKUpdateTest class search for new sdk if not found uses
>> the main one then it publish the eSDK into known folder
>> inside work and it starts a web server for serve the eSDK.
>>
>> Finally it executes sdk-update over http, the local test is
>> commented due to bug [1].
>>
>> [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=9043
>>
>> [YOCTO #9089]
>>
>> Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
>> ---
>>   meta/lib/oeqa/sdkext/sdk_update.py | 39
>> ++++++++++++++++++++++++++++++++++++++
>>   1 file changed, 39 insertions(+)
>>   create mode 100644 meta/lib/oeqa/sdkext/sdk_update.py
>>
>> diff --git a/meta/lib/oeqa/sdkext/sdk_update.py
>> b/meta/lib/oeqa/sdkext/sdk_update.py
>> new file mode 100644
>> index 0000000..16f5b10
>> --- /dev/null
>> +++ b/meta/lib/oeqa/sdkext/sdk_update.py
>> @@ -0,0 +1,39 @@
>> +import os
>> +import shutil
>> +import subprocess
>> +
>> +from oeqa.oetest import oeSDKExtTest
>> +from oeqa.utils.httpserver import HTTPService
>> +
>> +class SdkUpdateTest(oeSDKExtTest):
>> +
>> +    @classmethod
>> +    def setUpClass(self):
>> +        self.publish_dir = os.path.join(self.tc.sdktestdir,
>> 'esdk_publish')
>> +        if os.path.exists(self.publish_dir):
>> +            shutil.rmtree(self.publish_dir)
>> +        os.mkdir(self.publish_dir)
>> +
>> +        tcname_new = self.tc.d.expand(
>> +            "${SDK_DEPLOY}/${TOOLCHAINEXT_OUTPUTNAME}-new.sh")
>> +        if not os.path.exists(tcname_new):
>> +            tcname_new = self.tc.tcname
>> +
>> +        cmd = 'oe-publish-sdk %s %s' % (tcname_new, self.publish_dir)
>> +        subprocess.check_output(cmd, shell=True)
>> +
>> +        self.http_service = HTTPService(self.publish_dir)
>> +        self.http_service.start()
> 
> I think Paul and I briefly mentioned it, but SimpleHTTPServer fails as
> an sstate mirror if enough fetchers run. We think it was because of a
> limit on the number of simultaneous connections. So I would expect this
> to fail for instance if all packages were updated.

I tested running an update and didn't fail this implementation of
oeqa.utils open another process with multiprocessing for serve http only.

> 
> But really we shouldn't care too much about the sstate updating part,
> and more about the layer updating etc. Ideally this test would check to
> make sure the local sdk now matches the remote.

I could add a test for the output that i saw displays "Already updated"
or what you mean about match?


> 
>> +        self.http_url = "http://127.0.0.1:%d" % self.http_service.port
>> +
>> +    def test_sdk_update_http(self):
>> +        output = self._run("devtool sdk-update \"%s\"" % self.http_url)
>> +
>> +#    def test_sdk_update_local(self):
>> +#        output = self._run("devtool sdk-update \"%s\"" %
>> self.publish_dir)
>> +
>> +    @classmethod
>> +    def tearDownClass(self):
>> +        self.http_service.stop()
>> +        shutil.rmtree(self.publish_dir)
>>
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: [PATCH 4/4] oeqa/sdkext: Add sdk_update.SDKUpdateTest class.

[Randy Witt]

On 02/22/2016 08:26 AM, Aníbal Limón wrote:
>
>
> On 02/22/2016 10:18 AM, Randy Witt wrote:
>> On 02/22/2016 07:03 AM, Aníbal Limón wrote:
>>> From: Aníbal Limón <limon.anibal@gmail.com>
>>>
>>> The SDKUpdateTest class test devtool sdk-update mechanism inside
>>> eSDK.
>>>
>>> The SDKUpdateTest class search for new sdk if not found uses
>>> the main one then it publish the eSDK into known folder
>>> inside work and it starts a web server for serve the eSDK.
>>>
>>> Finally it executes sdk-update over http, the local test is
>>> commented due to bug [1].
>>>
>>> [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=9043
>>>
>>> [YOCTO #9089]
>>>
>>> Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
>>> ---
>>>    meta/lib/oeqa/sdkext/sdk_update.py | 39
>>> ++++++++++++++++++++++++++++++++++++++
>>>    1 file changed, 39 insertions(+)
>>>    create mode 100644 meta/lib/oeqa/sdkext/sdk_update.py
>>>
>>> diff --git a/meta/lib/oeqa/sdkext/sdk_update.py
>>> b/meta/lib/oeqa/sdkext/sdk_update.py
>>> new file mode 100644
>>> index 0000000..16f5b10
>>> --- /dev/null
>>> +++ b/meta/lib/oeqa/sdkext/sdk_update.py
>>> @@ -0,0 +1,39 @@
>>> +import os
>>> +import shutil
>>> +import subprocess
>>> +
>>> +from oeqa.oetest import oeSDKExtTest
>>> +from oeqa.utils.httpserver import HTTPService
>>> +
>>> +class SdkUpdateTest(oeSDKExtTest):
>>> +
>>> +    @classmethod
>>> +    def setUpClass(self):
>>> +        self.publish_dir = os.path.join(self.tc.sdktestdir,
>>> 'esdk_publish')
>>> +        if os.path.exists(self.publish_dir):
>>> +            shutil.rmtree(self.publish_dir)
>>> +        os.mkdir(self.publish_dir)
>>> +
>>> +        tcname_new = self.tc.d.expand(
>>> +            "${SDK_DEPLOY}/${TOOLCHAINEXT_OUTPUTNAME}-new.sh")
>>> +        if not os.path.exists(tcname_new):
>>> +            tcname_new = self.tc.tcname
>>> +
>>> +        cmd = 'oe-publish-sdk %s %s' % (tcname_new, self.publish_dir)
>>> +        subprocess.check_output(cmd, shell=True)
>>> +
>>> +        self.http_service = HTTPService(self.publish_dir)
>>> +        self.http_service.start()
>>
>> I think Paul and I briefly mentioned it, but SimpleHTTPServer fails as
>> an sstate mirror if enough fetchers run. We think it was because of a
>> limit on the number of simultaneous connections. So I would expect this
>> to fail for instance if all packages were updated.
>
> I tested running an update and didn't fail this implementation of
> oeqa.utils open another process with multiprocessing for serve http only.

My suspicion is because nothing actually updated. And even if something updated, 
unless you hit the max connection limit when pulling from sstate_mirror it would 
be fine.

>>
>> But really we shouldn't care too much about the sstate updating part,
>> and more about the layer updating etc. Ideally this test would check to
>> make sure the local sdk now matches the remote.
>
> I could add a test for the output that i saw displays "Already updated"
> or what you mean about match?

devtool sdk-update will check the remote conf/sdk-conf-manifest to see if any 
configuration changed. So for instance the creator of the sdk could add new 
configuration variables, or even items to bblayers.conf.

If the sdk-conf-manifest is changed, the updater should pull down the new layer 
by pulling(cloning if that fails) layers/.git on the remote and moving the 
layers to the appropriate local location. It should also update the local 
configuration to match what is on the remote.

So if you get "Already up-to-date" nothing changed and nothing was updated. To 
test the updating, we need to make sure the configuration actually changes, and 
potentially even add another layer, then update, and then make sure the local 
sdk matches what we think it should based on the remote sdk we published.

>
>>
>>> +        self.http_url = "http://127.0.0.1:%d" % self.http_service.port
>>> +
>>> +    def test_sdk_update_http(self):
>>> +        output = self._run("devtool sdk-update \"%s\"" % self.http_url)
>>> +
>>> +#    def test_sdk_update_local(self):
>>> +#        output = self._run("devtool sdk-update \"%s\"" %
>>> self.publish_dir)
>>> +
>>> +    @classmethod
>>> +    def tearDownClass(self):
>>> +        self.http_service.stop()
>>> +        shutil.rmtree(self.publish_dir)
>>>
>>
>

Re: [PATCH 4/4] oeqa/sdkext: Add sdk_update.SDKUpdateTest class.

[Aníbal Limón]

[-- Attachment #1: Type: text/plain, Size: 5034 bytes --]



On 02/22/2016 10:36 AM, Randy Witt wrote:
> On 02/22/2016 08:26 AM, Aníbal Limón wrote:
>>
>>
>> On 02/22/2016 10:18 AM, Randy Witt wrote:
>>> On 02/22/2016 07:03 AM, Aníbal Limón wrote:
>>>> From: Aníbal Limón <limon.anibal@gmail.com>
>>>>
>>>> The SDKUpdateTest class test devtool sdk-update mechanism inside
>>>> eSDK.
>>>>
>>>> The SDKUpdateTest class search for new sdk if not found uses
>>>> the main one then it publish the eSDK into known folder
>>>> inside work and it starts a web server for serve the eSDK.
>>>>
>>>> Finally it executes sdk-update over http, the local test is
>>>> commented due to bug [1].
>>>>
>>>> [1] https://bugzilla.yoctoproject.org/show_bug.cgi?id=9043
>>>>
>>>> [YOCTO #9089]
>>>>
>>>> Signed-off-by: Aníbal Limón <limon.anibal@gmail.com>
>>>> ---
>>>>    meta/lib/oeqa/sdkext/sdk_update.py | 39
>>>> ++++++++++++++++++++++++++++++++++++++
>>>>    1 file changed, 39 insertions(+)
>>>>    create mode 100644 meta/lib/oeqa/sdkext/sdk_update.py
>>>>
>>>> diff --git a/meta/lib/oeqa/sdkext/sdk_update.py
>>>> b/meta/lib/oeqa/sdkext/sdk_update.py
>>>> new file mode 100644
>>>> index 0000000..16f5b10
>>>> --- /dev/null
>>>> +++ b/meta/lib/oeqa/sdkext/sdk_update.py
>>>> @@ -0,0 +1,39 @@
>>>> +import os
>>>> +import shutil
>>>> +import subprocess
>>>> +
>>>> +from oeqa.oetest import oeSDKExtTest
>>>> +from oeqa.utils.httpserver import HTTPService
>>>> +
>>>> +class SdkUpdateTest(oeSDKExtTest):
>>>> +
>>>> +    @classmethod
>>>> +    def setUpClass(self):
>>>> +        self.publish_dir = os.path.join(self.tc.sdktestdir,
>>>> 'esdk_publish')
>>>> +        if os.path.exists(self.publish_dir):
>>>> +            shutil.rmtree(self.publish_dir)
>>>> +        os.mkdir(self.publish_dir)
>>>> +
>>>> +        tcname_new = self.tc.d.expand(
>>>> +            "${SDK_DEPLOY}/${TOOLCHAINEXT_OUTPUTNAME}-new.sh")
>>>> +        if not os.path.exists(tcname_new):
>>>> +            tcname_new = self.tc.tcname
>>>> +
>>>> +        cmd = 'oe-publish-sdk %s %s' % (tcname_new, self.publish_dir)
>>>> +        subprocess.check_output(cmd, shell=True)
>>>> +
>>>> +        self.http_service = HTTPService(self.publish_dir)
>>>> +        self.http_service.start()
>>>
>>> I think Paul and I briefly mentioned it, but SimpleHTTPServer fails as
>>> an sstate mirror if enough fetchers run. We think it was because of a
>>> limit on the number of simultaneous connections. So I would expect this
>>> to fail for instance if all packages were updated.
>>
>> I tested running an update and didn't fail this implementation of
>> oeqa.utils open another process with multiprocessing for serve http only.
> 
> My suspicion is because nothing actually updated. And even if something
> updated, unless you hit the max connection limit when pulling from
> sstate_mirror it would be fine.

Not it was updated the first time didn't display Already up to date that
was the second time also i saw the connections from sdk-update to local
web server.

> 
>>>
>>> But really we shouldn't care too much about the sstate updating part,
>>> and more about the layer updating etc. Ideally this test would check to
>>> make sure the local sdk now matches the remote.
>>
>> I could add a test for the output that i saw displays "Already updated"
>> or what you mean about match?
> 
> devtool sdk-update will check the remote conf/sdk-conf-manifest to see
> if any configuration changed. So for instance the creator of the sdk
> could add new configuration variables, or even items to bblayers.conf.
> 
> If the sdk-conf-manifest is changed, the updater should pull down the
> new layer by pulling(cloning if that fails) layers/.git on the remote
> and moving the layers to the appropriate local location. It should also
> update the local configuration to match what is on the remote.
> 
> So if you get "Already up-to-date" nothing changed and nothing was
> updated. To test the updating, we need to make sure the configuration
> actually changes, and potentially even add another layer, then update,
> and then make sure the local sdk matches what we think it should based
> on the remote sdk we published.

Up comment,

That's the reason for search new esdk (suffix -new) in the code because
that is another task (AB task) to generate the new eSDK, this test
satisfy the requeriment for install the eSDK launch the webserver and
launch sdk-update tool over it.


> 
>>
>>>
>>>> +        self.http_url = "http://127.0.0.1:%d" % self.http_service.port
>>>> +
>>>> +    def test_sdk_update_http(self):
>>>> +        output = self._run("devtool sdk-update \"%s\"" %
>>>> self.http_url)
>>>> +
>>>> +#    def test_sdk_update_local(self):
>>>> +#        output = self._run("devtool sdk-update \"%s\"" %
>>>> self.publish_dir)
>>>> +
>>>> +    @classmethod
>>>> +    def tearDownClass(self):
>>>> +        self.http_service.stop()
>>>> +        shutil.rmtree(self.publish_dir)
>>>>
>>>
>>
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]