[PATCH v2 0/5] Fix for 8004

[PATCH v2 0/5] Fix for 8004

[Ed Bartosh]

Hi,

This patchset fixes bug 8004: importing a layer from the recipes page
auto adds the 1st level dependencies but not the 2nd level.
Now dependent layers are discovered recursively and Toaster shows all of them
in Compatible layers view when user clicks 'Add layer' button.

The patchset includes fixes for orm test suite and new test case for
Layer_Version.get_alldeps API used in this bug fix.

Changes in v2: UI shows number of first level deps in compatible layers page.
               (Reverted two patches from v1)

The following changes since commit 7b10ae122d46891c734e786358ea5f36760e8d5e:

  toaster: Fix broken test case (2015-10-01 10:42:16 +0100)

are available in the git repository at:

  git://git.yoctoproject.org/poky-contrib ed/toaster/8004
  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/8004

Ed Bartosh (5):
  toaster: implement API to get full list of deps
  toaster: use get_alldeps in layerdetails renderer
  toaster: fix NameError
  toaster: fix orm tests
  toaster: test get_alldeps API

 bitbake/lib/toaster/orm/models.py       | 21 ++++++++++++-
 bitbake/lib/toaster/orm/tests.py        | 53 ++++++++++++++-------------------
 bitbake/lib/toaster/toastergui/views.py |  4 +--
 3 files changed, 45 insertions(+), 33 deletions(-)

--
Regards,
Ed

[PATCH 1/5] toaster: implement API to get full list of deps

[Ed Bartosh]

Implemented Layer_Version.get_alldeps API to recursively get
full list of dependencies for the layer. Dependencies that are
already in the project are filtered out from the result.
Result list of Layer_Version objects is sorted by layer name
for UI to look consistent.

This API is going to be used to show amount and list of
dependencies for the layer in the list of compatible layers
for the project.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 bitbake/lib/toaster/orm/models.py | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/bitbake/lib/toaster/orm/models.py b/bitbake/lib/toaster/orm/models.py
index 5aed158..c5e256f 100644
--- a/bitbake/lib/toaster/orm/models.py
+++ b/bitbake/lib/toaster/orm/models.py
@@ -1177,6 +1177,25 @@ class Layer_Version(models.Model):
     def get_detailspage_url(self, project_id):
         return reverse('layerdetails', args=(project_id, self.pk))
 
+    def get_alldeps(self, project_id):
+        """Get full list of unique layer dependencies."""
+        def gen_layerdeps(lver, project):
+            for ldep in lver.dependencies.all():
+                yield ldep.depends_on
+                # get next level of deps recursively calling gen_layerdeps
+                for subdep in gen_layerdeps(ldep.depends_on, project):
+                    yield subdep
+
+        project = Project.objects.get(pk=project_id)
+        result = []
+        projectlvers = [player.layercommit for player in project.projectlayer_set.all()]
+        for dep in gen_layerdeps(self, project):
+            # filter out duplicates and layers already belonging to the project
+            if dep not in result + projectlvers:
+                result.append(dep)
+
+        return sorted(result, key=lambda x: x.layer.name)
+
     def __unicode__(self):
         return "%d %s (VCS %s, Project %s)" % (self.pk, str(self.layer), self.get_vcs_reference(), self.build.project if self.build is not None else "No project")
 
-- 
2.1.4

[PATCH 2/5] toaster: use get_alldeps in layerdetails renderer

[Ed Bartosh]

Used Layer_Version.get_alldeps api in layerdetails template
renderer to get list of layer dependencies.

[YOCTO 8004]

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 bitbake/lib/toaster/toastergui/views.py | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/bitbake/lib/toaster/toastergui/views.py b/bitbake/lib/toaster/toastergui/views.py
index 9f16e8f..3168cfa 100755
--- a/bitbake/lib/toaster/toastergui/views.py
+++ b/bitbake/lib/toaster/toastergui/views.py
@@ -2760,8 +2760,8 @@ if True:
 
         context = { 'project' : project,
                    'layerversion' : layer_version,
-                   'layerdeps' : { "list": [
-                     [{"id": y.id, "name": y.layer.name} for y in x.depends_on.get_equivalents_wpriority(project)][0] for x in layer_version.dependencies.all()]},
+                   'layerdeps' : {"list": [{"id": dep.id, "name": dep.layer.name} \
+                                              for dep in layer_version.get_alldeps(project.id)]},
                    'projectlayers': map(lambda prjlayer: prjlayer.layercommit.id, ProjectLayer.objects.filter(project=project))
                   }
 
-- 
2.1.4

[PATCH 3/5] toaster: fix NameError

[Ed Bartosh]

Fixed exception: NameError: global name 'DoesNotExist' is not defined

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 bitbake/lib/toaster/orm/models.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/bitbake/lib/toaster/orm/models.py b/bitbake/lib/toaster/orm/models.py
index c5e256f..5bc7c6b 100644
--- a/bitbake/lib/toaster/orm/models.py
+++ b/bitbake/lib/toaster/orm/models.py
@@ -900,7 +900,7 @@ class LayerIndexLayerSource(LayerSource):
                     oe_core_l.save()
                     continue
 
-                except DoesNotExist:
+                except Layer.DoesNotExist:
                     pass
 
             l, created = Layer.objects.get_or_create(layer_source = self, name = li['name'])
-- 
2.1.4

[PATCH 4/5] toaster: fix orm tests

[Ed Bartosh]

Cleaned up and fixed orm tests. Removed test_build_layerversion as
it's not needed due to changed compatible_layer_versions API.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 bitbake/lib/toaster/orm/tests.py | 33 ++++-----------------------------
 1 file changed, 4 insertions(+), 29 deletions(-)

diff --git a/bitbake/lib/toaster/orm/tests.py b/bitbake/lib/toaster/orm/tests.py
index 783aea8..c30f99a 100644
--- a/bitbake/lib/toaster/orm/tests.py
+++ b/bitbake/lib/toaster/orm/tests.py
@@ -25,10 +25,9 @@ from django.test import TestCase, TransactionTestCase
 from orm.models import LocalLayerSource, LayerIndexLayerSource, ImportedLayerSource, LayerSource
 from orm.models import Branch
 
-from orm.models import Project, Build, Layer, Layer_Version, Branch, ProjectLayer
+from orm.models import Project, Layer, Layer_Version, Branch, ProjectLayer
 from orm.models import Release, ReleaseLayerSourcePriority, BitbakeVersion
 
-from django.utils import timezone
 from django.db import IntegrityError
 
 import os
@@ -153,35 +152,11 @@ class LayerVersionEquivalenceTestCase(TestCase):
         equivqs = self.lver.get_equivalents_wpriority(self.project)
         self.assertEqual(list(equivqs), [lver2, self.lver])
 
-    def test_build_layerversion(self):
+    def test_compatible_layer_versions(self):
         """
-        Any layer version coming from the build should show up
-        before any layer version coming from upstream
-        """
-        build = Build.objects.create(project=self.project,
-                                     started_on=timezone.now(),
-                                     completed_on=timezone.now())
-        lvb = Layer_Version.objects.create(layer=self.layer, build=build,
-                                           commit="deadbeef")
-
-        # a build layerversion must be in the equivalence
-        # list for the original layerversion
-        equivqs = self.lver.get_equivalents_wpriority(self.project)
-        self.assertTrue(len(equivqs) == 2)
-        self.assertTrue(equivqs[0] == self.lver)
-        self.assertTrue(equivqs[1] == lvb)
-
-        # getting the build layerversion equivalent list must
-        # return the same list as the original layer
-        bequivqs = lvb.get_equivalents_wpriority(self.project)
-
-        self.assertEqual(list(equivqs), list(bequivqs))
-
-    def test_compatible_layerversions(self):
-        """
-        When we have a 2 layer versions, compatible_layerversions()
+        When we have a 2 layer versions, get_all_compatible_layerversions()
         should return a queryset with both.
         """
-        compat_lv = self.project.compatible_layerversions()
+        compat_lv = self.project.get_all_compatible_layer_versions()
         self.assertEqual(list(compat_lv), [self.lver, self.lver2])
 
-- 
2.1.4

[PATCH 5/5] toaster: test get_alldeps API

[Ed Bartosh]

Added test case to test Layer_Version.get_alldeps API.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
---
 bitbake/lib/toaster/orm/tests.py | 20 +++++++++++++++++++-
 1 file changed, 19 insertions(+), 1 deletion(-)

diff --git a/bitbake/lib/toaster/orm/tests.py b/bitbake/lib/toaster/orm/tests.py
index c30f99a..719266e 100644
--- a/bitbake/lib/toaster/orm/tests.py
+++ b/bitbake/lib/toaster/orm/tests.py
@@ -23,7 +23,7 @@
 
 from django.test import TestCase, TransactionTestCase
 from orm.models import LocalLayerSource, LayerIndexLayerSource, ImportedLayerSource, LayerSource
-from orm.models import Branch
+from orm.models import Branch, LayerVersionDependency
 
 from orm.models import Project, Layer, Layer_Version, Branch, ProjectLayer
 from orm.models import Release, ReleaseLayerSourcePriority, BitbakeVersion
@@ -160,3 +160,21 @@ class LayerVersionEquivalenceTestCase(TestCase):
         compat_lv = self.project.get_all_compatible_layer_versions()
         self.assertEqual(list(compat_lv), [self.lver, self.lver2])
 
+    def test_layerversion_get_alldeps(self):
+        """Test Layer_Version.get_alldeps API."""
+        lvers = {}
+        for i in range(10):
+            name = "layer%d" % i
+            lvers[name] = Layer_Version.objects.create(layer=Layer.objects.create(name=name),
+                                                       project=self.project)
+            if i:
+                LayerVersionDependency.objects.create(layer_version=lvers["layer%d" % (i - 1)],
+                                                      depends_on=lvers[name])
+                # Check dinamically added deps
+                self.assertEqual(lvers['layer0'].get_alldeps(self.project.id),
+                                 [lvers['layer%d' % n] for n in range(1, i+1)])
+
+        # Check chain of deps created in previous loop
+        for i in range(10):
+            self.assertEqual(lvers['layer%d' % i].get_alldeps(self.project.id),
+                             [lvers['layer%d' % n] for n in range(i+1, 10)])
-- 
2.1.4

Re: [PATCH v2 0/5] Fix for 8004

[Barros Pena, Belen]

On 01/10/2015 14:56, "toaster-bounces@yoctoproject.org on behalf of Ed
Bartosh" <toaster-bounces@yoctoproject.org on behalf of
ed.bartosh@linux.intel.com> wrote:

>Hi,
>
>This patchset fixes bug 8004: importing a layer from the recipes page
>auto adds the 1st level dependencies but not the 2nd level.
>Now dependent layers are discovered recursively and Toaster shows all of
>them
>in Compatible layers view when user clicks 'Add layer' button.
>
>The patchset includes fixes for orm test suite and new test case for
>Layer_Version.get_alldeps API used in this bug fix.
>
>Changes in v2: UI shows number of first level deps in compatible layers
>page.
>               (Reverted two patches from v1)
>
>The following changes since commit
>7b10ae122d46891c734e786358ea5f36760e8d5e:

I am bit confused. Now the number of dependencies shown in the 'compatible
layers' table matches the dependencies shown in the modal dialog. But I
don't think this is the correct behaviour: the information in the table
should match the information in the layer details page. Those 2 places
(the table and the layer details page) show the 'objective' information
about the layer. The modal dialog, on the other hand, takes into account
the state of the project configuration, and shows you which layers you
still need to add in order to satisfy the layer dependencies.

So both the 'compatible layers' table and the layer details page should
show the fist-level dependencies for a layer. Sorry if I wasn't clear
before :/

Thanks!

Belén

>
>  toaster: Fix broken test case (2015-10-01 10:42:16 +0100)
>
>are available in the git repository at:
>
>  git://git.yoctoproject.org/poky-contrib ed/toaster/8004
>  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/8004
>
>Ed Bartosh (5):
>  toaster: implement API to get full list of deps
>  toaster: use get_alldeps in layerdetails renderer
>  toaster: fix NameError
>  toaster: fix orm tests
>  toaster: test get_alldeps API
>
> bitbake/lib/toaster/orm/models.py       | 21 ++++++++++++-
> bitbake/lib/toaster/orm/tests.py        | 53
>++++++++++++++-------------------
> bitbake/lib/toaster/toastergui/views.py |  4 +--
> 3 files changed, 45 insertions(+), 33 deletions(-)
>
>--
>Regards,
>Ed
>
>-- 
>_______________________________________________
>toaster mailing list
>toaster@yoctoproject.org
>https://lists.yoctoproject.org/listinfo/toaster

Re: [PATCH v2 0/5] Fix for 8004

[Brian Avery]

Hi,
I'm confused as it looks to me like ed's 8004 patch is behaving as you
outlined above;
To reproduce:
1) new proj off master
2) goto layers
3) search meta-shr
4) it shows 4 dependencies in the table which is the number of 1st
level deps it has (see
http://layers.openembedded.org/layerindex/branch/master/layer/meta-shr/
)
5) click add for meta-shr.  the dialog box pops up with the 4 missing
dependencies. Note, the dialog box does not have openembedded-core
which is already in the project and it adds the 2nd level dependency
for meta-systemd.
6) once meta-shr has been added there are  a total of 8 layers in the proj.
7) the meta-shr layer details page show 4 (1st level) deps which match
the ones in the compatible layers table.

This seems correct to me, am I missing something?
-b

On Thu, Oct 1, 2015 at 8:51 AM, Barros Pena, Belen
<belen.barros.pena@intel.com> wrote:
>
>
> On 01/10/2015 14:56, "toaster-bounces@yoctoproject.org on behalf of Ed
> Bartosh" <toaster-bounces@yoctoproject.org on behalf of
> ed.bartosh@linux.intel.com> wrote:
>
>>Hi,
>>
>>This patchset fixes bug 8004: importing a layer from the recipes page
>>auto adds the 1st level dependencies but not the 2nd level.
>>Now dependent layers are discovered recursively and Toaster shows all of
>>them
>>in Compatible layers view when user clicks 'Add layer' button.
>>
>>The patchset includes fixes for orm test suite and new test case for
>>Layer_Version.get_alldeps API used in this bug fix.
>>
>>Changes in v2: UI shows number of first level deps in compatible layers
>>page.
>>               (Reverted two patches from v1)
>>
>>The following changes since commit
>>7b10ae122d46891c734e786358ea5f36760e8d5e:
>
> I am bit confused. Now the number of dependencies shown in the 'compatible
> layers' table matches the dependencies shown in the modal dialog. But I
> don't think this is the correct behaviour: the information in the table
> should match the information in the layer details page. Those 2 places
> (the table and the layer details page) show the 'objective' information
> about the layer. The modal dialog, on the other hand, takes into account
> the state of the project configuration, and shows you which layers you
> still need to add in order to satisfy the layer dependencies.
>
> So both the 'compatible layers' table and the layer details page should
> show the fist-level dependencies for a layer. Sorry if I wasn't clear
> before :/
>
> Thanks!
>
> Belén
>
>>
>>  toaster: Fix broken test case (2015-10-01 10:42:16 +0100)
>>
>>are available in the git repository at:
>>
>>  git://git.yoctoproject.org/poky-contrib ed/toaster/8004
>>  http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/8004
>>
>>Ed Bartosh (5):
>>  toaster: implement API to get full list of deps
>>  toaster: use get_alldeps in layerdetails renderer
>>  toaster: fix NameError
>>  toaster: fix orm tests
>>  toaster: test get_alldeps API
>>
>> bitbake/lib/toaster/orm/models.py       | 21 ++++++++++++-
>> bitbake/lib/toaster/orm/tests.py        | 53
>>++++++++++++++-------------------
>> bitbake/lib/toaster/toastergui/views.py |  4 +--
>> 3 files changed, 45 insertions(+), 33 deletions(-)
>>
>>--
>>Regards,
>>Ed
>>
>>--
>>_______________________________________________
>>toaster mailing list
>>toaster@yoctoproject.org
>>https://lists.yoctoproject.org/listinfo/toaster
>
> --
> _______________________________________________
> toaster mailing list
> toaster@yoctoproject.org
> https://lists.yoctoproject.org/listinfo/toaster

Re: [PATCH v2 0/5] Fix for 8004

[Barros Pena, Belen]

On 01/10/2015 18:56, "Brian Avery" <avery.brian@gmail.com> wrote:

>Hi,
>I'm confused as it looks to me like ed's 8004 patch is behaving as you
>outlined above;
>To reproduce:
>1) new proj off master
>2) goto layers
>3) search meta-shr
>4) it shows 4 dependencies in the table which is the number of 1st
>level deps it has (see
>http://layers.openembedded.org/layerindex/branch/master/layer/meta-shr/
>)
>5) click add for meta-shr.  the dialog box pops up with the 4 missing
>dependencies. Note, the dialog box does not have openembedded-core
>which is already in the project and it adds the 2nd level dependency
>for meta-systemd.
>6) once meta-shr has been added there are  a total of 8 layers in the
>proj.
>7) the meta-shr layer details page show 4 (1st level) deps which match
>the ones in the compatible layers table.
>
>This seems correct to me, am I missing something?

No: I was missing something. This behaves correctly.

Sorry for the confusion.

Cheers

Belén



>-b
>
>On Thu, Oct 1, 2015 at 8:51 AM, Barros Pena, Belen
><belen.barros.pena@intel.com> wrote:
>>
>>
>> On 01/10/2015 14:56, "toaster-bounces@yoctoproject.org on behalf of Ed
>> Bartosh" <toaster-bounces@yoctoproject.org on behalf of
>> ed.bartosh@linux.intel.com> wrote:
>>
>>>Hi,
>>>
>>>This patchset fixes bug 8004: importing a layer from the recipes page
>>>auto adds the 1st level dependencies but not the 2nd level.
>>>Now dependent layers are discovered recursively and Toaster shows all of
>>>them
>>>in Compatible layers view when user clicks 'Add layer' button.
>>>
>>>The patchset includes fixes for orm test suite and new test case for
>>>Layer_Version.get_alldeps API used in this bug fix.
>>>
>>>Changes in v2: UI shows number of first level deps in compatible layers
>>>page.
>>>               (Reverted two patches from v1)
>>>
>>>The following changes since commit
>>>7b10ae122d46891c734e786358ea5f36760e8d5e:
>>
>> I am bit confused. Now the number of dependencies shown in the
>>'compatible
>> layers' table matches the dependencies shown in the modal dialog. But I
>> don't think this is the correct behaviour: the information in the table
>> should match the information in the layer details page. Those 2 places
>> (the table and the layer details page) show the 'objective' information
>> about the layer. The modal dialog, on the other hand, takes into account
>> the state of the project configuration, and shows you which layers you
>> still need to add in order to satisfy the layer dependencies.
>>
>> So both the 'compatible layers' table and the layer details page should
>> show the fist-level dependencies for a layer. Sorry if I wasn't clear
>> before :/
>>
>> Thanks!
>>
>> Belén
>>
>>>
>>>  toaster: Fix broken test case (2015-10-01 10:42:16 +0100)
>>>
>>>are available in the git repository at:
>>>
>>>  git://git.yoctoproject.org/poky-contrib ed/toaster/8004
>>>  
>>>http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/8004
>>>
>>>Ed Bartosh (5):
>>>  toaster: implement API to get full list of deps
>>>  toaster: use get_alldeps in layerdetails renderer
>>>  toaster: fix NameError
>>>  toaster: fix orm tests
>>>  toaster: test get_alldeps API
>>>
>>> bitbake/lib/toaster/orm/models.py       | 21 ++++++++++++-
>>> bitbake/lib/toaster/orm/tests.py        | 53
>>>++++++++++++++-------------------
>>> bitbake/lib/toaster/toastergui/views.py |  4 +--
>>> 3 files changed, 45 insertions(+), 33 deletions(-)
>>>
>>>--
>>>Regards,
>>>Ed
>>>
>>>--
>>>_______________________________________________
>>>toaster mailing list
>>>toaster@yoctoproject.org
>>>https://lists.yoctoproject.org/listinfo/toaster
>>
>> --
>> _______________________________________________
>> toaster mailing list
>> toaster@yoctoproject.org
>> https://lists.yoctoproject.org/listinfo/toaster

Re: [PATCH v2 0/5] Fix for 8004

[Brian Avery]

pushed to bitbake-devel
pushed to toaster-next

-b

On Fri, Oct 2, 2015 at 2:29 AM, Barros Pena, Belen
<belen.barros.pena@intel.com> wrote:
>
>
> On 01/10/2015 18:56, "Brian Avery" <avery.brian@gmail.com> wrote:
>
>>Hi,
>>I'm confused as it looks to me like ed's 8004 patch is behaving as you
>>outlined above;
>>To reproduce:
>>1) new proj off master
>>2) goto layers
>>3) search meta-shr
>>4) it shows 4 dependencies in the table which is the number of 1st
>>level deps it has (see
>>http://layers.openembedded.org/layerindex/branch/master/layer/meta-shr/
>>)
>>5) click add for meta-shr.  the dialog box pops up with the 4 missing
>>dependencies. Note, the dialog box does not have openembedded-core
>>which is already in the project and it adds the 2nd level dependency
>>for meta-systemd.
>>6) once meta-shr has been added there are  a total of 8 layers in the
>>proj.
>>7) the meta-shr layer details page show 4 (1st level) deps which match
>>the ones in the compatible layers table.
>>
>>This seems correct to me, am I missing something?
>
> No: I was missing something. This behaves correctly.
>
> Sorry for the confusion.
>
> Cheers
>
> Belén
>
>
>
>>-b
>>
>>On Thu, Oct 1, 2015 at 8:51 AM, Barros Pena, Belen
>><belen.barros.pena@intel.com> wrote:
>>>
>>>
>>> On 01/10/2015 14:56, "toaster-bounces@yoctoproject.org on behalf of Ed
>>> Bartosh" <toaster-bounces@yoctoproject.org on behalf of
>>> ed.bartosh@linux.intel.com> wrote:
>>>
>>>>Hi,
>>>>
>>>>This patchset fixes bug 8004: importing a layer from the recipes page
>>>>auto adds the 1st level dependencies but not the 2nd level.
>>>>Now dependent layers are discovered recursively and Toaster shows all of
>>>>them
>>>>in Compatible layers view when user clicks 'Add layer' button.
>>>>
>>>>The patchset includes fixes for orm test suite and new test case for
>>>>Layer_Version.get_alldeps API used in this bug fix.
>>>>
>>>>Changes in v2: UI shows number of first level deps in compatible layers
>>>>page.
>>>>               (Reverted two patches from v1)
>>>>
>>>>The following changes since commit
>>>>7b10ae122d46891c734e786358ea5f36760e8d5e:
>>>
>>> I am bit confused. Now the number of dependencies shown in the
>>>'compatible
>>> layers' table matches the dependencies shown in the modal dialog. But I
>>> don't think this is the correct behaviour: the information in the table
>>> should match the information in the layer details page. Those 2 places
>>> (the table and the layer details page) show the 'objective' information
>>> about the layer. The modal dialog, on the other hand, takes into account
>>> the state of the project configuration, and shows you which layers you
>>> still need to add in order to satisfy the layer dependencies.
>>>
>>> So both the 'compatible layers' table and the layer details page should
>>> show the fist-level dependencies for a layer. Sorry if I wasn't clear
>>> before :/
>>>
>>> Thanks!
>>>
>>> Belén
>>>
>>>>
>>>>  toaster: Fix broken test case (2015-10-01 10:42:16 +0100)
>>>>
>>>>are available in the git repository at:
>>>>
>>>>  git://git.yoctoproject.org/poky-contrib ed/toaster/8004
>>>>
>>>>http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/toaster/8004
>>>>
>>>>Ed Bartosh (5):
>>>>  toaster: implement API to get full list of deps
>>>>  toaster: use get_alldeps in layerdetails renderer
>>>>  toaster: fix NameError
>>>>  toaster: fix orm tests
>>>>  toaster: test get_alldeps API
>>>>
>>>> bitbake/lib/toaster/orm/models.py       | 21 ++++++++++++-
>>>> bitbake/lib/toaster/orm/tests.py        | 53
>>>>++++++++++++++-------------------
>>>> bitbake/lib/toaster/toastergui/views.py |  4 +--
>>>> 3 files changed, 45 insertions(+), 33 deletions(-)
>>>>
>>>>--
>>>>Regards,
>>>>Ed
>>>>
>>>>--
>>>>_______________________________________________
>>>>toaster mailing list
>>>>toaster@yoctoproject.org
>>>>https://lists.yoctoproject.org/listinfo/toaster
>>>
>>> --
>>> _______________________________________________
>>> toaster mailing list
>>> toaster@yoctoproject.org
>>> https://lists.yoctoproject.org/listinfo/toaster
>

[PATCH 1/5] toaster: implement API to get full list of deps

[brian avery]

From: Ed Bartosh <ed.bartosh@linux.intel.com>

Implemented Layer_Version.get_alldeps API to recursively get
full list of dependencies for the layer. Dependencies that are
already in the project are filtered out from the result.
Result list of Layer_Version objects is sorted by layer name
for UI to look consistent.

This API is going to be used to show amount and list of
dependencies for the layer in the list of compatible layers
for the project.

Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
Signed-off-by: brian avery <avery.brian@gmail.com>
---
 lib/toaster/orm/models.py | 19 +++++++++++++++++++
 1 file changed, 19 insertions(+)

diff --git a/lib/toaster/orm/models.py b/lib/toaster/orm/models.py
index ad12dd4..431bd8b 100644
--- a/lib/toaster/orm/models.py
+++ b/lib/toaster/orm/models.py
@@ -1178,6 +1178,25 @@ class Layer_Version(models.Model):
     def get_detailspage_url(self, project_id):
         return reverse('layerdetails', args=(project_id, self.pk))
 
+    def get_alldeps(self, project_id):
+        """Get full list of unique layer dependencies."""
+        def gen_layerdeps(lver, project):
+            for ldep in lver.dependencies.all():
+                yield ldep.depends_on
+                # get next level of deps recursively calling gen_layerdeps
+                for subdep in gen_layerdeps(ldep.depends_on, project):
+                    yield subdep
+
+        project = Project.objects.get(pk=project_id)
+        result = []
+        projectlvers = [player.layercommit for player in project.projectlayer_set.all()]
+        for dep in gen_layerdeps(self, project):
+            # filter out duplicates and layers already belonging to the project
+            if dep not in result + projectlvers:
+                result.append(dep)
+
+        return sorted(result, key=lambda x: x.layer.name)
+
     def __unicode__(self):
         return "%d %s (VCS %s, Project %s)" % (self.pk, str(self.layer), self.get_vcs_reference(), self.build.project if self.build is not None else "No project")
 
-- 
1.9.1

Re: [PATCH 1/5] toaster: implement API to get full list of deps

[Michael Wood]

On 01/10/15 14:56, Ed Bartosh wrote:
> Implemented Layer_Version.get_alldeps API to recursively get
> full list of dependencies for the layer. Dependencies that are
> already in the project are filtered out from the result.
> Result list of Layer_Version objects is sorted by layer name
> for UI to look consistent.
>
> This API is going to be used to show amount and list of
> dependencies for the layer in the list of compatible layers
> for the project.
>
> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
> ---
>   bitbake/lib/toaster/orm/models.py | 19 +++++++++++++++++++
>   1 file changed, 19 insertions(+)
>
> diff --git a/bitbake/lib/toaster/orm/models.py b/bitbake/lib/toaster/orm/models.py
> index 5aed158..c5e256f 100644
> --- a/bitbake/lib/toaster/orm/models.py
> +++ b/bitbake/lib/toaster/orm/models.py
> @@ -1177,6 +1177,25 @@ class Layer_Version(models.Model):
>       def get_detailspage_url(self, project_id):
>           return reverse('layerdetails', args=(project_id, self.pk))
>   
> +    def get_alldeps(self, project_id):
> +        """Get full list of unique layer dependencies."""
> +        def gen_layerdeps(lver, project):
> +            for ldep in lver.dependencies.all():
> +                yield ldep.depends_on
> +                # get next level of deps recursively calling gen_layerdeps
> +                for subdep in gen_layerdeps(ldep.depends_on, project):
> +                    yield subdep
> +
> +        project = Project.objects.get(pk=project_id)
> +        result = []
> +        projectlvers = [player.layercommit for player in project.projectlayer_set.all()]
> +        for dep in gen_layerdeps(self, project):
> +            # filter out duplicates and layers already belonging to the project
> +            if dep not in result + projectlvers:
> +                result.append(dep)
> +
> +        return sorted(result, key=lambda x: x.layer.name)
> +
>       def __unicode__(self):
>           return "%d %s (VCS %s, Project %s)" % (self.pk, str(self.layer), self.get_vcs_reference(), self.build.project if self.build is not None else "No project")
>   

Sorry for too late review, spotted a regression.

This will cause a regression in the layerdetails page, the API should 
not be filtering out any layer dependencies that are already added to 
the project.

- This is already done in the client side logic which is why the context 
has projectlayers in it
- The layerdetails page is supposed to list the dependencies for the 
layer regardless as to whether they have been added or not.

(btw you can do something like this 
self.dependencies.exclude(depends_on__in=prj.get_project_layer_versions()) 
if you want a list of the layers it depends on but are not added, rather 
than iterate through these and create a new list)

As it's been submitted would you be able to write a patch on top to fix 
this.

Michael

Re: [PATCH 1/5] toaster: implement API to get full list of deps

[Michael Wood]

On 07/10/15 17:17, Michael Wood wrote:
> On 01/10/15 14:56, Ed Bartosh wrote:
>> Implemented Layer_Version.get_alldeps API to recursively get
>> full list of dependencies for the layer. Dependencies that are
>> already in the project are filtered out from the result.
>> Result list of Layer_Version objects is sorted by layer name
>> for UI to look consistent.
>>
>> This API is going to be used to show amount and list of
>> dependencies for the layer in the list of compatible layers
>> for the project.
>>
>> Signed-off-by: Ed Bartosh <ed.bartosh@linux.intel.com>
>> ---
>>   bitbake/lib/toaster/orm/models.py | 19 +++++++++++++++++++
>>   1 file changed, 19 insertions(+)
>>
>> diff --git a/bitbake/lib/toaster/orm/models.py 
>> b/bitbake/lib/toaster/orm/models.py
>> index 5aed158..c5e256f 100644
>> --- a/bitbake/lib/toaster/orm/models.py
>> +++ b/bitbake/lib/toaster/orm/models.py
>> @@ -1177,6 +1177,25 @@ class Layer_Version(models.Model):
>>       def get_detailspage_url(self, project_id):
>>           return reverse('layerdetails', args=(project_id, self.pk))
>>   +    def get_alldeps(self, project_id):
>> +        """Get full list of unique layer dependencies."""
>> +        def gen_layerdeps(lver, project):
>> +            for ldep in lver.dependencies.all():
>> +                yield ldep.depends_on
>> +                # get next level of deps recursively calling 
>> gen_layerdeps
>> +                for subdep in gen_layerdeps(ldep.depends_on, project):
>> +                    yield subdep
>> +
>> +        project = Project.objects.get(pk=project_id)
>> +        result = []
>> +        projectlvers = [player.layercommit for player in 
>> project.projectlayer_set.all()]
>> +        for dep in gen_layerdeps(self, project):
>> +            # filter out duplicates and layers already belonging to 
>> the project
>> +            if dep not in result + projectlvers:
>> +                result.append(dep)
>> +
>> +        return sorted(result, key=lambda x: x.layer.name)
>> +
>>       def __unicode__(self):
>>           return "%d %s (VCS %s, Project %s)" % (self.pk, 
>> str(self.layer), self.get_vcs_reference(), self.build.project if 
>> self.build is not None else "No project")
>
> Sorry for too late review, spotted a regression.
>
> This will cause a regression in the layerdetails page, the API should 
> not be filtering out any layer dependencies that are already added to 
> the project.
>
> - This is already done in the client side logic which is why the 
> context has projectlayers in it
> - The layerdetails page is supposed to list the dependencies for the 
> layer regardless as to whether they have been added or not.
>
> (btw you can do something like this 
> self.dependencies.exclude(depends_on__in=prj.get_project_layer_versions()) 
> if you want a list of the layers it depends on but are not added, 
> rather than iterate through these and create a new list)
>
> As it's been submitted would you be able to write a patch on top to 
> fix this.
>
> Michael

Ah so I might be wrong here, later on in the thread Belen mentioned:

On 01/10/15 16:51, Barros Pena, Belen wrote:
> So both the 'compatible layers' table and the layer details page should
> show the fist-level dependencies for a layer

Which contradicts the bug I was looking at 
https://bugzilla.yoctoproject.org/show_bug.cgi?id=8042
I'll push that bug to you Belen so you can decide what to do.

Thanks,

Michael