[PATCH 0/1] archiver.bbclass: exclude recursive symlink for diff

[PATCH 0/1] archiver.bbclass: exclude recursive symlink for diff

[Robert Yang]

The following changes since commit 755f7d9afaab4e7424f5f57587aa45df3807acf0:

  pulseaudio: disable GTK+ by default (2013-09-02 18:04:59 +0100)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib robert/archiver
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/archiver

Robert Yang (1):
  archiver.bbclass: exclude recursive symlink for diff

 meta/classes/archiver.bbclass |   18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

-- 
1.7.10.4

[PATCH 1/1] archiver.bbclass: exclude recursive symlink for diff

[Robert Yang]

There would be errors if there are recursive symlinks in the ${S}:

diff: /xxx/udev/182-r8/udev-182/test/sys/block/loop0/bdi/subsystem/0:16/subsystem: recursive directory loop

We can check the recursive symlink and save it to the exclude file, then
the diff's --exclude-from argument can exclude them

[YOCTO #4986]

Signed-off-by: Valentin Cobelea <valentin.cobelea@enea.com>
Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/archiver.bbclass |   18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
index 66efe7d..f5ad562 100644
--- a/meta/classes/archiver.bbclass
+++ b/meta/classes/archiver.bbclass
@@ -493,6 +493,24 @@ def create_diff_gz(d):
     for i in exclude_from:
         f.write(i)
         f.write("\n")
+
+    symlinks = []
+    directories = []
+
+    # Add symlink and root dir to the separated lists for checking whether
+    # there is recursive symlink
+    for root, dirs, files in os.walk(os.getcwd(), followlinks=True):
+        if os.path.islink(root):
+            symlinks.append(root)
+        elif os.path.isdir(root):
+            directories.append(root)
+
+    # Check if there is a recursive symlink to a dir
+    for sym in symlinks:
+        if sym in directories:
+            # write the 'sym' in the 'exclude-from' file
+            f.write(os.path.basename(sym))
+
     f.close()
 
     s=d.getVar('S', True)
-- 
1.7.10.4

Re: [PATCH 1/1] archiver.bbclass: exclude recursive symlink for diff

[Robert Yang]

Please hold this patch for a while, seems that it didn't fix all of the things.

// Robert

On 09/03/2013 07:51 PM, Robert Yang wrote:
> There would be errors if there are recursive symlinks in the ${S}:
>
> diff: /xxx/udev/182-r8/udev-182/test/sys/block/loop0/bdi/subsystem/0:16/subsystem: recursive directory loop
>
> We can check the recursive symlink and save it to the exclude file, then
> the diff's --exclude-from argument can exclude them
>
> [YOCTO #4986]
>
> Signed-off-by: Valentin Cobelea <valentin.cobelea@enea.com>
> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
>   meta/classes/archiver.bbclass |   18 ++++++++++++++++++
>   1 file changed, 18 insertions(+)
>
> diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
> index 66efe7d..f5ad562 100644
> --- a/meta/classes/archiver.bbclass
> +++ b/meta/classes/archiver.bbclass
> @@ -493,6 +493,24 @@ def create_diff_gz(d):
>       for i in exclude_from:
>           f.write(i)
>           f.write("\n")
> +
> +    symlinks = []
> +    directories = []
> +
> +    # Add symlink and root dir to the separated lists for checking whether
> +    # there is recursive symlink
> +    for root, dirs, files in os.walk(os.getcwd(), followlinks=True):
> +        if os.path.islink(root):
> +            symlinks.append(root)
> +        elif os.path.isdir(root):
> +            directories.append(root)
> +
> +    # Check if there is a recursive symlink to a dir
> +    for sym in symlinks:
> +        if sym in directories:
> +            # write the 'sym' in the 'exclude-from' file
> +            f.write(os.path.basename(sym))
> +
>       f.close()
>
>       s=d.getVar('S', True)
>

Re: [PATCH 1/1] archiver.bbclass: exclude recursive symlink for diff

[Vali Cobelea]

Hi,

What seems to be the issue ?

BR,
Valentin


On 09/03/2013 03:51 PM, Robert Yang wrote:
>
> Please hold this patch for a while, seems that it didn't fix all of 
> the things.
>
> // Robert
>
> On 09/03/2013 07:51 PM, Robert Yang wrote:
>> There would be errors if there are recursive symlinks in the ${S}:
>>
>> diff: 
>> /xxx/udev/182-r8/udev-182/test/sys/block/loop0/bdi/subsystem/0:16/subsystem: 
>> recursive directory loop
>>
>> We can check the recursive symlink and save it to the exclude file, then
>> the diff's --exclude-from argument can exclude them
>>
>> [YOCTO #4986]
>>
>> Signed-off-by: Valentin Cobelea <valentin.cobelea@enea.com>
>> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
>> ---
>>   meta/classes/archiver.bbclass |   18 ++++++++++++++++++
>>   1 file changed, 18 insertions(+)
>>
>> diff --git a/meta/classes/archiver.bbclass 
>> b/meta/classes/archiver.bbclass
>> index 66efe7d..f5ad562 100644
>> --- a/meta/classes/archiver.bbclass
>> +++ b/meta/classes/archiver.bbclass
>> @@ -493,6 +493,24 @@ def create_diff_gz(d):
>>       for i in exclude_from:
>>           f.write(i)
>>           f.write("\n")
>> +
>> +    symlinks = []
>> +    directories = []
>> +
>> +    # Add symlink and root dir to the separated lists for checking 
>> whether
>> +    # there is recursive symlink
>> +    for root, dirs, files in os.walk(os.getcwd(), followlinks=True):
>> +        if os.path.islink(root):
>> +            symlinks.append(root)
>> +        elif os.path.isdir(root):
>> +            directories.append(root)
>> +
>> +    # Check if there is a recursive symlink to a dir
>> +    for sym in symlinks:
>> +        if sym in directories:
>> +            # write the 'sym' in the 'exclude-from' file
>> +            f.write(os.path.basename(sym))
>> +
>>       f.close()
>>
>>       s=d.getVar('S', True)
>>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH 1/1] archiver.bbclass: exclude recursive symlink for diff

[Robert Yang]

Hi Valentin,

It hangs up when build acl (which means that I can reproduce the
problem as you have told), but it worked well without this patch,
maybe python version related, I will do more investigation.

// Robert

On 09/03/2013 08:53 PM, Vali Cobelea wrote:
> Hi,
>
> What seems to be the issue ?
>
> BR,
> Valentin
>
>
> On 09/03/2013 03:51 PM, Robert Yang wrote:
>>
>> Please hold this patch for a while, seems that it didn't fix all of the things.
>>
>> // Robert
>>
>> On 09/03/2013 07:51 PM, Robert Yang wrote:
>>> There would be errors if there are recursive symlinks in the ${S}:
>>>
>>> diff:
>>> /xxx/udev/182-r8/udev-182/test/sys/block/loop0/bdi/subsystem/0:16/subsystem:
>>> recursive directory loop
>>>
>>> We can check the recursive symlink and save it to the exclude file, then
>>> the diff's --exclude-from argument can exclude them
>>>
>>> [YOCTO #4986]
>>>
>>> Signed-off-by: Valentin Cobelea <valentin.cobelea@enea.com>
>>> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
>>> ---
>>>   meta/classes/archiver.bbclass |   18 ++++++++++++++++++
>>>   1 file changed, 18 insertions(+)
>>>
>>> diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
>>> index 66efe7d..f5ad562 100644
>>> --- a/meta/classes/archiver.bbclass
>>> +++ b/meta/classes/archiver.bbclass
>>> @@ -493,6 +493,24 @@ def create_diff_gz(d):
>>>       for i in exclude_from:
>>>           f.write(i)
>>>           f.write("\n")
>>> +
>>> +    symlinks = []
>>> +    directories = []
>>> +
>>> +    # Add symlink and root dir to the separated lists for checking whether
>>> +    # there is recursive symlink
>>> +    for root, dirs, files in os.walk(os.getcwd(), followlinks=True):
>>> +        if os.path.islink(root):
>>> +            symlinks.append(root)
>>> +        elif os.path.isdir(root):
>>> +            directories.append(root)
>>> +
>>> +    # Check if there is a recursive symlink to a dir
>>> +    for sym in symlinks:
>>> +        if sym in directories:
>>> +            # write the 'sym' in the 'exclude-from' file
>>> +            f.write(os.path.basename(sym))
>>> +
>>>       f.close()
>>>
>>>       s=d.getVar('S', True)
>>>
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
>
>

Re: [PATCH 1/1] archiver.bbclass: exclude recursive symlink for diff

[Vali Cobelea]

Hi,

You can try even without the patch and see if you reproduce the issue.
We do have the symlinks recursivity problem as I said but on random 
packages, "udev" was the first we noticed.

Best regards,
Valentin

On 09/04/2013 03:57 AM, Robert Yang wrote:
>
> Hi Valentin,
>
> It hangs up when build acl (which means that I can reproduce the
> problem as you have told), but it worked well without this patch,
> maybe python version related, I will do more investigation.
>
> // Robert
>
> On 09/03/2013 08:53 PM, Vali Cobelea wrote:
>> Hi,
>>
>> What seems to be the issue ?
>>
>> BR,
>> Valentin
>>
>>
>> On 09/03/2013 03:51 PM, Robert Yang wrote:
>>>
>>> Please hold this patch for a while, seems that it didn't fix all of 
>>> the things.
>>>
>>> // Robert
>>>
>>> On 09/03/2013 07:51 PM, Robert Yang wrote:
>>>> There would be errors if there are recursive symlinks in the ${S}:
>>>>
>>>> diff:
>>>> /xxx/udev/182-r8/udev-182/test/sys/block/loop0/bdi/subsystem/0:16/subsystem: 
>>>>
>>>> recursive directory loop
>>>>
>>>> We can check the recursive symlink and save it to the exclude file, 
>>>> then
>>>> the diff's --exclude-from argument can exclude them
>>>>
>>>> [YOCTO #4986]
>>>>
>>>> Signed-off-by: Valentin Cobelea <valentin.cobelea@enea.com>
>>>> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
>>>> ---
>>>>   meta/classes/archiver.bbclass |   18 ++++++++++++++++++
>>>>   1 file changed, 18 insertions(+)
>>>>
>>>> diff --git a/meta/classes/archiver.bbclass 
>>>> b/meta/classes/archiver.bbclass
>>>> index 66efe7d..f5ad562 100644
>>>> --- a/meta/classes/archiver.bbclass
>>>> +++ b/meta/classes/archiver.bbclass
>>>> @@ -493,6 +493,24 @@ def create_diff_gz(d):
>>>>       for i in exclude_from:
>>>>           f.write(i)
>>>>           f.write("\n")
>>>> +
>>>> +    symlinks = []
>>>> +    directories = []
>>>> +
>>>> +    # Add symlink and root dir to the separated lists for checking 
>>>> whether
>>>> +    # there is recursive symlink
>>>> +    for root, dirs, files in os.walk(os.getcwd(), followlinks=True):
>>>> +        if os.path.islink(root):
>>>> +            symlinks.append(root)
>>>> +        elif os.path.isdir(root):
>>>> +            directories.append(root)
>>>> +
>>>> +    # Check if there is a recursive symlink to a dir
>>>> +    for sym in symlinks:
>>>> +        if sym in directories:
>>>> +            # write the 'sym' in the 'exclude-from' file
>>>> +            f.write(os.path.basename(sym))
>>>> +
>>>>       f.close()
>>>>
>>>>       s=d.getVar('S', True)
>>>>
>>> _______________________________________________
>>> Openembedded-core mailing list
>>> Openembedded-core@lists.openembedded.org
>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>
>>
>>

Re: [PATCH 1/1] archiver.bbclass: exclude recursive symlink for diff

[Robert Yang]

On 09/04/2013 03:22 PM, Vali Cobelea wrote:
> Hi,
>
> You can try even without the patch and see if you reproduce the issue.
> We do have the symlinks recursivity problem as I said but on random packages,
> "udev" was the first we noticed.

I can't reproduce the error, but it is really a problem, I've added a patch
in the bugzilla, and will send it here sooner.

// Robert

>
> Best regards,
> Valentin
>
> On 09/04/2013 03:57 AM, Robert Yang wrote:
>>
>> Hi Valentin,
>>
>> It hangs up when build acl (which means that I can reproduce the
>> problem as you have told), but it worked well without this patch,
>> maybe python version related, I will do more investigation.
>>
>> // Robert
>>
>> On 09/03/2013 08:53 PM, Vali Cobelea wrote:
>>> Hi,
>>>
>>> What seems to be the issue ?
>>>
>>> BR,
>>> Valentin
>>>
>>>
>>> On 09/03/2013 03:51 PM, Robert Yang wrote:
>>>>
>>>> Please hold this patch for a while, seems that it didn't fix all of the things.
>>>>
>>>> // Robert
>>>>
>>>> On 09/03/2013 07:51 PM, Robert Yang wrote:
>>>>> There would be errors if there are recursive symlinks in the ${S}:
>>>>>
>>>>> diff:
>>>>> /xxx/udev/182-r8/udev-182/test/sys/block/loop0/bdi/subsystem/0:16/subsystem:
>>>>> recursive directory loop
>>>>>
>>>>> We can check the recursive symlink and save it to the exclude file, then
>>>>> the diff's --exclude-from argument can exclude them
>>>>>
>>>>> [YOCTO #4986]
>>>>>
>>>>> Signed-off-by: Valentin Cobelea <valentin.cobelea@enea.com>
>>>>> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
>>>>> ---
>>>>>   meta/classes/archiver.bbclass |   18 ++++++++++++++++++
>>>>>   1 file changed, 18 insertions(+)
>>>>>
>>>>> diff --git a/meta/classes/archiver.bbclass b/meta/classes/archiver.bbclass
>>>>> index 66efe7d..f5ad562 100644
>>>>> --- a/meta/classes/archiver.bbclass
>>>>> +++ b/meta/classes/archiver.bbclass
>>>>> @@ -493,6 +493,24 @@ def create_diff_gz(d):
>>>>>       for i in exclude_from:
>>>>>           f.write(i)
>>>>>           f.write("\n")
>>>>> +
>>>>> +    symlinks = []
>>>>> +    directories = []
>>>>> +
>>>>> +    # Add symlink and root dir to the separated lists for checking whether
>>>>> +    # there is recursive symlink
>>>>> +    for root, dirs, files in os.walk(os.getcwd(), followlinks=True):
>>>>> +        if os.path.islink(root):
>>>>> +            symlinks.append(root)
>>>>> +        elif os.path.isdir(root):
>>>>> +            directories.append(root)
>>>>> +
>>>>> +    # Check if there is a recursive symlink to a dir
>>>>> +    for sym in symlinks:
>>>>> +        if sym in directories:
>>>>> +            # write the 'sym' in the 'exclude-from' file
>>>>> +            f.write(os.path.basename(sym))
>>>>> +
>>>>>       f.close()
>>>>>
>>>>>       s=d.getVar('S', True)
>>>>>
>>>> _______________________________________________
>>>> Openembedded-core mailing list
>>>> Openembedded-core@lists.openembedded.org
>>>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>>
>>>
>>>
>
>
>