[PATCH 0/3] make DESCRIPTION support newline

[PATCH 0/3] make DESCRIPTION support newline

[Robert Yang]

The following changes since commit 590010a6525b0e1bc1de73e794764e23404591df:

  core-image-weston: add weston-examples to the image (2013-06-18 17:33:17 +0100)

are available in the git repository at:

  git://git.pokylinux.org/poky-contrib robert/newline
  http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/newline

Robert Yang (3):
  package_rpm.bbclass: make DESCRIPTION support newline
  package_ipk.bbclass: make DESCRIPTION support newline
  package_deb.bbclass: make DESCRIPTION support newline

 meta/classes/package_deb.bbclass |    5 +++--
 meta/classes/package_ipk.bbclass |    5 +++--
 meta/classes/package_rpm.bbclass |   19 ++++++++++++++-----
 3 files changed, 20 insertions(+), 9 deletions(-)

-- 
1.7.10.4

[PATCH 1/3] package_rpm.bbclass: make DESCRIPTION support newline

[Robert Yang]

The recipe's DESCRIPTION is wrapped automatically by textwrap, make it
support newline ("\n") to let the user can wrap it manually, e.g.:

DESCRIPTION = "Foo1\nFoo2"

In the past, it would be:
Foo1\nFoo2

Now:
Foo1
Foo2

[YOCTO #4348]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/package_rpm.bbclass |   19 ++++++++++++++-----
 1 file changed, 14 insertions(+), 5 deletions(-)

diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
index c654cdb..01bfaf1 100644
--- a/meta/classes/package_rpm.bbclass
+++ b/meta/classes/package_rpm.bbclass
@@ -534,7 +534,6 @@ def write_rpm_perfiledata(srcname, d):
 
 
 python write_specfile () {
-    import textwrap
     import oe.packagedata
 
     # append information for logs and patches to %prep
@@ -668,6 +667,18 @@ python write_specfile () {
                 deps.append(depends)
         return " ".join(deps)
 
+    def append_description(spec_preamble, text):
+        """
+        Add the description to the spec file, respect to the rules:
+        - split the DESCRIPTION into multi-lines by newline (\n), and
+        - the text width is 75.
+        """
+        import textwrap
+        dedent_text = textwrap.dedent(text).strip()
+        # Bitbake saves "\n" as "\\n"
+        for t in dedent_text.split('\\n'):
+            spec_preamble.append('%s' % textwrap.fill(t.strip(), width=75))
+
     packages = d.getVar('PACKAGES', True)
     if not packages or packages == '':
         bb.debug(1, "No packages; nothing to do")
@@ -868,8 +879,7 @@ python write_specfile () {
         spec_preamble_bottom.append('')
 
         spec_preamble_bottom.append('%%description -n %s' % splitname)
-        dedent_text = textwrap.dedent(splitdescription).strip()
-        spec_preamble_bottom.append('%s' % textwrap.fill(dedent_text, width=75))
+        append_description(spec_preamble_bottom, splitdescription)
 
         spec_preamble_bottom.append('')
 
@@ -975,8 +985,7 @@ python write_specfile () {
     spec_preamble_top.append('')
 
     spec_preamble_top.append('%description')
-    dedent_text = textwrap.dedent(srcdescription).strip()
-    spec_preamble_top.append('%s' % textwrap.fill(dedent_text, width=75))
+    append_description(spec_preamble_top, srcdescription)
 
     spec_preamble_top.append('')
 
-- 
1.7.10.4

[PATCH 2/3] package_ipk.bbclass: make DESCRIPTION support newline

[Robert Yang]

The recipe's DESCRIPTION is wrapped automatically by textwrap, make it
support newline ("\n") to let the user can wrap it manually, e.g.:

DESCRIPTION = "Foo1\nFoo2"

In the past, it would be:
Foo1\nFoo2

Now:
Foo1
Foo2

[YOCTO #4348]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/package_ipk.bbclass |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
index 55628e4..e67f641 100644
--- a/meta/classes/package_ipk.bbclass
+++ b/meta/classes/package_ipk.bbclass
@@ -304,10 +304,11 @@ python do_package_ipk () {
                 # Special behavior for description...
                 if 'DESCRIPTION' in fs:
                     summary = localdata.getVar('SUMMARY', True) or localdata.getVar('DESCRIPTION', True) or "."
+                    ctrlfile.write('Description: %s\n' % summary)
                     description = localdata.getVar('DESCRIPTION', True) or "."
                     description = textwrap.dedent(description).strip()
-                    ctrlfile.write('Description: %s\n' % summary)
-                    ctrlfile.write('%s\n' % textwrap.fill(description, width=74, initial_indent=' ', subsequent_indent=' '))
+                    for t in description.split('\\n'):
+                        ctrlfile.write('%s\n' % textwrap.fill(t, width=74, initial_indent=' ', subsequent_indent=' '))
                 else:
                     ctrlfile.write(c % tuple(pullData(fs, localdata)))
         except KeyError:
-- 
1.7.10.4

Re: [PATCH 2/3] package_ipk.bbclass: make DESCRIPTION support newline

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 1907 bytes --]

On Wed, Jun 19, 2013 at 05:00:40AM -0400, Robert Yang wrote:
> The recipe's DESCRIPTION is wrapped automatically by textwrap, make it
> support newline ("\n") to let the user can wrap it manually, e.g.:
> 
> DESCRIPTION = "Foo1\nFoo2"
> 
> In the past, it would be:
> Foo1\nFoo2
> 
> Now:
> Foo1
> Foo2
> 
> [YOCTO #4348]
> 
> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
>  meta/classes/package_ipk.bbclass |    5 +++--
>  1 file changed, 3 insertions(+), 2 deletions(-)
> 
> diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
> index 55628e4..e67f641 100644
> --- a/meta/classes/package_ipk.bbclass
> +++ b/meta/classes/package_ipk.bbclass
> @@ -304,10 +304,11 @@ python do_package_ipk () {
>                  # Special behavior for description...
>                  if 'DESCRIPTION' in fs:
>                      summary = localdata.getVar('SUMMARY', True) or localdata.getVar('DESCRIPTION', True) or "."
> +                    ctrlfile.write('Description: %s\n' % summary)
>                      description = localdata.getVar('DESCRIPTION', True) or "."
>                      description = textwrap.dedent(description).strip()
> -                    ctrlfile.write('Description: %s\n' % summary)
> -                    ctrlfile.write('%s\n' % textwrap.fill(description, width=74, initial_indent=' ', subsequent_indent=' '))
> +                    for t in description.split('\\n'):
> +                        ctrlfile.write('%s\n' % textwrap.fill(t, width=74, initial_indent=' ', subsequent_indent=' '))
>                  else:

Isn't DESCRIPTION supposed to be short oneline and longer multiline only
in SUMMARY?

is opkg-utils (package-index) working with this? Newlines in SUMMARY
were causing incorrect parsing and not using cache IIRC, not sure if the
fix for that was generic enough to cover DESCRIPTION.

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: [PATCH 2/3] package_ipk.bbclass: make DESCRIPTION support newline

[Richard Purdie]

On Wed, 2013-06-19 at 11:29 +0200, Martin Jansa wrote:
> On Wed, Jun 19, 2013 at 05:00:40AM -0400, Robert Yang wrote:
> > The recipe's DESCRIPTION is wrapped automatically by textwrap, make it
> > support newline ("\n") to let the user can wrap it manually, e.g.:
> > 
> > DESCRIPTION = "Foo1\nFoo2"
> > 
> > In the past, it would be:
> > Foo1\nFoo2
> > 
> > Now:
> > Foo1
> > Foo2
> > 
> > [YOCTO #4348]
> > 
> > Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> > ---
> >  meta/classes/package_ipk.bbclass |    5 +++--
> >  1 file changed, 3 insertions(+), 2 deletions(-)
> > 
> > diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
> > index 55628e4..e67f641 100644
> > --- a/meta/classes/package_ipk.bbclass
> > +++ b/meta/classes/package_ipk.bbclass
> > @@ -304,10 +304,11 @@ python do_package_ipk () {
> >                  # Special behavior for description...
> >                  if 'DESCRIPTION' in fs:
> >                      summary = localdata.getVar('SUMMARY', True) or localdata.getVar('DESCRIPTION', True) or "."
> > +                    ctrlfile.write('Description: %s\n' % summary)
> >                      description = localdata.getVar('DESCRIPTION', True) or "."
> >                      description = textwrap.dedent(description).strip()
> > -                    ctrlfile.write('Description: %s\n' % summary)
> > -                    ctrlfile.write('%s\n' % textwrap.fill(description, width=74, initial_indent=' ', subsequent_indent=' '))
> > +                    for t in description.split('\\n'):
> > +                        ctrlfile.write('%s\n' % textwrap.fill(t, width=74, initial_indent=' ', subsequent_indent=' '))
> >                  else:
> 
> Isn't DESCRIPTION supposed to be short oneline and longer multiline only
> in SUMMARY?

No, SUMMARY is meant to be the short oneline and DESCRIPTION is the
multiline one afaik...

Cheers,

Richard

> is opkg-utils (package-index) working with this? Newlines in SUMMARY
> were causing incorrect parsing and not using cache IIRC, not sure if the
> fix for that was generic enough to cover DESCRIPTION.
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core

Re: [PATCH 2/3] package_ipk.bbclass: make DESCRIPTION support newline

[Phil Blundell]

On Wed, 2013-06-19 at 11:29 +0200, Martin Jansa wrote:
> On Wed, Jun 19, 2013 at 05:00:40AM -0400, Robert Yang wrote:
> > The recipe's DESCRIPTION is wrapped automatically by textwrap, make it
> > support newline ("\n") to let the user can wrap it manually, e.g.:
> > 
> > DESCRIPTION = "Foo1\nFoo2"
> > 
> > In the past, it would be:
> > Foo1\nFoo2
> > 
> > Now:
> > Foo1
> > Foo2
> > 
> > [YOCTO #4348]
> > 
> > Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> > ---
> >  meta/classes/package_ipk.bbclass |    5 +++--
> >  1 file changed, 3 insertions(+), 2 deletions(-)
> > 
> > diff --git a/meta/classes/package_ipk.bbclass b/meta/classes/package_ipk.bbclass
> > index 55628e4..e67f641 100644
> > --- a/meta/classes/package_ipk.bbclass
> > +++ b/meta/classes/package_ipk.bbclass
> > @@ -304,10 +304,11 @@ python do_package_ipk () {
> >                  # Special behavior for description...
> >                  if 'DESCRIPTION' in fs:
> >                      summary = localdata.getVar('SUMMARY', True) or localdata.getVar('DESCRIPTION', True) or "."
> > +                    ctrlfile.write('Description: %s\n' % summary)
> >                      description = localdata.getVar('DESCRIPTION', True) or "."
> >                      description = textwrap.dedent(description).strip()
> > -                    ctrlfile.write('Description: %s\n' % summary)
> > -                    ctrlfile.write('%s\n' % textwrap.fill(description, width=74, initial_indent=' ', subsequent_indent=' '))
> > +                    for t in description.split('\\n'):
> > +                        ctrlfile.write('%s\n' % textwrap.fill(t, width=74, initial_indent=' ', subsequent_indent=' '))
> >                  else:
> 
> Isn't DESCRIPTION supposed to be short oneline and longer multiline only
> in SUMMARY?

No, the reverse.  SUMMARY is (as the name suggests, and as the existing
code implements) meant to be the short synopsis and DESCRIPTION is the
longer text. 

> is opkg-utils (package-index) working with this? Newlines in SUMMARY
> were causing incorrect parsing and not using cache IIRC, not sure if the
> fix for that was generic enough to cover DESCRIPTION.

Newlines in SUMMARY would indeed break opkg-utils because the text after
the line break wouldn't be correctly indented.  It wouldn't be a bad
thing for package_ipk to check for that, but this shouldn't be an issue
with the patch above since it seems to be explicitly indenting all lines
of the long description.

I'm not entirely thrilled with Robert's patch because it seems rather
ugly to have embedded "\n" sequences in the recipe DESCRIPTION fields,
and also because package_ipk isn't the only consumer of that variable
and it isn't entirely obvious that all the other users will know what to
do.  But, absent a way to express string literals with embedded newlines
in bitbake, perhaps this is indeed the best compromise.

p.

Re: [PATCH 2/3] package_ipk.bbclass: make DESCRIPTION support newline

[Enrico Scholz]

Robert Yang <liezhi.yang-CWA4WttNNZF54TAoqtyWWQ@public.gmane.org>
writes:

> The recipe's DESCRIPTION is wrapped automatically by textwrap, make it
> support newline ("\n") to let the user can wrap it manually, e.g.:
>
> -                    ctrlfile.write('Description: %s\n' % summary)
> -                    ctrlfile.write('%s\n' % textwrap.fill(description, width=74, initial_indent=' ', subsequent_indent=' '))
> +                    for t in description.split('\\n'):
> +                        ctrlfile.write('%s\n' % textwrap.fill(t, width=74, initial_indent=' ', subsequent_indent=' '))

When user wrapped lines manually, why is textwrap.fill() called on the
lines again?  E.g. when user wrapped manually at 78 columns, this will
create long - short - long - short lines (in german, why call this
effect "Kammquoting" (comb quoting)) .

I suggest to avoid textwrap.fill() when text contains '\n'.



Enrico

Re: [PATCH 2/3] package_ipk.bbclass: make DESCRIPTION support newline

[Robert Yang]

Sorry, it's only replied to Enrico, now, reply to the list ...

On 06/19/2013 06:33 PM, Enrico Scholz wrote:
>
>
> Robert Yang <liezhi.yang-CWA4WttNNZF54TAoqtyWWQ@public.gmane.org>
> writes:
>
>> The recipe's DESCRIPTION is wrapped automatically by textwrap, make it
>> support newline ("\n") to let the user can wrap it manually, e.g.:
>>
>> -                    ctrlfile.write('Description: %s\n' % summary)
>> -                    ctrlfile.write('%s\n' % textwrap.fill(description, width=74, initial_indent=' ', subsequent_indent=' '))
>> +                    for t in description.split('\\n'):
>> +                        ctrlfile.write('%s\n' % textwrap.fill(t, width=74, initial_indent=' ', subsequent_indent=' '))
>
> When user wrapped lines manually, why is textwrap.fill() called on the
> lines again?  E.g. when user wrapped manually at 78 columns, this will
> create long - short - long - short lines (in german, why call this
> effect "Kammquoting" (comb quoting)) .
>
> I suggest to avoid textwrap.fill() when text contains '\n'.
>

Because I think that the "\n" is mainly used for splitting paragraph,
so the auto wrap is still useful when there is a "\n", e.g.:

DESCRIPTION = "FOO \n<Too many characters>"

The ouput is:
  FOO
  <74 characters>
  <74 characters>
  [snip]

I think that this is better than:
  FOO
  <Too many characters>

// Robert

>
>
> Enrico
>
>
>

[PATCH 3/3] package_deb.bbclass: make DESCRIPTION support newline

[Robert Yang]

The recipe's DESCRIPTION is wrapped automatically by textwrap, make it
support newline ("\n") to let the user can wrap it manually, e.g.:

DESCRIPTION = "Foo1\nFoo2"

In the past, it would be:
Foo1\nFoo2

Now:
Foo1
Foo2

[YOCTO #4348]

Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
---
 meta/classes/package_deb.bbclass |    5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/meta/classes/package_deb.bbclass b/meta/classes/package_deb.bbclass
index 949432e..74b7d25 100644
--- a/meta/classes/package_deb.bbclass
+++ b/meta/classes/package_deb.bbclass
@@ -279,10 +279,11 @@ python do_package_deb () {
                 # Special behavior for description...
                 if 'DESCRIPTION' in fs:
                      summary = localdata.getVar('SUMMARY', True) or localdata.getVar('DESCRIPTION', True) or "."
+                     ctrlfile.write('Description: %s\n' % unicode(summary))
                      description = localdata.getVar('DESCRIPTION', True) or "."
                      description = textwrap.dedent(description).strip()
-                     ctrlfile.write('Description: %s\n' % unicode(summary))
-                     ctrlfile.write('%s\n' % unicode(textwrap.fill(description, width=74, initial_indent=' ', subsequent_indent=' ')))
+                     for t in description.split('\\n'):
+                         ctrlfile.write('%s\n' % unicode(textwrap.fill(t, width=74, initial_indent=' ', subsequent_indent=' ')))
                 else:
                      ctrlfile.write(unicode(c % tuple(pullData(fs, localdata))))
         except KeyError:
-- 
1.7.10.4

Re: [PATCH 0/3] make DESCRIPTION support newline

[Robert Yang]

Ping for this one.

// Robert

On 06/19/2013 05:00 PM, Robert Yang wrote:
> The following changes since commit 590010a6525b0e1bc1de73e794764e23404591df:
>
>    core-image-weston: add weston-examples to the image (2013-06-18 17:33:17 +0100)
>
> are available in the git repository at:
>
>    git://git.pokylinux.org/poky-contrib robert/newline
>    http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/newline
>
> Robert Yang (3):
>    package_rpm.bbclass: make DESCRIPTION support newline
>    package_ipk.bbclass: make DESCRIPTION support newline
>    package_deb.bbclass: make DESCRIPTION support newline
>
>   meta/classes/package_deb.bbclass |    5 +++--
>   meta/classes/package_ipk.bbclass |    5 +++--
>   meta/classes/package_rpm.bbclass |   19 ++++++++++++++-----
>   3 files changed, 20 insertions(+), 9 deletions(-)
>

Re: [PATCH 0/3] make DESCRIPTION support newline

[Saul Wold]

On 06/25/2013 08:27 PM, Robert Yang wrote:
>
> Ping for this one.
>
Robert,

There were some comments on your patch set and I think an issue with the 
ipk one.

Sau!

> // Robert
>
> On 06/19/2013 05:00 PM, Robert Yang wrote:
>> The following changes since commit
>> 590010a6525b0e1bc1de73e794764e23404591df:
>>
>>    core-image-weston: add weston-examples to the image (2013-06-18
>> 17:33:17 +0100)
>>
>> are available in the git repository at:
>>
>>    git://git.pokylinux.org/poky-contrib robert/newline
>>    http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/newline
>>
>> Robert Yang (3):
>>    package_rpm.bbclass: make DESCRIPTION support newline
>>    package_ipk.bbclass: make DESCRIPTION support newline
>>    package_deb.bbclass: make DESCRIPTION support newline
>>
>>   meta/classes/package_deb.bbclass |    5 +++--
>>   meta/classes/package_ipk.bbclass |    5 +++--
>>   meta/classes/package_rpm.bbclass |   19 ++++++++++++++-----
>>   3 files changed, 20 insertions(+), 9 deletions(-)
>>
> _______________________________________________
> Openembedded-core mailing list
> Openembedded-core@lists.openembedded.org
> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>
>

Re: [PATCH 0/3] make DESCRIPTION support newline

[Robert Yang]

On 06/27/2013 12:05 AM, Saul Wold wrote:
> On 06/25/2013 08:27 PM, Robert Yang wrote:
>>
>> Ping for this one.
>>
> Robert,
>
> There were some comments on your patch set and I think an issue with the ipk one.
>

Hi Saul,

I think that you meant Martin's comment on the ipk one, Phil had explained to
him, and it worked well with the opkg-utils.

// Robert

> Sau!
>
>> // Robert
>>
>> On 06/19/2013 05:00 PM, Robert Yang wrote:
>>> The following changes since commit
>>> 590010a6525b0e1bc1de73e794764e23404591df:
>>>
>>>    core-image-weston: add weston-examples to the image (2013-06-18
>>> 17:33:17 +0100)
>>>
>>> are available in the git repository at:
>>>
>>>    git://git.pokylinux.org/poky-contrib robert/newline
>>>    http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/newline
>>>
>>> Robert Yang (3):
>>>    package_rpm.bbclass: make DESCRIPTION support newline
>>>    package_ipk.bbclass: make DESCRIPTION support newline
>>>    package_deb.bbclass: make DESCRIPTION support newline
>>>
>>>   meta/classes/package_deb.bbclass |    5 +++--
>>>   meta/classes/package_ipk.bbclass |    5 +++--
>>>   meta/classes/package_rpm.bbclass |   19 ++++++++++++++-----
>>>   3 files changed, 20 insertions(+), 9 deletions(-)
>>>
>> _______________________________________________
>> Openembedded-core mailing list
>> Openembedded-core@lists.openembedded.org
>> http://lists.openembedded.org/mailman/listinfo/openembedded-core
>>
>>
>
>

Re: [PATCH 0/3] make DESCRIPTION support newline

[Martin Jansa]

[-- Attachment #1: Type: text/plain, Size: 905 bytes --]

On Fri, Jun 28, 2013 at 10:04:30AM +0800, Robert Yang wrote:
> 
> 
> On 06/27/2013 12:05 AM, Saul Wold wrote:
> > On 06/25/2013 08:27 PM, Robert Yang wrote:
> >>
> >> Ping for this one.
> >>
> > Robert,
> >
> > There were some comments on your patch set and I think an issue with the ipk one.
> >
> 
> Hi Saul,
> 
> I think that you meant Martin's comment on the ipk one, Phil had explained to
> him, and it worked well with the opkg-utils.

I've confused DESCRIPTION and SUMMARY, but I was talking about this
change:
http://git.yoctoproject.org/cgit/cgit.cgi/opkg-utils/commit/?id=7f0a7976c29b34feaf4f6b00f0a63626614d1c7d

And longer description here:
https://lists.yoctoproject.org/pipermail/yocto/2012-March/007851.html

So please make sure that your change doesn't break it, but with this
applied it shouldn't.

-- 
Martin 'JaMa' Jansa     jabber: Martin.Jansa@gmail.com

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 205 bytes --]

Re: [PATCH 0/3] make DESCRIPTION support newline

[Robert Yang]

On 06/28/2013 02:50 PM, Martin Jansa wrote:
> On Fri, Jun 28, 2013 at 10:04:30AM +0800, Robert Yang wrote:
>>
>>
>> On 06/27/2013 12:05 AM, Saul Wold wrote:
>>> On 06/25/2013 08:27 PM, Robert Yang wrote:
>>>>
>>>> Ping for this one.
>>>>
>>> Robert,
>>>
>>> There were some comments on your patch set and I think an issue with the ipk one.
>>>
>>
>> Hi Saul,
>>
>> I think that you meant Martin's comment on the ipk one, Phil had explained to
>> him, and it worked well with the opkg-utils.
>
> I've confused DESCRIPTION and SUMMARY, but I was talking about this
> change:
> http://git.yoctoproject.org/cgit/cgit.cgi/opkg-utils/commit/?id=7f0a7976c29b34feaf4f6b00f0a63626614d1c7d
>
> And longer description here:
> https://lists.yoctoproject.org/pipermail/yocto/2012-March/007851.html
>
> So please make sure that your change doesn't break it, but with this
> applied it shouldn't.
>

Yes, I had taken the patch, and it doesn't break the opkg-utils.

// Robert