[U-Boot] [PATCH 0/3] Fix bugs of MAKEALL

[U-Boot] [PATCH 0/3] Fix bugs of MAKEALL

[Masahiro Yamada]

Commit 27af930e9a5c91365ca639ada580b338eabe4989
changed the boards.cfg format and introduced
some bugs to MAKEALL.

This series fixes them.

Cc: Albert ARIBAUD <albert.u.boot@aribaud.net>

Masahiro Yamada (3):
  MAKEALL: fix awk warning message
  MAKEALL: fix a bug to use CROSS_COMPILE_<ARCH>
  MAKEALL: fix boards_by_cpu function

 MAKEALL | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)

-- 
1.8.1.2

[U-Boot] [PATCH 1/3] MAKEALL: fix awk warning message

[Masahiro Yamada]

If you do `./MAKEALL -M ` or `./MAKEALL -m`
GNU awk would display warnings like follows:

    awk: warning: escape sequence `\ ' treated as plain ` '

In the first place, we do not explicitly set the field separator.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Cc: Albert ARIBAUD <albert.u.boot@aribaud.net>
---

See line 163 of ./MAKEALL

    SELECTED=$(awk '('"$FILTER"') { print $7 }' boards.cfg)

We already use awk without specifiying FS.



 MAKEALL | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/MAKEALL b/MAKEALL
index 93c0b3f..0d7893b 100755
--- a/MAKEALL
+++ b/MAKEALL
@@ -518,7 +518,7 @@ get_target_location() {
 	local vendor=""
 
 	# Automatic mode
-	local line=`awk -F '\ +' '\$7 == "'"$target"'" { print \$0 }' boards.cfg`
+	local line=`awk '\$7 == "'"$target"'" { print \$0 }' boards.cfg`
 	if [ -z "${line}" ] ; then echo "" ; return ; fi
 
 	set ${line}
@@ -556,7 +556,7 @@ get_target_location() {
 get_target_maintainers() {
 	local name=`echo $1 | cut -d : -f 3`
 
-	local line=`awk -F '\ +' '\$7 == "'"$target"'" { print \$0 }' boards.cfg`
+	local line=`awk '\$7 == "'"$target"'" { print \$0 }' boards.cfg`
 	if [ -z "${line}" ]; then
 		echo ""
 		return ;
-- 
1.8.1.2

[U-Boot] [PATCH 2/3] MAKEALL: fix a bug to use CROSS_COMPILE_<ARCH>

[Masahiro Yamada]

Commit 27af930e changed the boards.cfg format but
missed to change get_target_arch() fuction.
This commit adjusts it for CROSS_COMPILE_<ARCH>
to work correctly.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Cc: Albert ARIBAUD <albert.u.boot@aribaud.net>
---
 MAKEALL | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/MAKEALL b/MAKEALL
index 0d7893b..4f685e1 100755
--- a/MAKEALL
+++ b/MAKEALL
@@ -571,7 +571,7 @@ get_target_arch() {
 	local target=$1
 
 	# Automatic mode
-	local line=`egrep -i "^[[:space:]]*${target}[[:space:]]" boards.cfg`
+	local line=`awk '\$7 == "'"$target"'" { print \$0 }' boards.cfg`
 
 	if [ -z "${line}" ] ; then echo "" ; return ; fi
 
-- 
1.8.1.2

[U-Boot] [PATCH 3/3] MAKEALL: fix boards_by_field function

[Masahiro Yamada]

Commit 27af930e changed the boards.cfg format
and it changed boards_by_field() function incorrectly.
For tegra cpus it returned Board Name field,
not Target field.

Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
Cc: Albert ARIBAUD <albert.u.boot@aribaud.net>
---

Commit 27af930e adjusted this part like follows:


                    -v field="$1" \
                    -v select="$2" \
                    -F "$FS" \
    -               '($1 !~ /^#/ && $field == select) { print $1 }' \
    +               '($1 !~ /^#/ && $field == select) { print $7 }' \
                    boards.cfg
     }
     boards_by_arch() { boards_by_field 2 "$@" ; }
     boards_by_cpu()  { boards_by_field 3 "$@" "[: \t]+" ; }
    -boards_by_soc()  { boards_by_field 6 "$@" ; }
    +boards_by_soc()  { boards_by_field 4 "$@" ; }


TAB is also treated as a field speparator, so
we should have taken the 8th field for Tegra
whereas the 7th field for the other cpus.

Fortunately, Board Name field and Target filed are the same
for all Tegra LSIs.
But we should not expect it.



 MAKEALL | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)

diff --git a/MAKEALL b/MAKEALL
index 4f685e1..485721e 100755
--- a/MAKEALL
+++ b/MAKEALL
@@ -226,17 +226,17 @@ RC=0
 # Helper funcs for parsing boards.cfg
 boards_by_field()
 {
-	FS="[ \t]+"
-	[ -n "$3" ] && FS="$3"
 	awk \
 		-v field="$1" \
 		-v select="$2" \
-		-F "$FS" \
-		'($1 !~ /^#/ && $field == select) { print $7 }' \
+		-v cut="$3" \
+		'{sub(cut,"",$field)}
+		($1 !~ /^#/ && $field == select) { print $7 }' \
 		boards.cfg
 }
+
 boards_by_arch() { boards_by_field 2 "$@" ; }
-boards_by_cpu()  { boards_by_field 3 "$@" "[: \t]+" ; }
+boards_by_cpu()  { boards_by_field 3 "$@" ":.*" ; }
 boards_by_soc()  { boards_by_field 4 "$@" ; }
 
 #########################################################################
-- 
1.8.1.2

[U-Boot] [PATCH 2/3] MAKEALL: fix a bug to use CROSS_COMPILE_<ARCH>

[Albert ARIBAUD]

Hi Masahiro,

On Thu, 17 Oct 2013 16:37:41 +0900, Masahiro Yamada
<yamada.m@jp.panasonic.com> wrote:

> Commit 27af930e changed the boards.cfg format but
> missed to change get_target_arch() fuction.
> This commit adjusts it for CROSS_COMPILE_<ARCH>
> to work correctly.
> 
> Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
> Cc: Albert ARIBAUD <albert.u.boot@aribaud.net>
> ---
>  MAKEALL | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/MAKEALL b/MAKEALL
> index 0d7893b..4f685e1 100755
> --- a/MAKEALL
> +++ b/MAKEALL
> @@ -571,7 +571,7 @@ get_target_arch() {
>  	local target=$1
>  
>  	# Automatic mode
> -	local line=`egrep -i "^[[:space:]]*${target}[[:space:]]" boards.cfg`
> +	local line=`awk '\$7 == "'"$target"'" { print \$0 }' boards.cfg`
>  
>  	if [ -z "${line}" ] ; then echo "" ; return ; fi
>  

What issue does this change fix?

Amicalement,
-- 
Albert.

[U-Boot] [PATCH 2/3] MAKEALL: fix a bug to use CROSS_COMPILE_<ARCH>

[Masahiro Yamada]

Hello Albert.



> What issue does this change fix?


MAKEALL supports the environment variable CROSS_COMPILE_<ARCH>.



MAKEALL --help says like follows:

  CROSS_COMPILE_<ARCH> cross-compiler toolchain prefix for
   architecture "ARCH".  Substitute "ARCH" for any



This feature is useful when you want to build
various architectures at a time.


For example, you can use it like this:

    CROSS_COMPILE_ARM=arm-linux-gnueabi-       \
    CROSS_COMPILE_POWERPC=powerpc-linux-gnu-   \
    CROSS_COMPILE_SH=sh4-linux-                \
    ./MAKEALL  -a arm -a powerpc -a sh




Commit 27af930e broke this feature,
so I want to fix this.


Best Regards
Masahiro Yamada

[U-Boot] [PATCH 3/3] MAKEALL: fix boards_by_field function

[Albert ARIBAUD]

Hi Masahiro,

On Thu, 17 Oct 2013 16:37:42 +0900, Masahiro Yamada
<yamada.m@jp.panasonic.com> wrote:

> Commit 27af930e changed the boards.cfg format
> and it changed boards_by_field() function incorrectly.
> For tegra cpus it returned Board Name field,
> not Target field.
> 
> Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
> Cc: Albert ARIBAUD <albert.u.boot@aribaud.net>
> ---
> 
> Commit 27af930e adjusted this part like follows:
> 
> 
>                     -v field="$1" \
>                     -v select="$2" \
>                     -F "$FS" \
>     -               '($1 !~ /^#/ && $field == select) { print $1 }' \
>     +               '($1 !~ /^#/ && $field == select) { print $7 }' \
>                     boards.cfg
>      }
>      boards_by_arch() { boards_by_field 2 "$@" ; }
>      boards_by_cpu()  { boards_by_field 3 "$@" "[: \t]+" ; }
>     -boards_by_soc()  { boards_by_field 6 "$@" ; }
>     +boards_by_soc()  { boards_by_field 4 "$@" ; }
> 
> 
> TAB is also treated as a field speparator, so
> we should have taken the 8th field for Tegra
> whereas the 7th field for the other cpus.
> 
> Fortunately, Board Name field and Target filed are the same
> for all Tegra LSIs.
> But we should not expect it.

Not sure I am following here, as the commit you mention does not
change how tabs are processed.

Besides, the system should *not* be sensitive to tabs. If it is, this
must be fixed and tabs removed.

Amicalement,
-- 
Albert.

[U-Boot] [PATCH 3/3] MAKEALL: fix boards_by_field function

[Masahiro Yamada]

Hello Albert.


> > Commit 27af930e adjusted this part like follows:
> > 
> > 
> >                     -v field="$1" \
> >                     -v select="$2" \
> >                     -F "$FS" \
> >     -               '($1 !~ /^#/ && $field == select) { print $1 }' \
> >     +               '($1 !~ /^#/ && $field == select) { print $7 }' \
> >                     boards.cfg
> >      }
> >      boards_by_arch() { boards_by_field 2 "$@" ; }
> >      boards_by_cpu()  { boards_by_field 3 "$@" "[: \t]+" ; }
> >     -boards_by_soc()  { boards_by_field 6 "$@" ; }
> >     +boards_by_soc()  { boards_by_field 4 "$@" ; }
> > 
> > 
> > TAB is also treated as a field speparator, so
> > we should have taken the 8th field for Tegra
> > whereas the 7th field for the other cpus.
> > 
> > Fortunately, Board Name field and Target filed are the same
> > for all Tegra LSIs.
> > But we should not expect it.
> 
> Not sure I am following here, as the commit you mention does not
> change how tabs are processed.
> 
> Besides, the system should *not* be sensitive to tabs. If it is, this
> must be fixed and tabs removed.


Sorry, my mistake.

s/TAB/colon/


The comment should be:

Colon is also treated as a field speparator, so
we should have taken the 8th field for Tegra
whereas the 7th field for the other cpus.



Best Regards
Masahiro Yamada

[U-Boot] [PATCH 2/3] MAKEALL: fix a bug to use CROSS_COMPILE_<ARCH>

[Albert ARIBAUD]

Hi Masahiro,

On Thu, 17 Oct 2013 17:48:50 +0900, Masahiro Yamada
<yamada.m@jp.panasonic.com> wrote:

> Hello Albert.
> 
> 
> 
> > What issue does this change fix?
> 
> 
> MAKEALL supports the environment variable CROSS_COMPILE_<ARCH>.

> Commit 27af930e broke this feature,
> so I want to fix this.

Sorry, I have been unclear. How exactly does the commit break this
feature? What worked before it which does not work after?

> Best Regards
> Masahiro Yamada

Amicalement,
-- 
Albert.

[U-Boot] [PATCH 3/3] MAKEALL: fix boards_by_field function

[Albert ARIBAUD]

Hi Masahiro,

On Thu, 17 Oct 2013 16:37:42 +0900, Masahiro Yamada
<yamada.m@jp.panasonic.com> wrote:

> Commit 27af930e changed the boards.cfg format
> and it changed boards_by_field() function incorrectly.
> For tegra cpus it returned Board Name field,
> not Target field.
> 
> Signed-off-by: Masahiro Yamada <yamada.m@jp.panasonic.com>
> Cc: Albert ARIBAUD <albert.u.boot@aribaud.net>
> ---
> 
> Commit 27af930e adjusted this part like follows:
> 
> 
>                     -v field="$1" \
>                     -v select="$2" \
>                     -F "$FS" \
>     -               '($1 !~ /^#/ && $field == select) { print $1 }' \
>     +               '($1 !~ /^#/ && $field == select) { print $7 }' \
>                     boards.cfg
>      }
>      boards_by_arch() { boards_by_field 2 "$@" ; }
>      boards_by_cpu()  { boards_by_field 3 "$@" "[: \t]+" ; }
>     -boards_by_soc()  { boards_by_field 6 "$@" ; }
>     +boards_by_soc()  { boards_by_field 4 "$@" ; }
> 
> 
> TAB is also treated as a field speparator, so
> we should have taken the 8th field for Tegra
> whereas the 7th field for the other cpus.

(As per our discussion, 'tab' here should be 'colon')

Not sure I am getting the logic here. Colon is *not* a field separator,
precisely because it is not present on all lines; it is a sub-field
separator. At this low level, the only field separator should be spaces.

Therefore, I would prefer boards_by_field() and board_by_cpu() to *not*
handle colons and thus consider the CPU field as a whole even when it
consists in a cpu:splcpu pair.

Splitting that pair and using either cpu or splcpu depending on
whether building SPL or not should only happen when the CPU field is
actually used, not fetched.

Can you try and provide a v2 patch (set) along these lines?

Amicalement,
-- 
Albert.

[U-Boot] [PATCH 2/3] MAKEALL: fix a bug to use CROSS_COMPILE_<ARCH>

[Masahiro Yamada]

Hello Albert



> > Commit 27af930e broke this feature,
> > so I want to fix this.
> 
> Sorry, I have been unclear. How exactly does the commit break this
> feature? What worked before it which does not work after?


A quite simple test.



$ git checkout 27af930e^
$ CROSS_COMPILE_ARM=arm-linux-gnueabi-  ./MAKEALL -a arm
Configuring for integratorcp_cm1136 - Board: integratorcp, Options: CM1136
   text	   data	    bss	    dec	    hex	filename
 160402	   6164	  16156	 182722	  2c9c2	./u-boot
Configuring for imx31_phycore board...
   text	   data	    bss	    dec	    hex	filename
 144449	   5162	  21016	 170627	  29a83	./u-boot

...





$ git checkout 27af930e
$ CROSS_COMPILE_ARM=arm-linux-gnueabi-  ./MAKEALL -a arm
Configuring for integratorcp_cm1136 - Board: integratorcp, Options: CM1136
make: *** [lib/asm-offsets.s] Error 127
size: './u-boot': No such file
/bin/bash: arm-linux-gcc: command not found
/bin/bash: arm-linux-gcc: command not found
dirname: missing operand
Try 'dirname --help' for more information.
/bin/bash: line 3: arm-linux-gcc: command not found
/bin/bash: line 3: arm-linux-gcc: command not found
/bin/bash: arm-linux-gcc: command not found
/bin/bash: arm-linux-gcc: command not found

...





Note:
I am using arm-linux-gnueabi-gcc, not arm-linux-gcc





Best Regards
Masahiro Yamada

[U-Boot] [PATCH 3/3] MAKEALL: fix boards_by_field function

[Masahiro Yamada]

Hello Albert




> >                     -v field="$1" \
> >                     -v select="$2" \
> >                     -F "$FS" \
> >     -               '($1 !~ /^#/ && $field == select) { print $1 }' \
> >     +               '($1 !~ /^#/ && $field == select) { print $7 }' \
> >                     boards.cfg
> >      }
> >      boards_by_arch() { boards_by_field 2 "$@" ; }
> >      boards_by_cpu()  { boards_by_field 3 "$@" "[: \t]+" ; }
> >     -boards_by_soc()  { boards_by_field 6 "$@" ; }
> >     +boards_by_soc()  { boards_by_field 4 "$@" ; }
> > 
> > 
> > TAB is also treated as a field speparator, so
> > we should have taken the 8th field for Tegra
> > whereas the 7th field for the other cpus.
> 
> (As per our discussion, 'tab' here should be 'colon')

Yes, I answerd so in my previous reply.


> Not sure I am getting the logic here. Colon is *not* a field separator,
> precisely because it is not present on all lines; it is a sub-field
> separator. At this low level, the only field separator should be spaces.
> 
> Therefore, I would prefer boards_by_field() and board_by_cpu() to *not*
> handle colons and thus consider the CPU field as a whole even when it
> consists in a cpu:splcpu pair.

Yes. I think I already did this in my v1 patch.


  +		-v cut="$3" \
  +		'{sub(cut,"",$field)}

These lines split the pair into cpu and spl_cpu.
I simply cut down  spl_cpu because it is the same behaviour
as before Commit 27af930e.



> Splitting that pair and using either cpu or splcpu depending on
> whether building SPL or not should only happen when the CPU field is
> actually used, not fetched.
> 
> Can you try and provide a v2 patch (set) along these lines?


Sorry, I cannot understand what you mean.


Best Regards
Masahiro Yamada

[U-Boot] [PATCH 3/3] MAKEALL: fix boards_by_field function

[Albert ARIBAUD]

Hi Masahiro,

On Thu, 17 Oct 2013 19:32:31 +0900, Masahiro Yamada
<yamada.m@jp.panasonic.com> wrote:

> Hello Albert
> 
> 
> 
> 
> > >                     -v field="$1" \
> > >                     -v select="$2" \
> > >                     -F "$FS" \
> > >     -               '($1 !~ /^#/ && $field == select) { print $1 }' \
> > >     +               '($1 !~ /^#/ && $field == select) { print $7 }' \
> > >                     boards.cfg
> > >      }
> > >      boards_by_arch() { boards_by_field 2 "$@" ; }
> > >      boards_by_cpu()  { boards_by_field 3 "$@" "[: \t]+" ; }
> > >     -boards_by_soc()  { boards_by_field 6 "$@" ; }
> > >     +boards_by_soc()  { boards_by_field 4 "$@" ; }
> > > 
> > > 
> > > TAB is also treated as a field speparator, so
> > > we should have taken the 8th field for Tegra
> > > whereas the 7th field for the other cpus.
> > 
> > (As per our discussion, 'tab' here should be 'colon')
> 
> Yes, I answerd so in my previous reply.
> 
> 
> > Not sure I am getting the logic here. Colon is *not* a field separator,
> > precisely because it is not present on all lines; it is a sub-field
> > separator. At this low level, the only field separator should be spaces.
> > 
> > Therefore, I would prefer boards_by_field() and board_by_cpu() to *not*
> > handle colons and thus consider the CPU field as a whole even when it
> > consists in a cpu:splcpu pair.
> 
> Yes. I think I already did this in my v1 patch.
> 
> 
>   +		-v cut="$3" \
>   +		'{sub(cut,"",$field)}
> 
> These lines split the pair into cpu and spl_cpu.
> I simply cut down  spl_cpu because it is the same behaviour
> as before Commit 27af930e.
> 
> 
> 
> > Splitting that pair and using either cpu or splcpu depending on
> > whether building SPL or not should only happen when the CPU field is
> > actually used, not fetched.
> > 
> > Can you try and provide a v2 patch (set) along these lines?
> 
> 
> Sorry, I cannot understand what you mean.

I do understand that your patch wants to restore the behavior prior to
27af930e.

My problem is, while things worked prior to 27af930e, they were not
done the right way, because boards_by_<field>() functions should not
depend on the fact that a field contains a colon or not; they should
only depend on the fact that fields are space-separated.

IOW, target integratorcp_cm1136 on line 46 in boards.cfg has 9 fields,
and its CPU field is "arm1136"; and line 348, dalmore, *also* has 9
fields even though its CPU field is "armv7:arm720t".

The way the code is written now, board_by_field() has to do the job of 
board_by_cpu() and has to know the CPU field has colon-separated
subfields. What should be done is, board_by_field should not even worry
about colons at all, and it is board_by_cpu() which should know about
CPU dubfields and treat them properly.

Is this clearer?

> Best Regards
> Masahiro Yamada
> 


Amicalement,
-- 
Albert.

[U-Boot] [PATCH 3/3] MAKEALL: fix boards_by_field function

[Masahiro Yamada]

Hello Albert.


> The way the code is written now, board_by_field() has to do the job of 
> board_by_cpu() and has to know the CPU field has colon-separated
> subfields. What should be done is, board_by_field should not even worry
> about colons at all, and it is board_by_cpu() which should know about
> CPU dubfields and treat them properly.
> 
> Is this clearer?


OK. It's clear now.

I re-wrote like follows:


boards_by_field()
{
	field=$1
	regexp=$2

	awk '($1 !~ /^#/ && $'"$field"' ~ /^'"$regexp"'$/) { print $7 }' \
		boards.cfg
}

boards_by_arch() { boards_by_field 2 "$@" ; }
boards_by_cpu()  { boards_by_field 3 "$@" ; boards_by_field 3 "$@:.*" ; }
boards_by_soc()  { boards_by_field 4 "$@" ; }



Is this good enough?




BTW, I think these function names are misleading.


We want to get the 7th field (Target),
not 6th field (Board Name) of boards.cfg
by these functions.


So I think we should rename

s/boards_by_field/targets_by_field/
s/boards_by_arch/targets_by_arch/
s/boards_by_cpu/targets_by_cpu/
s/boards_by_soc/targets_by_soc/



Best Regards
Masahiro Yamada

[U-Boot] [PATCH 3/3] MAKEALL: fix boards_by_field function

[Masahiro Yamada]

Hello Albert.

> 
> The way the code is written now, board_by_field() has to do the job of 
> board_by_cpu() and has to know the CPU field has colon-separated
> subfields. What should be done is, board_by_field should not even worry
> about colons at all, and it is board_by_cpu() which should know about
> CPU dubfields and treat them properly.
> 
> Is this clearer?
> 

I posted v2.

Best Regards
Masahiro Yamada