[PATCH] Improve compatibility with other platforms

[PATCH] Improve compatibility with other platforms

[kevans-HZy0K5TPuP5AfugRpC6u6w]

This doesn't make things work out of the box necessarily, but it gets us a step
closer.

 - alloca(3) on FreeBSD and OpenBSD, at least, is provided as part of
<stdlib.h>. I make the change here for FreeBSD only, though, as I cannot test it
on OpenBSD.

 - stat -c %s's equivalent on FreeBSD is stat -f %Uz; these differ enough, allow
STATSZ in the environment to specify local replacement for a stat that outputs
size in bytes of an argument.

This greatly reduces, but not eliminates, the magnitude of patch needed to run
the test suite successfully on other platforms. It also helps denote where
changes may be required.

Signed-off-by: Kyle Evans <kevans-HZy0K5TPuP5AfugRpC6u6w@public.gmane.org>
---
 fdtoverlay.c       |  3 +++
 tests/run_tests.sh | 13 +++++++++----
 2 files changed, 12 insertions(+), 4 deletions(-)

diff --git a/fdtoverlay.c b/fdtoverlay.c
index 7f124fc..7d60021 100644
--- a/fdtoverlay.c
+++ b/fdtoverlay.c
@@ -26,7 +26,10 @@
 #include <stdio.h>
 #include <stdlib.h>
 #include <string.h>
+/* alloc(3) is provided by stdlib.h on some platforms */
+#ifndef __FreeBSD__
 #include <alloca.h>
+#endif
 #include <inttypes.h>
 
 #include <libfdt.h>
diff --git a/tests/run_tests.sh b/tests/run_tests.sh
index d36dffb..13c9f1c 100755
--- a/tests/run_tests.sh
+++ b/tests/run_tests.sh
@@ -6,6 +6,11 @@ if [ -z "$CC" ]; then
     CC=gcc
 fi
 
+# stat differs between platforms
+if [ -z "$STATSZ" ]; then
+	STATSZ="stat -c %s"
+fi
+
 export QUIET_TEST=1
 STOP_ON_FAIL=0
 
@@ -114,7 +119,7 @@ run_wrap_error_test () {
 # $2: align base
 check_align () {
     shorten_echo "check_align $@:	"
-    local size=$(stat -c %s "$1")
+    local size=$($STATSZ "$1")
     local align="$2"
     (
 	if [ $(($size % $align)) -eq 0 ] ;then
@@ -714,7 +719,7 @@ fdtput_tests () {
     text=lorem.txt
 
     # Allow just enough space for $text
-    run_dtc_test -O dtb -p $(stat -c %s $text) -o $dtb $dts
+    run_dtc_test -O dtb -p $($STATSZ $text) -o $dtb $dts
 
     # run_fdtput_test <expected-result> <file> <node> <property> <flags> <value>
     run_fdtput_test "a_model" $dtb / model -ts "a_model"
@@ -733,7 +738,7 @@ fdtput_tests () {
     run_fdtput_test "$(cat $text $text)" $dtb /randomnode blob -ts "$(cat $text $text)"
 
     # Start again with a fresh dtb
-    run_dtc_test -O dtb -p $(stat -c %s $text) -o $dtb $dts
+    run_dtc_test -O dtb -p $($STATSZ $text) -o $dtb $dts
 
     # Node creation
     run_wrap_error_test $DTPUT $dtb -c /baldrick sod
@@ -761,7 +766,7 @@ fdtput_tests () {
     run_wrap_test $DTPUT $dtb -cp /chosen/son
 
     # Start again with a fresh dtb
-    run_dtc_test -O dtb -p $(stat -c %s $text) -o $dtb $dts
+    run_dtc_test -O dtb -p $($STATSZ $text) -o $dtb $dts
 
     # Node delete
     run_wrap_test $DTPUT $dtb -c /chosen/node1 /chosen/node2 /chosen/node3
-- 
2.15.1

Re: [PATCH] Improve compatibility with other platforms

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 3538 bytes --]

On Sun, Dec 31, 2017 at 10:15:18PM -0600, kevans-HZy0K5TPuP5AfugRpC6u6w@public.gmane.org wrote:
> This doesn't make things work out of the box necessarily, but it gets us a step
> closer.
> 
>  - alloca(3) on FreeBSD and OpenBSD, at least, is provided as part of
> <stdlib.h>. I make the change here for FreeBSD only, though, as I cannot test it
> on OpenBSD.

TBH, rather than dicking around with #ifdefs and headers, I'd rather
use use malloc() instead of alloca().

>  - stat -c %s's equivalent on FreeBSD is stat -f %Uz; these differ enough, allow
> STATSZ in the environment to specify local replacement for a stat that outputs
> size in bytes of an argument.

Can you split this change out into a separate patch, please.

> 
> This greatly reduces, but not eliminates, the magnitude of patch needed to run
> the test suite successfully on other platforms. It also helps denote where
> changes may be required.
> 
> Signed-off-by: Kyle Evans <kevans-HZy0K5TPuP5AfugRpC6u6w@public.gmane.org>


> ---
>  fdtoverlay.c       |  3 +++
>  tests/run_tests.sh | 13 +++++++++----
>  2 files changed, 12 insertions(+), 4 deletions(-)
> 
> diff --git a/fdtoverlay.c b/fdtoverlay.c
> index 7f124fc..7d60021 100644
> --- a/fdtoverlay.c
> +++ b/fdtoverlay.c
> @@ -26,7 +26,10 @@
>  #include <stdio.h>
>  #include <stdlib.h>
>  #include <string.h>
> +/* alloc(3) is provided by stdlib.h on some platforms */
> +#ifndef __FreeBSD__
>  #include <alloca.h>
> +#endif
>  #include <inttypes.h>
>  
>  #include <libfdt.h>
> diff --git a/tests/run_tests.sh b/tests/run_tests.sh
> index d36dffb..13c9f1c 100755
> --- a/tests/run_tests.sh
> +++ b/tests/run_tests.sh
> @@ -6,6 +6,11 @@ if [ -z "$CC" ]; then
>      CC=gcc
>  fi
>  
> +# stat differs between platforms
> +if [ -z "$STATSZ" ]; then
> +	STATSZ="stat -c %s"
> +fi
> +
>  export QUIET_TEST=1
>  STOP_ON_FAIL=0
>  
> @@ -114,7 +119,7 @@ run_wrap_error_test () {
>  # $2: align base
>  check_align () {
>      shorten_echo "check_align $@:	"
> -    local size=$(stat -c %s "$1")
> +    local size=$($STATSZ "$1")
>      local align="$2"
>      (
>  	if [ $(($size % $align)) -eq 0 ] ;then
> @@ -714,7 +719,7 @@ fdtput_tests () {
>      text=lorem.txt
>  
>      # Allow just enough space for $text
> -    run_dtc_test -O dtb -p $(stat -c %s $text) -o $dtb $dts
> +    run_dtc_test -O dtb -p $($STATSZ $text) -o $dtb $dts
>  
>      # run_fdtput_test <expected-result> <file> <node> <property> <flags> <value>
>      run_fdtput_test "a_model" $dtb / model -ts "a_model"
> @@ -733,7 +738,7 @@ fdtput_tests () {
>      run_fdtput_test "$(cat $text $text)" $dtb /randomnode blob -ts "$(cat $text $text)"
>  
>      # Start again with a fresh dtb
> -    run_dtc_test -O dtb -p $(stat -c %s $text) -o $dtb $dts
> +    run_dtc_test -O dtb -p $($STATSZ $text) -o $dtb $dts
>  
>      # Node creation
>      run_wrap_error_test $DTPUT $dtb -c /baldrick sod
> @@ -761,7 +766,7 @@ fdtput_tests () {
>      run_wrap_test $DTPUT $dtb -cp /chosen/son
>  
>      # Start again with a fresh dtb
> -    run_dtc_test -O dtb -p $(stat -c %s $text) -o $dtb $dts
> +    run_dtc_test -O dtb -p $($STATSZ $text) -o $dtb $dts
>  
>      # Node delete
>      run_wrap_test $DTPUT $dtb -c /chosen/node1 /chosen/node2 /chosen/node3

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] Improve compatibility with other platforms

[Kyle Evans]

On Tue, Jan 2, 2018 at 6:13 PM, David Gibson
<david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
> On Sun, Dec 31, 2017 at 10:15:18PM -0600, kevans-HZy0K5TPuP5AfugRpC6u6w@public.gmane.org wrote:
>> This doesn't make things work out of the box necessarily, but it gets us a step
>> closer.
>>
>>  - alloca(3) on FreeBSD and OpenBSD, at least, is provided as part of
>> <stdlib.h>. I make the change here for FreeBSD only, though, as I cannot test it
>> on OpenBSD.
>
> TBH, rather than dicking around with #ifdefs and headers, I'd rather
> use use malloc() instead of alloca().

Alrighty- I'll prepare a patch for this, knowing that's a direction
you're comfortable heading in.

>>  - stat -c %s's equivalent on FreeBSD is stat -f %Uz; these differ enough, allow
>> STATSZ in the environment to specify local replacement for a stat that outputs
>> size in bytes of an argument.
>
> Can you split this change out into a separate patch, please.

Will do.

Re: [PATCH] Improve compatibility with other platforms

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 1569 bytes --]

On Tue, Jan 02, 2018 at 08:37:16PM -0600, Kyle Evans wrote:
> On Tue, Jan 2, 2018 at 6:13 PM, David Gibson
> <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
> > On Sun, Dec 31, 2017 at 10:15:18PM -0600, kevans-HZy0K5TPuP5AfugRpC6u6w@public.gmane.org wrote:
> >> This doesn't make things work out of the box necessarily, but it gets us a step
> >> closer.
> >>
> >>  - alloca(3) on FreeBSD and OpenBSD, at least, is provided as part of
> >> <stdlib.h>. I make the change here for FreeBSD only, though, as I cannot test it
> >> on OpenBSD.
> >
> > TBH, rather than dicking around with #ifdefs and headers, I'd rather
> > use use malloc() instead of alloca().
> 
> Alrighty- I'll prepare a patch for this, knowing that's a direction
> you're comfortable heading in.
> 
> >>  - stat -c %s's equivalent on FreeBSD is stat -f %Uz; these differ enough, allow
> >> STATSZ in the environment to specify local replacement for a stat that outputs
> >> size in bytes of an argument.
> >
> > Can you split this change out into a separate patch, please.
> 
> Will do.

Fwiw, some years ago I did a bunch of work to get dtc to compile and
pass the tests on FreeBSD.  Working out of the box on *BSD is
something I'd like it to have.  Unfortunately, because I can't easily
test on a BSD machine, the BSD support tends to bitrot rather rapidly.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] Improve compatibility with other platforms

[Kyle Evans]

On Tue, Jan 2, 2018 at 8:58 PM, David Gibson
<david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
> Fwiw, some years ago I did a bunch of work to get dtc to compile and
> pass the tests on FreeBSD.  Working out of the box on *BSD is
> something I'd like it to have.  Unfortunately, because I can't easily
> test on a BSD machine, the BSD support tends to bitrot rather rapidly.


Happy to hear this. =) You're not too far off, fortunately. This, the
alloca header situation, and an -ldl in tests/Makefile.tests are the
only obstacles to being able to do a simple "gmake && gmake check"
here. I've submitted patches for the first two, and as of r320872 [1]
we provide a libdl on -HEAD.

I will likely submit a patch in the near future to use `stat -f %Uz`
automatically if stat --version exits with a non-zero exit code or
indicates that it's GNU stat(1). This should cover the *BSDs, to
include MacOS.

[1] https://svnweb.freebsd.org/base?view=revision&revision=320872

Re: [PATCH] Improve compatibility with other platforms

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 2440 bytes --]

On Tue, Jan 02, 2018 at 09:29:14PM -0600, Kyle Evans wrote:
> On Tue, Jan 2, 2018 at 8:58 PM, David Gibson
> <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
> > Fwiw, some years ago I did a bunch of work to get dtc to compile and
> > pass the tests on FreeBSD.  Working out of the box on *BSD is
> > something I'd like it to have.  Unfortunately, because I can't easily
> > test on a BSD machine, the BSD support tends to bitrot rather rapidly.
> 
> 
> Happy to hear this. =) You're not too far off, fortunately.

Good to hear.

> This, the
> alloca header situation, and an -ldl in tests/Makefile.tests are the
> only obstacles to being able to do a simple "gmake && gmake check"
> here. I've submitted patches for the first two, and as of r320872 [1]
> we provide a libdl on -HEAD.

Ok sounds good.  I had a look at the tests using -ldl and it looks
like they couldn't easily be adapted to avoid it.  However, I'd be ok
with logic to skip those tests if libdl isn't available, if you want
to broaden support to older releases.

At the moment dtc seems to be just teetering on the edge of being
complex enough to need some sort of configuration system (whether it
be hand-rolled scripts, autoconf or whatever).  So far I've been
avoiding adding such a thing, because that introduces a bunch of
problems of its own: hand rolled scripts are a pain to maintain,
autoconf is ugly as sin, pretty much anything else is nonstandard and
liable to introduce a bunch of extra dependencies.  But, I might have
to bite the bullet at some point.

> I will likely submit a patch in the near future to use `stat -f %Uz`
> automatically if stat --version exits with a non-zero exit code or
> indicates that it's GNU stat(1). This should cover the *BSDs, to
> include MacOS.

Great.  MacOS support is interesting in particular, because unlike
FreeBSD, I can set up a Travis build to test it there, which means I'm
much more likely to catch regressions.  In fact, looks like I have a
stale branch here that added an OSX build to the travis.yml - I think
some of the errors you've found and didn't have time to debug them
(debugging when your only host is indirect through Travis is pretty
painful).

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] Improve compatibility with other platforms

[Kyle Evans]

[Resending with a proper mail client, because my initial response
didn't go to the lists]

On Tue, Jan 2, 2018 at 9:55 PM, David Gibson
<david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
> Ok sounds good.  I had a look at the tests using -ldl and it looks
> like they couldn't easily be adapted to avoid it.  However, I'd be ok
> with logic to skip those tests if libdl isn't available, if you want
> to broaden support to older releases.

The hard part is that libdl doesn't need to be available. With FreeBSD
(and generally, all BSDs if I understood right), the symbols expected
of libdl are available in libc.

If I have my terminology right, our libdl is in fact just a filter on
libc to isolate these symbols/functions for all of the software that
expected libdl. We'd spent enough time patching out -ldl that it
seemed a good idea, but these binaries *do* compile without it.

> At the moment dtc seems to be just teetering on the edge of being
> complex enough to need some sort of configuration system (whether it
> be hand-rolled scripts, autoconf or whatever).  So far I've been
> avoiding adding such a thing, because that introduces a bunch of
> problems of its own: hand rolled scripts are a pain to maintain,
> autoconf is ugly as sin, pretty much anything else is nonstandard and
> liable to introduce a bunch of extra dependencies.  But, I might have
> to bite the bullet at some point.

Ugh, build systems. =P

> Great.  MacOS support is interesting in particular, because unlike
> FreeBSD, I can set up a Travis build to test it there, which means I'm
> much more likely to catch regressions.  In fact, looks like I have a
> stale branch here that added an OSX build to the travis.yml - I think
> some of the errors you've found and didn't have time to debug them
> (debugging when your only host is indirect through Travis is pretty
> painful).

It'd be interesting to see those results as of recent. Their userland
should be similar enough in the ways you care about for the results to
likely be relevant to us, I would think.

Re: [PATCH] Improve compatibility with other platforms

[Rob Herring]

On Tue, Jan 2, 2018 at 9:55 PM, David Gibson
<david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
> On Tue, Jan 02, 2018 at 09:29:14PM -0600, Kyle Evans wrote:
>> On Tue, Jan 2, 2018 at 8:58 PM, David Gibson
>> <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
>> > Fwiw, some years ago I did a bunch of work to get dtc to compile and
>> > pass the tests on FreeBSD.  Working out of the box on *BSD is
>> > something I'd like it to have.  Unfortunately, because I can't easily
>> > test on a BSD machine, the BSD support tends to bitrot rather rapidly.
>>
>>
>> Happy to hear this. =) You're not too far off, fortunately.
>
> Good to hear.
>
>> This, the
>> alloca header situation, and an -ldl in tests/Makefile.tests are the
>> only obstacles to being able to do a simple "gmake && gmake check"
>> here. I've submitted patches for the first two, and as of r320872 [1]
>> we provide a libdl on -HEAD.
>
> Ok sounds good.  I had a look at the tests using -ldl and it looks
> like they couldn't easily be adapted to avoid it.  However, I'd be ok
> with logic to skip those tests if libdl isn't available, if you want
> to broaden support to older releases.
>
> At the moment dtc seems to be just teetering on the edge of being
> complex enough to need some sort of configuration system (whether it
> be hand-rolled scripts, autoconf or whatever).  So far I've been
> avoiding adding such a thing, because that introduces a bunch of
> problems of its own: hand rolled scripts are a pain to maintain,
> autoconf is ugly as sin, pretty much anything else is nonstandard and
> liable to introduce a bunch of extra dependencies.  But, I might have
> to bite the bullet at some point.

What about meson? All the cool kids are using it now.

Rob

Re: [PATCH] Improve compatibility with other platforms

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 3029 bytes --]

On Wed, Jan 03, 2018 at 08:03:27AM -0600, Kyle Evans wrote:
> [Resending with a proper mail client, because my initial response
> didn't go to the lists]
> 
> On Tue, Jan 2, 2018 at 9:55 PM, David Gibson
> <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
> > Ok sounds good.  I had a look at the tests using -ldl and it looks
> > like they couldn't easily be adapted to avoid it.  However, I'd be ok
> > with logic to skip those tests if libdl isn't available, if you want
> > to broaden support to older releases.
> 
> The hard part is that libdl doesn't need to be available. With FreeBSD
> (and generally, all BSDs if I understood right), the symbols expected
> of libdl are available in libc.
> 
> If I have my terminology right, our libdl is in fact just a filter on
> libc to isolate these symbols/functions for all of the software that
> expected libdl. We'd spent enough time patching out -ldl that it
> seemed a good idea, but these binaries *do* compile without it.

Ah, right.

I that case, I think the approach to take is to make a new LIBDL make
variable with the "-ldl" in it, so that it can be overridden on *BSD
to be empty.

> 
> > At the moment dtc seems to be just teetering on the edge of being
> > complex enough to need some sort of configuration system (whether it
> > be hand-rolled scripts, autoconf or whatever).  So far I've been
> > avoiding adding such a thing, because that introduces a bunch of
> > problems of its own: hand rolled scripts are a pain to maintain,
> > autoconf is ugly as sin, pretty much anything else is nonstandard and
> > liable to introduce a bunch of extra dependencies.  But, I might have
> > to bite the bullet at some point.
> 
> Ugh, build systems. =P

I know, right.

> > Great.  MacOS support is interesting in particular, because unlike
> > FreeBSD, I can set up a Travis build to test it there, which means I'm
> > much more likely to catch regressions.  In fact, looks like I have a
> > stale branch here that added an OSX build to the travis.yml - I think
> > some of the errors you've found and didn't have time to debug them
> > (debugging when your only host is indirect through Travis is pretty
> > painful).
> 
> It'd be interesting to see those results as of recent. Their userland
> should be similar enough in the ways you care about for the results to
> likely be relevant to us, I would think.

I rebased and tried it again:
    https://travis-ci.org/dgibson/dtc/jobs/324587391

Looks like we're getting problems with the interesting way I (ab)use
the assembler to produce a test blob.  I've hit a number of problems
with that over time, so I'd really like to replace it with something
else.  Unfortunately, I haven't yet thought of a replacement that
wouldn't be a pain in the arse.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] Improve compatibility with other platforms

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 2244 bytes --]

On Wed, Jan 03, 2018 at 10:51:23AM -0600, Rob Herring wrote:
> On Tue, Jan 2, 2018 at 9:55 PM, David Gibson
> <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
> > On Tue, Jan 02, 2018 at 09:29:14PM -0600, Kyle Evans wrote:
> >> On Tue, Jan 2, 2018 at 8:58 PM, David Gibson
> >> <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
> >> > Fwiw, some years ago I did a bunch of work to get dtc to compile and
> >> > pass the tests on FreeBSD.  Working out of the box on *BSD is
> >> > something I'd like it to have.  Unfortunately, because I can't easily
> >> > test on a BSD machine, the BSD support tends to bitrot rather rapidly.
> >>
> >>
> >> Happy to hear this. =) You're not too far off, fortunately.
> >
> > Good to hear.
> >
> >> This, the
> >> alloca header situation, and an -ldl in tests/Makefile.tests are the
> >> only obstacles to being able to do a simple "gmake && gmake check"
> >> here. I've submitted patches for the first two, and as of r320872 [1]
> >> we provide a libdl on -HEAD.
> >
> > Ok sounds good.  I had a look at the tests using -ldl and it looks
> > like they couldn't easily be adapted to avoid it.  However, I'd be ok
> > with logic to skip those tests if libdl isn't available, if you want
> > to broaden support to older releases.
> >
> > At the moment dtc seems to be just teetering on the edge of being
> > complex enough to need some sort of configuration system (whether it
> > be hand-rolled scripts, autoconf or whatever).  So far I've been
> > avoiding adding such a thing, because that introduces a bunch of
> > problems of its own: hand rolled scripts are a pain to maintain,
> > autoconf is ugly as sin, pretty much anything else is nonstandard and
> > liable to introduce a bunch of extra dependencies.  But, I might have
> > to bite the bullet at some point.
> 
> What about meson? All the cool kids are using it now.

Heh, hadn't heard of that one.  Too busy maintaining vital
infrastructure to follow what the cool kids are doing.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] Improve compatibility with other platforms

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 2764 bytes --]

On Thu, Jan 04, 2018 at 11:55:00AM +1100, David Gibson wrote:
> On Wed, Jan 03, 2018 at 10:51:23AM -0600, Rob Herring wrote:
> > On Tue, Jan 2, 2018 at 9:55 PM, David Gibson
> > <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
> > > On Tue, Jan 02, 2018 at 09:29:14PM -0600, Kyle Evans wrote:
> > >> On Tue, Jan 2, 2018 at 8:58 PM, David Gibson
> > >> <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
> > >> > Fwiw, some years ago I did a bunch of work to get dtc to compile and
> > >> > pass the tests on FreeBSD.  Working out of the box on *BSD is
> > >> > something I'd like it to have.  Unfortunately, because I can't easily
> > >> > test on a BSD machine, the BSD support tends to bitrot rather rapidly.
> > >>
> > >>
> > >> Happy to hear this. =) You're not too far off, fortunately.
> > >
> > > Good to hear.
> > >
> > >> This, the
> > >> alloca header situation, and an -ldl in tests/Makefile.tests are the
> > >> only obstacles to being able to do a simple "gmake && gmake check"
> > >> here. I've submitted patches for the first two, and as of r320872 [1]
> > >> we provide a libdl on -HEAD.
> > >
> > > Ok sounds good.  I had a look at the tests using -ldl and it looks
> > > like they couldn't easily be adapted to avoid it.  However, I'd be ok
> > > with logic to skip those tests if libdl isn't available, if you want
> > > to broaden support to older releases.
> > >
> > > At the moment dtc seems to be just teetering on the edge of being
> > > complex enough to need some sort of configuration system (whether it
> > > be hand-rolled scripts, autoconf or whatever).  So far I've been
> > > avoiding adding such a thing, because that introduces a bunch of
> > > problems of its own: hand rolled scripts are a pain to maintain,
> > > autoconf is ugly as sin, pretty much anything else is nonstandard and
> > > liable to introduce a bunch of extra dependencies.  But, I might have
> > > to bite the bullet at some point.
> > 
> > What about meson? All the cool kids are using it now.
> 
> Heh, hadn't heard of that one.  Too busy maintaining vital
> infrastructure to follow what the cool kids are doing.

Wait, no, I went to a talk about that once, and it did look pretty
nice.  Never had time or headspace to really do anything with it,
though.  Plus it would introduce Python as a build dep.  Not really a
problem for dtc per se, but given that libfdt is supposed to be
embeddable in weird limited firmware environments, I really want to
keep the build deps to a minimum.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH] Improve compatibility with other platforms

[Kyle Evans]

On Wed, Jan 3, 2018 at 6:54 PM, David Gibson
<david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
> On Wed, Jan 03, 2018 at 08:03:27AM -0600, Kyle Evans wrote:
>> On Tue, Jan 2, 2018 at 9:55 PM, David Gibson
>> <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
>> > [... snip ...]
>> > Great.  MacOS support is interesting in particular, because unlike
>> > FreeBSD, I can set up a Travis build to test it there, which means I'm
>> > much more likely to catch regressions.  In fact, looks like I have a
>> > stale branch here that added an OSX build to the travis.yml - I think
>> > some of the errors you've found and didn't have time to debug them
>> > (debugging when your only host is indirect through Travis is pretty
>> > painful).
>>
>> It'd be interesting to see those results as of recent. Their userland
>> should be similar enough in the ways you care about for the results to
>> likely be relevant to us, I would think.
>
> I rebased and tried it again:
>     https://travis-ci.org/dgibson/dtc/jobs/324587391
>
> Looks like we're getting problems with the interesting way I (ab)use
> the assembler to produce a test blob.  I've hit a number of problems
> with that over time, so I'd really like to replace it with something
> else.  Unfortunately, I haven't yet thought of a replacement that
> wouldn't be a pain in the arse.

That's interesting- I wonder what version of LLVM this maps to? Or are
they so far gone that it doesn't matter? We've got an effort over here
going to replace our system linker with LLD; the version of LLD that
we're optionally using on -HEAD right now has no problem with this:

$ ld.lld --version
LLD 5.0.1 (FreeBSD 320880) (compatible with GNU linkers)

Re: [PATCH] Improve compatibility with other platforms

[David Gibson]

[-- Attachment #1: Type: text/plain, Size: 3288 bytes --]

On Wed, Jan 03, 2018 at 07:47:55PM -0600, Kyle Evans wrote:
> On Wed, Jan 3, 2018 at 6:54 PM, David Gibson
> <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
> > On Wed, Jan 03, 2018 at 08:03:27AM -0600, Kyle Evans wrote:
> >> On Tue, Jan 2, 2018 at 9:55 PM, David Gibson
> >> <david-xT8FGy+AXnRB3Ne2BGzF6laj5H9X9Tb+@public.gmane.org> wrote:
> >> > [... snip ...]
> >> > Great.  MacOS support is interesting in particular, because unlike
> >> > FreeBSD, I can set up a Travis build to test it there, which means I'm
> >> > much more likely to catch regressions.  In fact, looks like I have a
> >> > stale branch here that added an OSX build to the travis.yml - I think
> >> > some of the errors you've found and didn't have time to debug them
> >> > (debugging when your only host is indirect through Travis is pretty
> >> > painful).
> >>
> >> It'd be interesting to see those results as of recent. Their userland
> >> should be similar enough in the ways you care about for the results to
> >> likely be relevant to us, I would think.
> >
> > I rebased and tried it again:
> >     https://travis-ci.org/dgibson/dtc/jobs/324587391
> >
> > Looks like we're getting problems with the interesting way I (ab)use
> > the assembler to produce a test blob.  I've hit a number of problems
> > with that over time, so I'd really like to replace it with something
> > else.  Unfortunately, I haven't yet thought of a replacement that
> > wouldn't be a pain in the arse.
> 
> That's interesting- I wonder what version of LLVM this maps to? Or are
> they so far gone that it doesn't matter? We've got an effort over here
> going to replace our system linker with LLD; the version of LLD that
> we're optionally using on -HEAD right now has no problem with this:
> 
> $ ld.lld --version
> LLD 5.0.1 (FreeBSD 320880) (compatible with GNU linkers)

I think the problem is at the asm level, rather than the linker level.
Specifically I think it's failing to realize that expressions of the
form (sym2 - sym1) where sym2 and sym1 are both local labels are
actually assemble time constant and don't need relocation, even if the
symbols themselves do.

If they actually did need relocation, I don't think they're in a form
I'd expect any linker to be able to cope with (or even the ELF format
itself).

I suppose it's possible there's something in the OSX ABI which means
these things really can't be assemble time evaluated, in which case
we'd be pretty stuck.

According to the Travis logs it's using:

$ gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin16.7.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

I'm assuming it will be using the same LLVM version for assembling as
compiling.  Given how out of date the Linux images often are on
Travis, it wouldn't suprise me of the OSX ones are as well.

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]