[uml-devel] UML build process uses perl? (With sed alternative.)

[uml-devel] UML build process uses perl? (With sed alternative.)

[Rob Landley]

I thought maybe the problem I'm having is due to running UML with a uclibc 
root on a glibc host system, so I tried to build UML in my chroot 
environment.

It died when it tried to use perl to build.

My uclibc environment hasn't got perl.  The normal linux kernel build doesn't 
use perl.  Neither do busybox, lilo, uClibc, zlib, dropbear, autoconf, 
automake, bin86, binutils, bison, cdrtools, e2fsprogs, flex, gcc, libtool, 
m4, make, nasm, the squashfs or zisofs tools, udev, or any of the other 
packages I build under the thing.

As far as I can tell, the following sed invocation will give more or less what 
the perl does, without a dependency on perl.  (Busybox has sed.)  It may need 
a bit of adapting to make the makefile happy (and adding config.tmp to make 
clean), but the concept seems to work...

sed -e 's/^.*$/"&\\n"/' /linux-2.6.9/.config > config.tmp
sed -e '/CONFIG/{' -e 's/"CONFIG"\;/""/' -e 'r config.tmp' -e 'a ""\;' \
	-e '}' config.c.in

Rob


-------------------------------------------------------
The SF.Net email is sponsored by: Beat the post-holiday blues
Get a FREE limited edition SourceForge.net t-shirt from ThinkGeek.
It's fun and FREE -- well, almost....http://www.thinkgeek.com/sfshirt
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] UML build process uses perl? (With sed alternative.)

[Blaisorblade]

[-- Attachment #1: Type: text/plain, Size: 3156 bytes --]

On Friday 14 January 2005 06:40, Rob Landley wrote:
> I thought maybe the problem I'm having is due to running UML with a uclibc
> root on a glibc host system, so I tried to build UML in my chroot
> environment.
>
> It died when it tried to use perl to build.
>
> My uclibc environment hasn't got perl.  The normal linux kernel build
> doesn't use perl.  Neither do busybox, lilo, uClibc, zlib, dropbear,
> autoconf, automake, bin86, binutils, bison, cdrtools, e2fsprogs, flex, gcc,
> libtool, m4, make, nasm, the squashfs or zisofs tools, udev, or any of the
> other packages I build under the thing.
>
> As far as I can tell, the following sed invocation will give more or less
> what the perl does, without a dependency on perl.  (Busybox has sed.)  It
> may need a bit of adapting to make the makefile happy (and adding
> config.tmp to make clean), but the concept seems to work...
>
> sed -e 's/^.*$/"&\\n"/' /linux-2.6.9/.config > config.tmp

It's ok, if the "&" stands for the matched string as I recall; I've done it 
through this equivalent form:

sed -e 's/^/"/' -e 's/$/\\n"/' .config > config.tmp

I've also modified this further below for a different requirement.

The only potential problem is the use of the temporary file which is not nice, 
however your need seems more important in fact...

> sed -e '/CONFIG/{' -e 's/"CONFIG"\;/""/' -e 'r config.tmp' -e 'a ""\;' \
>  -e '}' config.c.in
If you compare that with the real result, it does not match - and you see it 
because double quotes in .config (which exist, yes) are not escaped as here 
(the dollar is escaped in the original):

$config =~ s/"/\\"/g;

So I've done it this way:

sed -e 's/"/\\"/g' -e 's/^/"/' -e 's/$/\\n"/' .config > config.tmp
i.e. first escape the double quotes, then add unescaped double quotes at start 
and end, and then do what you do:

sed -e '/CONFIG/{' -e 's/"CONFIG"\;/""/' -e 'r config.tmp' -e 'a""\;' -e '}' 
arch/um/kernel/config.c.in > arch/um/kernel/config.c

Which gives the same result as the original method (there is one trivial 
difference at the beginning of the output which is, IMHO, purely cosmethical:

--- arch/um/kernel/config.c     2005-01-19 18:51:48.174446032 +0100
+++ ./config.c  2005-01-19 18:52:46.015652832 +0100
@@ -9,3 +9,4 @@

-static __initdata char *config = "#\n"
+static __initdata char *config = ""
+"#\n"
 "# Automatically generated make config: don't edit\n"

This actually should work, indeed, if translated in the Makefile... which 
should be more or less like the attached patch.

Also, I wanted to avoid writing more -e options for the same programs, to have 
it clearer... however I had problems to do it in the Makefile, so I

Please double-check that the patch works with your busybox sed implementation, 
and compare the original and the new obtained config.c carefully, since the 
sed code you supplied wasn't good.

In fact, I'm no expert in sed portability, so I'm not sure if it behaves well 
with busybox (actually I'm no sed expert, so it was difficult for me to read 
your code at first).
-- 
Paolo Giarrusso, aka Blaisorblade
Linux registered user n. 292729
http://www.user-mode-linux.org/~blaisorblade

[-- Attachment #2: uml-do-not-rely-on-perl-for-building.patch --]
[-- Type: text/x-diff, Size: 2314 bytes --]


To quote .config into config.c for building the result into the code, use sed
instead of perl, as requested by one "embedded" UML user (which notes that perl
is a big requirement, while busybox provides sed which is used in this patch).

I've tested that there are only cosmethical differences in the produced
config.c file, which don't change at all the result (i.e. "a" is replaced by
"" "a" at the beginning, which is non-significant).

Signed-off-by: Paolo 'Blaisorblade' Giarrusso <blaisorblade@yahoo.it>
---

 linux-2.6.11-paolo/arch/um/kernel/Makefile |   28 +++++++++++++++++++++-------
 1 files changed, 21 insertions(+), 7 deletions(-)

diff -puN arch/um/kernel/Makefile~uml-do-not-rely-on-perl-for-building arch/um/kernel/Makefile
--- linux-2.6.11/arch/um/kernel/Makefile~uml-do-not-rely-on-perl-for-building	2005-01-19 18:56:13.163161624 +0100
+++ linux-2.6.11-paolo/arch/um/kernel/Makefile	2005-01-19 19:52:39.296391128 +0100
@@ -4,7 +4,7 @@
 #
 
 extra-y := vmlinux.lds
-clean-files := vmlinux.lds.S
+clean-files := vmlinux.lds.S config.tmp
 
 obj-y = checksum.o config.o exec_kern.o exitcode.o \
 	helper.o init_task.o irq.o irq_user.o ksyms.o main.o mem.o mem_user.o \
@@ -33,11 +33,25 @@ CFLAGS_frame.o := -fno-omit-frame-pointe
 $(USER_OBJS) : %.o: %.c
 	$(CC) $(USER_CFLAGS) $(CFLAGS_$(notdir $@)) -c -o $@ $<
 
-QUOTE = 'my $$config=`cat $(TOPDIR)/.config`; $$config =~ s/"/\\"/g ; $$config =~ s/\n/\\n"\n"/g ; while(<STDIN>) { $$_ =~ s/CONFIG/$$config/; print $$_ }'
+targets += config.c
 
-quiet_cmd_quote = QUOTE   $@
-cmd_quote = $(PERL) -e $(QUOTE) < $< > $@
+#Be careful with the below Sed code - sed is pitfall-rich!
+#We use sed to lower build requirements, for "embedded" builders for instance.
 
-targets += config.c
-$(obj)/config.c : $(src)/config.c.in $(TOPDIR)/.config FORCE
-	$(call if_changed,quote)
+$(obj)/config.tmp: $(objtree)/.config FORCE
+	$(call if_changed,quote1)
+
+quiet_cmd_quote1 = QUOTE   $@
+      cmd_quote1 = sed -e 's/"/\\"/g' -e 's/^/"/' -e 's/$$/\\n"/' \
+		   $< > $@
+
+$(obj)/config.c: $(src)/config.c.in $(obj)/config.tmp FORCE
+	$(call if_changed,quote2)
+
+quiet_cmd_quote2 = QUOTE   $@
+      cmd_quote2 = sed -e '/CONFIG/{'		\
+     		   -e 's/"CONFIG"\;/""/'	\
+		   -e 'r $(obj)/config.tmp'	\
+		   -e 'a""\;'			\
+		   -e '}'			\
+		   $< > $@
_

Re: [uml-devel] UML build process uses perl? (With sed alternative.)

[Rob Landley]

On Wednesday 19 January 2005 01:55 pm, Blaisorblade wrote:

> > As far as I can tell, the following sed invocation will give more or less
> > what the perl does, without a dependency on perl.  (Busybox has sed.)  It
> > may need a bit of adapting to make the makefile happy (and adding
> > config.tmp to make clean), but the concept seems to work...
> >
> > sed -e 's/^.*$/"&\\n"/' /linux-2.6.9/.config > config.tmp
>
> It's ok, if the "&" stands for the matched string as I recall; I've done it
> through this equivalent form:
>
> sed -e 's/^/"/' -e 's/$/\\n"/' .config > config.tmp

Cool.

> I've also modified this further below for a different requirement.
>
> The only potential problem is the use of the temporary file which is not
> nice, however your need seems more important in fact...

There's bound to be a way to make the temporary file go away.  I couldn't 
quite figure out how to apply the in-place option for sed, but possibly the 
sed output could be assigned to an environment variable, and the shell could 
do the final splice of the data into place...

> > sed -e '/CONFIG/{' -e 's/"CONFIG"\;/""/' -e 'r config.tmp' -e 'a ""\;' \
> >  -e '}' config.c.in
>
> If you compare that with the real result, it does not match - and you see
> it because double quotes in .config (which exist, yes) are not escaped as
> here (the dollar is escaped in the original):

As I said, it needed more work.  I wanted to float the concept to see if 
people were agreeable to it.

> Which gives the same result as the original method (there is one trivial
> difference at the beginning of the output which is, IMHO, purely
> cosmethical:
>
> --- arch/um/kernel/config.c     2005-01-19 18:51:48.174446032 +0100
> +++ ./config.c  2005-01-19 18:52:46.015652832 +0100
> @@ -9,3 +9,4 @@
>
> -static __initdata char *config = "#\n"
> +static __initdata char *config = ""
> +"#\n"
>  "# Automatically generated make config: don't edit\n"
>
> This actually should work, indeed, if translated in the Makefile... which
> should be more or less like the attached patch.
>
> Also, I wanted to avoid writing more -e options for the same programs, to
> have it clearer... however I had problems to do it in the Makefile, so I

I probably could have gotten the result to match exactly, but the result would 
have been a more complex sed invocation for no real gain...

> Please double-check that the patch works with your busybox sed
> implementation, and compare the original and the new obtained config.c
> carefully, since the sed code you supplied wasn't good.

I maintain busybox sed, and if it can't handle something gnu sed can, I will 
fix it.

> In fact, I'm no expert in sed portability, so I'm not sure if it behaves
> well with busybox (actually I'm no sed expert, so it was difficult for me
> to read your code at first).

Sorry 'bout that.  I wasn't a sed expert before I extensively rewrote busybox 
sed, but I want to replace all the gnu command line tools with busybox ones 
(for general usage: the FSF took 833 lines of C to implement "cat".  That's 
just wrong).  Sed was heavily used in the ./configure steps of a lot of 
software packages.  The old one didn't work, now it does.

I'll take a closer look at the patch this evening.  Thanks.

Rob


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] UML build process uses perl? (With sed alternative.)

[Rob Landley]

[-- Attachment #1: Type: text/plain, Size: 397 bytes --]

On Wednesday 19 January 2005 01:55 pm, Blaisorblade wrote:

> Please double-check that the patch works with your busybox sed
> implementation, and compare the original and the new obtained config.c
> carefully, since the sed code you supplied wasn't good.

I've double checked, and it worked fine.

I still need the attached one-liner patch to get it to build under uclibc, 
though.

Thanks,

Rob

[-- Attachment #2: strlcpy.patch --]
[-- Type: text/x-diff, Size: 385 bytes --]

--- linux-2.6.9-old/arch/um/include/user.h	2005-01-20 00:38:43.000000000 -0500
+++ linux-2.6.9/arch/um/include/user.h	2005-01-20 00:39:00.000000000 -0500
@@ -14,7 +14,6 @@
 extern void kfree(void *ptr);
 extern int in_aton(char *str);
 extern int open_gdb_chan(void);
-extern int strlcpy(char *, const char *, int);
 extern void *um_vmalloc(int size);
 extern void vfree(void *ptr);
 

Re: [uml-devel] UML build process uses perl? (With sed alternative.)

[Jeff Dike]

rob@landley.net said:
> I still need the attached one-liner patch to get it to build under
> uclibc,  though. 

What does that break, exactly?

				Jeff



-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel

Re: [uml-devel] UML build process uses perl? (With sed alternative.)

[Rob Landley]

On Thursday 20 January 2005 07:31 pm, Jeff Dike wrote:
> rob@landley.net said:
> > I still need the attached one-liner patch to get it to build under
> > uclibc,  though.
>
> What does that break, exactly?
>
> 				Jeff

  CC      arch/um/kernel/init_task.o
  CC      arch/um/kernel/irq.o
gcc -Wall -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common 
-O2 -fomit-frame-pointer -U__i386__ -Ui386 -D__arch_um__ -DSUBARCH=\"i386\" 
-D_LARGEFILE64_SOURCE   -Iarch/um/include 
-I/linux-2.6.9/arch/um/kernel/tt/include 
-I/linux-2.6.9/arch/um/kernel/skas/include -D_GNU_SOURCE  -c -o 
arch/um/kernel/irq_user.o arch/um/kernel/irq_user.c
In file included from arch/um/kernel/irq_user.c:16:
arch/um/include/user.h:15: error: conflicting types for `strlcpy'
/usr/include/string.h:384: error: previous declaration of `strlcpy'
make[1]: *** [arch/um/kernel/irq_user.o] Error 1
make: *** [arch/um/kernel] Error 2

This is because uclibc's string.h has this snippet:

#ifdef	__USE_BSD
/* Two OpenBSD extension functions. */
extern size_t strlcat(char *__restrict dst, const char *__restrict src,
                      size_t n) __THROW;
extern size_t strlcpy(char *__restrict dst, const char *__restrict src,
                      size_t n) __THROW;
#endif

Rob


-------------------------------------------------------
This SF.Net email is sponsored by: IntelliVIEW -- Interactive Reporting
Tool for open source databases. Create drag-&-drop reports. Save time
by over 75%! Publish reports on the web. Export to DOC, XLS, RTF, etc.
Download a FREE copy at http://www.intelliview.com/go/osdn_nl
_______________________________________________
User-mode-linux-devel mailing list
User-mode-linux-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/user-mode-linux-devel