2.5: troubles with piping make output

2.5: troubles with piping make output

[Denis Vlasenko]

My favorite way of running make is

make "$@" 2>&1 | tee --append !make.log

but in 2.5.45 it does not work. Removing '| tee ...'
part fixes it, but I'd like to retain the old way
for obvious reasons.
--
vda

Re: 2.5: troubles with piping make output

[Sam Ravnborg]

On Sun, Nov 03, 2002 at 02:14:35PM -0200, Denis Vlasenko wrote:
> My favorite way of running make is
> 
> make "$@" 2>&1 | tee --append !make.log
> 
> but in 2.5.45 it does not work. Removing '| tee ...'
> part fixes it, but I'd like to retain the old way
> for obvious reasons.
Well, works for me, except that bash dislike '!'.

Could you try to dig a little furhter.
There were quite a lot of changes from 2.44 -> 2.45 but I cannot see
piping should be affected.

	Sam

Re: 2.5: troubles with piping make output

[Denis Vlasenko]

On 3 November 2002 16:28, Sam Ravnborg wrote:
> On Sun, Nov 03, 2002 at 02:14:35PM -0200, Denis Vlasenko wrote:
> > My favorite way of running make is
> >
> > make "$@" 2>&1 | tee --append !make.log
> >
> > but in 2.5.45 it does not work. Removing '| tee ...'
> > part fixes it, but I'd like to retain the old way
> > for obvious reasons.
>
> Well, works for me, except that bash dislike '!'.

Okay, look here:

# cat mk.sh
#!/bin/sh

>>0make.log echo
>>0make.log echo "Date: `date`"
>>0make.log echo "Directory: `pwd`"
>>0make.log echo "Command: make $@"
>>0make.log echo "============="

CFLAGS=-std=gnu99 \
GCC_EXEC_PREFIX=/usr/app/gcc-3.2/lib/gcc-lib/ \
QTDIR=/usr/app/qt-3.0.3posix \
make "$@" 2>&1 | tee --append 0make.log

Ok, I untarred fresh 2.5.45, copied ~2.4.18 .config and ran:

# ./mk.sh oldconfig
   rm -f scripts/built-in.o; ar rcs scripts/built-in.o
  gcc -Wp,-MD,scripts/.fixdep.d -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer   -o scripts/fixdep scripts/fixdep.c
  gcc -Wp,-MD,scripts/.split-include.d -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer   -o scripts/split-include scripts/split-include.c
  gcc -Wp,-MD,scripts/.docproc.d -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer   -o scripts/docproc scripts/docproc.c
  gcc -Wp,-MD,scripts/.conmakehash.d -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer   -o scripts/conmakehash scripts/conmakehash.c
  cat scripts/kconfig/zconf.tab.h_shipped > scripts/kconfig/zconf.tab.h
  gcc -Wp,-MD,scripts/kconfig/.conf.o.d -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer   -c -o scripts/kconfig/conf.o scripts/kconfig/conf.c
sed < scripts/kconfig/lkc_proto.h > scripts/kconfig/lkc_defs.h 's/P(\([^,]*\),.*/#define \1 (\*\1_p)/'
  gcc -Wp,-MD,scripts/kconfig/.kconfig_load.o.d -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer   -c -o scripts/kconfig/kconfig_load.o scripts/kconfig/kconfig_load.c
  gcc -Wp,-MD,scripts/kconfig/.mconf.o.d -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer   -c -o scripts/kconfig/mconf.o scripts/kconfig/mconf.c
  cat scripts/kconfig/zconf.tab.c_shipped > scripts/kconfig/zconf.tab.c
  cat scripts/kconfig/lex.zconf.c_shipped > scripts/kconfig/lex.zconf.c
  gcc -Wp,-MD,scripts/kconfig/.zconf.tab.o.d -fPIC -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer  -Iscripts/kconfig -c -o scripts/kconfig/zconf.tab.o scripts/kconfig/zconf.tab.c
  gcc  -shared -o scripts/kconfig/libkconfig.so scripts/kconfig/zconf.tab.o
  gcc  -o scripts/kconfig/conf scripts/kconfig/conf.o scripts/kconfig/libkconfig.so
./scripts/kconfig/conf -o arch/i386/Kconfig
.config:24: trying to assign nonexistent symbol LOLAT
.config:25: trying to assign nonexistent symbol LOLAT_SYSCTL
.config:172: trying to assign nonexistent symbol BLK_DEV_LVM
.config:258: trying to assign nonexistent symbol BLK_DEV_IDETAPE
.config:307: trying to assign nonexistent symbol BLK_DEV_ATARAID
.config:308: trying to assign nonexistent symbol BLK_DEV_ATARAID_PDC
.config:309: trying to assign nonexistent symbol BLK_DEV_ATARAID_HPT
.config:320: trying to assign nonexistent symbol SD_EXTRA_DEVS
.config:331: trying to assign nonexistent symbol SCSI_DEBUG_QUEUES
.config:579: trying to assign nonexistent symbol INPUT_KEYBDEV
.config:591: trying to assign nonexistent symbol SERIAL
.config:609: trying to assign nonexistent symbol ATIXL_BUSMOUSE
.config:610: trying to assign nonexistent symbol LOGIBUSMOUSE
.config:611: trying to assign nonexistent symbol MS_BUSMOUSE
.config:612: trying to assign nonexistent symbol MOUSE
.config:613: trying to assign nonexistent symbol PSMOUSE
.config:614: trying to assign nonexistent symbol 82C710_MOUSE
.config:615: trying to assign nonexistent symbol PC110_PAD
.config:765: trying to assign nonexistent symbol ZLIB_FS_INFLATE

as it seem to hang here.

Ok, let's do it without mk.sh:

# CFLAGS=-std=gnu99 GCC_EXEC_PREFIX=/usr/app/gcc-3.2/lib/gcc-lib/ QTDIR=/usr/app/qt-3.0.3posix make oldconfig   cat scripts/kconfig/zconf.tab.h_shipped > scripts/kconfig/zconf.tab.h
  gcc -Wp,-MD,scripts/kconfig/.conf.o.d -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer   -c -o scripts/kconfig/conf.o scripts/kconfig/conf.c
sed < scripts/kconfig/lkc_proto.h > scripts/kconfig/lkc_defs.h 's/P(\([^,]*\),.*/#define \1 (\*\1_p)/'
  gcc -Wp,-MD,scripts/kconfig/.kconfig_load.o.d -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer   -c -o scripts/kconfig/kconfig_load.o scripts/kconfig/kconfig_load.c
  gcc -Wp,-MD,scripts/kconfig/.mconf.o.d -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer   -c -o scripts/kconfig/mconf.o scripts/kconfig/mconf.c
  cat scripts/kconfig/zconf.tab.c_shipped > scripts/kconfig/zconf.tab.c
  cat scripts/kconfig/lex.zconf.c_shipped > scripts/kconfig/lex.zconf.c
  gcc -Wp,-MD,scripts/kconfig/.zconf.tab.o.d -fPIC -Wall -Wstrict-prototypes -O2 -fomit-frame-pointer  -Iscripts/kconfig -c -o scripts/kconfig/zconf.tab.o scripts/kconfig/zconf.tab.c
  gcc  -shared -o scripts/kconfig/libkconfig.so scripts/kconfig/zconf.tab.o
  gcc  -o scripts/kconfig/conf scripts/kconfig/conf.o scripts/kconfig/libkconfig.so
./scripts/kconfig/conf -o arch/i386/Kconfig
#
# using defaults found in .config
#
.config:24: trying to assign nonexistent symbol LOLAT
.config:25: trying to assign nonexistent symbol LOLAT_SYSCTL
.config:172: trying to assign nonexistent symbol BLK_DEV_LVM
.config:258: trying to assign nonexistent symbol BLK_DEV_IDETAPE
.config:307: trying to assign nonexistent symbol BLK_DEV_ATARAID
.config:308: trying to assign nonexistent symbol BLK_DEV_ATARAID_PDC
.config:309: trying to assign nonexistent symbol BLK_DEV_ATARAID_HPT
.config:320: trying to assign nonexistent symbol SD_EXTRA_DEVS
.config:331: trying to assign nonexistent symbol SCSI_DEBUG_QUEUES
.config:579: trying to assign nonexistent symbol INPUT_KEYBDEV
.config:591: trying to assign nonexistent symbol SERIAL
.config:609: trying to assign nonexistent symbol ATIXL_BUSMOUSE
.config:610: trying to assign nonexistent symbol LOGIBUSMOUSE
.config:611: trying to assign nonexistent symbol MS_BUSMOUSE
.config:612: trying to assign nonexistent symbol MOUSE
.config:613: trying to assign nonexistent symbol PSMOUSE
.config:614: trying to assign nonexistent symbol 82C710_MOUSE
.config:615: trying to assign nonexistent symbol PC110_PAD
.config:765: trying to assign nonexistent symbol ZLIB_FS_INFLATE
*
* Linux Kernel Configuration
*
*
* Code maturity level options
*
Prompt for development and/or incomplete code/drivers (EXPERIMENTAL) [Y/n/?] y
*
* General setup
*
Networking support (NET) [Y/n/?] y
System V IPC (SYSVIPC) [Y/n/?] y
BSD Process Accounting (BSD_PROCESS_ACCT) [Y/n/?] y
Sysctl support (SYSCTL) [Y/n/?] y
*
* Loadable module support
*
Enable loadable module support (MODULES) [Y/n/?] y
  Set version information on all module symbols (MODVERSIONS) [Y/n/?] y
  Kernel module loader (KMOD) [Y/n/?] y
*
* Processor type and features
*
Processor family (386, 486, 586/K5/5x86/6x86/6x86MX, Pentium-Classic, Pentium-MMX, Pentium-Pro/Celeron/Pentium-II, Pentium-III/Celeron(Coppermine), Pentium-4, K6/K6-II/K6-III, Athlon/Duron/K7, Elan, Crusoe, Winchip-C6, Winchip-2, Winchip-2A/Winchip-3, CyrixIII/VIA-C3) [386] (NEW)

and it politely waits for my input.

Looks like fflush() got forgotten somewhere ;)
--
vda

Re: 2.5: troubles with piping make output

[Roman Zippel]

Hi,

On Sun, 3 Nov 2002, Denis Vlasenko wrote:

> *
> * Processor type and features
> *
> Processor family (386, 486, 586/K5/5x86/6x86/6x86MX, Pentium-Classic, Pentium-MMX, Pentium-Pro/Celeron/Pentium-II, Pentium-III/Celeron(Coppermine), Pentium-4, K6/K6-II/K6-III, Athlon/Duron/K7, Elan, Crusoe, Winchip-C6, Winchip-2, Winchip-2A/Winchip-3, CyrixIII/VIA-C3) [386] (NEW)
> 
> and it politely waits for my input.
> 
> Looks like fflush() got forgotten somewhere ;)

What shell are you using?
This is what should happen:

$ make 2>&1 | tee
make[1]: `scripts/kconfig/conf' is up to date.
./scripts/kconfig/conf -s arch/i386/Kconfig
#
# using defaults found in .config
#
*
* Restart config...
*
*
* Processor type and features
*
Processor family (386, 486, 586/K5/5x86/6x86/6x86MX, Pentium-Classic, Pentium-MMX, Pentium-Pro/Celeron/Pentium-II, Pentium-III/Celeron(Coppermine), Pentium-4, K6/K6-II/K6-III, Athlon/Duron/K7, Elan, Crusoe, Winchip-C6, Winchip-2, Winchip-2A/Winchip-3, CyrixIII/VIA-C3) [Pentium-Pro/Celeron/Pentium-II] (NEW) aborted!

Console input/output is redirected. Run 'make oldconfig' to update configuration.

make: *** [include/linux/autoconf.h] Error 1

bye, Roman

Re: 2.5: troubles with piping make output

[Russell King]

On Sun, Nov 03, 2002 at 09:10:54PM +0100, Roman Zippel wrote:
> > Looks like fflush() got forgotten somewhere ;)
> 
> What shell are you using?
> This is what should happen:
> 
> $ make 2>&1 | tee
> make[1]: `scripts/kconfig/conf' is up to date.
> ./scripts/kconfig/conf -s arch/i386/Kconfig
> #
> # using defaults found in .config
> #
> *
> * Restart config...
> *
> *
> * Processor type and features
> *
> Processor family (386, 486, 586/K5/5x86/6x86/6x86MX, Pentium-Classic, Pentium-MMX, Pentium-Pro/Celeron/Pentium-II, Pentium-III/Celeron(Coppermine), Pentium-4, K6/K6-II/K6-III, Athlon/Duron/K7, Elan, Crusoe, Winchip-C6, Winchip-2, Winchip-2A/Winchip-3, CyrixIII/VIA-C3) [Pentium-Pro/Celeron/Pentium-II] (NEW) aborted!
> 
> Console input/output is redirected. Run 'make oldconfig' to update configuration.
> 
> make: *** [include/linux/autoconf.h] Error 1

No thanks.  That breaks my build scripts.  I don't want to go logging into
multiple machines just to run make oldconfig when the old system worked
perfectly well.

"perfectly well" here means that make oldconfig worked over ssh, with the
local end logging the stdout to a file as well as the terminal, with stdin
from the terminal.  It is quite reasonable to expect the configuration to
continue as normal.

So, here's a patch that adds the necessary fflush to make this situation
work (for me at least.)

diff -u orig/scripts/kconfig/conf.c linux/scripts/kconfig/conf.c
--- orig/scripts/kconfig/conf.c	Sat Nov  2 18:58:34 2002
+++ linux/scripts/kconfig/conf.c	Fri Nov  1 17:02:19 2002
@@ -115,6 +115,7 @@
 			exit(1);
 		}
 	case ask_all:
+		fflush(stdout);
 		fgets(line, 128, stdin);
 		return;
 	case set_default:


-- 
Russell King (rmk@arm.linux.org.uk)                The developer of ARM Linux
             http://www.arm.linux.org.uk/personal/aboutme.html

Re: 2.5: troubles with piping make output

[Roman Zippel]

Hi,

On Sun, 3 Nov 2002, Russell King wrote:

> No thanks.  That breaks my build scripts.  I don't want to go logging into
> multiple machines just to run make oldconfig when the old system worked
> perfectly well.
> 
> "perfectly well" here means that make oldconfig worked over ssh, with the
> local end logging the stdout to a file as well as the terminal, with stdin
> from the terminal.  It is quite reasonable to expect the configuration to
> continue as normal.

Huh? What do you mean? oldconfig still works as before, above only happens 
if you touch .config or a Kconfig file, kconfig tries to automatically 
update .config and will fail if stdio is redirected, but needs user input.
The problem is not a missing fflush, the question is why kconfig couldn't 
detect the pipe.

bye, Roman

Re: 2.5: troubles with piping make output

[Russell King]

On Sun, Nov 03, 2002 at 09:54:05PM +0100, Roman Zippel wrote:
> On Sun, 3 Nov 2002, Russell King wrote:
> > No thanks.  That breaks my build scripts.  I don't want to go logging into
> > multiple machines just to run make oldconfig when the old system worked
> > perfectly well.
> > 
> > "perfectly well" here means that make oldconfig worked over ssh, with the
> > local end logging the stdout to a file as well as the terminal, with stdin
> > from the terminal.  It is quite reasonable to expect the configuration to
> > continue as normal.
> 
> Huh? What do you mean? oldconfig still works as before, above only happens 
> if you touch .config or a Kconfig file, kconfig tries to automatically 
> update .config and will fail if stdio is redirected, but needs user input.
> The problem is not a missing fflush, the question is why kconfig couldn't 
> detect the pipe.

ssh host make -C $tree oldconfig ARCH=arm

that doesn't allocate a terminal.  I want such commands to _prompt_ for
input.  If they die because its not a terminal, I consider that _broken_.
Why?  The command is able to read input from a human, and write its output
to a human via the ssh pipes.

If you insist on breaking this, I'll insist on fixing it.  Its a misfeature
that you refuse to run in this situation.

-- 
Russell King (rmk@arm.linux.org.uk)                The developer of ARM Linux
             http://www.arm.linux.org.uk/personal/aboutme.html

Re: 2.5: troubles with piping make output

[Roman Zippel]

Hi,

On Sun, 3 Nov 2002, Russell King wrote:

> > Huh? What do you mean? oldconfig still works as before,
> 
> ssh host make -C $tree oldconfig ARCH=arm
> 
> that doesn't allocate a terminal.  I want such commands to _prompt_ for
> input.  If they die because its not a terminal, I consider that _broken_.
> Why?  The command is able to read input from a human, and write its output
> to a human via the ssh pipes.
> 
> If you insist on breaking this, I'll insist on fixing it.  Its a misfeature
> that you refuse to run in this situation.

As I already said, oldconfig still works as before. Maybe you should have 
tried it first?

bye, Roman

Re: 2.5: troubles with piping make output

[Russell King]

On Sun, Nov 03, 2002 at 10:29:38PM +0100, Roman Zippel wrote:
> As I already said, oldconfig still works as before. Maybe you should have 
> tried it first?

I have.  However, I thought you were about to change the oldconfig
behaviour.  My bad.

The patch is still required, though, to make sure stdout is flushed to
the user before asking a question, which doesn't happen in the case I
highlighted.

-- 
Russell King (rmk@arm.linux.org.uk)                The developer of ARM Linux
             http://www.arm.linux.org.uk/personal/aboutme.html

Re: 2.5: troubles with piping make output

[Roman Zippel]

Hi,

On Sun, 3 Nov 2002, Russell King wrote:

> > As I already said, oldconfig still works as before. Maybe you should have 
> > tried it first?
> 
> I have.  However, I thought you were about to change the oldconfig
> behaviour.  My bad.

Why should I? The original problem was with 'make | tee', which might run 
a silent version of oldconfig.

> The patch is still required, though, to make sure stdout is flushed to
> the user before asking a question, which doesn't happen in the case I
> highlighted.

Will add. Thanks.

bye, Roman