coloring stdout && stderr, small kbuild example

coloring stdout && stderr, small kbuild example

[Oleg Verych]

Those boring never-ending builds...

Can some dualCPU power be spent, if building is interactive (i.e. sent to
terminal), to have them a bit colored?

My last rant about "colored printk()" led away from kernel developers
and their solutions. But now, let me share something.

(maybe some terminal emulators have more colors, finally?) usage:

olecom@flower:/tmp/blinux$ stty -tostop ; mkfifo stdout.pipe stderr.pipe
olecom@flower:/tmp/blinux$ sh ./colorize-linux-build.sh <stdout.pipe &
[1] 22070
olecom@flower:/tmp/blinux$ COLORIZERR=y sh colorize-linux-build.sh <stderr.pipe &
[2] 22145
olecom@flower:/tmp/blinux$
olecom@flower:/tmp/blinux$ LANG=C make >stdout.pipe 2>stderr.pipe
make -C /mnt/zdev0/linux-2.6 O=/dev/shm/blinux/.
  Using /mnt/zdev0/linux-2.6 as source for kernel
  GEN     /dev/shm/blinux/Makefile
  CHK     include/linux/version.h
  CHK     include/linux/utsrelease.h
  CALL    /mnt/zdev0/linux-2.6/scripts/checksyscalls.sh
  CHK     include/linux/compile.h
  VDSOSYM arch/x86/vdso/vdso-syms.lds
  VDSOSYM arch/x86/vdso/vdso32-syscall-syms.lds
  VDSOSYM arch/x86/vdso/vdso32-sysenter-syms.lds
  VDSOSYM arch/x86/vdso/vdso32-syms.lds
  LD      arch/x86/vdso/built-in.o
  CC      mm/memory.o

olecom@flower:/tmp/blinux$

make[3]: *** [mm/memory.o] Interrupt
make[2]: *** [mm] Interrupt
make[1]: *** [sub-make] Interrupt
make: *** [all] Interrupt

[1]-  Done                    sh ./colorize-linux-build.sh <stdout.pipe
[2]+  Done                    COLORIZERR=yes sh colorize-linux-build.sh <stderr.pipe
olecom@flower:/tmp/blinux$

________________________________________________________________________
#!/bin/sh
# colorize linux build

# Time-stamp: Wed Apr  9 13:78:13 CEST 2008 olecom@flower.upol.cRAzY

color() {
    # $1 -- regex
    # $2 -- color code
    # $3 -- next color code (default is $D)
    # $4 -- regex match number (s///flags)
    printf %s "`eval echo $TERMINAL_OUTPUT`"
}

# for tty
[ -t 1 ] && {
    # escape symbol
    E=`printf '\033'`
    # default fg && bg rendering ; reset attributes
    D="7" ; RESET="$E[0m"
    # basic attributes (reset bg color and bold, set fg color)
    V="$E[40;22;3"
    # wrap regex with color ; goto exit
    TERMINAL_OUTPUT='{s-"$1"-"$V$2m&$V${3-'$D'}m"-$4 ";" b}'
}

# process stdout

[ -z "$COLORIZERR" ] && {
    sed "
/CC/` color CC '6;1'`
/LD/` color LD '2;1'`
/AS/` color AS '5;1'`
/GEN/`color GEN '2;1'`
/UPD/`color UPD '2;1'`
/CHK/`color CHK '3;1'`
/CA/` color CALL '6'`
s-^-${V}2m-
"
    printf "$RESET
"
}

# process stderr (once)

while read ERROR
do OUT=$OUT'
'$ERROR
done

printf "\033[1;37;41m$OUT$RESET
"
exit

Re: coloring stdout && stderr, small kbuild example

[Bernd Petrovitsch]

On Mit, 2008-04-09 at 17:02 +0200, Oleg Verych wrote:
> Those boring never-ending builds...
> 
> Can some dualCPU power be spent, if building is interactive (i.e. sent to
> terminal), to have them a bit colored?
> 
> My last rant about "colored printk()" led away from kernel developers
> and their solutions. But now, let me share something.

I don't know if it's of any use for you but there is already
*) color-make - http://bre.klaki.net/programs/colormake/ - and
*) color-gcc - http://schlueters.de/colorgcc.html -
out there.

	Bernd
-- 
Firmix Software GmbH                   http://www.firmix.at/
mobil: +43 664 4416156                 fax: +43 1 7890849-55
          Embedded Linux Development and Services

Re: coloring stdout && stderr, small kbuild example

[Oleg Verych]

> I don't know if it's of any use for you but there is already
> *) color-make - http://bre.klaki.net/programs/colormake/ - and
> *) color-gcc - http://schlueters.de/colorgcc.html -
> out there.

Yea (subject: small kbuild example).
_____

Re: coloring stdout && stderr, small kbuild example

[Oleg Verych]

Time flies...

> > My last rant about "colored printk()" led away from kernel developers
> > and their solutions. But now, let me share something.

That was about "The True Kernel Developer's Way to Coloring"
by Jonathan Corbet October 9, 2007 http://lwn.net/Articles/253726/

Bernd Petrovitsch @ Wed, Apr 09, 2008 at 05:14:00PM +0200:

> I don't know if it's of any use for you but there is already
> *) color-make - http://bre.klaki.net/programs/colormake/ - and
> *) color-gcc - http://schlueters.de/colorgcc.html -
> out there.

Please, show those to http://udrepper.livejournal.com/17109.html
`perl` is more welcome in Debian, btw.

Free versions of color_proc_cpuinfo && color_meminfo:

TERMINAL_OUTPUT='s-"$1"-"$V$2m&$V${3-'$D'}m"-$4'

`color '[^:]*' 4 3`
`color :       3 2`
`color AMD     6 2`
`color Intel   6 2`
`color Athlon  '6;44' 2`
`color Pentium '6;44' 2`

Invent Your Own Easily(R)
--
sed 'sed && sh + olecom = love' << ''
-o--=O`C
 #oo'L O
<___=E M

Re: coloring stdout && stderr, small kbuild example

[Pavel Machek]

Hi!

> > Those boring never-ending builds...
> > 
> > Can some dualCPU power be spent, if building is interactive (i.e. sent to
> > terminal), to have them a bit colored?
> > 
> > My last rant about "colored printk()" led away from kernel developers
> > and their solutions. But now, let me share something.
> 
> I don't know if it's of any use for you but there is already
> *) color-make - http://bre.klaki.net/programs/colormake/ - and
> *) color-gcc - http://schlueters.de/colorgcc.html -
> out there.

icecc seems to do the coloring, too...

-- 
(english) http://www.livejournal.com/~pavelmachek
(cesky, pictures) http://atrey.karlin.mff.cuni.cz/~pavel/picture/horses/blog.html

lide: linux ide, just concept idea (Re: coloring stdout && stderr, small kbuild example)

[Oleg Verych]

Hallo.

Pavel Machek @ Sat, Apr 12, 2008 at 7:32 PM:
>  icecc seems to do the coloring, too...

This one? http://en.opensuse.org/Icecream
,--
|0.7.3 (r552930):
|[..]
|        - builtin colorgcc
`--
Now i'm being pointed to C++ one. Alright!

== Check this out then. ==

Linus says, make it working somehow and release, so...
I'd like to have it in menuconfig window running gcc on every */M
(i want alot more, without ncurses, perl, C++ using kernel, sh, sed)

extract$         sed '1,/^__/d' >/tmp/lide.sh
use (in OBJDIR)$ clear ; sh /tmp/lide.sh kdefc

kdefc   is a user friendly thing for `make defconfig`
ksecmis is one for funny `make CONFIG_DEBUG_SECTION_MISMATCH=y`
any ordinary command will work, no command -- interactive mode.

Coloring and painting is for children, big daddies usually don't get it,
as well as most basic userspace possible(: /bin/sh ).

(any tty braindamage is a cool tty design from 70s, not sh + sed)
--
Text processing(for newbies): http://kernelnewbies.org/olecom
Tech VA: http://kerneltrap.org/blog/9104
-o--=O`C
 #oo'L O
<___=E M
________
#!/bin/sh
# linux integrated development environment (without text editor/pager)

# Time-stamp: Sun Apr 13 16:79:15 CEST 2008 olecom@flower.upol.cRAzY

[ "$lideRELOAD" ] || {
	[ -t 1 ] || exec dd count=1 # notty
#sed(){
# busybox -- cryptic errors from internal `sed`
# sed -- dash crashes
#}
	stty -tostop
	rm -rf /tmp/stderr.pipe /tmp/stdout.pipe
	mkfifo /tmp/stderr.pipe /tmp/stdout.pipe
# /tmp/stderr.txt is for `tee ` and off-line pager
}
set +i -e

# own syntax:
# $E   is  'ESC[' = '\233'
# fork tty service
sed "s-[$]E-`printf '\233'`-g" << '# here' | /bin/sh -s &
# ${E} is   ESC   ; Ignored symbol for stderr prefix
E=`printf '\033'` ; I=`printf '\177'`

# default fg;bg rendering ; reset attributes
D="7" ; RESET="$E0m"
# basic attributes (reset bg color and bold, set fg color)
V="$E40;22;3"

# wrap regex, multiple times per line
T=\''\"s-$1-$V$2m&$V${3-'$D'}m-$4;\"'\'
eval '
color_lide() {
# wrap regex with color
	# $1 -- regex ; $2 -- color code ; $3 -- next color code (default is $D)
	# $4 -- s///flags
	printf %s "`eval echo '"$T"'`"
}
scolor_lide() {
# color() with speed optimizing jump, thus, one match in line
	printf %s "{`eval echo '"$T"'` ; b_$REGION}"
}
'
T=color_lide


## setup of regions

# main load:  stdout region, content is cut on width = $W

# small load: stderr, no cut; stderr out is rare, thus full lines

# parameters: all have content line limits (*LINES - 1)
#             *ORIGIN is a row placing (FIXME: add colons)

# console output concurrency: whole region is out with cursor move
#		 (no save/restore of it)

outLINES=10  ; outORIGIN=1 ; W=77
errLINES=14  ; errORIGIN=$(($outLINES + 5))
PMT="$E$(($outLINES + 3));1Hprompt:"

sed_GET_CHUNK=':addnl ; $!{ s $ '$EK' ; N ; baddnl }'

[ "$lideRELOAD" ] || {
	sed "# sed does final new lines here
$sed_GET_CHUNK
s ,-- $E$outORIGIN;1H& ;
s/ . -/$E30m&/g
s 0 1 ; s 0 2 ; s 0 3 ; s 0 4 ; s 0 5 ; s 0 6 ;

s-___-$E$errORIGIN;1H$E1;34;40m&$E31m-1
s-___-$E34m&$E31m-2
s/ _ /$E37m&$E31m/g
" << "---"
,-- kbuild: s - t - d - o - u - t --
|
|
`--
___ kbuild: s _ t _ d _ e _ r _ r ___
---
	printf "$PMT\n"
}

# shift title string

outORIGIN=$(($outORIGIN + 1))
errORIGIN=$(($errORIGIN + 1))


## handling of content

region(){
# buffered output with no more than LINES in continuously running `sed`
    # $1 -- type (out, stderr etc.) ; $2 -- region''s final line decoration
    eval 'printf %s "
# get prev. buffer; append current line
x ; G

# remove overflowed head (this sed''s something:)
$'"$1"'LINES,19810702s-^[^\\n]*\\n--

#$'"$1"'LINES,\$s-[^\\n]*\\n--
# last line  --^^ address yields debbug#475464: lost cycle

# save buffer; move to ${1}ORIGIN and print whole region
h ; s .* $E$'"$1"'ORIGIN;1H&\\n'"$2"' ;
"'
}

lideSYNTAX="
/[.,%/;:]/s [.,%/;:] $E1;36m&$E22;3${D}m g
/'/s-'\([^']*\)'-$E35m'$E36m\1$E35m'$E3${D}m-g"'
/"/s-"\([^"]*\)"-$E1;32;40m"$E22;39;44m\1$E1;32;40m"$E22;3'$D'm-g'"
/(/s-(\([^)]*\))-$E1;33m($E22;3${D}m\1$E1;33m)$E22;3${D}m-g
"
REGION=err
sed_STDERR="
$lideSYNTAX

# hard error
/^$I/s-.*-$E1;5;41;37m&${V}1;25;m$EK-
# ordinary
/^$I/!s-.*-&$EK-

# example of being freindly to kernel developers and kbuild users
/make /{s~make CONFIG_DEBUG_SECTION_MISMATCH=y~ksecmis~;b_$REGION}

# UPPER symbols:
`$T '[[:upper:]_]\{3,\}' '2' '1' g`
/[Ww][Aa][Rr]/`$T '[Ww][Aa][Rr][Nn][Ii][Nn][Gg]' '7;41;1' '1;40;22'`
/:/`$T '^[^:]*:' '2;1' $D`

:_$REGION
`region $REGION '$PMT'`
"

REGION=out
sed_OUT="
s-\(.\{0,$W\}\).*-|\1$EK-
/^|#/`s$T '#.*' '3'`
$lideSYNTAX
/:/`$T '^[^:]*:' '2;1' $D`
/ CC/`s$T CC  '6;1'`
/ LD/`s$T LD  '2;1'`
/ AS/`s$T AS  '5;1'`
/ GE/`s$T GEN '2;1'`
/ UP/`s$T UPD '2;1'`
/ CH/`s$T CHK '3;1'`
/ CA/`s$T CALL  '6'`
s-^-${V}${D}m-
:_$REGION
`region $REGION '\\\`--$PMT'`
"
unset lideSYNTAX

trap '
    printf "$RESET$E$(($errORIGIN + $errLINES))H"
    kill -1 $JJ ; kill -KILL $JJ
' EXIT HUP INT TERM QUIT

# close script's input, remove own stderr

exec 0<&- 2>/dev/null
tee -a /tmp/stderr.txt </tmp/stderr.pipe |
sed -u "$sed_STDERR" & JJ=$!' '$JJ
sed "$sed_OUT"    </tmp/stdout.pipe #>out.pipe & JJ=$!' '$JJ

exit

# stdout/stderr coloring daemon ends
# here

JJ=$! # save its pid
unset lideRELOAD
HISTLINE=:

lideCHECK_EXIT_STATUS='
S=$? ; Ex=Ex ; [ 0 -ne "$S" ] && Ex="\177"$Ex
sleep 1  # fix braindamage due to cuncurrency, buffers, other tty crap
printf  1>&2 "${Ex}it status: $S\n"'
trap "$lideCHECK_EXIT_STATUS"'"user interface is over\n"
	kill -1 $JJ

	rm -rf /tmp/stderr.pipe /tmp/stdout.pipe
	exit
' EXIT HUP QUIT

trap 'echo got SIGINT or SIGTERM 1>&2' INT TERM

exec 1>/tmp/stdout.pipe 2>/tmp/stderr.pipe

set +e

echo '# examples of being freindly to kernel developers and kbuild users:
commands: ksecmis ; kdefc ; (add yours) ; `make` crap ; any other
leave:    hell (`exit` is ignored)
'
lideBUILTINS="
ksecmis) LINE='make CONFIG_DEBUG_SECTION_MISMATCH=y' ;;
kdefc)   LINE='make defconfig' ;;
# add yours here
"

eval '

batch() {
case $1 in
'"$lideBUILTINS"'
*) LINE=\"\$@\" ;;
esac
eval "echo \"$LINE\" ; $LINE ; "'"$lideCHECK_EXIT_STATUS"'
}

interactive() {
while read LINE
do echo "$LINE"
   case $LINE in
'"$lideBUILTINS"'
hell)    LINE=break ;;
exit)    LINE=""   ;;

#reload) kill -1 $JJ ; export lideRELOAD=y ; exec /bin/sh $0 ;;
esac

eval "${LINE:-$HISTLINE} ; "'"$lideCHECK_EXIT_STATUS"'
[ "$LINE" ] && HISTLINE=$LINE
done
}
'
# `eval` must complete itself, no infinite loops inside
unset lideCHECK_EXIT_STATUS lideBUILTINS

[ -z "$1" ] && interactive || batch "$@"

sed 'sed && sh + olecom = love' << '' >&2

WTFPLed-by:  olecom
see license: http://sam.zoy.org/wtfpl/

Re: coloring stdout && stderr, small kbuild example

[Oleg Verych]

Forgot, email can be piped ('|' in mutt) to have script ready

to this line: sed '1,/^___/d' >/tmp/colorize-linux-build.sh'
--
sed 'sed && sh + olecom = love' << ''
http://kernelnewbies.org/olecom
-o--=O`C
 #oo'L O
<___=E M