* [PATCH 0/5] pybootchartgui: make the build profiling in pictures
@ 2012-06-06 9:34 Robert Yang
2012-06-06 9:34 ` [PATCH 1/5] pybootchartgui: add the original code Robert Yang
` (5 more replies)
0 siblings, 6 replies; 7+ messages in thread
From: Robert Yang @ 2012-06-06 9:34 UTC (permalink / raw)
To: openembedded-core; +Cc: Zhenfeng.Zhao
Note: Removed the patch details for PATCH 1/5 and 2/5 since they are too
big(bigger than 2M) to send via email, please fetch the patch via
the git repo
// Robert
The following changes since commit df8f55a919b3cc602ce1f1c51630c7edf6e36b55:
ltp: Add patch to correct failing build (2012-06-05 23:05:00 +0100)
are available in the git repository at:
git://git.pokylinux.org/poky-contrib robert/chart
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/chart
Robert Yang (5):
pybootchartgui: add the original code
pybootchartgui: remove the original examples
pybootchartgui: make the build profiling in pictures
pybootchartgui: split the output chart into multiple ones
pybootchartgui: Fix the filename and add a default format
scripts/pybootchartgui/AUTHORS | 11 +
scripts/pybootchartgui/COPYING | 674 ++++++++++++++++++++
scripts/pybootchartgui/README | 36 +
scripts/pybootchartgui/pybootchartgui.py | 7 +
scripts/pybootchartgui/pybootchartgui/batch.py | 23 +
scripts/pybootchartgui/pybootchartgui/draw.py | 408 ++++++++++++
scripts/pybootchartgui/pybootchartgui/gui.py | 273 ++++++++
scripts/pybootchartgui/pybootchartgui/main.py | 76 +++
scripts/pybootchartgui/pybootchartgui/parsing.py | 264 ++++++++
.../pybootchartgui/pybootchartgui/process_tree.py | 270 ++++++++
scripts/pybootchartgui/pybootchartgui/samples.py | 93 +++
.../pybootchartgui/tests/parser_test.py | 93 +++
.../pybootchartgui/tests/process_tree_test.py | 78 +++
13 files changed, 2306 insertions(+), 0 deletions(-)
create mode 100644 scripts/pybootchartgui/AUTHORS
create mode 100644 scripts/pybootchartgui/COPYING
create mode 100644 scripts/pybootchartgui/README
create mode 100755 scripts/pybootchartgui/pybootchartgui.py
create mode 100644 scripts/pybootchartgui/pybootchartgui/__init__.py
create mode 100644 scripts/pybootchartgui/pybootchartgui/batch.py
create mode 100644 scripts/pybootchartgui/pybootchartgui/draw.py
create mode 100644 scripts/pybootchartgui/pybootchartgui/gui.py
create mode 100644 scripts/pybootchartgui/pybootchartgui/main.py
create mode 100644 scripts/pybootchartgui/pybootchartgui/parsing.py
create mode 100644 scripts/pybootchartgui/pybootchartgui/process_tree.py
create mode 100644 scripts/pybootchartgui/pybootchartgui/samples.py
create mode 100644 scripts/pybootchartgui/pybootchartgui/tests/parser_test.py
create mode 100644 scripts/pybootchartgui/pybootchartgui/tests/process_tree_test.py
^ permalink raw reply [flat|nested] 7+ messages in thread* [PATCH 1/5] pybootchartgui: add the original code 2012-06-06 9:34 [PATCH 0/5] pybootchartgui: make the build profiling in pictures Robert Yang @ 2012-06-06 9:34 ` Robert Yang 2012-06-06 9:34 ` [PATCH 2/5] pybootchartgui: remove the original examples Robert Yang ` (4 subsequent siblings) 5 siblings, 0 replies; 7+ messages in thread From: Robert Yang @ 2012-06-06 9:34 UTC (permalink / raw) To: openembedded-core; +Cc: Zhenfeng.Zhao This is from: http://pybootchartgui.googlecode.com/files/pybootchartgui-r124.tar.gz Will modify it to make the build profiling in pictures. [YOCTO #2403] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> --- scripts/pybootchartgui/AUTHORS | 11 + scripts/pybootchartgui/COPYING | 674 ++ scripts/pybootchartgui/README | 36 + .../examples/1/extract.proc_diskstats.log | 141 + .../pybootchartgui/examples/1/extract.proc_ps.log | 861 ++ .../examples/1/extract.proc_stat.log | 141 + .../examples/1/extract.processtree.1.log | 861 ++ .../examples/1/extract.processtree.2.log | 861 ++ .../examples/1/extract.processtree.3.log | 861 ++ .../examples/1/extract.processtree.3a.log | 861 ++ .../examples/1/extract.processtree.3b.log | 200 + .../examples/1/extract.processtree.3c.log | 200 + .../examples/1/extract.processtree.3d.log | 108 + .../examples/1/extract.processtree.3e.log | 100 + .../examples/1/extract.processtree.4.log | 100 + .../examples/1/extract.processtree.5.log | 100 + .../pybootchartgui/examples/1/extract2.proc_ps.log | 861 ++ scripts/pybootchartgui/examples/1/header | 7 + .../pybootchartgui/examples/1/java-png-result.png | Bin 0 -> 138999 bytes .../pybootchartgui/examples/1/proc_diskstats.log | 3171 +++++++ scripts/pybootchartgui/examples/1/proc_ps.log | 9041 ++++++++++++++++++++ scripts/pybootchartgui/examples/1/proc_stat.log | 1562 ++++ .../pybootchartgui/examples/1/pybootchartgui.png | Bin 0 -> 102909 bytes scripts/pybootchartgui/examples/2/header | 6 + .../pybootchartgui/examples/2/java-png-result.png | Bin 0 -> 173535 bytes scripts/pybootchartgui/examples/2/kernel_pacct | Bin 0 -> 321408 bytes .../pybootchartgui/examples/2/proc_diskstats.log | 5262 ++++++++++++ scripts/pybootchartgui/examples/2/proc_ps.log | 8940 +++++++++++++++++++ scripts/pybootchartgui/examples/2/proc_stat.log | 2050 +++++ .../pybootchartgui/examples/3/f11_bootchart.tgz | Bin 0 -> 77979 bytes .../pybootchartgui/examples/4/f11_bootchart2.tgz | Bin 0 -> 77979 bytes scripts/pybootchartgui/pybootchartgui.py | 7 + scripts/pybootchartgui/pybootchartgui/batch.py | 23 + scripts/pybootchartgui/pybootchartgui/draw.py | 355 + scripts/pybootchartgui/pybootchartgui/gui.py | 273 + scripts/pybootchartgui/pybootchartgui/main.py | 71 + scripts/pybootchartgui/pybootchartgui/parsing.py | 223 + .../pybootchartgui/pybootchartgui/process_tree.py | 270 + scripts/pybootchartgui/pybootchartgui/samples.py | 93 + .../pybootchartgui/tests/parser_test.py | 93 + .../pybootchartgui/tests/process_tree_test.py | 78 + 41 files changed, 38502 insertions(+), 0 deletions(-) create mode 100644 scripts/pybootchartgui/AUTHORS create mode 100644 scripts/pybootchartgui/COPYING create mode 100644 scripts/pybootchartgui/README create mode 100644 scripts/pybootchartgui/examples/1/extract.proc_diskstats.log create mode 100644 scripts/pybootchartgui/examples/1/extract.proc_ps.log create mode 100644 scripts/pybootchartgui/examples/1/extract.proc_stat.log create mode 100644 scripts/pybootchartgui/examples/1/extract.processtree.1.log create mode 100644 scripts/pybootchartgui/examples/1/extract.processtree.2.log create mode 100644 scripts/pybootchartgui/examples/1/extract.processtree.3.log create mode 100644 scripts/pybootchartgui/examples/1/extract.processtree.3a.log create mode 100644 scripts/pybootchartgui/examples/1/extract.processtree.3b.log create mode 100644 scripts/pybootchartgui/examples/1/extract.processtree.3c.log create mode 100644 scripts/pybootchartgui/examples/1/extract.processtree.3d.log create mode 100644 scripts/pybootchartgui/examples/1/extract.processtree.3e.log create mode 100644 scripts/pybootchartgui/examples/1/extract.processtree.4.log create mode 100644 scripts/pybootchartgui/examples/1/extract.processtree.5.log create mode 100644 scripts/pybootchartgui/examples/1/extract2.proc_ps.log create mode 100644 scripts/pybootchartgui/examples/1/header create mode 100644 scripts/pybootchartgui/examples/1/java-png-result.png create mode 100644 scripts/pybootchartgui/examples/1/proc_diskstats.log create mode 100644 scripts/pybootchartgui/examples/1/proc_ps.log create mode 100644 scripts/pybootchartgui/examples/1/proc_stat.log create mode 100644 scripts/pybootchartgui/examples/1/pybootchartgui.png create mode 100644 scripts/pybootchartgui/examples/2/header create mode 100644 scripts/pybootchartgui/examples/2/java-png-result.png create mode 100644 scripts/pybootchartgui/examples/2/kernel_pacct create mode 100644 scripts/pybootchartgui/examples/2/proc_diskstats.log create mode 100644 scripts/pybootchartgui/examples/2/proc_ps.log create mode 100644 scripts/pybootchartgui/examples/2/proc_stat.log create mode 100755 scripts/pybootchartgui/examples/3/f11_bootchart.tgz create mode 100755 scripts/pybootchartgui/examples/4/f11_bootchart2.tgz create mode 100755 scripts/pybootchartgui/pybootchartgui.py create mode 100644 scripts/pybootchartgui/pybootchartgui/__init__.py create mode 100644 scripts/pybootchartgui/pybootchartgui/batch.py create mode 100644 scripts/pybootchartgui/pybootchartgui/draw.py create mode 100644 scripts/pybootchartgui/pybootchartgui/gui.py create mode 100644 scripts/pybootchartgui/pybootchartgui/main.py create mode 100644 scripts/pybootchartgui/pybootchartgui/parsing.py create mode 100644 scripts/pybootchartgui/pybootchartgui/process_tree.py create mode 100644 scripts/pybootchartgui/pybootchartgui/samples.py create mode 100644 scripts/pybootchartgui/pybootchartgui/tests/parser_test.py create mode 100644 scripts/pybootchartgui/pybootchartgui/tests/process_tree_test.py # Remove the patch since it is too big to send via email, please fetch # the patch via the git repo ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 2/5] pybootchartgui: remove the original examples 2012-06-06 9:34 [PATCH 0/5] pybootchartgui: make the build profiling in pictures Robert Yang 2012-06-06 9:34 ` [PATCH 1/5] pybootchartgui: add the original code Robert Yang @ 2012-06-06 9:34 ` Robert Yang 2012-06-06 9:34 ` [PATCH 3/5] pybootchartgui: make the build profiling in pictures Robert Yang ` (3 subsequent siblings) 5 siblings, 0 replies; 7+ messages in thread From: Robert Yang @ 2012-06-06 9:34 UTC (permalink / raw) To: openembedded-core; +Cc: Zhenfeng.Zhao Remove the examples since they would not work any more, and they cost much disk space. [YOCTO #2403] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> --- .../examples/1/extract.proc_diskstats.log | 141 - .../pybootchartgui/examples/1/extract.proc_ps.log | 861 -- .../examples/1/extract.proc_stat.log | 141 - .../examples/1/extract.processtree.1.log | 861 -- .../examples/1/extract.processtree.2.log | 861 -- .../examples/1/extract.processtree.3.log | 861 -- .../examples/1/extract.processtree.3a.log | 861 -- .../examples/1/extract.processtree.3b.log | 200 - .../examples/1/extract.processtree.3c.log | 200 - .../examples/1/extract.processtree.3d.log | 108 - .../examples/1/extract.processtree.3e.log | 100 - .../examples/1/extract.processtree.4.log | 100 - .../examples/1/extract.processtree.5.log | 100 - .../pybootchartgui/examples/1/extract2.proc_ps.log | 861 -- scripts/pybootchartgui/examples/1/header | 7 - .../pybootchartgui/examples/1/java-png-result.png | Bin 138999 -> 0 bytes .../pybootchartgui/examples/1/proc_diskstats.log | 3171 ------- scripts/pybootchartgui/examples/1/proc_ps.log | 9041 -------------------- scripts/pybootchartgui/examples/1/proc_stat.log | 1562 ---- .../pybootchartgui/examples/1/pybootchartgui.png | Bin 102909 -> 0 bytes scripts/pybootchartgui/examples/2/header | 6 - .../pybootchartgui/examples/2/java-png-result.png | Bin 173535 -> 0 bytes scripts/pybootchartgui/examples/2/kernel_pacct | Bin 321408 -> 0 bytes .../pybootchartgui/examples/2/proc_diskstats.log | 5262 ------------ scripts/pybootchartgui/examples/2/proc_ps.log | 8940 ------------------- scripts/pybootchartgui/examples/2/proc_stat.log | 2050 ----- .../pybootchartgui/examples/3/f11_bootchart.tgz | Bin 77979 -> 0 bytes .../pybootchartgui/examples/4/f11_bootchart2.tgz | Bin 77979 -> 0 bytes 28 files changed, 0 insertions(+), 36295 deletions(-) delete mode 100644 scripts/pybootchartgui/examples/1/extract.proc_diskstats.log delete mode 100644 scripts/pybootchartgui/examples/1/extract.proc_ps.log delete mode 100644 scripts/pybootchartgui/examples/1/extract.proc_stat.log delete mode 100644 scripts/pybootchartgui/examples/1/extract.processtree.1.log delete mode 100644 scripts/pybootchartgui/examples/1/extract.processtree.2.log delete mode 100644 scripts/pybootchartgui/examples/1/extract.processtree.3.log delete mode 100644 scripts/pybootchartgui/examples/1/extract.processtree.3a.log delete mode 100644 scripts/pybootchartgui/examples/1/extract.processtree.3b.log delete mode 100644 scripts/pybootchartgui/examples/1/extract.processtree.3c.log delete mode 100644 scripts/pybootchartgui/examples/1/extract.processtree.3d.log delete mode 100644 scripts/pybootchartgui/examples/1/extract.processtree.3e.log delete mode 100644 scripts/pybootchartgui/examples/1/extract.processtree.4.log delete mode 100644 scripts/pybootchartgui/examples/1/extract.processtree.5.log delete mode 100644 scripts/pybootchartgui/examples/1/extract2.proc_ps.log delete mode 100644 scripts/pybootchartgui/examples/1/header delete mode 100644 scripts/pybootchartgui/examples/1/java-png-result.png delete mode 100644 scripts/pybootchartgui/examples/1/proc_diskstats.log delete mode 100644 scripts/pybootchartgui/examples/1/proc_ps.log delete mode 100644 scripts/pybootchartgui/examples/1/proc_stat.log delete mode 100644 scripts/pybootchartgui/examples/1/pybootchartgui.png delete mode 100644 scripts/pybootchartgui/examples/2/header delete mode 100644 scripts/pybootchartgui/examples/2/java-png-result.png delete mode 100644 scripts/pybootchartgui/examples/2/kernel_pacct delete mode 100644 scripts/pybootchartgui/examples/2/proc_diskstats.log delete mode 100644 scripts/pybootchartgui/examples/2/proc_ps.log delete mode 100644 scripts/pybootchartgui/examples/2/proc_stat.log delete mode 100755 scripts/pybootchartgui/examples/3/f11_bootchart.tgz delete mode 100755 scripts/pybootchartgui/examples/4/f11_bootchart2.tgz # Remove the patch since it is too big to send via email, please fetch # the code via the git repo ^ permalink raw reply [flat|nested] 7+ messages in thread
* [PATCH 3/5] pybootchartgui: make the build profiling in pictures 2012-06-06 9:34 [PATCH 0/5] pybootchartgui: make the build profiling in pictures Robert Yang 2012-06-06 9:34 ` [PATCH 1/5] pybootchartgui: add the original code Robert Yang 2012-06-06 9:34 ` [PATCH 2/5] pybootchartgui: remove the original examples Robert Yang @ 2012-06-06 9:34 ` Robert Yang 2012-06-06 9:34 ` [PATCH 4/5] pybootchartgui: split the output chart into multiple ones Robert Yang ` (2 subsequent siblings) 5 siblings, 0 replies; 7+ messages in thread From: Robert Yang @ 2012-06-06 9:34 UTC (permalink / raw) To: openembedded-core; +Cc: Zhenfeng.Zhao The original patch is from Richard, I rebased it to the up-to-date upstream code, here are the original messages from him: We have just merged Beth's initial buildstats logging work. I was sitting wondering how to actually evaluate the numbers as I wanted to know "where are we spending the time?". It occurred to me that I wanted a graph very similar to that generated by bootchart. I looked around and found pyboootchartgui and then hacked it around a bit and coerced it to start producing charts like: http://tim.rpsys.net/bootchart.png which is the initial "pseudo-native" part of the build. This was simple enough to test with. I then tried graphing a poky-image-sato. To get a graph I could actually read, I stripped out any task taking less than 8 seconds and scaled the x axis from 25 units per second to one unit per second. The result was: http://tim.rpsys.net/bootchart2.png (warning this is a 2.7MB png) I also added in a little bit of colour coding for the second chart. Interestingly it looks like there is more yellow than green meaning configure is a bigger drain on the build time not that its unexpected :/. I quite enjoyed playing with this and on a serious note, the gradient of the task graph makes me a little suspicious of whether the overhead of launching tasks in bitbake itself is having some effect on build time. Certainly on the first graph there are some interesting latencies showing up. Anyhow, I think this is the first time bitbake's task execution has been visualised and there are some interesting things we can learn from it. I'm hoping this is a start of a much more detailed understanding of the build process with respect to performance. [YOCTO #2403] Signed-off-by: Richard Purdie <richard.purdie@linuxfoundation.org> Signed-off-by: Robert Yang <liezhi.yang@windriver.com> --- scripts/pybootchartgui/pybootchartgui/batch.py | 4 +- scripts/pybootchartgui/pybootchartgui/draw.py | 131 +++++++++++++++------- scripts/pybootchartgui/pybootchartgui/gui.py | 4 +- scripts/pybootchartgui/pybootchartgui/parsing.py | 47 +++++---- 4 files changed, 122 insertions(+), 64 deletions(-) diff --git a/scripts/pybootchartgui/pybootchartgui/batch.py b/scripts/pybootchartgui/pybootchartgui/batch.py index bd67c93..3c1dbf8 100644 --- a/scripts/pybootchartgui/pybootchartgui/batch.py +++ b/scripts/pybootchartgui/pybootchartgui/batch.py @@ -14,10 +14,10 @@ def render(res, format, filename): return 10 make_surface, write_surface = handlers[format] - w,h = draw.extents(*res) + w,h = draw.extents(res) w = max(w, draw.MIN_IMG_W) surface = make_surface(w,h) ctx = cairo.Context(surface) - draw.render(ctx, *res) + draw.render(ctx, res) write_surface(surface) diff --git a/scripts/pybootchartgui/pybootchartgui/draw.py b/scripts/pybootchartgui/pybootchartgui/draw.py index 249cd2e..16830fa 100644 --- a/scripts/pybootchartgui/pybootchartgui/draw.py +++ b/scripts/pybootchartgui/pybootchartgui/draw.py @@ -37,7 +37,7 @@ FILE_OPEN_COLOR = (0.20, 0.71, 0.71, 1.0) # Process border color. PROC_BORDER_COLOR = (0.71, 0.71, 0.71, 1.0) -# Waiting process color. + PROC_COLOR_D = (0.76, 0.48, 0.48, 0.125) # Running process color. PROC_COLOR_R = CPU_COLOR @@ -72,6 +72,17 @@ DEP_STROKE = 1.0 # Process description date format. DESC_TIME_FORMAT = "mm:ss.SSS" +# Configure task color +TASK_COLOR_CONFIGURE = (1.0, 1.0, 0.00, 1.0) +# Compile task color. +TASK_COLOR_COMPILE = (0.0, 1.00, 0.00, 1.0) +# Install task color +TASK_COLOR_INSTALL = (1.0, 0.00, 1.00, 1.0) +# Package task color +TASK_COLOR_PACKAGE = (0.0, 1.00, 1.00, 1.0) +# Sysroot task color +TASK_COLOR_SYSROOT = (0.0, 0.00, 1.00, 1.0) + # Process states STATE_UNDEFINED = 0 STATE_RUNNING = 1 @@ -131,7 +142,7 @@ def draw_label_in_box(ctx, color, label, x, y, w, maxx): def draw_5sec_labels(ctx, rect, sec_w): ctx.set_font_size(AXIS_FONT_SIZE) for i in range(0, rect[2] + 1, sec_w): - if ((i / sec_w) % 5 == 0) : + if ((i / sec_w) % 30 == 0) : label = "%ds" % (i / sec_w) label_w = ctx.text_extents(label)[2] draw_text(ctx, label, TEXT_COLOR, rect[0] + i - label_w/2, rect[1] - 2) @@ -143,7 +154,7 @@ def draw_box_ticks(ctx, rect, sec_w): ctx.set_line_cap(cairo.LINE_CAP_SQUARE) for i in range(sec_w, rect[2] + 1, sec_w): - if ((i / sec_w) % 5 == 0) : + if ((i / sec_w) % 30 == 0) : ctx.set_source_rgba(*TICK_COLOR_BOLD) else : ctx.set_source_rgba(*TICK_COLOR) @@ -188,87 +199,129 @@ header_h = 280 bar_h = 55 # offsets off_x, off_y = 10, 10 -sec_w = 25 # the width of a second +sec_w = 1 # the width of a second proc_h = 16 # the height of a process leg_s = 10 MIN_IMG_W = 800 -def extents(headers, cpu_stats, disk_stats, proc_tree): - w = (proc_tree.duration * sec_w / 100) + 2*off_x - h = proc_h * proc_tree.num_proc + header_h + 2*off_y +def extents(res): + start = min(res.start.keys()) + end = max(res.end.keys()) + + w = ((end - start) * sec_w) + 2*off_x + h = proc_h * len(res.processes) + header_h + 2*off_y + return (w,h) # # Render the chart. # -def render(ctx, headers, cpu_stats, disk_stats, proc_tree): - (w, h) = extents(headers, cpu_stats, disk_stats, proc_tree) +def render(ctx, res): + (w, h) = extents(res) ctx.set_line_width(1.0) ctx.select_font_face(FONT_NAME) draw_fill_rect(ctx, WHITE, (0, 0, max(w, MIN_IMG_W), h)) w -= 2*off_x # draw the title and headers - curr_y = draw_header(ctx, headers, off_x, proc_tree.duration) + #curr_y = draw_header(ctx, headers, off_x, proc_tree.duration) + curr_y = 0 # render bar legend ctx.set_font_size(LEGEND_FONT_SIZE) - draw_legend_box(ctx, "CPU (user+sys)", CPU_COLOR, off_x, curr_y+20, leg_s) - draw_legend_box(ctx, "I/O (wait)", IO_COLOR, off_x + 120, curr_y+20, leg_s) + #print "w, h %s %s" % (w, h) + + #draw_legend_box(ctx, "CPU (user+sys)", CPU_COLOR, off_x, curr_y+20, leg_s) + #draw_legend_box(ctx, "I/O (wait)", IO_COLOR, off_x + 120, curr_y+20, leg_s) # render I/O wait - chart_rect = (off_x, curr_y+30, w, bar_h) - draw_box_ticks(ctx, chart_rect, sec_w) - draw_chart(ctx, IO_COLOR, True, chart_rect, [(sample.time, sample.user + sample.sys + sample.io) for sample in cpu_stats], proc_tree) + #chart_rect = (off_x, curr_y+30, w, bar_h) + #draw_box_ticks(ctx, chart_rect, sec_w) + #draw_chart(ctx, IO_COLOR, True, chart_rect, [(sample.time, sample.user + sample.sys + sample.io) for sample in cpu_stats], proc_tree) # render CPU load - draw_chart(ctx, CPU_COLOR, True, chart_rect, [(sample.time, sample.user + sample.sys) for sample in cpu_stats], proc_tree) + #draw_chart(ctx, CPU_COLOR, True, chart_rect, [(sample.time, sample.user + sample.sys) for sample in cpu_stats], proc_tree) - curr_y = curr_y + 30 + bar_h + #curr_y = curr_y + 30 + bar_h # render second chart - draw_legend_line(ctx, "Disk throughput", DISK_TPUT_COLOR, off_x, curr_y+20, leg_s) - draw_legend_box(ctx, "Disk utilization", IO_COLOR, off_x + 120, curr_y+20, leg_s) + #draw_legend_line(ctx, "Disk throughput", DISK_TPUT_COLOR, off_x, curr_y+20, leg_s) + #draw_legend_box(ctx, "Disk utilization", IO_COLOR, off_x + 120, curr_y+20, leg_s) # render I/O utilization - chart_rect = (off_x, curr_y+30, w, bar_h) - draw_box_ticks(ctx, chart_rect, sec_w) - draw_chart(ctx, IO_COLOR, True, chart_rect, [(sample.time, sample.util) for sample in disk_stats], proc_tree) + #chart_rect = (off_x, curr_y+30, w, bar_h) + #draw_box_ticks(ctx, chart_rect, sec_w) + #draw_chart(ctx, IO_COLOR, True, chart_rect, [(sample.time, sample.util) for sample in disk_stats], proc_tree) # render disk throughput - max_sample = max(disk_stats, key=lambda s: s.tput) - draw_chart(ctx, DISK_TPUT_COLOR, False, chart_rect, [(sample.time, sample.tput) for sample in disk_stats], proc_tree) + #max_sample = max(disk_stats, key=lambda s: s.tput) + #draw_chart(ctx, DISK_TPUT_COLOR, False, chart_rect, [(sample.time, sample.tput) for sample in disk_stats], proc_tree) - pos_x = off_x + ((max_sample.time - proc_tree.start_time) * w / proc_tree.duration) + #pos_x = off_x + ((max_sample.time - proc_tree.start_time) * w / proc_tree.duration) + pos_x = off_x shift_x, shift_y = -20, 20 if (pos_x < off_x + 245): shift_x, shift_y = 5, 40 - label = "%dMB/s" % round((max_sample.tput) / 1024.0) - draw_text(ctx, label, DISK_TPUT_COLOR, pos_x + shift_x, curr_y + shift_y) + #label = "%dMB/s" % round((max_sample.tput) / 1024.0) + #draw_text(ctx, label, DISK_TPUT_COLOR, pos_x + shift_x, curr_y + shift_y) - # draw process boxes - draw_process_bar_chart(ctx, proc_tree, curr_y + bar_h, w, h) - ctx.set_font_size(SIG_FONT_SIZE) - draw_text(ctx, SIGNATURE, SIG_COLOR, off_x + 5, h - off_y - 5) - -def draw_process_bar_chart(ctx, proc_tree, curr_y, w, h): - draw_legend_box(ctx, "Running (%cpu)", PROC_COLOR_R, off_x , curr_y + 45, leg_s) - draw_legend_box(ctx, "Unint.sleep (I/O)", PROC_COLOR_D, off_x+120, curr_y + 45, leg_s) - draw_legend_box(ctx, "Sleeping", PROC_COLOR_S, off_x+240, curr_y + 45, leg_s) - draw_legend_box(ctx, "Zombie", PROC_COLOR_Z, off_x+360, curr_y + 45, leg_s) + chart_rect = [off_x, curr_y+60, w, h - 2 * off_y - (curr_y+60) + proc_h] + + draw_legend_box(ctx, "Configure", TASK_COLOR_CONFIGURE, off_x , curr_y + 45, leg_s) + draw_legend_box(ctx, "Compile", TASK_COLOR_COMPILE, off_x+120, curr_y + 45, leg_s) + draw_legend_box(ctx, "Install", TASK_COLOR_INSTALL, off_x+240, curr_y + 45, leg_s) + draw_legend_box(ctx, "Package", TASK_COLOR_PACKAGE, off_x+360, curr_y + 45, leg_s) + draw_legend_box(ctx, "Populate Sysroot", TASK_COLOR_SYSROOT, off_x+480, curr_y + 45, leg_s) - chart_rect = [off_x, curr_y+60, w, h - 2 * off_y - (curr_y+60) + proc_h] ctx.set_font_size(PROC_TEXT_FONT_SIZE) draw_box_ticks(ctx, chart_rect, sec_w) draw_5sec_labels(ctx, chart_rect, sec_w) - + y = curr_y+60 + + offset = min(res.start.keys()) + for s in sorted(res.start.keys()): + task = res.start[s].split(":")[1] + #print res.start[s] + #print res.processes[res.start[s]][1] + #print s + x = (s - offset) * sec_w + w = ((res.processes[res.start[s]][1] - s) * sec_w) + + #print "proc at %s %s %s %s" % (x, y, w, proc_h) + col = None + if task == "do_compile": + col = TASK_COLOR_COMPILE + elif task == "do_configure": + col = TASK_COLOR_CONFIGURE + elif task == "do_install": + col = TASK_COLOR_INSTALL + elif task == "do_package": + col = TASK_COLOR_PACKAGE + elif task == "do_populate_sysroot": + col = TASK_COLOR_SYSROOT + + draw_rect(ctx, PROC_BORDER_COLOR, (x, y, w, proc_h)) + if col: + draw_fill_rect(ctx, col, (x, y, w, proc_h)) + + draw_label_in_box(ctx, PROC_TEXT_COLOR, res.start[s], x, y + proc_h - 4, w, proc_h) + y = y + proc_h + + # draw process boxes + #draw_process_bar_chart(ctx, proc_tree, curr_y + bar_h, w, h) + + ctx.set_font_size(SIG_FONT_SIZE) + draw_text(ctx, SIGNATURE, SIG_COLOR, off_x + 5, h - off_y - 5) + +def draw_process_bar_chart(ctx, proc_tree, curr_y, w, h): + for root in proc_tree.process_tree: draw_processes_recursively(ctx, root, proc_tree, y, proc_h, chart_rect) y = y + proc_h * proc_tree.num_nodes([root]) diff --git a/scripts/pybootchartgui/pybootchartgui/gui.py b/scripts/pybootchartgui/pybootchartgui/gui.py index 87081e3..310c3d1 100644 --- a/scripts/pybootchartgui/pybootchartgui/gui.py +++ b/scripts/pybootchartgui/pybootchartgui/gui.py @@ -35,7 +35,7 @@ class PyBootchartWidget(gtk.DrawingArea): self.zoom_ratio = 1.0 self.x, self.y = 0.0, 0.0 - self.chart_width, self.chart_height = draw.extents(*res) + self.chart_width, self.chart_height = draw.extents(res) self.hadj = None self.vadj = None @@ -56,7 +56,7 @@ class PyBootchartWidget(gtk.DrawingArea): cr.paint() cr.scale(self.zoom_ratio, self.zoom_ratio) cr.translate(-self.x, -self.y) - draw.render(cr, *self.res) + draw.render(cr, self.res) def position_changed(self): self.emit("position-changed", self.x, self.y) diff --git a/scripts/pybootchartgui/pybootchartgui/parsing.py b/scripts/pybootchartgui/pybootchartgui/parsing.py index a350a3e..11a0829 100644 --- a/scripts/pybootchartgui/pybootchartgui/parsing.py +++ b/scripts/pybootchartgui/pybootchartgui/parsing.py @@ -160,34 +160,39 @@ def get_num_cpus(headers): class ParserState: def __init__(self): - self.headers = None - self.disk_stats = None - self.ps_stats = None - self.cpu_stats = None + self.processes = {} + self.start = {} + self.end = {} def valid(self): - return self.headers != None and self.disk_stats != None and self.ps_stats != None and self.cpu_stats != None + return len(self.processes) != 0 _relevant_files = set(["header", "proc_diskstats.log", "proc_ps.log", "proc_stat.log"]) -def _do_parse(state, name, file): - if name == "header": - state.headers = _parse_headers(file) - elif name == "proc_diskstats.log": - state.disk_stats = _parse_proc_disk_stat_log(file, get_num_cpus(state.headers)) - elif name == "proc_ps.log": - state.ps_stats = _parse_proc_ps_log(file) - elif name == "proc_stat.log": - state.cpu_stats = _parse_proc_stat_log(file) +def _do_parse(state, filename, file): + #print filename + #writer.status("parsing '%s'" % filename) + paths = filename.split("/") + task = paths[-1] + pn = paths[-2] + start = None + end = None + for line in file: + if line.startswith("Started:"): + start = int(float(line.split()[-1])) + elif line.startswith("Ended:"): + end = int(float(line.split()[-1])) + if start and end and (end - start) > 8: + state.processes[pn + ":" + task] = [start, end] + state.start[start] = pn + ":" + task + state.end[end] = pn + ":" + task return state def parse_file(state, filename): basename = os.path.basename(filename) - if not(basename in _relevant_files): - return state with open(filename, "rb") as file: - return _do_parse(state, basename, file) + return _do_parse(state, filename, file) def parse_paths(state, paths): for path in paths: @@ -196,7 +201,7 @@ def parse_paths(state, paths): print "warning: path '%s' does not exist, ignoring." % path continue if os.path.isdir(path): - files = [ f for f in [os.path.join(path, f) for f in os.listdir(path)] if os.path.isfile(f) ] + files = [ f for f in [os.path.join(path, f) for f in os.listdir(path)] ] files.sort() state = parse_paths(state, files) elif extension in [".tar", ".tgz", ".tar.gz"]: @@ -218,6 +223,6 @@ def parse(paths, prune): state = parse_paths(ParserState(), paths) if not state.valid(): raise ParseError("empty state: '%s' does not contain a valid bootchart" % ", ".join(paths)) - monitored_app = state.headers.get("profile.process") - proc_tree = ProcessTree(state.ps_stats, monitored_app, prune) - return (state.headers, state.cpu_stats, state.disk_stats, proc_tree) + #monitored_app = state.headers.get("profile.process") + #proc_tree = ProcessTree(state.ps_stats, monitored_app, prune) + return state -- 1.7.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 4/5] pybootchartgui: split the output chart into multiple ones 2012-06-06 9:34 [PATCH 0/5] pybootchartgui: make the build profiling in pictures Robert Yang ` (2 preceding siblings ...) 2012-06-06 9:34 ` [PATCH 3/5] pybootchartgui: make the build profiling in pictures Robert Yang @ 2012-06-06 9:34 ` Robert Yang 2012-06-06 9:34 ` [PATCH 5/5] pybootchartgui: Fix the filename and add a default format Robert Yang 2012-06-15 18:05 ` [PATCH 0/5] pybootchartgui: make the build profiling in pictures Saul Wold 5 siblings, 0 replies; 7+ messages in thread From: Robert Yang @ 2012-06-06 9:34 UTC (permalink / raw) To: openembedded-core; +Cc: Zhenfeng.Zhao Split the output chart into multiple ones to make it more readable, it only works with "-o path", which means that it doesn't work if the user doesn't want to save the chart to the disk. For example: $ ./pybootchartgui.py /path/to/tmp/buildstats/core-image-sato-qemux86/201205301810/ -f svg -s 5 -o /tmp/ bootchart written to /tmp/bootchart_1.svg bootchart written to /tmp/bootchart_2.svg bootchart written to /tmp/bootchart_3.svg bootchart written to /tmp/bootchart_4.svg bootchart written to /tmp/bootchart_5.svg [YOCTO #2403] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> --- scripts/pybootchartgui/pybootchartgui/main.py | 14 +++++++- scripts/pybootchartgui/pybootchartgui/parsing.py | 36 ++++++++++++++++++++++ 2 files changed, 48 insertions(+), 2 deletions(-) diff --git a/scripts/pybootchartgui/pybootchartgui/main.py b/scripts/pybootchartgui/pybootchartgui/main.py index bf50afb..fce8dd3 100644 --- a/scripts/pybootchartgui/pybootchartgui/main.py +++ b/scripts/pybootchartgui/pybootchartgui/main.py @@ -17,6 +17,8 @@ def _mk_options_parser(): help="image format (...); default format ...") parser.add_option("-o", "--output", dest="output", metavar="PATH", default=None, help="output path (file or directory) where charts are stored") + parser.add_option("-s", "--split", dest="num", type=int, default=1, + help="split the output chart into <NUM> charts, only works with \"-o PATH\"") parser.add_option("-n", "--no-prune", action="store_false", dest="prune", default=True, help="do not prune the process tree") parser.add_option("-q", "--quiet", action="store_true", dest="quiet", default=False, @@ -59,8 +61,16 @@ def main(argv=None): gui.show(res) else: filename = _get_filename(args, options) - batch.render(res, options.format, filename) - print "bootchart written to", filename + res_list = parsing.split_res(res, options.num) + n = 1 + for r in res_list: + if len(res_list) == 1: + f = filename + "." + options.format + else: + f = filename + "_" + str(n) + "." + options.format + n = n + 1 + batch.render(r, options.format, f) + print "bootchart written to", f return 0 except parsing.ParseError, ex: print("Parse error: %s" % ex) diff --git a/scripts/pybootchartgui/pybootchartgui/parsing.py b/scripts/pybootchartgui/pybootchartgui/parsing.py index 11a0829..c64eba0 100644 --- a/scripts/pybootchartgui/pybootchartgui/parsing.py +++ b/scripts/pybootchartgui/pybootchartgui/parsing.py @@ -226,3 +226,39 @@ def parse(paths, prune): #monitored_app = state.headers.get("profile.process") #proc_tree = ProcessTree(state.ps_stats, monitored_app, prune) return state + +def split_res(res, n): + """ Split the res into n pieces """ + res_list = [] + if n > 1: + s_list = sorted(res.start.keys()) + frag_size = len(s_list) / float(n) + # Need the top value + if frag_size > int(frag_size): + frag_size = int(frag_size + 1) + else: + frag_size = int(frag_size) + + start = 0 + end = frag_size + while start < end: + state = ParserState() + for i in range(start, end): + # Add these lines for reference + #state.processes[pn + ":" + task] = [start, end] + #state.start[start] = pn + ":" + task + #state.end[end] = pn + ":" + task + p = res.start[s_list[i]] + s = s_list[i] + e = res.processes[p][1] + state.processes[p] = [s, e] + state.start[s] = p + state.end[e] = p + start = end + end = end + frag_size + if end > len(s_list): + end = len(s_list) + res_list.append(state) + else: + res_list.append(res) + return res_list -- 1.7.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* [PATCH 5/5] pybootchartgui: Fix the filename and add a default format 2012-06-06 9:34 [PATCH 0/5] pybootchartgui: make the build profiling in pictures Robert Yang ` (3 preceding siblings ...) 2012-06-06 9:34 ` [PATCH 4/5] pybootchartgui: split the output chart into multiple ones Robert Yang @ 2012-06-06 9:34 ` Robert Yang 2012-06-15 18:05 ` [PATCH 0/5] pybootchartgui: make the build profiling in pictures Saul Wold 5 siblings, 0 replies; 7+ messages in thread From: Robert Yang @ 2012-06-06 9:34 UTC (permalink / raw) To: openembedded-core; +Cc: Zhenfeng.Zhao * Fix teh output filename to make it easy to use * Add a default output format (svg) * Fix the usage message * Fix the version to v1.0.0 Currently, the help messages are: $ ./pybootchartgui.py --help Usage: pybootchartgui.py [options] /path/to/tmp/buildstats/<recipe-machine>/<BUILDNAME>/ Options: --version show program's version number and exit -h, --help show this help message and exit -i, --interactive start in active mode -f FORMAT, --format=FORMAT image format: svg, pdf, png, [default: svg] -o PATH, --output=PATH output path (file or directory) where charts are stored -s NUM, --split=NUM split the output chart into <NUM> charts, only works with "-o PATH" -n, --no-prune do not prune the process tree -q, --quiet suppress informational messages --very-quiet suppress all messages except errors --verbose print all messages [YOCTO #2403] Signed-off-by: Robert Yang <liezhi.yang@windriver.com> --- scripts/pybootchartgui/pybootchartgui/main.py | 33 ++++++++++-------------- 1 files changed, 14 insertions(+), 19 deletions(-) diff --git a/scripts/pybootchartgui/pybootchartgui/main.py b/scripts/pybootchartgui/pybootchartgui/main.py index fce8dd3..e70ab13 100644 --- a/scripts/pybootchartgui/pybootchartgui/main.py +++ b/scripts/pybootchartgui/pybootchartgui/main.py @@ -8,13 +8,13 @@ import batch def _mk_options_parser(): """Make an options parser.""" - usage = "%prog [options] PATH, ..., PATH" - version = "%prog v0.0.0" + usage = "%prog [options] /path/to/tmp/buildstats/<recipe-machine>/<BUILDNAME>/" + version = "%prog v1.0.0" parser = optparse.OptionParser(usage, version=version) parser.add_option("-i", "--interactive", action="store_true", dest="interactive", default=False, help="start in active mode") - parser.add_option("-f", "--format", dest="format", default = None, - help="image format (...); default format ...") + parser.add_option("-f", "--format", dest="format", default="svg", choices=["svg", "pdf", "png"], + help="image format: svg, pdf, png, [default: %default]") parser.add_option("-o", "--output", dest="output", metavar="PATH", default=None, help="output path (file or directory) where charts are stored") parser.add_option("-s", "--split", dest="num", type=int, default=1, @@ -29,20 +29,15 @@ def _mk_options_parser(): help="print all messages") return parser -def _get_filename(paths, options): - """Construct a usable filename for outputs based on the paths and options given on the commandline.""" - dir = "" +def _get_filename(path): + """Construct a usable filename for outputs""" + dir = "." file = "bootchart" - if options.output != None and not(os.path.isdir(options.output)): - return options.output - if options.output != None: - dir = options.output - if len(paths) == 1: - if os.path.isdir(paths[0]): - file = os.path.split(paths[0])[-1] - elif os.path.splitext(paths[0])[1] in [".tar", ".tgz", ".tar.gz"]: - file = os.path.splitext(paths[0])[0] - return os.path.join(dir, file + "." + options.format) + if os.path.isdir(path): + dir = path + elif path != None: + file = path + return os.path.join(dir, file) def main(argv=None): try: @@ -57,10 +52,10 @@ def main(argv=None): return 2 res = parsing.parse(args, options.prune) - if options.interactive or options.format == None: + if options.interactive or options.output == None: gui.show(res) else: - filename = _get_filename(args, options) + filename = _get_filename(options.output) res_list = parsing.split_res(res, options.num) n = 1 for r in res_list: -- 1.7.1 ^ permalink raw reply related [flat|nested] 7+ messages in thread
* Re: [PATCH 0/5] pybootchartgui: make the build profiling in pictures 2012-06-06 9:34 [PATCH 0/5] pybootchartgui: make the build profiling in pictures Robert Yang ` (4 preceding siblings ...) 2012-06-06 9:34 ` [PATCH 5/5] pybootchartgui: Fix the filename and add a default format Robert Yang @ 2012-06-15 18:05 ` Saul Wold 5 siblings, 0 replies; 7+ messages in thread From: Saul Wold @ 2012-06-15 18:05 UTC (permalink / raw) To: Patches and discussions about the oe-core layer; +Cc: Zhenfeng.Zhao On 06/06/2012 02:34 AM, Robert Yang wrote: > Note: Removed the patch details for PATCH 1/5 and 2/5 since they are too > big(bigger than 2M) to send via email, please fetch the patch via > the git repo > > // Robert > > The following changes since commit df8f55a919b3cc602ce1f1c51630c7edf6e36b55: > > ltp: Add patch to correct failing build (2012-06-05 23:05:00 +0100) > > are available in the git repository at: > git://git.pokylinux.org/poky-contrib robert/chart > http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/chart > > Robert Yang (5): > pybootchartgui: add the original code > pybootchartgui: remove the original examples > pybootchartgui: make the build profiling in pictures > pybootchartgui: split the output chart into multiple ones > pybootchartgui: Fix the filename and add a default format > > scripts/pybootchartgui/AUTHORS | 11 + > scripts/pybootchartgui/COPYING | 674 ++++++++++++++++++++ > scripts/pybootchartgui/README | 36 + > scripts/pybootchartgui/pybootchartgui.py | 7 + > scripts/pybootchartgui/pybootchartgui/batch.py | 23 + > scripts/pybootchartgui/pybootchartgui/draw.py | 408 ++++++++++++ > scripts/pybootchartgui/pybootchartgui/gui.py | 273 ++++++++ > scripts/pybootchartgui/pybootchartgui/main.py | 76 +++ > scripts/pybootchartgui/pybootchartgui/parsing.py | 264 ++++++++ > .../pybootchartgui/pybootchartgui/process_tree.py | 270 ++++++++ > scripts/pybootchartgui/pybootchartgui/samples.py | 93 +++ > .../pybootchartgui/tests/parser_test.py | 93 +++ > .../pybootchartgui/tests/process_tree_test.py | 78 +++ > 13 files changed, 2306 insertions(+), 0 deletions(-) > create mode 100644 scripts/pybootchartgui/AUTHORS > create mode 100644 scripts/pybootchartgui/COPYING > create mode 100644 scripts/pybootchartgui/README > create mode 100755 scripts/pybootchartgui/pybootchartgui.py > create mode 100644 scripts/pybootchartgui/pybootchartgui/__init__.py > create mode 100644 scripts/pybootchartgui/pybootchartgui/batch.py > create mode 100644 scripts/pybootchartgui/pybootchartgui/draw.py > create mode 100644 scripts/pybootchartgui/pybootchartgui/gui.py > create mode 100644 scripts/pybootchartgui/pybootchartgui/main.py > create mode 100644 scripts/pybootchartgui/pybootchartgui/parsing.py > create mode 100644 scripts/pybootchartgui/pybootchartgui/process_tree.py > create mode 100644 scripts/pybootchartgui/pybootchartgui/samples.py > create mode 100644 scripts/pybootchartgui/pybootchartgui/tests/parser_test.py > create mode 100644 scripts/pybootchartgui/pybootchartgui/tests/process_tree_test.py > > > _______________________________________________ > Openembedded-core mailing list > Openembedded-core@lists.openembedded.org > http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core > Merged into OE-Core Thanks Sau! ^ permalink raw reply [flat|nested] 7+ messages in thread
end of thread, other threads:[~2012-06-15 18:16 UTC | newest] Thread overview: 7+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2012-06-06 9:34 [PATCH 0/5] pybootchartgui: make the build profiling in pictures Robert Yang 2012-06-06 9:34 ` [PATCH 1/5] pybootchartgui: add the original code Robert Yang 2012-06-06 9:34 ` [PATCH 2/5] pybootchartgui: remove the original examples Robert Yang 2012-06-06 9:34 ` [PATCH 3/5] pybootchartgui: make the build profiling in pictures Robert Yang 2012-06-06 9:34 ` [PATCH 4/5] pybootchartgui: split the output chart into multiple ones Robert Yang 2012-06-06 9:34 ` [PATCH 5/5] pybootchartgui: Fix the filename and add a default format Robert Yang 2012-06-15 18:05 ` [PATCH 0/5] pybootchartgui: make the build profiling in pictures Saul Wold
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox
