From mboxrd@z Thu Jan 1 00:00:00 1970 From: Stephen Warren Date: Tue, 24 Nov 2015 14:28:27 -0700 Subject: [U-Boot] [PATCH] Implement pytest-based test infrastructure In-Reply-To: References: <1447570381-1361-1-git-send-email-swarren@wwwdotorg.org> <564E0030.9090808@wwwdotorg.org> <564E1E6B.7010708@wwwdotorg.org> <5651FBB0.9010202@wwwdotorg.org> <5653EB13.60506@wwwdotorg.org> Message-ID: <5654D67B.2090307@wwwdotorg.org> List-Id: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: u-boot@lists.denx.de On 11/24/2015 12:04 PM, Simon Glass wrote: > Hi Stephen, > > On 23 November 2015 at 21:44, Stephen Warren wrote: >> On 11/23/2015 06:45 PM, Simon Glass wrote: >>> On 22 November 2015 at 10:30, Stephen Warren wrote: >>>> On 11/21/2015 09:49 AM, Simon Glass wrote: >>>>> OK I got it working thank you. It is horribly slow though - do you >>>>> know what is holding it up? For me to takes 12 seconds to run the >>>>> (very basic) tests. .. >> I put a bit of time measurement into run_command() and found that on my >> system at work, for p.send("the shell command to execute") was actually >> (marginally) slower on sandbox than on real HW, despite real HW being a >> 115200 baud serial port, and the code splitting the shell commands into >> chunks that are sent and waited for synchronously to avoid overflowing >> UART FIFOs. I'm not sure why this is. Looking at U-Boot's console, it >> seems to be non-blocking, so I don't think termios VMIN/VTIME come into >> play (setting them to 0 made no difference), and the two raw modes took >> the same time. I meant to look into pexpect's termios settings to see if >> there was anything to tweak there, but forgot today. >> >> I did do one experiment to compare expect (the Tcl version) and pexpect. >> If I do roughly the following in both: >> >> spawn u-boot (sandbox) >> wait for prompt >> 100 times: >> send "echo $foo\n" >> wait for "echo $foo" >> wait for shell prompt >> send "reset" >> wait for "reset" >> send "\n" >> >> ... then Tcl is about 3x faster on my system (IIRC 0.5 vs. 1.5s). If I >> remove all the "wait"s, then IIRC Tcl was about 15x faster or more. >> That's a pity. Still, I'm sure as heck not going to rewrite all this in >> Tcl:-( I wonder if something similar to pexpect but more targetted at >> simple "interactive shell" cases would remove any of that overhead. > > It is possible that we should use sandbox in 'cooked' mode so that > lines an entered synchronously. The -t option might help here, or we > may need something else. I don't think cooked mode will work, since I believe cooked is line-buffered, yet when U-Boot emits the shell prompt there's no \n printed afterwards. FWIW, I hacked out pexpect and replaced it with some custom code. That reduced by sandbox execution time from ~5.1s to ~2.3s. Execution time against real HW didn't seem to be affected at all. Some features like timeouts and complete error handling are still missing, but I don't think that would affect the execution time. See my github tree for the WIP patch.