* [U-Boot] [PATCH 2/8] test/py: move U-Boot respawn trigger to the test core
2016-01-20 22:15 [U-Boot] [PATCH 1/8] test/py: fix timeout to be absolute Stephen Warren
@ 2016-01-20 22:15 ` Stephen Warren
2016-01-22 3:35 ` Simon Glass
2016-01-20 22:15 ` [U-Boot] [PATCH 3/8] test/py: drain console log at the end of any failed test Stephen Warren
` (7 subsequent siblings)
8 siblings, 1 reply; 22+ messages in thread
From: Stephen Warren @ 2016-01-20 22:15 UTC (permalink / raw)
To: u-boot
From: Stephen Warren <swarren@nvidia.com>
Prior to this change, U-Boot was lazilly (re-)spawned if/when a test
attempted to interact with it, and no active connection existed. This
approach was simple, yet had the disadvantage that U-Boot might be
spawned in the middle of a test function, e.g. after the test had already
performed actions such as creating data files, etc. In that case, this
could cause the log to contain the sequence (1) some test logs, (2)
U-Boot's boot process, (3) the rest of that test's logs. This isn't
optimally readable. This issue will affect the upcoming DFU and enhanced
UMS tests.
This change converts u_boot_console to be a function-scoped fixture, so
that pytest attempts to re-create the object for each test invocation.
This allows the fixture factory function to ensure that U-Boot is spawned
prior to every test. In practice, the same object is returned each time
so there is essentially no additional overhead due to this change.
This allows us to remove:
- The explicit ensure_spawned() call from test_sleep, since the core now
ensures that the spawn happens before the test code is executed.
- The laxy calls to ensure_spawned() in the u_boot_console_*
implementations.
The one downside is that test_env's "state_ttest_env" fixture must be
converted to a function-scoped fixture too, since a module-scoped fixture
cannot use a function-scoped fixture. To avoid overhead, we use the same
trick of returning the same object each time.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
---
test/py/conftest.py | 3 ++-
test/py/tests/test_env.py | 8 ++++++--
test/py/tests/test_sandbox_exit.py | 2 --
test/py/tests/test_sleep.py | 4 ----
test/py/u_boot_console_base.py | 2 --
test/py/u_boot_console_sandbox.py | 1 -
6 files changed, 8 insertions(+), 12 deletions(-)
diff --git a/test/py/conftest.py b/test/py/conftest.py
index e1674dfce053..38aa3f922a86 100644
--- a/test/py/conftest.py
+++ b/test/py/conftest.py
@@ -227,7 +227,7 @@ def pytest_generate_tests(metafunc):
vals = subconfig.get(fn + 's', [])
metafunc.parametrize(fn, vals)
- at pytest.fixture(scope='session')
+ at pytest.fixture(scope='function')
def u_boot_console(request):
'''Generate the value of a test's u_boot_console fixture.
@@ -238,6 +238,7 @@ def u_boot_console(request):
The fixture value.
'''
+ console.ensure_spawned()
return console
tests_not_run = set()
diff --git a/test/py/tests/test_env.py b/test/py/tests/test_env.py
index a3e8dd30330b..557c3afe5c0a 100644
--- a/test/py/tests/test_env.py
+++ b/test/py/tests/test_env.py
@@ -77,11 +77,15 @@ class StateTestEnv(object):
return var
n += 1
- at pytest.fixture(scope='module')
+ste = None
+ at pytest.fixture(scope='function')
def state_test_env(u_boot_console):
'''pytest fixture to provide a StateTestEnv object to tests.'''
- return StateTestEnv(u_boot_console)
+ global ste
+ if not ste:
+ ste = StateTestEnv(u_boot_console)
+ return ste
def unset_var(state_test_env, var):
'''Unset an environment variable.
diff --git a/test/py/tests/test_sandbox_exit.py b/test/py/tests/test_sandbox_exit.py
index 2aa8eb4abc68..1ec3607eb28a 100644
--- a/test/py/tests/test_sandbox_exit.py
+++ b/test/py/tests/test_sandbox_exit.py
@@ -13,7 +13,6 @@ def test_reset(u_boot_console):
u_boot_console.run_command('reset', wait_for_prompt=False)
assert(u_boot_console.validate_exited())
- u_boot_console.ensure_spawned()
@pytest.mark.boardspec('sandbox')
def test_ctrl_c(u_boot_console):
@@ -21,4 +20,3 @@ def test_ctrl_c(u_boot_console):
u_boot_console.kill(signal.SIGINT)
assert(u_boot_console.validate_exited())
- u_boot_console.ensure_spawned()
diff --git a/test/py/tests/test_sleep.py b/test/py/tests/test_sleep.py
index 64f1ddf9a09f..437b6bb9fea9 100644
--- a/test/py/tests/test_sleep.py
+++ b/test/py/tests/test_sleep.py
@@ -9,10 +9,6 @@ def test_sleep(u_boot_console):
'''Test the sleep command, and validate that it sleeps for approximately
the correct amount of time.'''
- # Do this before we time anything, to make sure U-Boot is already running.
- # Otherwise, the system boot time is included in the time measurement.
- u_boot_console.ensure_spawned()
-
# 3s isn't too long, but is enough to cross a few second boundaries.
sleep_time = 3
tstart = time.time()
diff --git a/test/py/u_boot_console_base.py b/test/py/u_boot_console_base.py
index 520f9a9e9f31..10fe3dbdd372 100644
--- a/test/py/u_boot_console_base.py
+++ b/test/py/u_boot_console_base.py
@@ -144,8 +144,6 @@ class ConsoleBase(object):
command string and emitted the subsequent command prompts.
'''
- self.ensure_spawned()
-
if self.at_prompt and \
self.at_prompt_logevt != self.logstream.logfile.cur_evt:
self.logstream.write(self.prompt, implicit=True)
diff --git a/test/py/u_boot_console_sandbox.py b/test/py/u_boot_console_sandbox.py
index 88b137e8c3a1..eb84150a1e44 100644
--- a/test/py/u_boot_console_sandbox.py
+++ b/test/py/u_boot_console_sandbox.py
@@ -51,7 +51,6 @@ class ConsoleSandbox(ConsoleBase):
Nothing.
'''
- self.ensure_spawned()
self.log.action('kill %d' % sig)
self.p.kill(sig)
--
2.7.0
^ permalink raw reply related [flat|nested] 22+ messages in thread* [U-Boot] [PATCH 2/8] test/py: move U-Boot respawn trigger to the test core
2016-01-20 22:15 ` [U-Boot] [PATCH 2/8] test/py: move U-Boot respawn trigger to the test core Stephen Warren
@ 2016-01-22 3:35 ` Simon Glass
0 siblings, 0 replies; 22+ messages in thread
From: Simon Glass @ 2016-01-22 3:35 UTC (permalink / raw)
To: u-boot
On 20 January 2016 at 15:15, Stephen Warren <swarren@wwwdotorg.org> wrote:
> From: Stephen Warren <swarren@nvidia.com>
>
> Prior to this change, U-Boot was lazilly (re-)spawned if/when a test
> attempted to interact with it, and no active connection existed. This
> approach was simple, yet had the disadvantage that U-Boot might be
> spawned in the middle of a test function, e.g. after the test had already
> performed actions such as creating data files, etc. In that case, this
> could cause the log to contain the sequence (1) some test logs, (2)
> U-Boot's boot process, (3) the rest of that test's logs. This isn't
> optimally readable. This issue will affect the upcoming DFU and enhanced
> UMS tests.
>
> This change converts u_boot_console to be a function-scoped fixture, so
> that pytest attempts to re-create the object for each test invocation.
> This allows the fixture factory function to ensure that U-Boot is spawned
> prior to every test. In practice, the same object is returned each time
> so there is essentially no additional overhead due to this change.
>
> This allows us to remove:
>
> - The explicit ensure_spawned() call from test_sleep, since the core now
> ensures that the spawn happens before the test code is executed.
>
> - The laxy calls to ensure_spawned() in the u_boot_console_*
> implementations.
>
> The one downside is that test_env's "state_ttest_env" fixture must be
> converted to a function-scoped fixture too, since a module-scoped fixture
> cannot use a function-scoped fixture. To avoid overhead, we use the same
> trick of returning the same object each time.
>
> Signed-off-by: Stephen Warren <swarren@nvidia.com>
> ---
> test/py/conftest.py | 3 ++-
> test/py/tests/test_env.py | 8 ++++++--
> test/py/tests/test_sandbox_exit.py | 2 --
> test/py/tests/test_sleep.py | 4 ----
> test/py/u_boot_console_base.py | 2 --
> test/py/u_boot_console_sandbox.py | 1 -
> 6 files changed, 8 insertions(+), 12 deletions(-)
Acked-by: Simon Glass <sjg@chromium.org>
^ permalink raw reply [flat|nested] 22+ messages in thread
* [U-Boot] [PATCH 3/8] test/py: drain console log at the end of any failed test
2016-01-20 22:15 [U-Boot] [PATCH 1/8] test/py: fix timeout to be absolute Stephen Warren
2016-01-20 22:15 ` [U-Boot] [PATCH 2/8] test/py: move U-Boot respawn trigger to the test core Stephen Warren
@ 2016-01-20 22:15 ` Stephen Warren
2016-01-22 3:36 ` Simon Glass
2016-01-20 22:15 ` [U-Boot] [PATCH 4/8] test/py: log when tests send CTRL-C Stephen Warren
` (6 subsequent siblings)
8 siblings, 1 reply; 22+ messages in thread
From: Stephen Warren @ 2016-01-20 22:15 UTC (permalink / raw)
To: u-boot
From: Stephen Warren <swarren@nvidia.com>
Tests may fail for a number of reasons, and in particular for reasons
other than a timeout waiting for U-Boot to print expected data. If the
last operation that a failed test performs is not waiting for U-Boot to
print something, then any trailing output from U-Boot during that test's
operation will not be logged as part of that test, but rather either
along with the next test, or even thrown away, potentiall hiding clues
re: the test failure reason.
Solve this by explicitly draining (and hence logging) the U-Boot output
in the case of failed tests.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
---
test/py/conftest.py | 1 +
test/py/u_boot_console_base.py | 38 ++++++++++++++++++++++++++++++++++++++
2 files changed, 39 insertions(+)
diff --git a/test/py/conftest.py b/test/py/conftest.py
index 38aa3f922a86..c1f19cee65a3 100644
--- a/test/py/conftest.py
+++ b/test/py/conftest.py
@@ -386,6 +386,7 @@ def pytest_runtest_protocol(item, nextitem):
skipped = report
if failed:
+ console.drain_console()
tests_failed.add(item.name)
elif skipped:
tests_skipped.add(item.name)
diff --git a/test/py/u_boot_console_base.py b/test/py/u_boot_console_base.py
index 10fe3dbdd372..418a26bb8e40 100644
--- a/test/py/u_boot_console_base.py
+++ b/test/py/u_boot_console_base.py
@@ -14,6 +14,7 @@ import os
import pytest
import re
import sys
+import u_boot_spawn
# Regexes for text we expect U-Boot to send to the console.
pattern_u_boot_spl_signon = re.compile('(U-Boot SPL \\d{4}\\.\\d{2}-[^\r\n]*)')
@@ -213,6 +214,43 @@ class ConsoleBase(object):
self.run_command(chr(3), wait_for_echo=False, send_nl=False)
+ def drain_console(self):
+ '''Read from and log the U-Boot console for a short time.
+
+ U-Boot's console output is only logged when the test code actively
+ waits for U-Boot to emit specific data. There are cases where tests
+ can fail without doing this. For example, if a test asks U-Boot to
+ enable USB device mode, then polls until a host-side device node
+ exists. In such a case, it is useful to log U-Boot's console output
+ in case U-Boot printed clues as to why the host-side even did not
+ occur. This function will do that.
+
+ Args:
+ None.
+
+ Returns:
+ Nothing.
+ '''
+
+ # If we are already not connected to U-Boot, there's nothing to drain.
+ # This should only happen when a previous call to run_command() or
+ # wait_for() failed (and hence the output has already been logged), or
+ # the system is shutting down.
+ if not self.p:
+ return
+
+ orig_timeout = self.p.timeout
+ try:
+ # Drain the log for a relatively short time.
+ self.p.timeout = 1000
+ # Wait for something U-Boot will likely never send. This will
+ # cause the console output to be read and logged.
+ self.p.expect(['This should never match U-Boot output'])
+ except u_boot_spawn.Timeout:
+ pass
+ finally:
+ self.p.timeout = orig_timeout
+
def ensure_spawned(self):
'''Ensure a connection to a correctly running U-Boot instance.
--
2.7.0
^ permalink raw reply related [flat|nested] 22+ messages in thread* [U-Boot] [PATCH 3/8] test/py: drain console log at the end of any failed test
2016-01-20 22:15 ` [U-Boot] [PATCH 3/8] test/py: drain console log at the end of any failed test Stephen Warren
@ 2016-01-22 3:36 ` Simon Glass
0 siblings, 0 replies; 22+ messages in thread
From: Simon Glass @ 2016-01-22 3:36 UTC (permalink / raw)
To: u-boot
On 20 January 2016 at 15:15, Stephen Warren <swarren@wwwdotorg.org> wrote:
> From: Stephen Warren <swarren@nvidia.com>
>
> Tests may fail for a number of reasons, and in particular for reasons
> other than a timeout waiting for U-Boot to print expected data. If the
> last operation that a failed test performs is not waiting for U-Boot to
> print something, then any trailing output from U-Boot during that test's
> operation will not be logged as part of that test, but rather either
> along with the next test, or even thrown away, potentiall hiding clues
> re: the test failure reason.
>
> Solve this by explicitly draining (and hence logging) the U-Boot output
> in the case of failed tests.
>
> Signed-off-by: Stephen Warren <swarren@nvidia.com>
> ---
> test/py/conftest.py | 1 +
> test/py/u_boot_console_base.py | 38 ++++++++++++++++++++++++++++++++++++++
> 2 files changed, 39 insertions(+)
Acked-by: Simon Glass <sjg@chromium.org>
^ permalink raw reply [flat|nested] 22+ messages in thread
* [U-Boot] [PATCH 4/8] test/py: log when tests send CTRL-C
2016-01-20 22:15 [U-Boot] [PATCH 1/8] test/py: fix timeout to be absolute Stephen Warren
2016-01-20 22:15 ` [U-Boot] [PATCH 2/8] test/py: move U-Boot respawn trigger to the test core Stephen Warren
2016-01-20 22:15 ` [U-Boot] [PATCH 3/8] test/py: drain console log at the end of any failed test Stephen Warren
@ 2016-01-20 22:15 ` Stephen Warren
2016-01-22 3:36 ` Simon Glass
2016-01-20 22:15 ` [U-Boot] [PATCH 5/8] test/py: optionally ignore errors from shell commands Stephen Warren
` (5 subsequent siblings)
8 siblings, 1 reply; 22+ messages in thread
From: Stephen Warren @ 2016-01-20 22:15 UTC (permalink / raw)
To: u-boot
From: Stephen Warren <swarren@nvidia.com>
Write a note to the log file when a test sends CTRL-C to U-Boot. This
makes it easier to follow what's happening in the logs, especially since
U-Boot doesn't echo the character back to its output, so there's no other
signal of what's going on.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
---
test/py/u_boot_console_base.py | 1 +
1 file changed, 1 insertion(+)
diff --git a/test/py/u_boot_console_base.py b/test/py/u_boot_console_base.py
index 418a26bb8e40..433bec6e9fdd 100644
--- a/test/py/u_boot_console_base.py
+++ b/test/py/u_boot_console_base.py
@@ -212,6 +212,7 @@ class ConsoleBase(object):
Nothing.
'''
+ self.log.action('Sending Ctrl-C')
self.run_command(chr(3), wait_for_echo=False, send_nl=False)
def drain_console(self):
--
2.7.0
^ permalink raw reply related [flat|nested] 22+ messages in thread* [U-Boot] [PATCH 4/8] test/py: log when tests send CTRL-C
2016-01-20 22:15 ` [U-Boot] [PATCH 4/8] test/py: log when tests send CTRL-C Stephen Warren
@ 2016-01-22 3:36 ` Simon Glass
0 siblings, 0 replies; 22+ messages in thread
From: Simon Glass @ 2016-01-22 3:36 UTC (permalink / raw)
To: u-boot
On 20 January 2016 at 15:15, Stephen Warren <swarren@wwwdotorg.org> wrote:
> From: Stephen Warren <swarren@nvidia.com>
>
> Write a note to the log file when a test sends CTRL-C to U-Boot. This
> makes it easier to follow what's happening in the logs, especially since
> U-Boot doesn't echo the character back to its output, so there's no other
> signal of what's going on.
>
> Signed-off-by: Stephen Warren <swarren@nvidia.com>
> ---
> test/py/u_boot_console_base.py | 1 +
> 1 file changed, 1 insertion(+)
Acked-by: Simon Glass <sjg@chromium.org>
^ permalink raw reply [flat|nested] 22+ messages in thread
* [U-Boot] [PATCH 5/8] test/py: optionally ignore errors from shell commands
2016-01-20 22:15 [U-Boot] [PATCH 1/8] test/py: fix timeout to be absolute Stephen Warren
` (2 preceding siblings ...)
2016-01-20 22:15 ` [U-Boot] [PATCH 4/8] test/py: log when tests send CTRL-C Stephen Warren
@ 2016-01-20 22:15 ` Stephen Warren
2016-01-22 3:36 ` Simon Glass
2016-01-20 22:15 ` [U-Boot] [PATCH 6/8] test/py: add various utility code Stephen Warren
` (4 subsequent siblings)
8 siblings, 1 reply; 22+ messages in thread
From: Stephen Warren @ 2016-01-20 22:15 UTC (permalink / raw)
To: u-boot
From: Stephen Warren <swarren@nvidia.com>
Sometimes it's useful to run shell commands and ignore any errors. One
example might be cleanup logic; if a test-case experiences an error, the
cleanup logic might experience an error too, and we don't want that error
to mask the original error, so we want to ignore the subsequent error.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
---
test/py/multiplexed_log.py | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/test/py/multiplexed_log.py b/test/py/multiplexed_log.py
index 48f2b51de15f..5059bbfb99c3 100644
--- a/test/py/multiplexed_log.py
+++ b/test/py/multiplexed_log.py
@@ -106,13 +106,17 @@ class RunAndLog(object):
'''Clean up any resources managed by this object.'''
pass
- def run(self, cmd, cwd=None):
+ def run(self, cmd, cwd=None, ignore_errors=False):
'''Run a command as a sub-process, and log the results.
Args:
cmd: The command to execute.
cwd: The directory to run the command in. Can be None to use the
current directory.
+ ignore_errors: Indicate whether to ignore errors. If True, the
+ function will simply return if the command cannot be executed
+ or exits with an error code, otherwise an exception will be
+ raised if such problems occur.
Returns:
Nothing.
@@ -148,7 +152,7 @@ class RunAndLog(object):
exception = e
if output and not output.endswith('\n'):
output += '\n'
- if exit_status and not exception:
+ if exit_status and not exception and not ignore_errors:
exception = Exception('Exit code: ' + str(exit_status))
if exception:
output += str(exception) + '\n'
--
2.7.0
^ permalink raw reply related [flat|nested] 22+ messages in thread* [U-Boot] [PATCH 5/8] test/py: optionally ignore errors from shell commands
2016-01-20 22:15 ` [U-Boot] [PATCH 5/8] test/py: optionally ignore errors from shell commands Stephen Warren
@ 2016-01-22 3:36 ` Simon Glass
0 siblings, 0 replies; 22+ messages in thread
From: Simon Glass @ 2016-01-22 3:36 UTC (permalink / raw)
To: u-boot
On 20 January 2016 at 15:15, Stephen Warren <swarren@wwwdotorg.org> wrote:
> From: Stephen Warren <swarren@nvidia.com>
>
> Sometimes it's useful to run shell commands and ignore any errors. One
> example might be cleanup logic; if a test-case experiences an error, the
> cleanup logic might experience an error too, and we don't want that error
> to mask the original error, so we want to ignore the subsequent error.
>
> Signed-off-by: Stephen Warren <swarren@nvidia.com>
> ---
> test/py/multiplexed_log.py | 8 ++++++--
> 1 file changed, 6 insertions(+), 2 deletions(-)
Acked-by: Simon Glass <sjg@chromium.org>
^ permalink raw reply [flat|nested] 22+ messages in thread
* [U-Boot] [PATCH 6/8] test/py: add various utility code
2016-01-20 22:15 [U-Boot] [PATCH 1/8] test/py: fix timeout to be absolute Stephen Warren
` (3 preceding siblings ...)
2016-01-20 22:15 ` [U-Boot] [PATCH 5/8] test/py: optionally ignore errors from shell commands Stephen Warren
@ 2016-01-20 22:15 ` Stephen Warren
2016-01-22 3:36 ` Simon Glass
2016-01-20 22:15 ` [U-Boot] [PATCH 7/8] test/py: ums: add filesystem-based testing Stephen Warren
` (3 subsequent siblings)
8 siblings, 1 reply; 22+ messages in thread
From: Stephen Warren @ 2016-01-20 22:15 UTC (permalink / raw)
To: u-boot
From: Stephen Warren <swarren@nvidia.com>
Add various common utility functions. These will be used by a forthcoming
re-written UMS test, and a brand-new DFU test.
Signed-off-by: Stephen Warren <swarren@nvidia.com>
---
test/py/u_boot_console_base.py | 19 +++++
test/py/u_boot_utils.py | 171 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 190 insertions(+)
create mode 100644 test/py/u_boot_utils.py
diff --git a/test/py/u_boot_console_base.py b/test/py/u_boot_console_base.py
index 433bec6e9fdd..06f61f987180 100644
--- a/test/py/u_boot_console_base.py
+++ b/test/py/u_boot_console_base.py
@@ -215,6 +215,25 @@ class ConsoleBase(object):
self.log.action('Sending Ctrl-C')
self.run_command(chr(3), wait_for_echo=False, send_nl=False)
+ def wait_for(self, text):
+ '''Wait for a pattern to be emitted by U-Boot.
+
+ This is useful when a long-running command such as "dfu" is executing,
+ and it periodically emits some text that should show up at a specific
+ location in the log file.
+
+ Args:
+ text: The text to wait for; either a string (containing raw text,
+ not a regular expression) or an re object.
+
+ Returns:
+ Nothing.
+ '''
+
+ if type(text) == type(''):
+ text = re.escape(text)
+ self.p.expect([text])
+
def drain_console(self):
'''Read from and log the U-Boot console for a short time.
diff --git a/test/py/u_boot_utils.py b/test/py/u_boot_utils.py
new file mode 100644
index 000000000000..539af618dbf2
--- /dev/null
+++ b/test/py/u_boot_utils.py
@@ -0,0 +1,171 @@
+# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+#
+# SPDX-License-Identifier: GPL-2.0
+
+# Utility code shared across multiple tests.
+
+import hashlib
+import os
+import os.path
+import sys
+import time
+
+def md5sum_data(data):
+ '''Calculate the MD5 hash of some data.
+
+ Args:
+ data: The data to hash.
+
+ Returns:
+ The hash of the data, as a binary string.
+ '''
+
+ h = hashlib.md5()
+ h.update(data)
+ return h.digest()
+
+def md5sum_file(fn, max_length=None):
+ '''Calculate the MD5 hash of the contents of a file.
+
+ Args:
+ fn: The filename of the file to hash.
+ max_length: The number of bytes to hash. If the file has more
+ bytes than this, they will be ignored. If None or omitted, the
+ entire file will be hashed.
+
+ Returns:
+ The hash of the file content, as a binary string.
+ '''
+
+ with open(fn, 'rb') as fh:
+ if max_length:
+ params = [max_length]
+ else:
+ params = []
+ data = fh.read(*params)
+ return md5sum_data(data)
+
+class PersistentRandomFile(object):
+ '''Generate and store information about a persistent file containing
+ random data.'''
+
+ def __init__(self, u_boot_console, fn, size):
+ '''Create or process the persistent file.
+
+ If the file does not exist, it is generated.
+
+ If the file does exist, its content is hashed for later comparison.
+
+ These files are always located in the "persistent data directory" of
+ the current test run.
+
+ Args:
+ u_boot_console: A console connection to U-Boot.
+ fn: The filename (without path) to create.
+ size: The desired size of the file in bytes.
+
+ Returns:
+ Nothing.
+ '''
+
+ self.fn = fn
+
+ self.abs_fn = u_boot_console.config.persistent_data_dir + '/' + fn
+
+ if os.path.exists(self.abs_fn):
+ u_boot_console.log.action('Persistent data file ' + self.abs_fn +
+ ' already exists')
+ self.content_hash = md5sum_file(self.abs_fn)
+ else:
+ u_boot_console.log.action('Generating ' + self.abs_fn +
+ ' (random, persistent, %d bytes)' % size)
+ data = os.urandom(size)
+ with open(self.abs_fn, 'wb') as fh:
+ fh.write(data)
+ self.content_hash = md5sum_data(data)
+
+def attempt_to_open_file(fn):
+ '''Attempt to open a file, without throwing exceptions.
+
+ Any errors (exceptions) that occur during the attempt to open the file
+ are ignored. This is useful in order to test whether a file (in
+ particular, a device node) exists and can be successfully opened, in order
+ to poll for e.g. USB enumeration completion.
+
+ Args:
+ fn: The filename to attempt to open.
+
+ Returns:
+ An open file handle to the file, or None if the file could not be
+ opened.
+ '''
+
+ try:
+ return open(fn, 'rb')
+ except:
+ return None
+
+def wait_until_open_succeeds(fn):
+ '''Poll until a file can be opened, or a timeout occurs.
+
+ Continually attempt to open a file, and return when this succeeds, or
+ raise an exception after a timeout.
+
+ Args:
+ fn: The filename to attempt to open.
+
+ Returns:
+ An open file handle to the file.
+ '''
+
+ for i in xrange(100):
+ fh = attempt_to_open_file(fn)
+ if fh:
+ return fh
+ time.sleep(0.1)
+ raise Exception('File could not be opened')
+
+def wait_until_file_open_fails(fn, ignore_errors):
+ '''Poll until a file cannot be opened, or a timeout occurs.
+
+ Continually attempt to open a file, and return when this fails, or
+ raise an exception after a timeout.
+
+ Args:
+ fn: The filename to attempt to open.
+ ignore_errors: Indicate whether to ignore timeout errors. If True, the
+ function will simply return if a timeout occurs, otherwise an
+ exception will be raised.
+
+ Returns:
+ Nothing.
+ '''
+
+ for i in xrange(100):
+ fh = attempt_to_open_file(fn)
+ if not fh:
+ return
+ fh.close()
+ time.sleep(0.1)
+ if ignore_errors:
+ return
+ raise Exception('File can still be opened')
+
+def run_and_log(u_boot_console, cmd, ignore_errors=False):
+ '''Run a command and log its output.
+
+ Args:
+ u_boot_console: A console connection to U-Boot.
+ cmd: The command to run, as an array of argv[].
+ ignore_errors: Indicate whether to ignore errors. If True, the function
+ will simply return if the command cannot be executed or exits with
+ an error code, otherwise an exception will be raised if such
+ problems occur.
+
+ Returns:
+ Nothing.
+ '''
+
+ runner = u_boot_console.log.get_runner(cmd[0], sys.stdout)
+ runner.run(cmd, ignore_errors=ignore_errors)
+ runner.close()
--
2.7.0
^ permalink raw reply related [flat|nested] 22+ messages in thread* [U-Boot] [PATCH 6/8] test/py: add various utility code
2016-01-20 22:15 ` [U-Boot] [PATCH 6/8] test/py: add various utility code Stephen Warren
@ 2016-01-22 3:36 ` Simon Glass
2016-01-22 16:45 ` Stephen Warren
2016-01-22 16:49 ` Stephen Warren
0 siblings, 2 replies; 22+ messages in thread
From: Simon Glass @ 2016-01-22 3:36 UTC (permalink / raw)
To: u-boot
Hi Stephen,
On 20 January 2016 at 15:15, Stephen Warren <swarren@wwwdotorg.org> wrote:
> From: Stephen Warren <swarren@nvidia.com>
>
> Add various common utility functions. These will be used by a forthcoming
> re-written UMS test, and a brand-new DFU test.
>
> Signed-off-by: Stephen Warren <swarren@nvidia.com>
> ---
> test/py/u_boot_console_base.py | 19 +++++
> test/py/u_boot_utils.py | 171 +++++++++++++++++++++++++++++++++++++++++
> 2 files changed, 190 insertions(+)
> create mode 100644 test/py/u_boot_utils.py
Acked-by: Simon Glass <sjg@chromium.org>
>
> diff --git a/test/py/u_boot_console_base.py b/test/py/u_boot_console_base.py
> index 433bec6e9fdd..06f61f987180 100644
> --- a/test/py/u_boot_console_base.py
> +++ b/test/py/u_boot_console_base.py
> @@ -215,6 +215,25 @@ class ConsoleBase(object):
> self.log.action('Sending Ctrl-C')
> self.run_command(chr(3), wait_for_echo=False, send_nl=False)
>
> + def wait_for(self, text):
> + '''Wait for a pattern to be emitted by U-Boot.
I meant to say we should use """ for function comments to keep it
consistent with the rest of U-Boot. Maybe could adjust this in a
follow-on patch?
> +
> + This is useful when a long-running command such as "dfu" is executing,
> + and it periodically emits some text that should show up at a specific
> + location in the log file.
> +
> + Args:
> + text: The text to wait for; either a string (containing raw text,
> + not a regular expression) or an re object.
> +
> + Returns:
> + Nothing.
> + '''
> +
> + if type(text) == type(''):
> + text = re.escape(text)
> + self.p.expect([text])
Does this potentially wait forever?
> +
> def drain_console(self):
> '''Read from and log the U-Boot console for a short time.
>
> diff --git a/test/py/u_boot_utils.py b/test/py/u_boot_utils.py
> new file mode 100644
> index 000000000000..539af618dbf2
> --- /dev/null
> +++ b/test/py/u_boot_utils.py
> @@ -0,0 +1,171 @@
> +# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
> +#
> +# SPDX-License-Identifier: GPL-2.0
> +
> +# Utility code shared across multiple tests.
> +
> +import hashlib
> +import os
> +import os.path
> +import sys
> +import time
> +
> +def md5sum_data(data):
> + '''Calculate the MD5 hash of some data.
> +
> + Args:
> + data: The data to hash.
> +
> + Returns:
> + The hash of the data, as a binary string.
> + '''
> +
> + h = hashlib.md5()
> + h.update(data)
> + return h.digest()
Or just:
return hashlib.md5().update(data).digest()
Regards,
Simon
^ permalink raw reply [flat|nested] 22+ messages in thread* [U-Boot] [PATCH 6/8] test/py: add various utility code
2016-01-22 3:36 ` Simon Glass
@ 2016-01-22 16:45 ` Stephen Warren
2016-01-22 16:49 ` Stephen Warren
1 sibling, 0 replies; 22+ messages in thread
From: Stephen Warren @ 2016-01-22 16:45 UTC (permalink / raw)
To: u-boot
On 01/21/2016 08:36 PM, Simon Glass wrote:
> Hi Stephen,
>
> On 20 January 2016 at 15:15, Stephen Warren <swarren@wwwdotorg.org> wrote:
>> From: Stephen Warren <swarren@nvidia.com>
>>
>> Add various common utility functions. These will be used by a forthcoming
>> re-written UMS test, and a brand-new DFU test.
>>
>> Signed-off-by: Stephen Warren <swarren@nvidia.com>
>> ---
>> test/py/u_boot_console_base.py | 19 +++++
>> test/py/u_boot_utils.py | 171 +++++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 190 insertions(+)
>> create mode 100644 test/py/u_boot_utils.py
>
> Acked-by: Simon Glass <sjg@chromium.org>
>
>> diff --git a/test/py/u_boot_console_base.py b/test/py/u_boot_console_base.py
>> index 433bec6e9fdd..06f61f987180 100644
>> --- a/test/py/u_boot_console_base.py
>> +++ b/test/py/u_boot_console_base.py
>> @@ -215,6 +215,25 @@ class ConsoleBase(object):
>> self.log.action('Sending Ctrl-C')
>> self.run_command(chr(3), wait_for_echo=False, send_nl=False)
>>
>> + def wait_for(self, text):
>> + '''Wait for a pattern to be emitted by U-Boot.
>
> I meant to say we should use """ for function comments to keep it
> consistent with the rest of U-Boot. Maybe could adjust this in a
> follow-on patch?
That feels inconsistent with using ' for strings everywhere else. I
don't see a good reason why the docstrings should use a different quote
character. Should the existing Python code be made consistent instead?
>> +
>> + This is useful when a long-running command such as "dfu" is executing,
>> + and it periodically emits some text that should show up at a specific
>> + location in the log file.
>> +
>> + Args:
>> + text: The text to wait for; either a string (containing raw text,
>> + not a regular expression) or an re object.
>> +
>> + Returns:
>> + Nothing.
>> + '''
>> +
>> + if type(text) == type(''):
>> + text = re.escape(text)
>> + self.p.expect([text])
>
> Does this potentially wait forever?
The expect() function throws a Timeout exception if none of the
strings/regexs passed to it match within the defined timeout (which is
stored in self.p.timeout).
^ permalink raw reply [flat|nested] 22+ messages in thread* [U-Boot] [PATCH 6/8] test/py: add various utility code
2016-01-22 3:36 ` Simon Glass
2016-01-22 16:45 ` Stephen Warren
@ 2016-01-22 16:49 ` Stephen Warren
1 sibling, 0 replies; 22+ messages in thread
From: Stephen Warren @ 2016-01-22 16:49 UTC (permalink / raw)
To: u-boot
On 01/21/2016 08:36 PM, Simon Glass wrote:
> Hi Stephen,
>
> On 20 January 2016 at 15:15, Stephen Warren <swarren@wwwdotorg.org> wrote:
>> From: Stephen Warren <swarren@nvidia.com>
>>
>> Add various common utility functions. These will be used by a forthcoming
>> re-written UMS test, and a brand-new DFU test.
>> diff --git a/test/py/u_boot_utils.py b/test/py/u_boot_utils.py
>> +def md5sum_data(data):
>> + '''Calculate the MD5 hash of some data.
>> +
>> + Args:
>> + data: The data to hash.
>> +
>> + Returns:
>> + The hash of the data, as a binary string.
>> + '''
>> +
>> + h = hashlib.md5()
>> + h.update(data)
>> + return h.digest()
>
> Or just:
>
> return hashlib.md5().update(data).digest()
I was going to do that, but it doesn't look like update() returns the
hash object, so I can't chain at least the .digest() call together, so I
may as well leave it as-is.
^ permalink raw reply [flat|nested] 22+ messages in thread
* [U-Boot] [PATCH 7/8] test/py: ums: add filesystem-based testing
2016-01-20 22:15 [U-Boot] [PATCH 1/8] test/py: fix timeout to be absolute Stephen Warren
` (4 preceding siblings ...)
2016-01-20 22:15 ` [U-Boot] [PATCH 6/8] test/py: add various utility code Stephen Warren
@ 2016-01-20 22:15 ` Stephen Warren
2016-01-21 11:26 ` Lukasz Majewski
2016-01-20 22:15 ` [U-Boot] [PATCH 8/8] test/py: add DFU test Stephen Warren
` (2 subsequent siblings)
8 siblings, 1 reply; 22+ messages in thread
From: Stephen Warren @ 2016-01-20 22:15 UTC (permalink / raw)
To: u-boot
From: Stephen Warren <swarren@nvidia.com>
Enhance the UMS test to optionally mount a partition and read/write a file
to it, validating that the content written and read back are identical.
This enhancement is backwards-compatible; old boardenv contents that don't
define the new configuration data will cause the test code to perform as
before.
test/ums/ is deleted since the Python test now performs the same testing
that it did.
The code is also re-written to make use of the recently added utility
module, and split it up into nested functions so the overall logic of
the test process can be followed more easily without the details
cluttering the code.
Cc: Lukasz Majewski <l.majewski@samsung.com>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
---
test/py/tests/test_ums.py | 212 +++++++++++++++++++++++++++++++++++---------
test/ums/README | 30 -------
test/ums/ums_gadget_test.sh | 183 --------------------------------------
3 files changed, 169 insertions(+), 256 deletions(-)
delete mode 100644 test/ums/README
delete mode 100755 test/ums/ums_gadget_test.sh
diff --git a/test/py/tests/test_ums.py b/test/py/tests/test_ums.py
index a137221c7a5b..cb6e5ef8c20b 100644
--- a/test/py/tests/test_ums.py
+++ b/test/py/tests/test_ums.py
@@ -7,8 +7,11 @@
# should be enhanced to validate disk IO.
import os
+import os.path
import pytest
+import re
import time
+import u_boot_utils
'''
Note: This test relies on:
@@ -17,13 +20,36 @@ a) boardenv_* to contain configuration values to define which USB ports are
available for testing. Without this, this test will be automatically skipped.
For example:
+# Leave this list empty if you have no block_devs below with writable
+# partitions defined.
+env__mount_points = (
+ "/mnt/ubtest-mnt-p2371-2180-na",
+)
+
env__usb_dev_ports = (
- {'tgt_usb_ctlr': '0', 'host_ums_dev_node': '/dev/disk/by-path/pci-0000:00:14.0-usb-0:13:1.0-scsi-0:0:0:0'},
+ {
+ "tgt_usb_ctlr": "0",
+ "host_ums_dev_node": "/dev/disk/by-path/pci-0000:00:14.0-usb-0:13:1.0-scsi-0:0:0:0",
+ },
)
env__block_devs = (
- {'type': 'mmc', 'id': '0'}, # eMMC; always present
- {'type': 'mmc', 'id': '1'}, # SD card; present since I plugged one in
+ # eMMC; always present
+ {
+ "type": "mmc",
+ "id": "0",
+ # The following two properties are optional.
+ # If present, the partition will be mounted and a file written-to and
+ # read-from it. If missing, only a simple block read test will be
+ # performed.
+ "writable_fs_partition": 1,
+ "writable_fs_subdir": "tmp/",
+ },
+ # SD card; present since I plugged one in
+ {
+ "type": "mmc",
+ "id": "1"
+ },
)
b) udev rules to set permissions on devices nodes, so that sudo is not
@@ -34,41 +60,15 @@ ACTION=="add", SUBSYSTEM=="block", SUBSYSTEMS=="usb", KERNELS=="3-13", MODE:="66
(You may wish to change the group ID instead of setting the permissions wide
open. All that matters is that the user ID running the test can access the
device.)
-'''
-def open_ums_device(host_ums_dev_node):
- '''Attempt to open a device node, returning either the opened file handle,
- or None on any error.'''
+c) /etc/fstab entries to allow the block device to be mounted without requiring
+root permissions. For example:
- try:
- return open(host_ums_dev_node, 'rb')
- except:
- return None
-
-def wait_for_ums_device(host_ums_dev_node):
- '''Continually attempt to open the device node exported by the "ums"
- command, and either return the opened file handle, or raise an exception
- after a timeout.'''
-
- for i in xrange(100):
- fh = open_ums_device(host_ums_dev_node)
- if fh:
- return fh
- time.sleep(0.1)
- raise Exception('UMS device did not appear')
-
-def wait_for_ums_device_gone(host_ums_dev_node):
- '''Continually attempt to open the device node exported by the "ums"
- command, and either return once the device has disappeared, or raise an
- exception if it does not before a timeout occurs.'''
-
- for i in xrange(100):
- fh = open_ums_device(host_ums_dev_node)
- if not fh:
- return
- fh.close()
- time.sleep(0.1)
- raise Exception('UMS device did not disappear')
+/dev/disk/by-path/pci-0000:00:14.0-usb-0:13:1.0-scsi-0:0:0:0-part1 /mnt/ubtest-mnt-p2371-2180-na ext4 noauto,user,nosuid,nodev
+
+This entry is only needed if any block_devs above contain a
+writable_fs_partition value.
+'''
@pytest.mark.buildconfigspec('cmd_usb_mass_storage')
def test_ums(u_boot_console, env__usb_dev_port, env__block_devs):
@@ -76,6 +76,14 @@ def test_ums(u_boot_console, env__usb_dev_port, env__block_devs):
device when "ums" is running, and this device must disappear when "ums" is
aborted.'''
+ have_writable_fs_partition = 'writable_fs_partition' in env__block_devs[0]
+ if not have_writable_fs_partition:
+ # If 'writable_fs_subdir' is missing, we'll skip all parts of the
+ # testing which mount filesystems.
+ u_boot_console.log.warning(
+ 'boardenv missing "writable_fs_partition"; ' +
+ 'UMS testing will be limited.')
+
tgt_usb_ctlr = env__usb_dev_port['tgt_usb_ctlr']
host_ums_dev_node = env__usb_dev_port['host_ums_dev_node']
@@ -84,11 +92,129 @@ def test_ums(u_boot_console, env__usb_dev_port, env__block_devs):
# device list here. We'll test each block device somewhere else.
tgt_dev_type = env__block_devs[0]['type']
tgt_dev_id = env__block_devs[0]['id']
+ if have_writable_fs_partition:
+ mount_point = u_boot_console.config.env['env__mount_points'][0]
+ mount_subdir = env__block_devs[0]['writable_fs_subdir']
+ part_num = env__block_devs[0]['writable_fs_partition']
+ host_ums_part_node = '%s-part%d' % (host_ums_dev_node, part_num)
+ else:
+ host_ums_part_node = host_ums_dev_node
+
+ test_f = u_boot_utils.PersistentRandomFile(u_boot_console, 'ums.bin',
+ 1024 * 1024);
+ if have_writable_fs_partition:
+ mounted_test_fn = mount_point + '/' + mount_subdir + test_f.fn
+
+ def start_ums():
+ '''Start U-Boot's ums shell command.
+
+ This also waits for the host-side USB enumeration process to complete.
+
+ Args:
+ None.
+
+ Returns:
+ Nothing.
+ '''
+
+ u_boot_console.log.action(
+ 'Starting long-running U-Boot ums shell command')
+ cmd = 'ums %s %s %s' % (tgt_usb_ctlr, tgt_dev_type, tgt_dev_id)
+ u_boot_console.run_command(cmd, wait_for_prompt=False)
+ u_boot_console.wait_for(re.compile('UMS: LUN.*[\r\n]'))
+ fh = u_boot_utils.wait_until_open_succeeds(host_ums_part_node)
+ u_boot_console.log.action('Reading raw data from UMS device')
+ fh.read(4096)
+ fh.close()
+
+ def mount():
+ '''Mount the block device that U-Boot exports.
+
+ Args:
+ None.
+
+ Returns:
+ Nothing.
+ '''
- cmd = 'ums %s %s %s' % (tgt_usb_ctlr, tgt_dev_type, tgt_dev_id)
- u_boot_console.run_command('ums 0 mmc 0', wait_for_prompt=False)
- fh = wait_for_ums_device(host_ums_dev_node)
- fh.read(4096)
- fh.close()
- u_boot_console.ctrlc()
- wait_for_ums_device_gone(host_ums_dev_node)
+ u_boot_console.log.action('Mounting exported UMS device')
+ cmd = ('/bin/mount', host_ums_part_node)
+ u_boot_utils.run_and_log(u_boot_console, cmd)
+
+ def umount(ignore_errors):
+ '''Unmount the block device that U-Boot exports.
+
+ Args:
+ ignore_errors: Ignore any errors. This is useful if an error has
+ already been detected, and the code is performing best-effort
+ cleanup. In this case, we do not want to mask the original
+ error by "honoring" any new errors.
+
+ Returns:
+ Nothing.
+ '''
+
+ u_boot_console.log.action('Unmounting UMS device')
+ cmd = ('/bin/umount', host_ums_part_node)
+ u_boot_utils.run_and_log(u_boot_console, cmd, ignore_errors)
+
+ def stop_ums(ignore_errors):
+ '''Stop U-Boot's ums shell command from executing.
+
+ This also waits for the host-side USB de-enumeration process to
+ complete.
+
+ Args:
+ ignore_errors: Ignore any errors. This is useful if an error has
+ already been detected, and the code is performing best-effort
+ cleanup. In this case, we do not want to mask the original
+ error by "honoring" any new errors.
+
+ Returns:
+ Nothing.
+ '''
+
+ u_boot_console.log.action(
+ 'Stopping long-running U-Boot ums shell command')
+ u_boot_console.ctrlc()
+ u_boot_utils.wait_until_file_open_fails(host_ums_part_node,
+ ignore_errors)
+
+ ignore_cleanup_errors = True
+ try:
+ start_ums()
+ if not have_writable_fs_partition:
+ # Skip filesystem-based testing if not configured
+ return
+ try:
+ mount()
+ u_boot_console.log.action('Writing test file via UMS')
+ cmd = ('rm', '-f', mounted_test_fn)
+ u_boot_utils.run_and_log(u_boot_console, cmd)
+ if os.path.exists(mounted_test_fn):
+ raise Exception('Could not rm target UMS test file')
+ cmd = ('cp', test_f.abs_fn, mounted_test_fn)
+ u_boot_utils.run_and_log(u_boot_console, cmd)
+ ignore_cleanup_errors = False
+ finally:
+ umount(ignore_errors=ignore_cleanup_errors)
+ finally:
+ stop_ums(ignore_errors=ignore_cleanup_errors)
+
+ ignore_cleanup_errors = True
+ try:
+ start_ums()
+ try:
+ mount()
+ u_boot_console.log.action('Reading test file back via UMS')
+ read_back_hash = u_boot_utils.md5sum_file(mounted_test_fn)
+ cmd = ('rm', '-f', mounted_test_fn)
+ u_boot_utils.run_and_log(u_boot_console, cmd)
+ ignore_cleanup_errors = False
+ finally:
+ umount(ignore_errors=ignore_cleanup_errors)
+ finally:
+ stop_ums(ignore_errors=ignore_cleanup_errors)
+
+ written_hash = test_f.content_hash
+ assert(written_hash == read_back_hash)
diff --git a/test/ums/README b/test/ums/README
deleted file mode 100644
index c80fbfefbf52..000000000000
--- a/test/ums/README
+++ /dev/null
@@ -1,30 +0,0 @@
-UMS test script.
-
-ums_gadget_test.sh
-==================
-
-Example usage:
-1. On the target:
- create UMS exportable partitions (with e.g. gpt write), or specify a
- partition number (PART_NUM) as "-" to use the entire device
- ums 0 mmc 0
-2. On the host:
- sudo test/ums/ums_gadget_test.sh VID PID PART_NUM [-f FILE_SYSTEM] [test_file]
- e.g. sudo test/ums/ums_gadget_test.sh 0525 a4a5 6 -f vfat ./dat_14M.img
-
-... where:
- VID - UMS device USB Vendor ID
- PID - UMS device USB Product ID
- PART_NUM - is the partition number on which UMS operates or "-" to use the
- whole device
-
-Information about available partitions on the target one can read with using
-the 'mmc part' or 'part list' commands.
-
-The partition num (PART_NUM) can be specified as '-' for using the whole device.
-
-The [-f FILE_SYSTEM] optional switch allows for formatting target partition to
-FILE_SYSTEM.
-
-The last, optional [test_file] parameter is for specifying the exact test file
-to use.
diff --git a/test/ums/ums_gadget_test.sh b/test/ums/ums_gadget_test.sh
deleted file mode 100755
index 9da486b266ce..000000000000
--- a/test/ums/ums_gadget_test.sh
+++ /dev/null
@@ -1,183 +0,0 @@
-#! /bin/bash
-
-# Copyright (C) 2014 Samsung Electronics
-# Lukasz Majewski <l.majewski@samsung.com>
-#
-# UMS operation test script
-#
-# SPDX-License-Identifier: GPL-2.0+
-
-clear
-
-COLOUR_RED="\33[31m"
-COLOUR_GREEN="\33[32m"
-COLOUR_ORANGE="\33[33m"
-COLOUR_DEFAULT="\33[0m"
-
-DIR=./
-SUFFIX=img
-RCV_DIR=rcv/
-LOG_FILE=./log/log-`date +%d-%m-%Y_%H-%M-%S`
-
-cd `dirname $0`
-../dfu/dfu_gadget_test_init.sh 33M 97M
-
-cleanup () {
- rm -rf $RCV_DIR $MNT_DIR
-}
-
-control_c()
-# run if user hits control-c
-{
- echo -en "\n*** CTRL+C ***\n"
- umount $MNT_DIR
- cleanup
- exit 0
-}
-
-# trap keyboard interrupt (control-c)
-trap control_c SIGINT
-
-die () {
- printf " $COLOUR_RED FAILED $COLOUR_DEFAULT \n"
- cleanup
- exit 1
-}
-
-calculate_md5sum () {
- MD5SUM=`md5sum $1`
- MD5SUM=`echo $MD5SUM | cut -d ' ' -f1`
- echo "md5sum:"$MD5SUM
-}
-
-ums_test_file () {
- printf "$COLOUR_GREEN========================================================================================= $COLOUR_DEFAULT\n"
- printf "File:$COLOUR_GREEN %s $COLOUR_DEFAULT\n" $1
-
- mount /dev/$MEM_DEV $MNT_DIR
- if [ -f $MNT_DIR/dat_* ]; then
- rm $MNT_DIR/dat_*
- fi
-
- cp ./$1 $MNT_DIR
-
- while true; do
- umount $MNT_DIR > /dev/null 2>&1
- if [ $? -eq 0 ]; then
- break
- fi
- printf "$COLOUR_ORANGE\tSleeping to wait for umount...$COLOUR_DEFAULT\n"
- sleep 1
- done
-
- echo -n "TX: "
- calculate_md5sum $1
-
- MD5_TX=$MD5SUM
- sleep 1
- N_FILE=$DIR$RCV_DIR${1:2}"_rcv"
-
- mount /dev/$MEM_DEV $MNT_DIR
- cp $MNT_DIR/$1 $N_FILE || die $?
- rm $MNT_DIR/$1
- umount $MNT_DIR
-
- echo -n "RX: "
- calculate_md5sum $N_FILE
- MD5_RX=$MD5SUM
-
- if [ "$MD5_TX" == "$MD5_RX" ]; then
- printf " $COLOUR_GREEN -------> OK $COLOUR_DEFAULT \n"
- else
- printf " $COLOUR_RED -------> FAILED $COLOUR_DEFAULT \n"
- cleanup
- exit 1
- fi
-}
-
-printf "$COLOUR_GREEN========================================================================================= $COLOUR_DEFAULT\n"
-echo "U-boot UMS test program"
-
-if [ $EUID -ne 0 ]; then
- echo "You must be root to do this." 1>&2
- exit 100
-fi
-
-if [ $# -lt 3 ]; then
- echo "Wrong number of arguments"
- echo "Example:"
- echo "sudo ./ums_gadget_test.sh VID PID PART_NUM [-f ext4] [test_file]"
- die
-fi
-
-MNT_DIR="/mnt/tmp-ums-test"
-
-VID=$1; shift
-PID=$1; shift
-PART_NUM=$1; shift
-
-if [ "$1" == "-f" ]; then
- shift
- FS_TO_FORMAT=$1; shift
-fi
-
-TEST_FILE=$1
-
-for f in `find /sys -type f -name idProduct`; do
- d=`dirname ${f}`
- if [ `cat ${d}/idVendor` != "${VID}" ]; then
- continue
- fi
- if [ `cat ${d}/idProduct` != "${PID}" ]; then
- continue
- fi
- USB_DEV=${d}
- break
-done
-
-if [ -z "${USB_DEV}" ]; then
- echo "Connect target"
- echo "e.g. ums 0 mmc 0"
- exit 1
-fi
-
-MEM_DEV=`find $USB_DEV -type d -name "sd[a-z]" | awk -F/ '{print $(NF)}' -`
-
-mkdir -p $RCV_DIR
-if [ ! -d $MNT_DIR ]; then
- mkdir -p $MNT_DIR
-fi
-
-if [ "$PART_NUM" == "-" ]; then
- PART_NUM=""
-fi
-MEM_DEV=$MEM_DEV$PART_NUM
-
-if [ -n "$FS_TO_FORMAT" ]; then
- echo -n "Formatting partition /dev/$MEM_DEV to $FS_TO_FORMAT"
- mkfs -t $FS_TO_FORMAT /dev/$MEM_DEV > /dev/null 2>&1
- if [ $? -eq 0 ]; then
- printf " $COLOUR_GREEN DONE $COLOUR_DEFAULT \n"
- else
- die
- fi
-fi
-
-printf "Mount: /dev/$MEM_DEV \n"
-
-if [ -n "$TEST_FILE" ]; then
- if [ ! -e $TEST_FILE ]; then
- echo "No file: $TEST_FILE"
- die
- fi
- ums_test_file $TEST_FILE
-else
- for file in $DIR*.$SUFFIX
- do
- ums_test_file $file
- done
-fi
-
-cleanup
-
-exit 0
--
2.7.0
^ permalink raw reply related [flat|nested] 22+ messages in thread* [U-Boot] [PATCH 7/8] test/py: ums: add filesystem-based testing
2016-01-20 22:15 ` [U-Boot] [PATCH 7/8] test/py: ums: add filesystem-based testing Stephen Warren
@ 2016-01-21 11:26 ` Lukasz Majewski
2016-01-22 3:36 ` Simon Glass
0 siblings, 1 reply; 22+ messages in thread
From: Lukasz Majewski @ 2016-01-21 11:26 UTC (permalink / raw)
To: u-boot
Hi Stephen,
> From: Stephen Warren <swarren@nvidia.com>
>
> Enhance the UMS test to optionally mount a partition and read/write a
> file to it, validating that the content written and read back are
> identical.
>
> This enhancement is backwards-compatible; old boardenv contents that
> don't define the new configuration data will cause the test code to
> perform as before.
>
> test/ums/ is deleted since the Python test now performs the same
> testing that it did.
>
> The code is also re-written to make use of the recently added utility
> module, and split it up into nested functions so the overall logic of
> the test process can be followed more easily without the details
> cluttering the code.
>
> Cc: Lukasz Majewski <l.majewski@samsung.com>
> Signed-off-by: Stephen Warren <swarren@nvidia.com>
> ---
> test/py/tests/test_ums.py | 212
> +++++++++++++++++++++++++++++++++++---------
> test/ums/README | 30 ------- test/ums/ums_gadget_test.sh
> | 183 -------------------------------------- 3 files changed, 169
> insertions(+), 256 deletions(-) delete mode 100644 test/ums/README
> delete mode 100755 test/ums/ums_gadget_test.sh
>
> diff --git a/test/py/tests/test_ums.py b/test/py/tests/test_ums.py
> index a137221c7a5b..cb6e5ef8c20b 100644
> --- a/test/py/tests/test_ums.py
> +++ b/test/py/tests/test_ums.py
> @@ -7,8 +7,11 @@
> # should be enhanced to validate disk IO.
>
> import os
> +import os.path
> import pytest
> +import re
> import time
> +import u_boot_utils
>
> '''
> Note: This test relies on:
> @@ -17,13 +20,36 @@ a) boardenv_* to contain configuration values to
> define which USB ports are available for testing. Without this, this
> test will be automatically skipped. For example:
>
> +# Leave this list empty if you have no block_devs below with writable
> +# partitions defined.
> +env__mount_points = (
> + "/mnt/ubtest-mnt-p2371-2180-na",
> +)
> +
> env__usb_dev_ports = (
> - {'tgt_usb_ctlr': '0', 'host_ums_dev_node':
> '/dev/disk/by-path/pci-0000:00:14.0-usb-0:13:1.0-scsi-0:0:0:0'},
> + {
> + "tgt_usb_ctlr": "0",
> + "host_ums_dev_node":
> "/dev/disk/by-path/pci-0000:00:14.0-usb-0:13:1.0-scsi-0:0:0:0",
> + },
> )
>
> env__block_devs = (
> - {'type': 'mmc', 'id': '0'}, # eMMC; always present
> - {'type': 'mmc', 'id': '1'}, # SD card; present since I plugged
> one in
> + # eMMC; always present
> + {
> + "type": "mmc",
> + "id": "0",
> + # The following two properties are optional.
> + # If present, the partition will be mounted and a file
> written-to and
> + # read-from it. If missing, only a simple block read test
> will be
> + # performed.
> + "writable_fs_partition": 1,
> + "writable_fs_subdir": "tmp/",
> + },
> + # SD card; present since I plugged one in
> + {
> + "type": "mmc",
> + "id": "1"
> + },
> )
>
> b) udev rules to set permissions on devices nodes, so that sudo is
> not @@ -34,41 +60,15 @@ ACTION=="add", SUBSYSTEM=="block",
> SUBSYSTEMS=="usb", KERNELS=="3-13", MODE:="66 (You may wish to change
> the group ID instead of setting the permissions wide open. All that
> matters is that the user ID running the test can access the device.)
> -'''
>
> -def open_ums_device(host_ums_dev_node):
> - '''Attempt to open a device node, returning either the opened
> file handle,
> - or None on any error.'''
> +c) /etc/fstab entries to allow the block device to be mounted
> without requiring +root permissions. For example:
>
> - try:
> - return open(host_ums_dev_node, 'rb')
> - except:
> - return None
> -
> -def wait_for_ums_device(host_ums_dev_node):
> - '''Continually attempt to open the device node exported by the
> "ums"
> - command, and either return the opened file handle, or raise an
> exception
> - after a timeout.'''
> -
> - for i in xrange(100):
> - fh = open_ums_device(host_ums_dev_node)
> - if fh:
> - return fh
> - time.sleep(0.1)
> - raise Exception('UMS device did not appear')
> -
> -def wait_for_ums_device_gone(host_ums_dev_node):
> - '''Continually attempt to open the device node exported by the
> "ums"
> - command, and either return once the device has disappeared, or
> raise an
> - exception if it does not before a timeout occurs.'''
> -
> - for i in xrange(100):
> - fh = open_ums_device(host_ums_dev_node)
> - if not fh:
> - return
> - fh.close()
> - time.sleep(0.1)
> - raise Exception('UMS device did not disappear')
> +/dev/disk/by-path/pci-0000:00:14.0-usb-0:13:1.0-scsi-0:0:0:0-part1 /mnt/ubtest-mnt-p2371-2180-na
> ext4 noauto,user,nosuid,nodev +
> +This entry is only needed if any block_devs above contain a
> +writable_fs_partition value.
> +'''
>
> @pytest.mark.buildconfigspec('cmd_usb_mass_storage')
> def test_ums(u_boot_console, env__usb_dev_port, env__block_devs):
> @@ -76,6 +76,14 @@ def test_ums(u_boot_console, env__usb_dev_port,
> env__block_devs): device when "ums" is running, and this device must
> disappear when "ums" is aborted.'''
>
> + have_writable_fs_partition = 'writable_fs_partition' in
> env__block_devs[0]
> + if not have_writable_fs_partition:
> + # If 'writable_fs_subdir' is missing, we'll skip all parts
> of the
> + # testing which mount filesystems.
> + u_boot_console.log.warning(
> + 'boardenv missing "writable_fs_partition"; ' +
> + 'UMS testing will be limited.')
> +
> tgt_usb_ctlr = env__usb_dev_port['tgt_usb_ctlr']
> host_ums_dev_node = env__usb_dev_port['host_ums_dev_node']
>
> @@ -84,11 +92,129 @@ def test_ums(u_boot_console, env__usb_dev_port,
> env__block_devs): # device list here. We'll test each block device
> somewhere else. tgt_dev_type = env__block_devs[0]['type']
> tgt_dev_id = env__block_devs[0]['id']
> + if have_writable_fs_partition:
> + mount_point =
> u_boot_console.config.env['env__mount_points'][0]
> + mount_subdir = env__block_devs[0]['writable_fs_subdir']
> + part_num = env__block_devs[0]['writable_fs_partition']
> + host_ums_part_node = '%s-part%d' % (host_ums_dev_node,
> part_num)
> + else:
> + host_ums_part_node = host_ums_dev_node
> +
> + test_f = u_boot_utils.PersistentRandomFile(u_boot_console,
> 'ums.bin',
> + 1024 * 1024);
> + if have_writable_fs_partition:
> + mounted_test_fn = mount_point + '/' + mount_subdir +
> test_f.fn +
> + def start_ums():
> + '''Start U-Boot's ums shell command.
> +
> + This also waits for the host-side USB enumeration process to
> complete. +
> + Args:
> + None.
> +
> + Returns:
> + Nothing.
> + '''
> +
> + u_boot_console.log.action(
> + 'Starting long-running U-Boot ums shell command')
> + cmd = 'ums %s %s %s' % (tgt_usb_ctlr, tgt_dev_type,
> tgt_dev_id)
> + u_boot_console.run_command(cmd, wait_for_prompt=False)
> + u_boot_console.wait_for(re.compile('UMS: LUN.*[\r\n]'))
> + fh =
> u_boot_utils.wait_until_open_succeeds(host_ums_part_node)
> + u_boot_console.log.action('Reading raw data from UMS device')
> + fh.read(4096)
> + fh.close()
> +
> + def mount():
> + '''Mount the block device that U-Boot exports.
> +
> + Args:
> + None.
> +
> + Returns:
> + Nothing.
> + '''
>
> - cmd = 'ums %s %s %s' % (tgt_usb_ctlr, tgt_dev_type, tgt_dev_id)
> - u_boot_console.run_command('ums 0 mmc 0', wait_for_prompt=False)
> - fh = wait_for_ums_device(host_ums_dev_node)
> - fh.read(4096)
> - fh.close()
> - u_boot_console.ctrlc()
> - wait_for_ums_device_gone(host_ums_dev_node)
> + u_boot_console.log.action('Mounting exported UMS device')
> + cmd = ('/bin/mount', host_ums_part_node)
> + u_boot_utils.run_and_log(u_boot_console, cmd)
> +
> + def umount(ignore_errors):
> + '''Unmount the block device that U-Boot exports.
> +
> + Args:
> + ignore_errors: Ignore any errors. This is useful if an
> error has
> + already been detected, and the code is performing
> best-effort
> + cleanup. In this case, we do not want to mask the
> original
> + error by "honoring" any new errors.
> +
> + Returns:
> + Nothing.
> + '''
> +
> + u_boot_console.log.action('Unmounting UMS device')
> + cmd = ('/bin/umount', host_ums_part_node)
> + u_boot_utils.run_and_log(u_boot_console, cmd, ignore_errors)
> +
> + def stop_ums(ignore_errors):
> + '''Stop U-Boot's ums shell command from executing.
> +
> + This also waits for the host-side USB de-enumeration process
> to
> + complete.
> +
> + Args:
> + ignore_errors: Ignore any errors. This is useful if an
> error has
> + already been detected, and the code is performing
> best-effort
> + cleanup. In this case, we do not want to mask the
> original
> + error by "honoring" any new errors.
> +
> + Returns:
> + Nothing.
> + '''
> +
> + u_boot_console.log.action(
> + 'Stopping long-running U-Boot ums shell command')
> + u_boot_console.ctrlc()
> + u_boot_utils.wait_until_file_open_fails(host_ums_part_node,
> + ignore_errors)
> +
> + ignore_cleanup_errors = True
> + try:
> + start_ums()
> + if not have_writable_fs_partition:
> + # Skip filesystem-based testing if not configured
> + return
> + try:
> + mount()
> + u_boot_console.log.action('Writing test file via UMS')
> + cmd = ('rm', '-f', mounted_test_fn)
> + u_boot_utils.run_and_log(u_boot_console, cmd)
> + if os.path.exists(mounted_test_fn):
> + raise Exception('Could not rm target UMS test file')
> + cmd = ('cp', test_f.abs_fn, mounted_test_fn)
> + u_boot_utils.run_and_log(u_boot_console, cmd)
> + ignore_cleanup_errors = False
> + finally:
> + umount(ignore_errors=ignore_cleanup_errors)
> + finally:
> + stop_ums(ignore_errors=ignore_cleanup_errors)
> +
> + ignore_cleanup_errors = True
> + try:
> + start_ums()
> + try:
> + mount()
> + u_boot_console.log.action('Reading test file back via
> UMS')
> + read_back_hash =
> u_boot_utils.md5sum_file(mounted_test_fn)
> + cmd = ('rm', '-f', mounted_test_fn)
> + u_boot_utils.run_and_log(u_boot_console, cmd)
> + ignore_cleanup_errors = False
> + finally:
> + umount(ignore_errors=ignore_cleanup_errors)
> + finally:
> + stop_ums(ignore_errors=ignore_cleanup_errors)
> +
> + written_hash = test_f.content_hash
> + assert(written_hash == read_back_hash)
> diff --git a/test/ums/README b/test/ums/README
> deleted file mode 100644
> index c80fbfefbf52..000000000000
> --- a/test/ums/README
> +++ /dev/null
> @@ -1,30 +0,0 @@
> -UMS test script.
> -
> -ums_gadget_test.sh
> -==================
> -
> -Example usage:
> -1. On the target:
> - create UMS exportable partitions (with e.g. gpt write), or
> specify a
> - partition number (PART_NUM) as "-" to use the entire device
> - ums 0 mmc 0
> -2. On the host:
> - sudo test/ums/ums_gadget_test.sh VID PID PART_NUM [-f
> FILE_SYSTEM] [test_file]
> - e.g. sudo test/ums/ums_gadget_test.sh 0525 a4a5 6 -f
> vfat ./dat_14M.img -
> -... where:
> - VID - UMS device USB Vendor ID
> - PID - UMS device USB Product ID
> - PART_NUM - is the partition number on which UMS operates or "-"
> to use the
> - whole device
> -
> -Information about available partitions on the target one can read
> with using -the 'mmc part' or 'part list' commands.
> -
> -The partition num (PART_NUM) can be specified as '-' for using the
> whole device. -
> -The [-f FILE_SYSTEM] optional switch allows for formatting target
> partition to -FILE_SYSTEM.
> -
> -The last, optional [test_file] parameter is for specifying the exact
> test file -to use.
> diff --git a/test/ums/ums_gadget_test.sh b/test/ums/ums_gadget_test.sh
> deleted file mode 100755
> index 9da486b266ce..000000000000
> --- a/test/ums/ums_gadget_test.sh
> +++ /dev/null
> @@ -1,183 +0,0 @@
> -#! /bin/bash
> -
> -# Copyright (C) 2014 Samsung Electronics
> -# Lukasz Majewski <l.majewski@samsung.com>
> -#
> -# UMS operation test script
> -#
> -# SPDX-License-Identifier: GPL-2.0+
> -
> -clear
> -
> -COLOUR_RED="\33[31m"
> -COLOUR_GREEN="\33[32m"
> -COLOUR_ORANGE="\33[33m"
> -COLOUR_DEFAULT="\33[0m"
> -
> -DIR=./
> -SUFFIX=img
> -RCV_DIR=rcv/
> -LOG_FILE=./log/log-`date +%d-%m-%Y_%H-%M-%S`
> -
> -cd `dirname $0`
> -../dfu/dfu_gadget_test_init.sh 33M 97M
> -
> -cleanup () {
> - rm -rf $RCV_DIR $MNT_DIR
> -}
> -
> -control_c()
> -# run if user hits control-c
> -{
> - echo -en "\n*** CTRL+C ***\n"
> - umount $MNT_DIR
> - cleanup
> - exit 0
> -}
> -
> -# trap keyboard interrupt (control-c)
> -trap control_c SIGINT
> -
> -die () {
> - printf " $COLOUR_RED FAILED $COLOUR_DEFAULT \n"
> - cleanup
> - exit 1
> -}
> -
> -calculate_md5sum () {
> - MD5SUM=`md5sum $1`
> - MD5SUM=`echo $MD5SUM | cut -d ' ' -f1`
> - echo "md5sum:"$MD5SUM
> -}
> -
> -ums_test_file () {
> - printf
> "$COLOUR_GREEN=========================================================================================
> $COLOUR_DEFAULT\n"
> - printf "File:$COLOUR_GREEN %s $COLOUR_DEFAULT\n" $1
> -
> - mount /dev/$MEM_DEV $MNT_DIR
> - if [ -f $MNT_DIR/dat_* ]; then
> - rm $MNT_DIR/dat_*
> - fi
> -
> - cp ./$1 $MNT_DIR
> -
> - while true; do
> - umount $MNT_DIR > /dev/null 2>&1
> - if [ $? -eq 0 ]; then
> - break
> - fi
> - printf "$COLOUR_ORANGE\tSleeping to wait for
> umount...$COLOUR_DEFAULT\n"
> - sleep 1
> - done
> -
> - echo -n "TX: "
> - calculate_md5sum $1
> -
> - MD5_TX=$MD5SUM
> - sleep 1
> - N_FILE=$DIR$RCV_DIR${1:2}"_rcv"
> -
> - mount /dev/$MEM_DEV $MNT_DIR
> - cp $MNT_DIR/$1 $N_FILE || die $?
> - rm $MNT_DIR/$1
> - umount $MNT_DIR
> -
> - echo -n "RX: "
> - calculate_md5sum $N_FILE
> - MD5_RX=$MD5SUM
> -
> - if [ "$MD5_TX" == "$MD5_RX" ]; then
> - printf " $COLOUR_GREEN -------> OK $COLOUR_DEFAULT \n"
> - else
> - printf " $COLOUR_RED -------> FAILED $COLOUR_DEFAULT \n"
> - cleanup
> - exit 1
> - fi
> -}
> -
> -printf
> "$COLOUR_GREEN=========================================================================================
> $COLOUR_DEFAULT\n" -echo "U-boot UMS test program" -
> -if [ $EUID -ne 0 ]; then
> - echo "You must be root to do this." 1>&2
> - exit 100
> -fi
> -
> -if [ $# -lt 3 ]; then
> - echo "Wrong number of arguments"
> - echo "Example:"
> - echo "sudo ./ums_gadget_test.sh VID PID PART_NUM [-f ext4]
> [test_file]"
> - die
> -fi
> -
> -MNT_DIR="/mnt/tmp-ums-test"
> -
> -VID=$1; shift
> -PID=$1; shift
> -PART_NUM=$1; shift
> -
> -if [ "$1" == "-f" ]; then
> - shift
> - FS_TO_FORMAT=$1; shift
> -fi
> -
> -TEST_FILE=$1
> -
> -for f in `find /sys -type f -name idProduct`; do
> - d=`dirname ${f}`
> - if [ `cat ${d}/idVendor` != "${VID}" ]; then
> - continue
> - fi
> - if [ `cat ${d}/idProduct` != "${PID}" ]; then
> - continue
> - fi
> - USB_DEV=${d}
> - break
> -done
> -
> -if [ -z "${USB_DEV}" ]; then
> - echo "Connect target"
> - echo "e.g. ums 0 mmc 0"
> - exit 1
> -fi
> -
> -MEM_DEV=`find $USB_DEV -type d -name "sd[a-z]" | awk -F/ '{print
> $(NF)}' -` -
> -mkdir -p $RCV_DIR
> -if [ ! -d $MNT_DIR ]; then
> - mkdir -p $MNT_DIR
> -fi
> -
> -if [ "$PART_NUM" == "-" ]; then
> - PART_NUM=""
> -fi
> -MEM_DEV=$MEM_DEV$PART_NUM
> -
> -if [ -n "$FS_TO_FORMAT" ]; then
> - echo -n "Formatting partition /dev/$MEM_DEV to $FS_TO_FORMAT"
> - mkfs -t $FS_TO_FORMAT /dev/$MEM_DEV > /dev/null 2>&1
> - if [ $? -eq 0 ]; then
> - printf " $COLOUR_GREEN DONE $COLOUR_DEFAULT \n"
> - else
> - die
> - fi
> -fi
> -
> -printf "Mount: /dev/$MEM_DEV \n"
> -
> -if [ -n "$TEST_FILE" ]; then
> - if [ ! -e $TEST_FILE ]; then
> - echo "No file: $TEST_FILE"
> - die
> - fi
> - ums_test_file $TEST_FILE
> -else
> - for file in $DIR*.$SUFFIX
> - do
> - ums_test_file $file
> - done
> -fi
> -
> -cleanup
> -
> -exit 0
Acked-by: Lukasz Majewski <l.majewski@samsung.com>
Stephen, thanks for converting DFU and UMS to pytest code.
--
Best regards,
Lukasz Majewski
Samsung R&D Institute Poland (SRPOL) | Linux Platform Group
^ permalink raw reply [flat|nested] 22+ messages in thread* [U-Boot] [PATCH 7/8] test/py: ums: add filesystem-based testing
2016-01-21 11:26 ` Lukasz Majewski
@ 2016-01-22 3:36 ` Simon Glass
0 siblings, 0 replies; 22+ messages in thread
From: Simon Glass @ 2016-01-22 3:36 UTC (permalink / raw)
To: u-boot
On 21 January 2016 at 04:26, Lukasz Majewski <l.majewski@samsung.com> wrote:
> Hi Stephen,
>
>> From: Stephen Warren <swarren@nvidia.com>
>>
>> Enhance the UMS test to optionally mount a partition and read/write a
>> file to it, validating that the content written and read back are
>> identical.
>>
>> This enhancement is backwards-compatible; old boardenv contents that
>> don't define the new configuration data will cause the test code to
>> perform as before.
>>
>> test/ums/ is deleted since the Python test now performs the same
>> testing that it did.
>>
>> The code is also re-written to make use of the recently added utility
>> module, and split it up into nested functions so the overall logic of
>> the test process can be followed more easily without the details
>> cluttering the code.
>>
>> Cc: Lukasz Majewski <l.majewski@samsung.com>
>> Signed-off-by: Stephen Warren <swarren@nvidia.com>
>> ---
>> test/py/tests/test_ums.py | 212
>> +++++++++++++++++++++++++++++++++++---------
>> test/ums/README | 30 ------- test/ums/ums_gadget_test.sh
>> | 183 -------------------------------------- 3 files changed, 169
>> insertions(+), 256 deletions(-) delete mode 100644 test/ums/README
>> delete mode 100755 test/ums/ums_gadget_test.sh
>>
Acked-by: Simon Glass <sjg@chromium.org>
^ permalink raw reply [flat|nested] 22+ messages in thread
* [U-Boot] [PATCH 8/8] test/py: add DFU test
2016-01-20 22:15 [U-Boot] [PATCH 1/8] test/py: fix timeout to be absolute Stephen Warren
` (5 preceding siblings ...)
2016-01-20 22:15 ` [U-Boot] [PATCH 7/8] test/py: ums: add filesystem-based testing Stephen Warren
@ 2016-01-20 22:15 ` Stephen Warren
2016-01-21 10:50 ` Lukasz Majewski
2016-01-21 9:40 ` [U-Boot] [PATCH 1/8] test/py: fix timeout to be absolute Lukasz Majewski
2016-01-22 3:35 ` Simon Glass
8 siblings, 1 reply; 22+ messages in thread
From: Stephen Warren @ 2016-01-20 22:15 UTC (permalink / raw)
To: u-boot
From: Stephen Warren <swarren@nvidia.com>
Add a test of DFU functionality to the Python test suite. The test
starts DFU in U-Boot, waits for USB device enumeration on the host,
executes dfu-util multiple times to test various transfer sizes, many
of which trigger USB driver edge cases, and finally aborts the DFU
command in U-Boot.
This test mirrors the functionality previously available via the shell
scripts in test/dfu, and hence those are removed too.
Cc: Lukasz Majewski <l.majewski@majess.pl>
Signed-off-by: Stephen Warren <swarren@nvidia.com>
---
test/dfu/README | 44 -------
test/dfu/dfu_gadget_test.sh | 108 ----------------
test/dfu/dfu_gadget_test_init.sh | 45 -------
test/py/tests/test_dfu.py | 257 +++++++++++++++++++++++++++++++++++++++
4 files changed, 257 insertions(+), 197 deletions(-)
delete mode 100644 test/dfu/README
delete mode 100755 test/dfu/dfu_gadget_test.sh
delete mode 100755 test/dfu/dfu_gadget_test_init.sh
create mode 100644 test/py/tests/test_dfu.py
diff --git a/test/dfu/README b/test/dfu/README
deleted file mode 100644
index 408d5594219a..000000000000
--- a/test/dfu/README
+++ /dev/null
@@ -1,44 +0,0 @@
-DFU TEST CASE DESCRIPTION:
-
-The prerequisites for running this script are assured by
-dfu_gadget_test_init.sh, which is automatically invoked by dfu_gadget_test.sh.
-In this file user is able to generate their own set of test files by altering
-the default set of TEST_FILES_SIZES variable.
-The dfu_gadget_test_init.sh would generate test images only if they are not
-already generated.
-
-On the target device, environment variable "dfu_alt_info" must contain at
-least:
-
- dfu_test.bin fat 0 6;dfudummy.bin fat 0 6
-
-Depending on your device, you may need to replace "fat" with
-"ext4", and "6" with the relevant partition number. For reference please
-consult the config file for TRATS/TRATS2 devices
-(../../include/configs/trats{2}.h)
-
-One can use fat, ext4 or any other supported file system supported by U-Boot.
-These can be created by exporting storage devices via UMS (ums 0 mmc 0) and
-using standard tools on host (like mkfs.ext4).
-
-Example usage:
-1. On the target:
- setenv dfu_alt_info dfu_test.bin fat 0 6\;dfudummy.bin fat 0 6
- dfu 0 mmc 0
-2. On the host:
- test/dfu/dfu_gadget_test.sh X Y [test file name] [usb device vendor:product]
- e.g. test/dfu/dfu_gadget_test.sh 0 1
- or
- e.g. test/dfu/dfu_gadget_test.sh 0 1 ./dat_960.img
- or
- e.g. test/dfu/dfu_gadget_test.sh 0 1 0451:d022
- or
- e.g. test/dfu/dfu_gadget_test.sh 0 1 ./dat_960.img 0451:d022
-
-... where X and Y are dfu_test.bin's and dfudummy.bin's alt setting numbers.
-They can be obtained from dfu-util -l or $dfu_alt_info.
-It is also possible to pass optional [test file name] to force the script to
-test one particular file.
-If many DFU devices are connected, it may be useful to filter on USB
-vendor/product ID (0451:d022).
-One can get them by running "lsusb" command on a host PC.
diff --git a/test/dfu/dfu_gadget_test.sh b/test/dfu/dfu_gadget_test.sh
deleted file mode 100755
index 9c7942257b44..000000000000
--- a/test/dfu/dfu_gadget_test.sh
+++ /dev/null
@@ -1,108 +0,0 @@
-#! /bin/bash
-
-# Copyright (C) 2014 Samsung Electronics
-# Lukasz Majewski <l.majewski@samsung.com>
-#
-# Script fixes, enhancements and testing:
-# Stephen Warren <swarren@nvidia.com>
-#
-# DFU operation test script
-#
-# SPDX-License-Identifier: GPL-2.0+
-
-set -e # any command return if not equal to zero
-clear
-
-COLOUR_RED="\33[31m"
-COLOUR_GREEN="\33[32m"
-COLOUR_DEFAULT="\33[0m"
-
-DIR=./
-SUFFIX=img
-RCV_DIR=rcv/
-LOG_FILE=./log/log-`date +%d-%m-%Y_%H-%M-%S`
-
-cd `dirname $0`
-./dfu_gadget_test_init.sh
-
-cleanup () {
- rm -rf $DIR$RCV_DIR
-}
-
-die () {
- printf " $COLOUR_RED FAILED $COLOUR_DEFAULT \n"
- cleanup
- exit 1
-}
-
-calculate_md5sum () {
- MD5SUM=`md5sum $1`
- MD5SUM=`echo $MD5SUM | cut -d ' ' -f1`
- echo "md5sum:"$MD5SUM
-}
-
-dfu_test_file () {
- printf "$COLOUR_GREEN ========================================================================================= $COLOUR_DEFAULT\n"
- printf "File:$COLOUR_GREEN %s $COLOUR_DEFAULT\n" $1
-
- dfu-util $USB_DEV -D $1 -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die $?
-
- echo -n "TX: "
- calculate_md5sum $1
-
- MD5_TX=$MD5SUM
-
- dfu-util $USB_DEV -D ${DIR}/dfudummy.bin -a $TARGET_ALT_SETTING_B >> $LOG_FILE 2>&1 || die $?
-
- N_FILE=$DIR$RCV_DIR${1:2}"_rcv"
-
- dfu-util $USB_DEV -U $N_FILE -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1 || die $?
-
- echo -n "RX: "
- calculate_md5sum $N_FILE
- MD5_RX=$MD5SUM
-
- if [ "$MD5_TX" == "$MD5_RX" ]; then
- printf " $COLOUR_GREEN -------> OK $COLOUR_DEFAULT \n"
- else
- printf " $COLOUR_RED -------> FAILED $COLOUR_DEFAULT \n"
- cleanup
- exit 1
- fi
-
-}
-
-printf "$COLOUR_GREEN========================================================================================= $COLOUR_DEFAULT\n"
-echo "DFU EP0 transmission test program"
-echo "Trouble shoot -> disable DBG (even the KERN_DEBUG) in the UDC driver"
-echo "@ -> TRATS2 # dfu 0 mmc 0"
-cleanup
-mkdir -p $DIR$RCV_DIR
-touch $LOG_FILE
-
-if [ $# -eq 0 ]
-then
- printf " $COLOUR_RED Please pass alt setting number!! $COLOUR_DEFAULT \n"
- exit 0
-fi
-
-TARGET_ALT_SETTING=$1
-TARGET_ALT_SETTING_B=$2
-
-file=$3
-[[ $3 == *':'* ]] && USB_DEV="-d $3" && file=""
-[ $# -eq 4 ] && USB_DEV="-d $4"
-
-if [ -n "$file" ]
-then
- dfu_test_file $file
-else
- for f in $DIR*.$SUFFIX
- do
- dfu_test_file $f
- done
-fi
-
-cleanup
-
-exit 0
diff --git a/test/dfu/dfu_gadget_test_init.sh b/test/dfu/dfu_gadget_test_init.sh
deleted file mode 100755
index 640628eecb7a..000000000000
--- a/test/dfu/dfu_gadget_test_init.sh
+++ /dev/null
@@ -1,45 +0,0 @@
-#! /bin/bash
-
-# Copyright (C) 2014 Samsung Electronics
-# Lukasz Majewski <l.majewski@samsung.com>
-#
-# Script fixes, enhancements and testing:
-# Stephen Warren <swarren@nvidia.com>
-#
-# Script for test files generation
-#
-# SPDX-License-Identifier: GPL-2.0+
-
-set -e # any command return if not equal to zero
-clear
-
-COLOUR_RED="\33[31m"
-COLOUR_GREEN="\33[32m"
-COLOUR_DEFAULT="\33[0m"
-
-LOG_DIR="./log"
-
-if [ $# -eq 0 ]; then
- TEST_FILES_SIZES="63 64 65 127 128 129 4095 4096 4097 959 960 961 1048575 1048576 8M"
-else
- TEST_FILES_SIZES=$@
-fi
-
-printf "Init script for generating data necessary for DFU test script"
-
-if [ ! -d $LOG_DIR ]; then
- `mkdir $LOG_DIR`
-fi
-
-for size in $TEST_FILES_SIZES
-do
- FILE="./dat_$size.img"
- if [ ! -f $FILE ]; then
- dd if=/dev/urandom of="./dat_$size.img" bs=$size count=1 > /dev/null 2>&1 || exit $?
- fi
-done
-dd if=/dev/urandom of="./dfudummy.bin" bs=1024 count=1 > /dev/null 2>&1 || exit $?
-
-printf "$COLOUR_GREEN OK $COLOUR_DEFAULT \n"
-
-exit 0
diff --git a/test/py/tests/test_dfu.py b/test/py/tests/test_dfu.py
new file mode 100644
index 000000000000..e86ea9a1887c
--- /dev/null
+++ b/test/py/tests/test_dfu.py
@@ -0,0 +1,257 @@
+# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
+#
+# SPDX-License-Identifier: GPL-2.0
+
+# Test U-Boot's "ums" command. At present, this test only ensures that a UMS
+# device can be enumerated by the host/test machine. In the future, this test
+# should be enhanced to validate disk IO.
+
+import os
+import os.path
+import pytest
+import u_boot_utils
+
+'''
+Note: This test relies on:
+
+a) boardenv_* to contain configuration values to define which USB ports are
+available for testing. Without this, this test will be automatically skipped.
+For example:
+
+env__usb_dev_ports = (
+ {
+ "tgt_usb_ctlr": "0",
+ "host_usb_dev_node": "/dev/usbdev-p2371-2180",
+ # This parameter is optional /if/ you only have a single board
+ # attached to your host at a time.
+ "host_usb_port_path": "3-13",
+ },
+)
+
+env__dfu_configs = (
+ # eMMC, partition 1
+ {
+ "alt_info": "/dfu_test.bin ext4 0 1;/dfu_dummy.bin ext4 0 1",
+ "cmd_params": "mmc 0",
+ },
+)
+b) udev rules to set permissions on devices nodes, so that sudo is not
+required. For example:
+
+ACTION=="add", SUBSYSTEM=="block", SUBSYSTEMS=="usb", KERNELS=="3-13", MODE:="666"
+
+(You may wish to change the group ID instead of setting the permissions wide
+open. All that matters is that the user ID running the test can access the
+device.)
+'''
+
+# The set of file sizes to test. These values trigger various edge-cases such
+# as one less than, equal to, and one greater than typical USB max packet
+# sizes, and similar boundary conditions.
+test_sizes = (
+ 64 - 1,
+ 64,
+ 64 + 1,
+ 128 - 1,
+ 128,
+ 128 + 1,
+ 960 - 1,
+ 960,
+ 960 + 1,
+ 4096 - 1,
+ 4096,
+ 4096 + 1,
+ 1024 * 1024 - 1,
+ 1024 * 1024,
+ 8 * 1024 * 1024,
+)
+
+first_usb_dev_port = None
+
+ at pytest.mark.buildconfigspec('cmd_dfu')
+def test_dfu(u_boot_console, env__usb_dev_port, env__dfu_config):
+ '''Test DFU functionality, using numerous file sizes.
+
+ Args:
+ u_boot_console: A U-Boot console connection.
+ env__usb_dev_port: The single USB device-mode port specification on
+ which to run the test.
+ env__dfu_config: The single DFU (memory region) configuration on which
+ to run the test.
+
+ Returns:
+ Nothing.
+ '''
+
+ def start_dfu():
+ '''Start U-Boot's dfu shell command.
+
+ This also waits for the host-side USB enumeration process to complete.
+
+ Args:
+ None.
+
+ Returns:
+ Nothing.
+ '''
+
+ u_boot_console.log.action(
+ 'Starting long-running U-Boot dfu shell command')
+
+ cmd = 'setenv dfu_alt_info "%s"' % env__dfu_config['alt_info']
+ u_boot_console.run_command(cmd)
+
+ cmd = 'dfu 0 ' + env__dfu_config['cmd_params']
+ u_boot_console.run_command(cmd, wait_for_prompt=False)
+ u_boot_console.log.action('Waiting for DFU USB device to appear')
+ fh = u_boot_utils.wait_until_open_succeeds(
+ env__usb_dev_port['host_usb_dev_node'])
+ fh.close()
+
+ def stop_dfu(ignore_errors):
+ '''Stop U-Boot's dfu shell command from executing.
+
+ This also waits for the host-side USB de-enumeration process to
+ complete.
+
+ Args:
+ ignore_errors: Ignore any errors. This is useful if an error has
+ already been detected, and the code is performing best-effort
+ cleanup. In this case, we do not want to mask the original
+ error by "honoring" any new errors.
+
+ Returns:
+ Nothing.
+ '''
+
+ try:
+ u_boot_console.log.action(
+ 'Stopping long-running U-Boot dfu shell command')
+ u_boot_console.ctrlc()
+ u_boot_console.log.action(
+ 'Waiting for DFU USB device to disappear')
+ u_boot_utils.wait_until_file_open_fails(
+ env__usb_dev_port['host_usb_dev_node'], ignore_errors)
+ except:
+ if not ignore_errors:
+ raise
+
+ def run_dfu_util(alt_setting, fn, up_dn_load_arg):
+ '''Invoke dfu-util on the host.
+
+ Args:
+ alt_setting: The DFU "alternate setting" identifier to interact
+ with.
+ fn: The host-side file name to transfer.
+ up_dn_load_arg: '-U' or '-D' depending on whether a DFU upload or
+ download operation should be performed.
+
+ Returns:
+ Nothing.
+ '''
+
+ cmd = ['dfu-util', '-a', str(alt_setting), up_dn_load_arg, fn]
+ if 'host_usb_port_path' in env__usb_dev_port:
+ cmd += ['-p', env__usb_dev_port['host_usb_port_path']]
+ u_boot_utils.run_and_log(u_boot_console, cmd)
+ u_boot_console.wait_for('Ctrl+C to exit ...')
+
+ def dfu_write(alt_setting, fn):
+ '''Write a file to the target board using DFU.
+
+ Args:
+ alt_setting: The DFU "alternate setting" identifier to interact
+ with.
+ fn: The host-side file name to transfer.
+
+ Returns:
+ Nothing.
+ '''
+
+ run_dfu_util(alt_setting, fn, '-D')
+
+ def dfu_read(alt_setting, fn):
+ '''Read a file from the target board using DFU.
+
+ Args:
+ alt_setting: The DFU "alternate setting" identifier to interact
+ with.
+ fn: The host-side file name to transfer.
+
+ Returns:
+ Nothing.
+ '''
+
+ # dfu-util fails reads/uploads if the host file already exists
+ if os.path.exists(fn):
+ os.remove(fn)
+ run_dfu_util(alt_setting, fn, '-U')
+
+ def dfu_write_read_check(size):
+ '''Test DFU transfers of a specific size of data
+
+ This function first writes data to the board then reads it back and
+ compares the written and read back data. Measures are taken to avoid
+ certain types of false positives.
+
+ Args:
+ size: The data size to test.
+
+ Returns:
+ Nothing.
+ '''
+
+ test_f = u_boot_utils.PersistentRandomFile(u_boot_console,
+ 'dfu_%d.bin' % size, size)
+ readback_fn = u_boot_console.config.result_dir + '/dfu_readback.bin'
+
+ u_boot_console.log.action('Writing test data to DFU primary ' +
+ 'altsetting')
+ dfu_write(0, test_f.abs_fn)
+
+ u_boot_console.log.action('Writing dummy data to DFU secondary ' +
+ 'altsetting to clear DFU buffers')
+ dfu_write(1, dummy_f.abs_fn)
+
+ u_boot_console.log.action('Reading DFU primary altsetting for ' +
+ 'comparison')
+ dfu_read(0, readback_fn)
+
+ u_boot_console.log.action('Comparing written and read data')
+ written_hash = test_f.content_hash
+ read_back_hash = u_boot_utils.md5sum_file(readback_fn, size)
+ assert(written_hash == read_back_hash)
+
+ # This test may be executed against multiple USB ports. The test takes a
+ # long time, so we don't want to do the whole thing each time. Instead,
+ # execute the full test on the first USB port, and perform a very limited
+ # test on other ports. In the limited case, we solely validate that the
+ # host PC can enumerate the U-Boot USB device.
+ global first_usb_dev_port
+ if not first_usb_dev_port:
+ first_usb_dev_port = env__usb_dev_port
+ if env__usb_dev_port == first_usb_dev_port:
+ sizes = test_sizes
+ else:
+ sizes = []
+
+ dummy_f = u_boot_utils.PersistentRandomFile(u_boot_console,
+ 'dfu_dummy.bin', 1024)
+
+ ignore_cleanup_errors = True
+ try:
+ start_dfu()
+
+ u_boot_console.log.action(
+ 'Overwriting DFU primary altsetting with dummy data')
+ dfu_write(0, dummy_f.abs_fn)
+
+ for size in sizes:
+ with u_boot_console.log.section("Data size %d" % size):
+ dfu_write_read_check(size)
+ # Make the status of each sub-test obvious. If the test didn't
+ # pass, an exception was thrown so this code isn't executed.
+ u_boot_console.log.status_pass('OK')
+ ignore_cleanup_errors = False
+ finally:
+ stop_dfu(ignore_cleanup_errors)
--
2.7.0
^ permalink raw reply related [flat|nested] 22+ messages in thread* [U-Boot] [PATCH 8/8] test/py: add DFU test
2016-01-20 22:15 ` [U-Boot] [PATCH 8/8] test/py: add DFU test Stephen Warren
@ 2016-01-21 10:50 ` Lukasz Majewski
2016-01-21 18:17 ` Stephen Warren
0 siblings, 1 reply; 22+ messages in thread
From: Lukasz Majewski @ 2016-01-21 10:50 UTC (permalink / raw)
To: u-boot
Hi Stephen,
> From: Stephen Warren <swarren@nvidia.com>
>
> Add a test of DFU functionality to the Python test suite. The test
> starts DFU in U-Boot, waits for USB device enumeration on the host,
> executes dfu-util multiple times to test various transfer sizes, many
> of which trigger USB driver edge cases, and finally aborts the DFU
> command in U-Boot.
>
> This test mirrors the functionality previously available via the shell
> scripts in test/dfu, and hence those are removed too.
>
> Cc: Lukasz Majewski <l.majewski@majess.pl>
> Signed-off-by: Stephen Warren <swarren@nvidia.com>
> ---
> test/dfu/README | 44 -------
> test/dfu/dfu_gadget_test.sh | 108 ----------------
> test/dfu/dfu_gadget_test_init.sh | 45 -------
> test/py/tests/test_dfu.py | 257
> +++++++++++++++++++++++++++++++++++++++ 4 files changed, 257
> insertions(+), 197 deletions(-) delete mode 100644 test/dfu/README
> delete mode 100755 test/dfu/dfu_gadget_test.sh
> delete mode 100755 test/dfu/dfu_gadget_test_init.sh
> create mode 100644 test/py/tests/test_dfu.py
>
> diff --git a/test/dfu/README b/test/dfu/README
> deleted file mode 100644
> index 408d5594219a..000000000000
> --- a/test/dfu/README
> +++ /dev/null
> @@ -1,44 +0,0 @@
> -DFU TEST CASE DESCRIPTION:
> -
> -The prerequisites for running this script are assured by
> -dfu_gadget_test_init.sh, which is automatically invoked by
> dfu_gadget_test.sh. -In this file user is able to generate their own
> set of test files by altering -the default set of TEST_FILES_SIZES
> variable. -The dfu_gadget_test_init.sh would generate test images
> only if they are not -already generated.
> -
> -On the target device, environment variable "dfu_alt_info" must
> contain at -least:
> -
> - dfu_test.bin fat 0 6;dfudummy.bin fat 0 6
> -
> -Depending on your device, you may need to replace "fat" with
> -"ext4", and "6" with the relevant partition number. For reference
> please -consult the config file for TRATS/TRATS2 devices
> -(../../include/configs/trats{2}.h)
> -
> -One can use fat, ext4 or any other supported file system supported
> by U-Boot. -These can be created by exporting storage devices via UMS
> (ums 0 mmc 0) and -using standard tools on host (like mkfs.ext4).
> -
> -Example usage:
> -1. On the target:
> - setenv dfu_alt_info dfu_test.bin fat 0 6\;dfudummy.bin fat 0 6
> - dfu 0 mmc 0
> -2. On the host:
> - test/dfu/dfu_gadget_test.sh X Y [test file name] [usb device
> vendor:product]
> - e.g. test/dfu/dfu_gadget_test.sh 0 1
> - or
> - e.g. test/dfu/dfu_gadget_test.sh 0 1 ./dat_960.img
> - or
> - e.g. test/dfu/dfu_gadget_test.sh 0 1 0451:d022
> - or
> - e.g. test/dfu/dfu_gadget_test.sh 0 1 ./dat_960.img 0451:d022
> -
> -... where X and Y are dfu_test.bin's and dfudummy.bin's alt setting
> numbers. -They can be obtained from dfu-util -l or $dfu_alt_info.
> -It is also possible to pass optional [test file name] to force the
> script to -test one particular file.
> -If many DFU devices are connected, it may be useful to filter on USB
> -vendor/product ID (0451:d022).
> -One can get them by running "lsusb" command on a host PC.
> diff --git a/test/dfu/dfu_gadget_test.sh b/test/dfu/dfu_gadget_test.sh
> deleted file mode 100755
> index 9c7942257b44..000000000000
> --- a/test/dfu/dfu_gadget_test.sh
> +++ /dev/null
> @@ -1,108 +0,0 @@
> -#! /bin/bash
> -
> -# Copyright (C) 2014 Samsung Electronics
> -# Lukasz Majewski <l.majewski@samsung.com>
> -#
> -# Script fixes, enhancements and testing:
> -# Stephen Warren <swarren@nvidia.com>
> -#
> -# DFU operation test script
> -#
> -# SPDX-License-Identifier: GPL-2.0+
> -
> -set -e # any command return if not equal to zero
> -clear
> -
> -COLOUR_RED="\33[31m"
> -COLOUR_GREEN="\33[32m"
> -COLOUR_DEFAULT="\33[0m"
> -
> -DIR=./
> -SUFFIX=img
> -RCV_DIR=rcv/
> -LOG_FILE=./log/log-`date +%d-%m-%Y_%H-%M-%S`
> -
> -cd `dirname $0`
> -./dfu_gadget_test_init.sh
> -
> -cleanup () {
> - rm -rf $DIR$RCV_DIR
> -}
> -
> -die () {
> - printf " $COLOUR_RED FAILED $COLOUR_DEFAULT \n"
> - cleanup
> - exit 1
> -}
> -
> -calculate_md5sum () {
> - MD5SUM=`md5sum $1`
> - MD5SUM=`echo $MD5SUM | cut -d ' ' -f1`
> - echo "md5sum:"$MD5SUM
> -}
> -
> -dfu_test_file () {
> - printf "$COLOUR_GREEN
> =========================================================================================
> $COLOUR_DEFAULT\n"
> - printf "File:$COLOUR_GREEN %s $COLOUR_DEFAULT\n" $1
> -
> - dfu-util $USB_DEV -D $1 -a $TARGET_ALT_SETTING >> $LOG_FILE 2>&1
> || die $? -
> - echo -n "TX: "
> - calculate_md5sum $1
> -
> - MD5_TX=$MD5SUM
> -
> - dfu-util $USB_DEV -D ${DIR}/dfudummy.bin -a
> $TARGET_ALT_SETTING_B >> $LOG_FILE 2>&1 || die $? -
> - N_FILE=$DIR$RCV_DIR${1:2}"_rcv"
> -
> - dfu-util $USB_DEV -U $N_FILE -a $TARGET_ALT_SETTING >> $LOG_FILE
> 2>&1 || die $? -
> - echo -n "RX: "
> - calculate_md5sum $N_FILE
> - MD5_RX=$MD5SUM
> -
> - if [ "$MD5_TX" == "$MD5_RX" ]; then
> - printf " $COLOUR_GREEN -------> OK $COLOUR_DEFAULT \n"
> - else
> - printf " $COLOUR_RED -------> FAILED $COLOUR_DEFAULT \n"
> - cleanup
> - exit 1
> - fi
> -
> -}
> -
> -printf
> "$COLOUR_GREEN=========================================================================================
> $COLOUR_DEFAULT\n" -echo "DFU EP0 transmission test program" -echo
> "Trouble shoot -> disable DBG (even the KERN_DEBUG) in the UDC
> driver" -echo "@ -> TRATS2 # dfu 0 mmc 0" -cleanup
> -mkdir -p $DIR$RCV_DIR
> -touch $LOG_FILE
> -
> -if [ $# -eq 0 ]
> -then
> - printf " $COLOUR_RED Please pass alt setting number!!
> $COLOUR_DEFAULT \n"
> - exit 0
> -fi
> -
> -TARGET_ALT_SETTING=$1
> -TARGET_ALT_SETTING_B=$2
> -
> -file=$3
> -[[ $3 == *':'* ]] && USB_DEV="-d $3" && file=""
> -[ $# -eq 4 ] && USB_DEV="-d $4"
> -
> -if [ -n "$file" ]
> -then
> - dfu_test_file $file
> -else
> - for f in $DIR*.$SUFFIX
> - do
> - dfu_test_file $f
> - done
> -fi
> -
> -cleanup
> -
> -exit 0
> diff --git a/test/dfu/dfu_gadget_test_init.sh
> b/test/dfu/dfu_gadget_test_init.sh deleted file mode 100755
> index 640628eecb7a..000000000000
> --- a/test/dfu/dfu_gadget_test_init.sh
> +++ /dev/null
> @@ -1,45 +0,0 @@
> -#! /bin/bash
> -
> -# Copyright (C) 2014 Samsung Electronics
> -# Lukasz Majewski <l.majewski@samsung.com>
> -#
> -# Script fixes, enhancements and testing:
> -# Stephen Warren <swarren@nvidia.com>
> -#
> -# Script for test files generation
> -#
> -# SPDX-License-Identifier: GPL-2.0+
> -
> -set -e # any command return if not equal to zero
> -clear
> -
> -COLOUR_RED="\33[31m"
> -COLOUR_GREEN="\33[32m"
> -COLOUR_DEFAULT="\33[0m"
> -
> -LOG_DIR="./log"
> -
> -if [ $# -eq 0 ]; then
> - TEST_FILES_SIZES="63 64 65 127 128 129 4095 4096 4097 959 960
> 961 1048575 1048576 8M" -else
> - TEST_FILES_SIZES=$@
> -fi
> -
> -printf "Init script for generating data necessary for DFU test
> script" -
> -if [ ! -d $LOG_DIR ]; then
> - `mkdir $LOG_DIR`
> -fi
> -
> -for size in $TEST_FILES_SIZES
> -do
> - FILE="./dat_$size.img"
> - if [ ! -f $FILE ]; then
> - dd if=/dev/urandom of="./dat_$size.img" bs=$size count=1
> > /dev/null 2>&1 || exit $?
> - fi
> -done
> -dd if=/dev/urandom of="./dfudummy.bin" bs=1024 count=1 > /dev/null
> 2>&1 || exit $? -
> -printf "$COLOUR_GREEN OK $COLOUR_DEFAULT \n"
> -
> -exit 0
> diff --git a/test/py/tests/test_dfu.py b/test/py/tests/test_dfu.py
> new file mode 100644
> index 000000000000..e86ea9a1887c
> --- /dev/null
> +++ b/test/py/tests/test_dfu.py
> @@ -0,0 +1,257 @@
> +# Copyright (c) 2016, NVIDIA CORPORATION. All rights reserved.
> +#
> +# SPDX-License-Identifier: GPL-2.0
> +
> +# Test U-Boot's "ums" command. At present, this test only ensures
> that a UMS +# device can be enumerated by the host/test machine. In
> the future, this test +# should be enhanced to validate disk IO.
Please update this comment to regard dfu, not ums.
> +
> +import os
> +import os.path
> +import pytest
> +import u_boot_utils
> +
> +'''
> +Note: This test relies on:
> +
> +a) boardenv_* to contain configuration values to define which USB
> ports are +available for testing. Without this, this test will be
> automatically skipped. +For example:
> +
> +env__usb_dev_ports = (
> + {
> + "tgt_usb_ctlr": "0",
> + "host_usb_dev_node": "/dev/usbdev-p2371-2180",
> + # This parameter is optional /if/ you only have a single
> board
> + # attached to your host at a time.
> + "host_usb_port_path": "3-13",
> + },
> +)
> +
> +env__dfu_configs = (
> + # eMMC, partition 1
> + {
> + "alt_info": "/dfu_test.bin ext4 0 1;/dfu_dummy.bin ext4 0 1",
> + "cmd_params": "mmc 0",
> + },
> +)
> +b) udev rules to set permissions on devices nodes, so that sudo is
> not +required. For example:
> +
> +ACTION=="add", SUBSYSTEM=="block", SUBSYSTEMS=="usb",
> KERNELS=="3-13", MODE:="666" +
> +(You may wish to change the group ID instead of setting the
> permissions wide +open. All that matters is that the user ID running
> the test can access the +device.)
> +'''
> +
> +# The set of file sizes to test. These values trigger various
> edge-cases such +# as one less than, equal to, and one greater than
> typical USB max packet +# sizes, and similar boundary conditions.
> +test_sizes = (
> + 64 - 1,
> + 64,
> + 64 + 1,
> + 128 - 1,
> + 128,
> + 128 + 1,
> + 960 - 1,
> + 960,
> + 960 + 1,
> + 4096 - 1,
> + 4096,
> + 4096 + 1,
> + 1024 * 1024 - 1,
> + 1024 * 1024,
> + 8 * 1024 * 1024,
> +)
> +
> +first_usb_dev_port = None
> +
> + at pytest.mark.buildconfigspec('cmd_dfu')
> +def test_dfu(u_boot_console, env__usb_dev_port, env__dfu_config):
> + '''Test DFU functionality, using numerous file sizes.
> +
> + Args:
> + u_boot_console: A U-Boot console connection.
> + env__usb_dev_port: The single USB device-mode port
> specification on
> + which to run the test.
> + env__dfu_config: The single DFU (memory region)
> configuration on which
> + to run the test.
> +
> + Returns:
> + Nothing.
> + '''
> +
> + def start_dfu():
> + '''Start U-Boot's dfu shell command.
> +
> + This also waits for the host-side USB enumeration process to
> complete. +
> + Args:
> + None.
> +
> + Returns:
> + Nothing.
> + '''
> +
> + u_boot_console.log.action(
> + 'Starting long-running U-Boot dfu shell command')
> +
> + cmd = 'setenv dfu_alt_info "%s"' %
> env__dfu_config['alt_info']
> + u_boot_console.run_command(cmd)
> +
> + cmd = 'dfu 0 ' + env__dfu_config['cmd_params']
> + u_boot_console.run_command(cmd, wait_for_prompt=False)
> + u_boot_console.log.action('Waiting for DFU USB device to
> appear')
> + fh = u_boot_utils.wait_until_open_succeeds(
> + env__usb_dev_port['host_usb_dev_node'])
> + fh.close()
> +
> + def stop_dfu(ignore_errors):
> + '''Stop U-Boot's dfu shell command from executing.
> +
> + This also waits for the host-side USB de-enumeration process
> to
> + complete.
> +
> + Args:
> + ignore_errors: Ignore any errors. This is useful if an
> error has
> + already been detected, and the code is performing
> best-effort
> + cleanup. In this case, we do not want to mask the
> original
> + error by "honoring" any new errors.
> +
> + Returns:
> + Nothing.
> + '''
> +
> + try:
> + u_boot_console.log.action(
> + 'Stopping long-running U-Boot dfu shell command')
> + u_boot_console.ctrlc()
> + u_boot_console.log.action(
> + 'Waiting for DFU USB device to disappear')
> + u_boot_utils.wait_until_file_open_fails(
> + env__usb_dev_port['host_usb_dev_node'],
> ignore_errors)
> + except:
> + if not ignore_errors:
> + raise
> +
> + def run_dfu_util(alt_setting, fn, up_dn_load_arg):
> + '''Invoke dfu-util on the host.
> +
> + Args:
> + alt_setting: The DFU "alternate setting" identifier to
> interact
> + with.
> + fn: The host-side file name to transfer.
> + up_dn_load_arg: '-U' or '-D' depending on whether a DFU
> upload or
> + download operation should be performed.
> +
> + Returns:
> + Nothing.
> + '''
> +
> + cmd = ['dfu-util', '-a', str(alt_setting), up_dn_load_arg,
> fn]
> + if 'host_usb_port_path' in env__usb_dev_port:
> + cmd += ['-p', env__usb_dev_port['host_usb_port_path']]
> + u_boot_utils.run_and_log(u_boot_console, cmd)
> + u_boot_console.wait_for('Ctrl+C to exit ...')
> +
> + def dfu_write(alt_setting, fn):
> + '''Write a file to the target board using DFU.
> +
> + Args:
> + alt_setting: The DFU "alternate setting" identifier to
> interact
> + with.
> + fn: The host-side file name to transfer.
> +
> + Returns:
> + Nothing.
> + '''
> +
> + run_dfu_util(alt_setting, fn, '-D')
> +
> + def dfu_read(alt_setting, fn):
> + '''Read a file from the target board using DFU.
> +
> + Args:
> + alt_setting: The DFU "alternate setting" identifier to
> interact
> + with.
> + fn: The host-side file name to transfer.
> +
> + Returns:
> + Nothing.
> + '''
> +
> + # dfu-util fails reads/uploads if the host file already
> exists
> + if os.path.exists(fn):
> + os.remove(fn)
> + run_dfu_util(alt_setting, fn, '-U')
> +
> + def dfu_write_read_check(size):
> + '''Test DFU transfers of a specific size of data
> +
> + This function first writes data to the board then reads it
> back and
> + compares the written and read back data. Measures are taken
> to avoid
> + certain types of false positives.
> +
> + Args:
> + size: The data size to test.
> +
> + Returns:
> + Nothing.
> + '''
> +
> + test_f = u_boot_utils.PersistentRandomFile(u_boot_console,
> + 'dfu_%d.bin' % size, size)
> + readback_fn = u_boot_console.config.result_dir +
> '/dfu_readback.bin' +
> + u_boot_console.log.action('Writing test data to DFU primary
> ' +
> + 'altsetting')
> + dfu_write(0, test_f.abs_fn)
> +
> + u_boot_console.log.action('Writing dummy data to DFU
> secondary ' +
> + 'altsetting to clear DFU buffers')
> + dfu_write(1, dummy_f.abs_fn)
> +
> + u_boot_console.log.action('Reading DFU primary altsetting
> for ' +
> + 'comparison')
> + dfu_read(0, readback_fn)
> +
> + u_boot_console.log.action('Comparing written and read data')
> + written_hash = test_f.content_hash
> + read_back_hash = u_boot_utils.md5sum_file(readback_fn, size)
> + assert(written_hash == read_back_hash)
> +
> + # This test may be executed against multiple USB ports. The test
> takes a
> + # long time, so we don't want to do the whole thing each time.
> Instead,
> + # execute the full test on the first USB port, and perform a
> very limited
> + # test on other ports. In the limited case, we solely validate
> that the
> + # host PC can enumerate the U-Boot USB device.
> + global first_usb_dev_port
> + if not first_usb_dev_port:
> + first_usb_dev_port = env__usb_dev_port
> + if env__usb_dev_port == first_usb_dev_port:
> + sizes = test_sizes
> + else:
> + sizes = []
> +
> + dummy_f = u_boot_utils.PersistentRandomFile(u_boot_console,
> + 'dfu_dummy.bin', 1024)
> +
> + ignore_cleanup_errors = True
> + try:
> + start_dfu()
> +
> + u_boot_console.log.action(
> + 'Overwriting DFU primary altsetting with dummy data')
> + dfu_write(0, dummy_f.abs_fn)
> +
> + for size in sizes:
> + with u_boot_console.log.section("Data size %d" % size):
> + dfu_write_read_check(size)
> + # Make the status of each sub-test obvious. If the
> test didn't
> + # pass, an exception was thrown so this code isn't
> executed.
> + u_boot_console.log.status_pass('OK')
> + ignore_cleanup_errors = False
> + finally:
> + stop_dfu(ignore_cleanup_errors)
Acked-by: Lukasz Majewski <l.majewski@samsung.com>
Great work Stephen, Thanks !
--
Best regards,
Lukasz Majewski
Samsung R&D Institute Poland (SRPOL) | Linux Platform Group
^ permalink raw reply [flat|nested] 22+ messages in thread* [U-Boot] [PATCH 8/8] test/py: add DFU test
2016-01-21 10:50 ` Lukasz Majewski
@ 2016-01-21 18:17 ` Stephen Warren
2016-01-22 3:36 ` Simon Glass
0 siblings, 1 reply; 22+ messages in thread
From: Stephen Warren @ 2016-01-21 18:17 UTC (permalink / raw)
To: u-boot
On 01/21/2016 03:50 AM, Lukasz Majewski wrote:
> Hi Stephen,
>
>> From: Stephen Warren <swarren@nvidia.com>
>>
>> Add a test of DFU functionality to the Python test suite. The test
>> starts DFU in U-Boot, waits for USB device enumeration on the host,
>> executes dfu-util multiple times to test various transfer sizes, many
>> of which trigger USB driver edge cases, and finally aborts the DFU
>> command in U-Boot.
>>
>> This test mirrors the functionality previously available via the shell
>> scripts in test/dfu, and hence those are removed too.
...
> Acked-by: Lukasz Majewski <l.majewski@samsung.com>
>
> Great work Stephen, Thanks !
Thanks for the review. I'm glad you're OK with the patch, given it
deletes your previous script.
I'll hold off on posting v2 (for the comment fix) for a few days in case
there are other comments.
^ permalink raw reply [flat|nested] 22+ messages in thread
* [U-Boot] [PATCH 8/8] test/py: add DFU test
2016-01-21 18:17 ` Stephen Warren
@ 2016-01-22 3:36 ` Simon Glass
0 siblings, 0 replies; 22+ messages in thread
From: Simon Glass @ 2016-01-22 3:36 UTC (permalink / raw)
To: u-boot
On 21 January 2016 at 11:17, Stephen Warren <swarren@wwwdotorg.org> wrote:
> On 01/21/2016 03:50 AM, Lukasz Majewski wrote:
>>
>> Hi Stephen,
>>
>>> From: Stephen Warren <swarren@nvidia.com>
>>>
>>> Add a test of DFU functionality to the Python test suite. The test
>>> starts DFU in U-Boot, waits for USB device enumeration on the host,
>>> executes dfu-util multiple times to test various transfer sizes, many
>>> of which trigger USB driver edge cases, and finally aborts the DFU
>>> command in U-Boot.
>>>
>>> This test mirrors the functionality previously available via the shell
>>> scripts in test/dfu, and hence those are removed too.
>
> ...
>>
>> Acked-by: Lukasz Majewski <l.majewski@samsung.com>
>>
>> Great work Stephen, Thanks !
>
>
> Thanks for the review. I'm glad you're OK with the patch, given it deletes
> your previous script.
>
> I'll hold off on posting v2 (for the comment fix) for a few days in case
> there are other comments.
Acked-by: Simon Glass <sjg@chromium.org>
^ permalink raw reply [flat|nested] 22+ messages in thread
* [U-Boot] [PATCH 1/8] test/py: fix timeout to be absolute
2016-01-20 22:15 [U-Boot] [PATCH 1/8] test/py: fix timeout to be absolute Stephen Warren
` (6 preceding siblings ...)
2016-01-20 22:15 ` [U-Boot] [PATCH 8/8] test/py: add DFU test Stephen Warren
@ 2016-01-21 9:40 ` Lukasz Majewski
2016-01-22 3:35 ` Simon Glass
8 siblings, 0 replies; 22+ messages in thread
From: Lukasz Majewski @ 2016-01-21 9:40 UTC (permalink / raw)
To: u-boot
Hi Stephen,
> From: Stephen Warren <swarren@nvidia.com>
>
> Currently, Spawn.expect() imposes its timeout solely upon receipt of
> new data, not on its overall operation. In theory, this could cause
> the timeout not to fire if U-Boot continually generated output that
> did not match the expected patterns.
>
> Fix the code to additionally impose a timeout on overall operation,
> which is the intended mode of operation.
>
> Signed-off-by: Stephen Warren <swarren@nvidia.com>
> ---
> test/py/u_boot_spawn.py | 7 ++++++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/test/py/u_boot_spawn.py b/test/py/u_boot_spawn.py
> index 1baee63df25c..df4c67597cab 100644
> --- a/test/py/u_boot_spawn.py
> +++ b/test/py/u_boot_spawn.py
> @@ -122,6 +122,7 @@ class Spawn(object):
> if type(patterns[pi]) == type(''):
> patterns[pi] = re.compile(patterns[pi])
>
> + tstart_s = time.time()
> try:
> while True:
> earliest_m = None
> @@ -142,7 +143,11 @@ class Spawn(object):
> self.after = self.buf[pos:posafter]
> self.buf = self.buf[posafter:]
> return earliest_pi
> - events = self.poll.poll(self.timeout)
> + tnow_s = time.time()
> + tdelta_ms = (tnow_s - tstart_s) * 1000
> + if tdelta_ms > self.timeout:
> + raise Timeout()
> + events = self.poll.poll(self.timeout - tdelta_ms)
> if not events:
> raise Timeout()
> c = os.read(self.fd, 1024)
Reviewed-by: Lukasz Majewski <l.majewski@samsung.com>
--
Best regards,
Lukasz Majewski
Samsung R&D Institute Poland (SRPOL) | Linux Platform Group
^ permalink raw reply [flat|nested] 22+ messages in thread* [U-Boot] [PATCH 1/8] test/py: fix timeout to be absolute
2016-01-20 22:15 [U-Boot] [PATCH 1/8] test/py: fix timeout to be absolute Stephen Warren
` (7 preceding siblings ...)
2016-01-21 9:40 ` [U-Boot] [PATCH 1/8] test/py: fix timeout to be absolute Lukasz Majewski
@ 2016-01-22 3:35 ` Simon Glass
8 siblings, 0 replies; 22+ messages in thread
From: Simon Glass @ 2016-01-22 3:35 UTC (permalink / raw)
To: u-boot
On 20 January 2016 at 15:15, Stephen Warren <swarren@wwwdotorg.org> wrote:
> From: Stephen Warren <swarren@nvidia.com>
>
> Currently, Spawn.expect() imposes its timeout solely upon receipt of new
> data, not on its overall operation. In theory, this could cause the
> timeout not to fire if U-Boot continually generated output that did not
> match the expected patterns.
>
> Fix the code to additionally impose a timeout on overall operation, which
> is the intended mode of operation.
>
> Signed-off-by: Stephen Warren <swarren@nvidia.com>
> ---
> test/py/u_boot_spawn.py | 7 ++++++-
> 1 file changed, 6 insertions(+), 1 deletion(-)
Acked-by: Simon Glass <sjg@chromium.org>
^ permalink raw reply [flat|nested] 22+ messages in thread