diff options
Diffstat (limited to 'doc')
| -rw-r--r-- | doc/develop/tests_sandbox.rst | 69 | ||||
| -rw-r--r-- | doc/usage/cmd/ut.rst | 11 |
2 files changed, 79 insertions, 1 deletions
diff --git a/doc/develop/tests_sandbox.rst b/doc/develop/tests_sandbox.rst index 8e42a32afb..bfd3bdb927 100644 --- a/doc/develop/tests_sandbox.rst +++ b/doc/develop/tests_sandbox.rst @@ -143,6 +143,75 @@ For example:: Test dm_test_rtc_reset failed 3 times +Isolating a test that breaks another +------------------------------------ + +When running unit tests, some may have side effects which cause a subsequent +test to break. This can sometimes be seen when using 'ut dm' or similar. + +You can use the `-I` argument to the `ut` command to isolate this problem. +First use `ut info` to see how many tests there are, then use a binary search to +home in on the problem. Note that you might need to restart U-Boot after each +iteration, so the `-c` argument to U-Boot is useful. + +For example, let's stay that dm_test_host() is failing:: + + => ut dm + ... + Test: dm_test_get_stats: core.c + Test: dm_test_get_stats: core.c (flat tree) + Test: dm_test_host: host.c + test/dm/host.c:71, dm_test_host(): 0 == ut_check_delta(mem_start): Expected 0x0 (0), got 0xffffcbb0 (-13392) + Test: dm_test_host: host.c (flat tree) + Test <NULL> failed 1 times + Test: dm_test_host_dup: host.c + Test: dm_test_host_dup: host.c (flat tree) + ... + +You can then tell U-Boot to run the failing test at different points in the +sequence: + + => ut info + Test suites: 21 + Total tests: 645 + +:: + + $ ./u-boot -T -c "ut dm -I300:dm_test_host" + ... + Test: dm_test_pinctrl_single: pinmux.c (flat tree) + Test: dm_test_host: host.c + test/dm/host.c:71, dm_test_host(): 0 == ut_check_delta(mem_start): Expected 0x0 (0), got 0xfffffdb0 (-592) + Test: dm_test_host: host.c (flat tree) + Test dm_test_host failed 1 times (position 300) + Failures: 4 + +So it happened before position 300. Trying 150 shows it failing, so we try 75:: + + $ ./u-boot -T -c "ut dm -I75:dm_test_host" + ... + Test: dm_test_autoprobe: core.c + Test: dm_test_autoprobe: core.c (flat tree) + Test: dm_test_host: host.c + Test: dm_test_host: host.c (flat tree) + Failures: 0 + +That succeeds, so we try 120, etc. until eventually we can figure out that the +problem first happens at position 82. + + $ ./u-boot -T -c "ut dm -I82:dm_test_host" + ... + Test: dm_test_blk_flags: blk.c + Test: dm_test_blk_flags: blk.c (flat tree) + Test: dm_test_host: host.c + test/dm/host.c:71, dm_test_host(): 0 == ut_check_delta(mem_start): Expected 0x0 (0), got 0xffffc960 (-13984) + Test: dm_test_host: host.c (flat tree) + Test dm_test_host failed 1 times (position 82) + Failures: 1 + +From this we can deduce that `dm_test_blk_flags()` causes the problem with +`dm_test_host()`. + Running sandbox_spl tests directly ---------------------------------- diff --git a/doc/usage/cmd/ut.rst b/doc/usage/cmd/ut.rst index 11c64a1779..a3039634f2 100644 --- a/doc/usage/cmd/ut.rst +++ b/doc/usage/cmd/ut.rst @@ -8,10 +8,12 @@ Synopis :: - ut [-r<runs>] [-f] [<suite> [<test>]] + ut [-r<runs>] [-f] [-I<n>:<one_test>] [<suite> [<test>]] <runs> Number of times to run each test -f Force 'manual' tests to run as well + <n> Run <one test> after <n> other tests have run + <one_test> Name of the 'one' test to run <suite> Test suite to run, or `all` <test> Name of single test to run @@ -35,6 +37,13 @@ Manual tests are normally skipped by this command. Use `-f` to run them. See See :ref:`develop/tests_writing:mixing python and c` for more information on manual test. +When running unit tests, some may have side effects which cause a subsequent +test to break. This can sometimes be seen when using 'ut dm' or similar. To +fix this, select the 'one' test which breaks. Then tell the 'ut' command to +run this one test after a certain number of other tests have run. Using a +binary search method with `-I` you can quickly figure one which test is causing +the problem. + Generally all tests in the suite are run. To run just a single test from the suite, provide the <test> argument. |