Make the test runner return a 0 exit code when only
flaky tests fail and --flaky-tests=dontcare is specified.
PR-URL: https://github.com/joyent/node/pull/25686
Reviewed-By: Julien Gilli <julien.gilli@joyent.com>
This is a minimal effort to support test output written both to
stdout and file in order to get our buildbots understanding
test output.
Cherry picked from 31940738e2
Original commit message follows:
PR-URL: https://github.com/iojs/io.js/pull/934
Reviewed-By: Chris Dickinson <christopher.s.dickinson@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Conflicts:
tools/test.py
Conflicts:
tools/test.py
PR-URL: https://github.com/joyent/node/pull/25686
Reviewed-By: Julien Gilli <julien.gilli@joyent.com>
Adding --flaky-tests option, to allow regarding flaky tests failures
as non-fatal.
Currently only observed by the TapProgressIndicator, which will
add a # TODO directive to tests classified as flaky. According to the
TAP specification, the test harness is supposed to treat failures
that have a # TODO directive as non-fatal.
PR-URL: https://github.com/joyent/node/pull/25686
Reviewed-By: Julien Gilli <julien.gilli@joyent.com>
test.py imports deps/v8/tools/utils.py but that file is gone after the
upgrade to 3.18.4 in commit 2f75785. Resurrect the file in tools/
PR-URL: https://github.com/joyent/node/pull/25686
Reviewed-By: Julien Gilli <julien.gilli@joyent.com>
Tests can leave the tty in non-blocking mode. If the test runner tries
to print to stdout/stderr after that and the tty buffer is full, it'll
die with a EAGAIN OSError. Ergo, put the tty back in blocking mode
before proceeding.
Test suite is often leaving `stray` processes on failure. They are
harmless, but may cause future test runs fail because those `stray`
processes are occupying `common.PORT` or due to some other reasons.
Killing whole process group on test suite failure should help in such
cases.
Mostly just upgraded tools/test.py to the latest one that's in V8. But also
fixing the before and after hooks to preserve the test/tmp directory so that
running tests manually usually works.
Add a setUp and tearDown function to the test case class, and use it to
create and remove the test/tmp directory for each test.
TODO: amend other tests.