In CI we previously passed `NODE_COMMON_PIPE` to the test runner to
avoid long filenames. Add an option to the test runner that allows the
user to change the temporary directory instead. This also allows us to
run test suites in parallel since `NODE_COMMON_PIPE` otherwise would
have been used from multiple tests at the same time.
PR-URL: https://github.com/nodejs/node/pull/3325
Reviewed-By: Joao Reis <reis@janeasystems.com>
In some virtualized environments the amount of available resources
are misleading; for instance `multiprocessing.cpu_count()` on our current
4-core smartos vm's returns `48`. This is not a bug, merely how the vm
host provides information about available hardware.
Avoid running into issues by overriding `cpu_count()` with `JOBS`.
PR-URL: https://github.com/nodejs/node/pull/4495
Reviewed-By: Brian White <mscdex@mscdex.net>
Tests on SmartOS are sometimes retried due to a SmartOS issue on CI.
When this happens, a TAP diagnostic message is written.
PR-URL: https://github.com/nodejs/node/pull/3960
Reviewed-By: Fedor Indutny <fedor@indutny.com>
Be slightly more verbose in cases where tests time out.
PR-URL: https://github.com/nodejs/node/pull/2647
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
By default the logfile is opened in append mode. This commit makes sure
that the file is opened in write-binary mode, so that the file will be
created if it doesn't exist or overwrite if it exists.
Fixes: #2834
PR-URL: #2837
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Alexis Campailla <orangemocha@nodejs.org>
As it is, the TAP result shows an extra hyphen in front of test names.
Sample:
ci.nodejs.org/job/node-test-commit-osx/nodes=osx1010/454/tapResults/
This patch removes the extra hyphen.
PR-URL: https://github.com/nodejs/node/pull/2718
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Make the test runner return a 0 exit code when only
flaky tests fail and --flaky-tests=dontcare is specified.
Ported from a9b642cf5b
PR-URL: https://github.com/nodejs/node/pull/2424
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: João Reis <reis@janeasystems.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Adding --flaky-tests option, to allow regarding flaky tests failures
as non-fatal.
Currently only observed by the TapProgressIndicator, which will
add a # TODO directive to tests classified as flaky. According to the
TAP specification, the test harness is supposed to treat failures
that have a # TODO directive as non-fatal.
Ported from df3a2b2cf2
PR-URL: https://github.com/nodejs/node/pull/2424
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: João Reis <reis@janeasystems.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
This commit replaces instances of io.js with Node.js, based on the
recent convergence. There are some remaining instances of io.js,
related to build and the installer.
Fixes: https://github.com/nodejs/node/issues/2361
PR-URL: https://github.com/nodejs/node/pull/2367
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: João Reis <reis@janeasystems.com>
With this option, tests can be divided in groups and only one is run
from each group. This enables us to parallelize the tests across
multiple machines.
PR-URL: https://github.com/nodejs/io.js/pull/2260
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Alexis Campailla <alexis@janeasystems.com>
These are the core changes that allow pLinux BE/LE compile. They
don't include all of the changes needed for AIX which will follow
once we have pLinux up and running in the builds
PR-URL: https://github.com/nodejs/io.js/pull/2124
Reviewed-By: Ben Noordhuis <ben@strongloop.com>
Reviewed-By: Johan Bergstrom <bugs@bergstroem.nu>
In the TAP protocol, skips are flagged as ok. Expose more
information so we can understand if the test was skipped or not.
PR-URL: https://github.com/nodejs/io.js/pull/2130
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
This commit introduces platform-specific test timeouts for the ARM
architectures. ARMv6 is notoriously slow so gets very large timeouts on
both the timeout value for each test, as well as certain problematic
individual tests. ARMv7 and ARMv8 also get slightly increased headroom.
PR-URL: https://github.com/iojs/io.js/pull/1366
Fixes: https://github.com/iojs/io.js/issues/1343
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
The ARM buildbots are notoriously slow. Update the test runner to
double the per-test time limit when it's running on one of them.
PR-URL: https://github.com/iojs/io.js/pull/1357
Reviewed-By: Roman Reiss <me@silverwind.io>
This is a minimal effort to support test output written both to
stdout and file in order to get our buildbots understanding
test output.
PR-URL: https://github.com/iojs/io.js/pull/934
Reviewed-By: Chris Dickinson <christopher.s.dickinson@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
test: add test/addons to default test list
`make test-all` and `python tools/test.py` will now also run the addon
tests in test/addons.
test: remove test-npm from test-all make target
The test-npm target builds documentation, changes versioned files,
clutters the current working directory with artifacts, etc. In short,
it doesn't seem quite ready for inclusion in `make test-all`.
test: add test-ci target, reduced test-all
Add a test-ci target that is like test-all minus the (slow) pummel and
gc test suites.
This is primarily intended for the continuous integration, where we want
decent coverage but don't want to wait for ages for tests to complete.
cpplint: add -license/copyright to default filters
Commit 3e1b1dd ("Remove excessive copyright/license boilerplate") trips
up the copyright boilerplate style check. Disable it.
PR-URL: https://github.com/iojs/io.js/pull/314
Reviewed-By: Rod Vagg <rod@vagg.org>
Now that parallel tests are enabled, the test runner spits out a ton of
'access denied' errors while running the tests. These happen because a
virus scanner or the indexing service temporarily open the file after it
has been updated, and the test runner tries to unlink() them at the same
time.
This patch resolves this issue by attempting to unlink the file until it
succeeds.
PR-URL: https://github.com/iojs/io.js/pull/284
Reviewed-by: Ben Noordhuis <info@bnoordhuis.nl>
* rename the build targets
* update the test runner to use `out/{Debug,Release}/iojs`
* update the installer to install the iojs binary
* update one test that explicitly checks for the binary name
PR-URL: https://github.com/iojs/io.js/pull/262
Reviewed-By: Bert Belder <bertbelder@gmail.com>
The flag was used for a short while during the v0.5 development cycle,
four years ago. Remove it, it's long overdue.
PR-URL: https://github.com/iojs/io.js/pull/262
Reviewed-By: Bert Belder <bertbelder@gmail.com>
* Allow running tests in mixed parallel/sequential modes
* Add -J flag for running tests on all available CPUs
* Support TEST_THREAD_ID in test/common.js and use it for tmpDir and PORT
* make: use -J flag
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
PR-URL: https://github.com/iojs/io.js/pull/172Fix: iojs/io.js#139
Adding --flaky-tests option, to allow regarding flaky tests failures
as non-fatal.
Currently only observed by the TapProgressIndicator, which will
add a # TODO directive to tests classified as flaky. According to the
TAP specification, the test harness is supposed to treat failures
that have a # TODO directive as non-fatal.
Adding --flaky-tests option, to allow regarding flaky tests failures
as non-fatal.
Currently only observed by the TapProgressIndicator, which will
add a # TODO directive to tests classified as flaky. According to the
TAP specification, the test harness is supposed to treat failures
that have a # TODO directive as non-fatal.
Make `python tools/test.py --arch=ia32,x64 --mode=debug,release` work.
The test runner looks for the `node` binary in `out/${arch}.${mode}/`.
Running tools/test.py without --arch makes it use `out/Release/node` or
`out/Debug/node` like before.
This commit removes `test/simple/test-executable-path.js` because the
assumptions it makes about the locations of the debug and release
binaries are now outdated.
PR-URL: https://github.com/node-forward/node/pull/24
Reviewed-By: Fedor Indutny <fedor@indutny.com>
Tests can leave the tty in non-blocking mode. If the test runner tries
to print to stdout/stderr after that and the tty buffer is full, it'll
die with a EAGAIN OSError. Ergo, put the tty back in blocking mode
before proceeding.
Test suite is often leaving `stray` processes on failure. They are
harmless, but may cause future test runs fail because those `stray`
processes are occupying `common.PORT` or due to some other reasons.
Killing whole process group on test suite failure should help in such
cases.
Johan Bergström
Rich Trott
Karl Skomski
Sakthipriyan Vairamani
Alexis Campailla
cjihrig
João Reis
Michael Dawson
Roman Reiss
Ben Noordhuis
Rod Vagg
Bert Belder
Chris Dickinson
Fedor Indutny
Timothy J Fontaine
isaacs
Fedor Indutny
Ryan Dahl