Allow test.py to run tests with a 'tests/' prefix or a '.js' postfix
PR-URL: https://github.com/nodejs/node/pull/9694
Fixes: https://github.com/nodejs/node/issues/9684
Reviewed-By: Prince John Wesley <princejohnwesley@gmail.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
The repeat option in test.py did not work as expected if `-j` was set to
more than one. Repeated tests running at the same time could share temp
directories and cause test failures. This was observed with:
tools/test.py -J --repeat=10 parallel/test-fs-watch-recursive
By using copy.deepCopy(), the repeated tests are separate objects and
not references to the same objects. Setting `thread_id` on one of them
will now not change the `thread_id` on all of them. And `thread_id` is
how the temp directory (and common.PORT as well) are determined.
Refs: https://github.com/nodejs/node/pull/9228
PR-URL: https://github.com/nodejs/node/pull/9249
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Produce a tap13-compatible output which makes it
simpler to parse. Output is still readable by
the jenkins tap plugin.
PR-URL: https://github.com/nodejs/node/pull/9262
Reviewed-By: Gibson Fahnestock <gibfahn@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
On every platform but `Windows`. Also, print the crash information when
using the tap reporter.
PR-URL: https://github.com/nodejs/node/pull/7859
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Manual backport from master to 4.x stream, original
commit message follows.
I often want to run a test many times to see if a failure
can be recreated and I believe this is a common
use case. We even have this job in the CI
https://ci.nodejs.org/job/node-stress-single-test/configure
but often you want to run it on a specific machine.
This patch adds the --repeat option so that
you can repeat the selected set of tests a
number of times. Given existing options
in test.py this will allow you to run
one or more tests for the number of
repeats specified. For example:
tools/test.py -j8 --repeat 1000 parallel/test-process-exec-argv
runs the test-process-exec-argv test 1000 times,
running 8 copies in parallel
tools/test.py --repeat 2
would run the entire test suite twice.
PR-URL: https://github.com/nodejs/node/pull/6700
Reviewed-By: Ben Noorhduis <info@bnoordhuis.nl>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Fedor Indutny <fedor@indutny.com>
Reviewed-By: thefourtheye - Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: joaocgreis - João Reis <reis@janeasystems.com>
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Commit 084b2ec ("test: include component in tap output") introduced
an in hindsight glaringly obvious but fortunately not very critical
Windows-specific bug by failing to take the path separator into account.
This commit rectifies that, the prefix is now correctly stripped.
PR-URL: https://github.com/nodejs/node/pull/6915
Refs: https://github.com/nodejs/node/pull/6653
Reviewed-By: Anna Henningsen <anna@addaleax.net>
This makes it so you can see why the check fails if it does.
Typically that sort of thing can happen if you are modifying
bootstrapping or `process`.
PR-URL: https://github.com/nodejs/node/pull/6786
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Changes to Node core in order to allow compilation for linuxOne.
The ../archs/linux32-s390x/opensslconf.h and
../archs/linux64-s390x/opensslconf.h were automatically
generated by running make linux-ppc linux-ppc64 in the
deps/openssl/config directory as per our standard
practice
After these changes we still need a version of v8
which supports linuxOne but that will be coming soon
in the 5.1 version of v8. Until then with these changes
we'll be able to create a hybrid build which pulls in
v8 from the http://github/andrewlow repo.
PR-URL: https://github.com/nodejs/node/pull/5941
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
* Test the toHTML function in html.js. Check that given valid markdown
it produces the expected html. One test case will prevent regressions
of #5873.
* Check that when given valid markdown toJSON produces valid JSON with
the expected schema.
* Add doctool to the list of built in tests so it runs in CI.
PR-URL: https://github.com/nodejs/node/pull/6031
Fixes: https://github.com/nodejs/node/issues/5955
Reviewed-By: Roman Reiss <me@silverwind.io>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Print test name as (for example) "parallel/test-assert". Tests that are
scraped from the addons documentation are all named test.js, making it
hard to decipher what test is running when only the filename is printed.
Fixes: https://github.com/nodejs/node/issues/6651
PR-URL: https://github.com/nodejs/node/pull/6653
Reviewed-By: James M Snell <jasnell@gmail.com>
The value is retrieved from `process.arch` in node itself.
PR-URL: https://github.com/nodejs/node/pull/5997
Reviewed-By: João Reis <reis@janeasystems.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
This commit adds a known_issues directory to the test directory
for scripts that reproduce known bugs. Since these scripts are
expected to fail, it also adds a --expect-fail flag to test.py
which reports tests as successful when they fail.
Refs: https://github.com/nodejs/testing/issues/18
Backport-URL: https://github.com/nodejs/node/pull/5785
PR-URL: https://github.com/nodejs/node/pull/5528
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Wyatt Preul <wpreul@gmail.com>
Reviewed-By: Rich Trott <rtrott@gmail.com>
Conflicts:
tools/test.py
In CI we previously passed `NODE_COMMON_PIPE` to the test runner to
avoid long filenames. Add an option to the test runner that allows the
user to change the temporary directory instead. This also allows us to
run test suites in parallel since `NODE_COMMON_PIPE` otherwise would
have been used from multiple tests at the same time.
PR-URL: https://github.com/nodejs/node/pull/3325
Reviewed-By: Joao Reis <reis@janeasystems.com>
In some virtualized environments the amount of available resources
are misleading; for instance `multiprocessing.cpu_count()` on our current
4-core smartos vm's returns `48`. This is not a bug, merely how the vm
host provides information about available hardware.
Avoid running into issues by overriding `cpu_count()` with `JOBS`.
PR-URL: https://github.com/nodejs/node/pull/4495
Reviewed-By: Brian White <mscdex@mscdex.net>
Tests on SmartOS are sometimes retried due to a SmartOS issue on CI.
When this happens, a TAP diagnostic message is written.
PR-URL: https://github.com/nodejs/node/pull/3960
Reviewed-By: Fedor Indutny <fedor@indutny.com>
Be slightly more verbose in cases where tests time out.
PR-URL: https://github.com/nodejs/node/pull/2647
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
By default the logfile is opened in append mode. This commit makes sure
that the file is opened in write-binary mode, so that the file will be
created if it doesn't exist or overwrite if it exists.
Fixes: #2834
PR-URL: #2837
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Alexis Campailla <orangemocha@nodejs.org>
As it is, the TAP result shows an extra hyphen in front of test names.
Sample:
ci.nodejs.org/job/node-test-commit-osx/nodes=osx1010/454/tapResults/
This patch removes the extra hyphen.
PR-URL: https://github.com/nodejs/node/pull/2718
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Make the test runner return a 0 exit code when only
flaky tests fail and --flaky-tests=dontcare is specified.
Ported from a9b642cf5b
PR-URL: https://github.com/nodejs/node/pull/2424
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: João Reis <reis@janeasystems.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Adding --flaky-tests option, to allow regarding flaky tests failures
as non-fatal.
Currently only observed by the TapProgressIndicator, which will
add a # TODO directive to tests classified as flaky. According to the
TAP specification, the test harness is supposed to treat failures
that have a # TODO directive as non-fatal.
Ported from df3a2b2cf2
PR-URL: https://github.com/nodejs/node/pull/2424
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: João Reis <reis@janeasystems.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
This commit replaces instances of io.js with Node.js, based on the
recent convergence. There are some remaining instances of io.js,
related to build and the installer.
Fixes: https://github.com/nodejs/node/issues/2361
PR-URL: https://github.com/nodejs/node/pull/2367
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: João Reis <reis@janeasystems.com>
With this option, tests can be divided in groups and only one is run
from each group. This enables us to parallelize the tests across
multiple machines.
PR-URL: https://github.com/nodejs/io.js/pull/2260
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Alexis Campailla <alexis@janeasystems.com>
These are the core changes that allow pLinux BE/LE compile. They
don't include all of the changes needed for AIX which will follow
once we have pLinux up and running in the builds
PR-URL: https://github.com/nodejs/io.js/pull/2124
Reviewed-By: Ben Noordhuis <ben@strongloop.com>
Reviewed-By: Johan Bergstrom <bugs@bergstroem.nu>
In the TAP protocol, skips are flagged as ok. Expose more
information so we can understand if the test was skipped or not.
PR-URL: https://github.com/nodejs/io.js/pull/2130
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
This commit introduces platform-specific test timeouts for the ARM
architectures. ARMv6 is notoriously slow so gets very large timeouts on
both the timeout value for each test, as well as certain problematic
individual tests. ARMv7 and ARMv8 also get slightly increased headroom.
PR-URL: https://github.com/iojs/io.js/pull/1366
Fixes: https://github.com/iojs/io.js/issues/1343
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
The ARM buildbots are notoriously slow. Update the test runner to
double the per-test time limit when it's running on one of them.
PR-URL: https://github.com/iojs/io.js/pull/1357
Reviewed-By: Roman Reiss <me@silverwind.io>
This is a minimal effort to support test output written both to
stdout and file in order to get our buildbots understanding
test output.
PR-URL: https://github.com/iojs/io.js/pull/934
Reviewed-By: Chris Dickinson <christopher.s.dickinson@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
test: add test/addons to default test list
`make test-all` and `python tools/test.py` will now also run the addon
tests in test/addons.
test: remove test-npm from test-all make target
The test-npm target builds documentation, changes versioned files,
clutters the current working directory with artifacts, etc. In short,
it doesn't seem quite ready for inclusion in `make test-all`.
test: add test-ci target, reduced test-all
Add a test-ci target that is like test-all minus the (slow) pummel and
gc test suites.
This is primarily intended for the continuous integration, where we want
decent coverage but don't want to wait for ages for tests to complete.
cpplint: add -license/copyright to default filters
Commit 3e1b1dd ("Remove excessive copyright/license boilerplate") trips
up the copyright boilerplate style check. Disable it.
PR-URL: https://github.com/iojs/io.js/pull/314
Reviewed-By: Rod Vagg <rod@vagg.org>
Now that parallel tests are enabled, the test runner spits out a ton of
'access denied' errors while running the tests. These happen because a
virus scanner or the indexing service temporarily open the file after it
has been updated, and the test runner tries to unlink() them at the same
time.
This patch resolves this issue by attempting to unlink the file until it
succeeds.
PR-URL: https://github.com/iojs/io.js/pull/284
Reviewed-by: Ben Noordhuis <info@bnoordhuis.nl>
* rename the build targets
* update the test runner to use `out/{Debug,Release}/iojs`
* update the installer to install the iojs binary
* update one test that explicitly checks for the binary name
PR-URL: https://github.com/iojs/io.js/pull/262
Reviewed-By: Bert Belder <bertbelder@gmail.com>
The flag was used for a short while during the v0.5 development cycle,
four years ago. Remove it, it's long overdue.
PR-URL: https://github.com/iojs/io.js/pull/262
Reviewed-By: Bert Belder <bertbelder@gmail.com>
* Allow running tests in mixed parallel/sequential modes
* Add -J flag for running tests on all available CPUs
* Support TEST_THREAD_ID in test/common.js and use it for tmpDir and PORT
* make: use -J flag
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
PR-URL: https://github.com/iojs/io.js/pull/172Fix: iojs/io.js#139
Adding --flaky-tests option, to allow regarding flaky tests failures
as non-fatal.
Currently only observed by the TapProgressIndicator, which will
add a # TODO directive to tests classified as flaky. According to the
TAP specification, the test harness is supposed to treat failures
that have a # TODO directive as non-fatal.
Adding --flaky-tests option, to allow regarding flaky tests failures
as non-fatal.
Currently only observed by the TapProgressIndicator, which will
add a # TODO directive to tests classified as flaky. According to the
TAP specification, the test harness is supposed to treat failures
that have a # TODO directive as non-fatal.
Make `python tools/test.py --arch=ia32,x64 --mode=debug,release` work.
The test runner looks for the `node` binary in `out/${arch}.${mode}/`.
Running tools/test.py without --arch makes it use `out/Release/node` or
`out/Debug/node` like before.
This commit removes `test/simple/test-executable-path.js` because the
assumptions it makes about the locations of the debug and release
binaries are now outdated.
PR-URL: https://github.com/node-forward/node/pull/24
Reviewed-By: Fedor Indutny <fedor@indutny.com>
Francis Gulotta
Rich Trott
Johan Bergström
Santiago Gimeno
Michael Dawson
Ben Noordhuis
Jeremiah Senkpiel
Ian Kronquist
cjihrig
Karl Skomski
Sakthipriyan Vairamani
Alexis Campailla
João Reis
Roman Reiss
Rod Vagg
Bert Belder
Chris Dickinson
Fedor Indutny
Timothy J Fontaine