Imprecision in process.hrtime() in some situations can result in a zero
duration being used as a denominator in benchmark tests. This would
almost certainly never happen in real benchmarks. It is only likely in
very short benchmarks like the type we run in our test suite to just
make sure that the benchmark code is runnable.
So, if the environment variable that we use in tests to indicate "allow
ludicrously short benchmarks" is set, convert a zero duration for
a benchmark to 1 nano-second.
PR-URL: https://github.com/nodejs/node/pull/13110
Fixes: https://github.com/nodejs/node/issues/13102
Fixes: https://github.com/nodejs/node/issues/12433
Reviewed-By: Refael Ackermann <refack@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Some benchmark tests are failing intermittently, possibly due to
hrtime() imprecision on particular hosts. This change will confirm or
refute that as the root cause the next time the test fails on CI. Either
way, it's a valid check.
PR-URL: https://github.com/nodejs/node/pull/12934
Ref: https://github.com/nodejs/node/issues/12497
Ref: https://github.com/nodejs/node/issues/12433
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/12429
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/12429
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
This allows running a benchmark with two or more values for the same
config rather than just one or all of them, for example:
```
node benchmark/buffers/buffer-creation.js type=buffer() type=fast-alloc type=fast-alloc-fill
```
PR-URL: https://github.com/nodejs/node/pull/11819
Reviewed-By: Joyee Cheung <joyeec9h3@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Franziska Hinkelmann <franziska.hinkelmann@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
This removes all instances of %OptimizeFunctionOnNextCall from common.js
and README.md
PR-URL: https://github.com/nodejs/node/pull/9615
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Some benchmarks' results are small values, so keeping decimals when
running them manually (not comparing) can be helpful.
PR-URL: https://github.com/nodejs/node/pull/10559
Reviewed-By: James M Snell <jasnell@gmail.com>
* Give createBenchmark and the Benchmark constructor
a third argument for specifying the command line flags
that this benchmark should be run with.
The benchmarks are no longer run with --expose-internals
by default, they will need to explicitly pass the flags.
* Rename options to configs in createBenchmark and the Benchmark
constructor to match the documentation since they are not optional.
* Comment the properties of a Benchmark object
Also improve the documentation about creating benchmarks
* Add detailed description of the arguments of `createBenchmark`
* Describe the two passes of running the benchmarks
* Suggest what kind of code should go where in the benchmark example
PR-URL: https://github.com/nodejs/node/pull/10448
Reviewed-By: Andreas Madsen <amwebdk@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
Adds --expose_internals switch to benchmark runner. This makes
misc/freelist.js benchmark run properly
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Andreas Madsen <amwebdk@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/8547
This adds support for multiple HTTP benchmarkers. Adds autocannon
as the secondary benchmarker.
PR-URL: https://github.com/nodejs/node/pull/8140
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
I noticed some typos and the lack of {} following an if.
PR-URL: https://github.com/nodejs/node/pull/7961
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Andreas Madsen <amwebdk@gmail.com>
This removes the need for parsing stdout from the benchmarks. If the
process wasn't executed by fork, it will just print like it used to.
This also fixes the parsing of CLI arguments, by inferring the type
from the options object instead of the value content.
Only two benchmarks had to be changed:
* http/http_server_for_chunky_client.js this previously used a spawn
now it uses a fork and relays the messages using common.sendResult.
* misc/v8-bench.js this utilized that v8/benchmark/run.js called
global.print and reformatted the input. It now interfaces directly
with the benchmark runner global.BenchmarkSuite.
PR-URL: https://github.com/nodejs/node/pull/7094
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
It wasn't obviouse that common.js was the main cli tool.
PR-URL: https://github.com/nodejs/node/pull/7094
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
This commit optimizes checkInvalidHeaderChar() by unrolling the
character checking loop a bit.
Additionally, some changes to the benchmark runner are needed in
order for the included benchmark to be run correctly. Specifically,
the regexp used to parse `key=value` parameters contained a greedy
quantifier that was causing the `key` to match part of the `value`
if `value` contained an equals sign.
PR-URL: https://github.com/nodejs/node/pull/6570
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
It's useful to be able to force optimization of a function.
Rather than duplicating the code everywhere for it, let's
make a utility available.
PR-URL: https://github.com/nodejs/node/pull/6222
Reviewed-By: Brian White <mscdex@mscdex.net>
In the hopes of soon having the benchmark code linted, this change
groups all the likely non-controversial lint-compliance changes such as
indentation, semi-colon usage, and single-vs.-double quotation marks.
Other lint rules may have subtle performance implications in the V8
currently shipped with Node.js. Those changes will require more careful
review and will be in a separate change.
PR-URL: https://github.com/nodejs/node/pull/5429
Reviewed-By: Roman Reiss <me@silverwind.io>
Reviewed-By: Brian White <mscdex@mscdex.net>
The benchmark runner spawns new processes for each configuration. The
specific configuration is transfered by process.argv. This means that
the values have to be parsed. As of right now only numbers and strings
are parsed correctly. However other values such as objects where used.
This fixes the benchmarks that used non-string/number values and
prevents future issues by asserting the type.
PR-URL: https://github.com/nodejs/node/pull/5177
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
Apply strict mode to benchmark code.
PR-URL: https://github.com/nodejs/node/pull/5336
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Reviewed-By: Roman Reiss <me@silverwind.io>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Very minor update in benchmark/common.js
Not exactly a critical change, just continued cleaning
out of old joyent/node PRs that never landed.
Ref: https://github.com/nodejs/node-v0.x-archive/pull/8515
PR-URL: https://github.com/nodejs/node/pull/2399
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Brendan Ashworth <brendan.ashworth@me.com>
Reviewed-By: Michaël Zasso <mic.besace@gmail.com>
This commit replaces instances of io.js with Node.js, based on the
recent convergence. There are some remaining instances of io.js,
related to build and the installer.
Fixes: https://github.com/nodejs/node/issues/2361
PR-URL: https://github.com/nodejs/node/pull/2367
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: João Reis <reis@janeasystems.com>
When running a non-http benchmark, there is no need the check for the
wrk tool so move the wrk check into the http method.
PR-URL: https://github.com/iojs/io.js/pull/1368
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
This commit adds an `OUTPUT_FORMAT` environment variable option for
all benchmark tests that allow either 'csv' or 'default' output. Default
output has been left unchanged, and csv output prints out the csv
headers along with the csv formatted per-test output, each test also
seperated by a newline.
It can be used like the following:
$ OUTPUT_FORMAT=csv iojs benchmark/common.js http
Not specifying the OUTPUT_FORMAT env var will default to 'default'.
Specifying a bad value will throw an error.
PR-URL: https://github.com/iojs/io.js/pull/777
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
wrk is an optional tool that some of the http benchmarks uses. The removal
doesn't affect any users. Developers are assumed to install it before running
the tests.
This change reduces the tarball by 5%
PR-URL: https://github.com/iojs/io.js/pull/982
Reviewed-By: Rod Vagg <rod@vagg.org>
Benchmarker should pass exec flags (e.g. --no-crankshaft,
--turbofan-filter, --trace-opt etc) to the benchmarking process
Reviewed-By: Sam Roberts <vieuxtech@gmail.com>
Reviewed-By: Chris Dickinson <christopher.s.dickinson@gmail.com>
PR-URL: https://github.com/iojs/io.js/pull/928
When running benchmark/compare.js, it is typical to run a version of
node that does not support template strings. This provides backwards
compatibility for comparing benchmarks using older versions of node.
PR-URL: https://github.com/iojs/io.js/pull/714
Reviewed-By: Chris Dickinson <christopher.s.dickinson@gmail.com>
This commit cleans up `benchmark/common.js` with a few generic changes
such as the following:
- declare all `require()`'d libraries at the top instead of in the
middle
- add some empty whitespace where it helps readability
- changes ambiguous variable names
- standardizes most if / else blocks
- missing semicolons
PR-URL: https://github.com/iojs/io.js/pull/662
Reviewed-By: Nicu Micleușanu <micnic90@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Before:
# common.js executes all tests in net directory.
$ ./iojs common.js net
After:
# common.js executes only "dgram" tests in net directory.
$ ./iojs common.js net dgram
PR-URL: https://github.com/iojs/io.js/pull/488
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Don't use Number#toPrecision(), it switches to scientific notation for
numbers with more digits than the precision; use Number#toFixed().
PR-URL: https://github.com/iojs/io.js/pull/185
Reviewed-By: Chris Dickinson <christopher.s.dickinson@gmail.com>
Rich Trott
Vse Mozhet Byt
James M Snell
Nikolai Vavilov
Bartosz Sosnowski
Joyee Cheung
Brian White
Andreas Madsen
Adrian Nitu
Minwoo Jung
cjihrig
Jackson Tian
Brendan Ashworth
Johan Bergström
Petka Antonov
Evan Lucas
Yosuke Furukawa
Ben Noordhuis
isaacs