* Speed up buffer.swap16 and swap32 by using builtins. Up to ~6x gain.
Drop transition point between JS and C++ implementations accordingly.
Amount of performance improvement not only depends on buffer size but
also memory alignment.
* Fix tests: C++ impl tests were testing 0-filled buffers so were
always passing.
* Add similar buffer.swap64 method.
* Make buffer-swap benchmark mirror JS impl.
doc/api/buffer.markdown has an entry of "added: REPLACEME" that should
be changed to the correct release number before tagged.
Because node is currently using a very old version of cpplint.py it
doesn't know that std::swap() has moved from <algorithm> to <utility> in
c++11. So until cpplint.py is updated simply NOLINT the line.
Technically it should be NOLINT(build/include_what_you_use), but that
puts the line over 80 characters causing another lint error.
PR-URL: https://github.com/nodejs/node/pull/7157
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Buffer.isEncoding and string_decoder.normalizeEncoding shared
quite a bit of logic. This moves the primary logic into
internal/util. The userland modules that monkey patch Buffer.isEncoding
should still work.
PR-URL: https://github.com/nodejs/node/pull/7207
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Fixes regression where creating a new Buffer from an
empty ArrayBuffer would fail.
Ref: 85ab4a5f12
PR-URL: https://github.com/nodejs/node/pull/7176
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
Reviewed-By: Ron Korving <ron@ronkorving.nl>
This comment applied to a line that was removed in
dd67608bfd
and is no longer relevant.
PR-URL: https://github.com/nodejs/node/pull/7264
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
When node began using the OneByte API (f150d56) it also switched to
officially supporting ISO-8859-1. Though at the time no new encoding
string was introduced.
Introduce the new encoding string 'latin1' to be more explicit. The
previous 'binary' and documented as an alias to 'latin1'. While many
tests have switched to use 'latin1', there are still plenty that do both
'binary' and 'latin1' checks side-by-side to ensure there is no
regression.
PR-URL: https://github.com/nodejs/node/pull/7111
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Improves performance of allocating unsafe buffers, creating buffers from
an existing ArrayBuffer and creating .slice(...) from existing Buffer by
avoiding deoptimizing change of prototype after Uint8Array allocation
in favor of ES6 native subclassing.
This is done through an internal ES6 class that extends Uint8Array and
is used for allocations, but the regular Buffer function is exposed, so
calling Buffer(...) with or without `new` continues to work as usual
and prototype chains are also preserved.
Performance wins for .slice are +120% (2.2x), and, consequently, for
unsafe allocations up to +95% (1.9x) for small buffers, and for safe
allocations (zero-filled) up to +30% (1.3x).
PR-URL: https://github.com/nodejs/node/pull/6893
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
Exceptions thrown from the Uint8Array constructor would leave it
disabled.
Regression introduced in commit 27e84dd ("lib,src: clean up
ArrayBufferAllocator") from two days ago. A follow-up commit
will add a regression test.
PR-URL: https://github.com/nodejs/node/pull/7093
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Remove the direct dependency on node::Environment (which is per-context)
from node::ArrayBufferAllocator (which is per-isolate.)
Contexts that want to toggle the zero fill flag, now do so through a
field that is owned by ArrayBufferAllocator. Better, still not great.
PR-URL: https://github.com/nodejs/node/pull/7082
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Treat negative length arguments to `Buffer()`/`allocUnsafe()`
as if they were zero so the allocation does not affect the
pool’s offset.
Fixes: https://github.com/nodejs/node/issues/7047
PR-URL: https://github.com/nodejs/node/pull/7051
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
This makes sure that `kNoZeroFill` flag is not accidentally set by
moving the all the flag operations directly inside `createBuffer()`.
It safeguards against logical errors like
https://github.com/nodejs/node/issues/6006.
This also ensures that `kNoZeroFill` flag is always restored to 0 using
a try-finally block, as it could be not restored to 0 in cases of failed
or zero-size `Uint8Array` allocation.
It safeguards against errors like
https://github.com/nodejs/node/issues/2930.
It also makes the `size > 0` check not needed there.
PR-URL: https://github.com/nodejs/node-private/pull/30
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
* Remove unnecessary templating from SearchString
SearchString used to have separate PatternChar and SubjectChar template type
arguments, apparently to support things like searching for an 8-bit string
inside a 16-bit string or vice versa. However, SearchString is only used from
node_buffer.cc, where PatternChar and SubjectChar are always the same. Since
this is extra complexity that's unused and untested (simplifying to a single
Char template argument still compiles and didn't break any unit tests), I
removed it.
* Use Boyer-Hoore[-Horspool] for both indexOf and lastIndexOf
Add test cases for lastIndexOf. Test the fallback from BMH to
Boyer-Moore, which looks like it was totally untested before.
* Extra bounds checks in node_buffer.cc
* Extra asserts in string_search.h
* Buffer.lastIndexOf: clean up, enforce consistency w/ String.lastIndexOf
* Polyfill memrchr(3) for non-GNU systems
PR-URL: https://github.com/nodejs/node/pull/4846
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Aligns the functionality of SlowBuffer with the new Buffer
constructor API. Next step is to docs-only deprecate
SlowBuffer.
Replace the internal uses of SlowBuffer with
`Buffer.allocUnsafeSlow(size)`
PR-URL: https://github.com/nodejs/node/pull/5833
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Adds additional `targetStart`, `targetEnd`, `sourceStart,
and `sourceEnd` arguments to `Buffer.prototype.compare`
to allow comparison of sub-ranges of two Buffers without
requiring Buffer.prototype.slice()
Fixes: https://github.com/nodejs/node/issues/521
PR-URL: https://github.com/nodejs/node/pull/5880
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
If `kNoZeroFill` is set here, it won't be reset in case of
pooled allocation. In case of "slow" allocation it will be
set later anyway.
Fixes: https://github.com/nodejs/node/issues/6006
PR-URL: https://github.com/nodejs/node/pull/6007
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Convert anything to string, but Buffer, TypedArray and ArrayBuffer
```
var uint8 = new Uint8Array([0xf0, 0x9f, 0x90]);
Buffer.byteLength(uint8); // should be 3, but returns 11
Buffer.byteLength(uint8.buffer); // should be 3, but return 20
```
PR-URL: https://github.com/nodejs/node/pull/5255
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
When create Buffer from empty string will touch
C++ binding also.
This patch can improve edge case ~70% faster.
PR-URL: https://github.com/nodejs/node/pull/4414
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Using let in for loops showed a regression in 4.4.0. @ofrobots
suggested that we avoid using let in for loops until TurboFan becomes
the default optimiser.
The regression that was detected was when looking at how long it took
to create a new buffer from an array of data.
When using `for (let i=0; i<length; i++) ` we saw the operation take
almost 40% longer compared to `var i=0`.
PR-URL: https://github.com/nodejs/node/pull/5819
Reviewed-By: Benjamin Gruenbaum <benjamingr@gmail.com>
Reviewed-By: Michael Dawson <michael_dawson@ca.ibm.com>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Reviewed-By: Trevor Norris <trevnorris@gmail.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
Ref: http://github.com/nodejs/benchmarking/issues/38
Several changes:
* Soft-Deprecate Buffer() constructors
* Add `Buffer.from()`, `Buffer.alloc()`, and `Buffer.allocUnsafe()`
* Add `--zero-fill-buffers` command line option
* Add byteOffset and length to `new Buffer(arrayBuffer)` constructor
* buffer.fill('') previously had no effect, now zero-fills
* Update the docs
PR-URL: https://github.com/nodejs/node/pull/4682
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Can now call fill() using following parameters if value is a String:
fill(string[, start[, end]][, encoding])
And with the following if value is a Buffer:
fill(buffer[, start[, end]])
The encoding is ignored if value is not a String. All other non-Buffer
values are coerced to a uint32.
A multibyte strings will simply be copied into the Buffer until the
number of bytes run out. Meaning partial strings can be left behind:
Buffer(3).fill('\u0222');
// returns: <Buffer c8 a2 c8>
In some encoding cases, such as 'hex', fill() will throw if the input
string is not valid.
PR-URL: https://github.com/nodejs/node/pull/4935
Reviewed-By: James M Snell <jasnell@gmail.com>
Add fromArrayLike() to handle logic of copying in values from array-like
argument.
PR-URL: https://github.com/nodejs/node/pull/4948
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Without this change, if any of the elements in the list to be concatenated is
not a Buffer instance, the method fails with "buf.copy is not a function".
Make an isBuffer check before using the copy method so that we can throw with
a better message.
Fixes: https://github.com/nodejs/node/issues/4949
PR-URL: https://github.com/nodejs/node/pull/4951
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Roman Klauke <romaaan.git@gmail.com>
A handful of variable declarations in `lib/buffer.js` redeclare the same
variable in the same scope. This change removes each redeclaration by
switching to `const`, switching to `let`, or explicitly hoisting the
`var` declaration.
PR-URL: https://github.com/nodejs/node/pull/4886
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Roman Klauke <romaaan.git@gmail.com>
Currently the signature is indexOf(val[, byteOffset[, encoding]])
Instead allow indexOf(val[, byteOffset][, encoding])
so that byteOffset does not need to be passed.
PR-URL: https://github.com/nodejs/node/pull/4803
Reviewed-By: James M Snell <jasnell@gmail.com>
Make the byteLength work correctly when input is Buffer.
e.g:
```js
// The incomplete unicode string
Buffer.byteLength(new Buffer([0xe4, 0xb8, 0xad, 0xe6, 0x96]))
```
The old output: 9
The new output: 5
PR-URL: https://github.com/nodejs/node/pull/4738
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
These have been deprecated since Apr 27, 2013, and the plan was to
remove them in "node v0.13".
buffer.get(index) is superseded by buffer[index].
buffer.set(index, value) is superseded by buffer[index] = value.
These have never been documented at any point in node's history.
PR-URL: https://github.com/nodejs/node/pull/4594
Fixes: https://github.com/nodejs/node/issues/4587
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: Roman Reiss <me@silverwind.io>
When create Buffer from a Buffer will copy data
from old to new even though length is zero.
This patch can improve edge case 4x faster.
following is benchmark results.
new: buffers/buffer_zero.js n=1024: 2463.53891
old: buffers/buffer_zero.js n=1024: 618.70801
PR-URL: https://github.com/nodejs/node/pull/4326
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Add Buffer#includes() by wrapping an indexOf and performing a strict
equals check to -1.
The includes method takes the search value, byteOffset, and encoding as
arguments.
The test is a modified version of the indexOf test.
Fixes: https://github.com/nodejs/node/issues/3552
PR-URL: https://github.com/nodejs/node/pull/3567
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
The Buffer constructor will generally get inlined, but any call to the Buffer
constructor for a string without encoding will cause an eager deoptimization
of any function that inlined the Buffer constructor. This is due to a an
out-of-bounds read on `arguments[1]`. This change prevents that deopt.
PR-URL: https://github.com/nodejs/node/pull/4158
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: Minwoo Jung <jmwsoft@gmail.com>
The algorithm used to convert negative values to hex generates incorrect
values when the low byte(s) of the value are zero because a carried
subtraction is applied prematurely.
Fixes: https://github.com/nodejs/node/issues/3992
PR-URL: https://github.com/nodejs/node/pull/3994
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
If an undefined encoding is passed to byteLength(),
assume that it is UTF8 immediately. This yields a
small speedup, as it prevents string operations on
the encoding argument.
PR-URL: https://github.com/nodejs/node/pull/4010
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
The type and range checks performed by this function can be done more
efficiently in native code.
PR-URL: https://github.com/nodejs/node/pull/3763
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
This commit fixes some error messages that are not consistent with
some general rules which most of the error messages follow.
PR-URL: https://github.com/nodejs/node/pull/3374
Reviewed-By: Roman Reiss <me@silverwind.io>
Accessing prototype properties directly on a typed array will throw. So
do an extra check in Buffer's own getters to verify it is being called
on an instance.
Fixes: https://github.com/nodejs/node/issues/3297
PR-URL: https://github.com/nodejs/node/pull/3302
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Adds the string search implementation from v8
which uses naive search if pattern length < 8
or to a specific badness then uses Boyer-Moore-Horspool
Added benchmark shows the expected improvements
Added option to use ucs2 encoding with Buffer::IndexOf
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/2539
Instantiating a Buffer of length zero would set the kNoZeroFill flag to
true but never actually call ArrayBuffer::Allocator(). Which means the
flag was never set back to false. The result was that the next
allocation would unconditionally not be zero filled.
Add test to ensure Uint8Array's are zero-filled after creating a Buffer
of length zero. This test may falsely succeed, but will not falsely fail.
Fix: https://github.com/nodejs/node/issues/2930
PR-URL: https://github.com/nodejs/node/pull/2931
Reviewed-By: Rod Vagg <rod@vagg.org>
As `raw` and `raws` encodings are deprecated for such a long time, and
they both are undocumented, this patch removes the support for those
encodings completely.
Previous discussion: https://github.com/nodejs/node/pull/2829
PR-URL: https://github.com/nodejs/node/pull/2859
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Overall construction time of Typed Arrays is faster in JS, but the
problem with using it normally is zero-fill of memory. Get around this
by using a flag in the ArrayBuffer::Allocator to trigger when memory
should or shouldn't be zero-filled.
Remove Buffer::Create() as it is no longer called.
The creation of the Uint8Array() was done at each callsite because at
the time of this patch there was a performance penalty for centralizing
the call in a single function.
PR-URL: https://github.com/nodejs/node/pull/2866
Reviewed-By: Fedor Indutny <fedor@indutny.com>
Fixes a regression that appeared with the new Buffer implementation in v3.
Without this change, calling the SlowBuffer constructor with something else
than a number would abort on the C++ side. This makes sure that the length
argument is coerced to number or is 0.
Fixes: https://github.com/nodejs/node/issues/2634
PR-URL: https://github.com/nodejs/node/pull/2635
Reviewed-By: trevnorris - Trevor Norris <trev.norris@gmail.com>
Reviewed-By: cjihrig - Colin Ihrig <cjihrig@gmail.com>
v8 will silently return an empty handle
which doesn't delete our data if string length is
above String::kMaxLength
Fixes: https://github.com/nodejs/node/issues/1374
PR-URL: https://github.com/nodejs/node/pull/2402
Reviewed-By: trevnorris - Trevor Norris <trev.norris@gmail.com>
Reviewed-By: indutny - Fedor Indutny <fedor.indutny@gmail.com>
Reviewed-By: bnoordhuis - Ben Noordhuis <info@bnoordhuis.nl>
Amended by @rvagg to change author date from
"1970-08-16 16:09:02 +0200"
to
"2015-08-16 16:09:02 +0200"
as per discussion @ https://github.com/nodejs/node/issues/2713
Original commit message:
buffer: align chunks on 8-byte boundary
When slicing global pool - ensure that the underlying buffer's data
ptr is 8-byte alignment to do not ruin expectations of 3rd party C++
addons.
NOTE: 0.10 node.js always returned aligned pointers and io.js should
do this too for compatibility.
PR-URL: https://github.com/nodejs/node/pull/2487
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Zach Bjornson
James M Snell
Ingvar Stepanyan
Nikolai Vavilov
Trevor Norris
Ben Noordhuis
Anna Henningsen
Justin Sprigg
Сковорода Никита Андреевич
dcposch@dcpos.ch
Vladimir Kurchatkin
Jackson Tian
Gareth Ellis
HUANG Wei
Michaël Zasso
Rich Trott
Peter Geiss
Feross Aboukhadijeh
Mathias Buus
Alexander Martin
Matt Loring
Bryce Baril
Peter A. Bigot
Tom Gallacher
micnic
Karl Skomski
Sakthipriyan Vairamani
Fedor Indutny