Buffer.isEncoding and string_decoder.normalizeEncoding shared
quite a bit of logic. This moves the primary logic into
internal/util. The userland modules that monkey patch Buffer.isEncoding
should still work.
PR-URL: https://github.com/nodejs/node/pull/7207
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Fixes regression where creating a new Buffer from an
empty ArrayBuffer would fail.
Ref: 85ab4a5f12
PR-URL: https://github.com/nodejs/node/pull/7176
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
Reviewed-By: Ron Korving <ron@ronkorving.nl>
When node began using the OneByte API (f150d56) it also switched to
officially supporting ISO-8859-1. Though at the time no new encoding
string was introduced.
Introduce the new encoding string 'latin1' to be more explicit. The
previous 'binary' and documented as an alias to 'latin1'. While many
tests have switched to use 'latin1', there are still plenty that do both
'binary' and 'latin1' checks side-by-side to ensure there is no
regression.
PR-URL: https://github.com/nodejs/node/pull/7111
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: James M Snell <jasnell@gmail.com>
The tap skipping output is so prevalent yet obscure in nature that we
ought to move it into it's own function in test/common.js
PR-URL: https://github.com/nodejs/node/pull/6697
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: Santiago Gimeno <santiago.gimeno@gmail.com>
Reviewed-By: Fedor Indutny <fedor.indutny@gmail.com>
In preparation for stricter linting, remove extra spaces.
PR-URL: https://github.com/nodejs/node/pull/6645
Reviewed-By: Michaël Zasso <mic.besace@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: Roman Reiss <me@silverwind.io>
In preparation for a lint rule enforcing function argument alignment,
adjust function arguments to be aligned.
PR-URL: https://github.com/nodejs/node/pull/6390
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: Imran Iqbal <imran@imraniqbal.org>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Ryan Graham <r.m.graham@gmail.com>
In preparation for a lint rule that will enforce
assert.deepStrictEqual() over assert.deepEqual(), change tests and
benchmarks accordingly. For tests and benchmarks that are testing or
benchmarking assert.deepEqual() itself, apply a comment to ignore the
upcoming rule.
PR-URL: https://github.com/nodejs/node/pull/6213
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Correct alignment on variable assignments that span multiple lines in
preparation for lint rule to enforce such alignment.
PR-URL: https://github.com/nodejs/node/pull/6242
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: James M Snell <jasnell@gmail.com>
Aligns the functionality of SlowBuffer with the new Buffer
constructor API. Next step is to docs-only deprecate
SlowBuffer.
Replace the internal uses of SlowBuffer with
`Buffer.allocUnsafeSlow(size)`
PR-URL: https://github.com/nodejs/node/pull/5833
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
ParseArrayIndex() was requesting a Uint32Value(), but assigning it to an
in32_t. This caused slight differences in error message reported in edge
cases of argument parsing. Fixed by getting the IntegerValue() before
checking if the value is < 0. Added test of API that was affected.
PR-URL: https://github.com/nodejs/node/pull/6084
Reviewed-By: James M Snell <jasnell@gmail.com>
Several changes:
* Soft-Deprecate Buffer() constructors
* Add `Buffer.from()`, `Buffer.alloc()`, and `Buffer.allocUnsafe()`
* Add `--zero-fill-buffers` command line option
* Add byteOffset and length to `new Buffer(arrayBuffer)` constructor
* buffer.fill('') previously had no effect, now zero-fills
* Update the docs
PR-URL: https://github.com/nodejs/node/pull/4682
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Add fromArrayLike() to handle logic of copying in values from array-like
argument.
PR-URL: https://github.com/nodejs/node/pull/4948
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Many variables in the buffer tests are redeclared. Change them so that
they are scoped appropriately.
PR-URL: https://github.com/nodejs/node/pull/4893
Reviewed-By: Roman Reiss <me@silverwind.io>
Reviewed-By: James M Snell <jasnell@gmail.com>
The algorithm used to convert negative values to hex generates incorrect
values when the low byte(s) of the value are zero because a carried
subtraction is applied prematurely.
Fixes: https://github.com/nodejs/node/issues/3992
PR-URL: https://github.com/nodejs/node/pull/3994
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Signed-off-by: Peter A. Bigot <pab@pabigot.com>
20285ad177 changed the format
of error messages throughout lib. However, the tests were not
updated to reflect these changes. This commit makes those
changes.
PR-URL: https://github.com/nodejs/node/pull/3727
Reviewed-By: Roman Reiss <me@silverwind.io>
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: James M Snell <jasnell@gmail.com>
malloc(0) and realloc(ptr, 0) have implementation-defined behavior in
that the standard allows them to either return a unique pointer or a
nullptr for zero-sized allocation requests. Normalize by always using
a nullptr.
Fixes: https://github.com/nodejs/node/issues/3496
PR-URL: https://github.com/nodejs/node/pull/3499
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Accessing prototype properties directly on a typed array will throw. So
do an extra check in Buffer's own getters to verify it is being called
on an instance.
Fixes: https://github.com/nodejs/node/issues/3297
PR-URL: https://github.com/nodejs/node/pull/3302
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Jeremiah Senkpiel <fishrock123@rocketmail.com>
Both pointer arguments to memcmp are defined as non-null
and compiler optimizes upon that.
PR-URL: https://github.com/nodejs/node/pull/2544
Reviewed-By: trevnorris - Trevor Norris <trev.norris@gmail.com>
Reviewed-By: thefourtheye - Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
This commit replaces instances of io.js with Node.js, based on the
recent convergence. There are some remaining instances of io.js,
related to build and the installer.
Fixes: https://github.com/nodejs/node/issues/2361
PR-URL: https://github.com/nodejs/node/pull/2367
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: João Reis <reis@janeasystems.com>
The new implementation of Buffer missed the check for null/undefined as
the first argument to new Buffer(). Reintroduce the check and add test.
Fix: e8734c0 "buffer: implement Uint8Array backed Buffer"
Fix: https://github.com/nodejs/io.js/issues/2194
PR-URL: https://github.com/nodejs/io.js/pull/2195
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
Upgrade the bundled V8 and update code in src/ and lib/ to the new API.
Notable backwards incompatible changes are the removal of the smalloc
module and dropped support for CESU-8 decoding. CESU-8 support can be
brought back if necessary by doing UTF-8 decoding ourselves.
This commit includes https://codereview.chromium.org/1192973004 to fix
a build error on python 2.6 systems. The original commit log follows:
Use optparse in js2c.py for python compatibility
Without this change, V8 won't build on RHEL/CentOS 6 because the
distro python is too old to know about the argparse module.
PR-URL: https://github.com/nodejs/io.js/pull/2022
Reviewed-By: Rod Vagg <rod@vagg.org>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
With V8 4.4 removing the external array data API currently used by
Buffer, the new implementation uses the Uint8Array to back Buffer.
Buffers now have a maximum size of Smi::kMaxLength, as defined by V8.
Which is ~2 GB on 64 bit and ~1 GB on 32 bit.
The flag --use-old-buffer allows using the old Buffer implementation.
This flag will be removed once V8 4.4 has landed.
The two JS Buffer implementations have been split into two files for
simplicity.
Use getter to return expected .parent/.offset values for backwards
compatibility.
PR-URL: https://github.com/nodejs/io.js/pull/1825
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
parallel/test-buffer called `Buffer.prototype.toString()` on a buffer
with uninitialized memory. Call `Buffer.prototype.fill()` on it first.
PR-URL: https://github.com/nodejs/io.js/pull/2193
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Buffer.byteLength is important for speed because it is called whenever a
new Buffer is created from a string.
This commit optimizes Buffer.byteLength execution by:
- moving base64 length calculation into JS-land, which is now much
faster
- remove redundant code and streamline the UTF8 length calculation
It also adds a benchmark and better tests.
PR-URL: https://github.com/nodejs/io.js/pull/1713
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Enable linting for the test directory. A number of changes was made so
all tests conform the current rules used by lib and src directories. The
only exception for tests is that unreachable (dead) code is allowed.
test-fs-non-number-arguments-throw had to be excluded from the changes
because of a weird issue on Windows CI.
PR-URL: https://github.com/nodejs/io.js/pull/1721
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Buffer#copy() immediately does a ToObject() on the first argument before
it checks if it's even an Object. This causes
Object::HasIndexedPropertiesInExternalArrayData() to be run on nothing,
triggering the segfault. Instead run HasInstance() on the args Value.
Which will check if it's actually an Object, before checking if it
contains data.
Fixes: https://github.com/iojs/io.js/issues/1519
PR-URL: https://github.com/iojs/io.js/pull/1520
Reviewed-by: Evan Lucas <evanlucas@me.com>
The .parent property of the allocated buffer should remain undefined in
the case that it's not a slice. Also included test to verify this.
PR-URL: https://github.com/iojs/io.js/pull/1109
Reviewed-By: Chris Dickinson <christopher.s.dickinson@gmail.com>
Reviewed-By: Fedor Indutny <fedor@indutny.com>
we had a few ways versions of looking for support before executing a test. this
commit unifies them as well as add the check for all tests that previously
lacked them. found by running `./configure --without-ssl && make test`. also,
produce tap skip output if the test is skipped.
PR-URL: https://github.com/iojs/io.js/pull/1049
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Shigeki Ohtsu <ohtsu@iij.ad.jp>
A number -> uint32 type coercion bug made buffer sizes
larger than kMaxLength (0x3fffffff) wrap around.
Instead of rejecting the requested size with an exception,
the constructor created a buffer with the wrong size.
PR-URL: https://github.com/iojs/io.js/pull/657
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
The copyright and license notice is already in the LICENSE file. There
is no justifiable reason to also require that it be included in every
file, since the individual files are not individually distributed except
as part of the entire package.
Add generic functions for (U)Int read/write operations on Buffers. These
support up to and including 48 bit reads and writes.
Include documentation and tests.
Additional work done by Trevor Norris to include 40 and 48 bit write
support. Because bitwise operations cannot be used on values greater
than 32 bits, the operations have been replaced with mathematical
calculations. Regardless, they are still faster than floating point
operations.
Reviewed-by: Trevor Norris <trev.norris@gmail.com>
Running fill() with an empty string would cause Node to hang
indefinitely. Now it will return without having operated on the buffer.
User facing function has been pulled into JS to perform all initial
value checks and coercions. The C++ method has been placed on the
"internal" object.
Coerced non-string values to numbers to match v0.10 support.
Simplified logic and changed a couple variable names.
Added tests for fill() and moved them all to the beginning of
buffer-test.js since many other tests depend on fill() working properly.
Fixes: https://github.com/joyent/node/issues/8469
Signed-off-by: Trevor Norris <trev.norris@gmail.com>
Creating a new buffer from the toJSON() output of another
buffer does not currently work. This commit adds that
support. Closes#7849.
Signed-off-by: Fedor Indutny <fedor@indutny.com>
Previously v8's WriteUtf8 function would produce invalid utf-8 output
when encountering unmatched surrogate code units [1]. The new
REPLACE_INVALID_UTF8 option fixes that by replacing invalid code points
with the unicode replacement character.
[1]: JS Strings are defined as arrays of 16 bit unsigned integers. There
is no unicode enforcement, so one can easily end up with invalid unicode
code unit sequences inside a string.
A recent change to v8's API now makes it impossible to memcpy to a
v8::ArrayBuffer without causing it to be externalized. This means that
the garbage collector will not automatically free the memory when the
object is collected.
When/If the necessary API is included to allow the above
Buffer#toArrayBuffer() will be reintroduced.
Signed-off-by: Timothy J Fontaine <tjfontaine@gmail.com>
Because of differences in memcmp() implementation, normalize output to
return -1, 0 or 1 only.
Signed-off-by: Timothy J Fontaine <tjfontaine@gmail.com>
compare() works like String.localeCompare such that:
Buffer.compare(a, b) === a.compare(b);
equals() does a native check to see if two buffers are equal.
Signed-off-by: Trevor Norris <trev.norris@gmail.com>
James M Snell
Ingvar Stepanyan
Trevor Norris
Jeremiah Senkpiel
Rich Trott
dcposch@dcpos.ch
HUANG Wei
Michaël Zasso
Sakthipriyan Vairamani
Peter A. Bigot
cjihrig
Ben Noordhuis
Karl Skomski
Roman Reiss
Brendan Ashworth
Frederic Hemberger
Johan Bergström
Vladimir Kurchatkin
isaacs
Fedor Indutny
Yazhong Liu
Felix Geisendörfer
Michael Kebe
Feross Aboukhadijeh
Sean McArthur