Since 2e568d9 there is a bug where unpiping a stream
from a readable stream that has `_readableState.pipesCount > 1`
will cause it to remove the first stream in the
`_.readableState.pipes` array no matter where in the list the
`dest` stream was.
This patch corrects that problem.
PR-URL: https://github.com/nodejs/node/pull/9171
Fixes: https://github.com/nodejs/node/issues/9170
Reviewed-By: Evan Lucas <evanlucas@me.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Myles Borins <myles.borins@gmail.com>
This is a known de-opt. It may not be 100% necessary in all cases but it
seems like a decent enough idea to avoid it.
PR-URL: https://github.com/nodejs/node/pull/8873
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: Ilkka Myller <ilkka.myller@nodefield.com>
Reviewed-By: Johan Bergström <bugs@bergstroem.nu>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Michaël Zasso <targos@protonmail.com>
Improve message when tranform._transform() method is not implemented
Improve error message when Readable._read() is not implemented
Remove extra word in err msg when Writable._write() when not implemented
Remove extra word in err msg when Transform._transform() when not implemented
PR-URL: https://github.com/nodejs/node/pull/8801
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ilkka Myller <ilkka.myller@nodefield.com>
This removes the cached check for EE.prototype.prependListener
because we can't have nice things. More specifically some
libraries will bundle their own event emitter implementation.
PR-URL: https://github.com/nodejs/node/pull/8018
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
This change is in preparation for lint-enforced brace style.
PR-URL: https://github.com/nodejs/node/pull/7630
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: Michaël Zasso <mic.besace@gmail.com>
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Roman Reiss <me@silverwind.io>
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: Rod Vagg <rod@vagg.org>
Guard against the call to write() inside pipe's ondata pushing more data
back onto the Readable, thus causing ondata to be called again.
This is fine but results in awaitDrain being increased more than once.
The problem with that is when the destination does drain, only a single
'drain' event is emitted, so awaitDrain in this case will never reach
zero and we end up with a permanently paused stream.
Fixes: https://github.com/nodejs/node/issues/7278
PR-URL: https://github.com/nodejs/node/pull/7292
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: Anna Henningsen <anna@addaleax.net>
read() performance is improved most by switching from an array to
a linked list for storing buffered data. However, other changes that
also contribute include: making some hot functions inlinable, faster
read() argument checking, and misc code rearrangement to avoid
unnecessary code execution.
PR-URL: https://github.com/nodejs/node/pull/7077
Reviewed-By: Calvin Metcalf <calvin.metcalf@gmail.com>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reset the `readableState.awaitDrain` counter after manual calls to
`.resume()`.
What might happen otherwise is that a slow consumer at the end of the
pipe could end up stalling the piping in the following scenario:
1. The writable stream indicates that its buffer is full.
2. This leads the readable stream to `pause()` and increase its
`awaitDrain` counter, which will be decreased by the writable’s next
`drain` event.
3. Something calls `.resume()` manually.
4. The readable continues to pipe to the writable, but once again
the writable stream indicates that the buffer is full.
5. The `awaitDrain` counter is thus increased again, but since it has
now been increased twice for a single piping destination, the next
`drain` event will not be able to reset `awaitDrain` to zero.
6. The pipe is stalled and no data is passed along anymore.
The solution in this commit is to reset the `awaitDrain` counter to
zero when `resume()` is called.
Fixes: https://github.com/nodejs/node/issues/7159
PR-URL: https://github.com/nodejs/node/pull/7160
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
A handful of modules (including readable-streams) make
inappropriate use of the internal _events property. One
such use is to prepend an event listener to the front
of the array of listeners.
This adds EE.prototype.prependListener() and
EE.prototype.prependOnceListener() methods to add handlers
to the *front* of the listener array.
Doc update and test case is included.
Fixes: https://github.com/nodejs/node/issues/1817
PR-URL: https://github.com/nodejs/node/pull/6032
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
In 68990948fe (https://github.com/nodejs/node/pull/2325),
the conditions for increasing `readableState.awaitDrain` when
writing to a piping destination returns false were changed so
that they could not actually be met, effectively leaving
`readableState.awaitDrain` with a constant value of 0.
This patch changes the conditions to testing whether the
stream for which `.write()` returned false is still a piping
destination, which was likely the intention of the original
patch.
Fixes: https://github.com/nodejs/node/issues/5820
Fixes: https://github.com/nodejs/node/issues/5257
PR-URL: https://github.com/nodejs/node/pull/6023
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Readable.resume() schedules the resume operation onto the next tick,
whereas pause() has immediate effect. This means that in a sequence
stream.resume();
stream.pause();
.. the 'pause' event will be triggered before the resume operation
is performed.
For process.stdin, we are relying on the 'pause' event to stop reading
on the underlying handle. This fix ensures that reads are started and
stopped in the same order as resume() and pause() are called.
PR-URL: https://github.com/nodejs/node/pull/5776
Reviewed-By: cjihrig - Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: jasnell - James M Snell <jasnell@gmail.com>
Several changes:
* Soft-Deprecate Buffer() constructors
* Add `Buffer.from()`, `Buffer.alloc()`, and `Buffer.allocUnsafe()`
* Add `--zero-fill-buffers` command line option
* Add byteOffset and length to `new Buffer(arrayBuffer)` constructor
* buffer.fill('') previously had no effect, now zero-fills
* Update the docs
PR-URL: https://github.com/nodejs/node/pull/4682
Reviewed-By: Сковорода Никита Андреевич <chalkerx@gmail.com>
Reviewed-By: Stephen Belanger <admin@stephenbelanger.com>
Before this commit, it was possible to push a partial character
to a readable stream where it was decoded as an empty string and
then added to the internal buffer. This caused the stream to not
emit any data, even when the rest of the character bytes were pushed
separately, because of a non-zero length check of the first chunk in
the internal buffer.
Fixes: https://github.com/nodejs/node/issues/5223
PR-URL: https://github.com/nodejs/node/pull/5226
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
`lib/_stream_readable.js` contained three instances of `var`
declarations occurring twice in the same scope. Refactored to `const` or
`let` as appropriate.
PR-URL: https://github.com/nodejs/node/pull/4816
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
ReadableState has the resumeScheduled property that helps determine if
a stream should be resumed. It was not assigned in the constructor.
When stream.resume is called on a readable stream that is not flowing,
it is set to true. This changes the property map of the ReadableState
which can cause a deopt in onEofChunk and needMoreData.
PR-URL: https://github.com/nodejs/node/pull/4761
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
Add a bytesRead property for readable is
useful in some use cases.
When user want know how many bytes read of
readable, need to caculate it in userland.
If encoding is specificed, get the value is
very slowly.
PR-URL: https://github.com/nodejs/node/pull/4372
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
As of 34b535f4c, test-child-process-flush-stdio was failing
on CentOS 5 systems in CI due to the change in stream state
checking in `child_process`. This commit fixes those failures
by making readable streams less eager in setting their readable
flag on EOF.
Fixes: https://github.com/nodejs/node/issues/4125
PR-URL: https://github.com/nodejs/node/pull/4141
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
These properties were initially used to determine stream status
back in node v0.8 and earlier. Since streams2 however, these
properties were *always* true, which can be misleading for
example if you are trying to immediately determine whether
a Writable stream is still writable or not (to avoid a "write after
end" exception).
PR-URL: https://github.com/nodejs/node/pull/4083
Reviewed-By: Chris Dickinson <christopher.s.dickinson@gmail.com>
This commit fixes some error messages that are not consistent with
some general rules which most of the error messages follow.
PR-URL: https://github.com/nodejs/node/pull/3374
Reviewed-By: Roman Reiss <me@silverwind.io>
Avoids doing a buffer.concat on the internal buffer
when that array has only a single thing in it.
Reviewed-By: Chris Dickinson <chris@neversaw.us>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/3300
The `events` module already exports `EventEmitter` constructor function
So, we don't have to use `events.EventEmitter` to access it.
Refer: https://github.com/nodejs/node/pull/2896
PR-URL: https://github.com/nodejs/node/pull/2921
Reviewed-By: Roman Reiss <me@silverwind.io>
Reviewed-By: Michaël Zasso <mic.besace@gmail.com>
Now parts of our public and public-ish APIs fall back to old-style
listenerCount() if the emitter does not have a listenerCount function.
Fixes: https://github.com/nodejs/node/issues/2655
Refs: 8f58fb92ff
PR-URL: https://github.com/nodejs/node/pull/2661
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
roundUpToNextPowerOf2() does more than just rounding up to the next
power of two. Rename it to computeNewHighWaterMark().
PR-URL: https://github.com/nodejs/node/pull/2479
Reviewed-By: Chris Dickinson <christopher.s.dickinson@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
Don't iterate over all 32 bits, use some hacker's delight bit twiddling
to compute the next power of two.
The logic can be reduced to `n = 1 << 32 - Math.clz32(n)` but then it
can't easily be backported to v2.x; Math.clz32() was added in V8 4.3.
PR-URL: https://github.com/nodejs/node/pull/2479
Reviewed-By: Chris Dickinson <christopher.s.dickinson@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
The high watermark is capped at 8 MB, not 128 MB like the comment
in lib/_stream_readable.js said.
PR-URL: https://github.com/nodejs/node/pull/2479
Reviewed-By: Chris Dickinson <christopher.s.dickinson@gmail.com>
Reviewed-By: Sakthipriyan Vairamani <thechargingvolcano@gmail.com>
As per the discussion in #734, this patch deprecates the usage of
`EventEmitter.listenerCount` static function in the docs, and introduces
the `listenerCount` function in the prototype of `EventEmitter` itself.
PR-URL: https://github.com/nodejs/node/pull/2349
Reviewed-By: Trevor Norris <trev.norris@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
Many of the util.is*() methods used to check data types
simply compare against a single value or the result of
typeof. This commit replaces calls to these methods with
equivalent checks. This commit does not touch calls to the
more complex methods (isRegExp(), isDate(), etc.).
Fixes: https://github.com/iojs/io.js/issues/607
PR-URL: https://github.com/iojs/io.js/pull/647
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
This commit replaces a number of var statements throughout
the lib code with const statements.
PR-URL: https://github.com/iojs/io.js/pull/541
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
The copyright and license notice is already in the LICENSE file. There
is no justifiable reason to also require that it be included in every
file, since the individual files are not individually distributed except
as part of the entire package.
Turn on strict mode for the files in the lib/ directory. It helps
catch bugs and can have a positive effect on performance.
PR-URL: https://github.com/node-forward/node/pull/64
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Fedor Indutny <fedor@indutny.com>
net Sockets were calling read(0) to start reading, without
checking to see if they were paused first. This would result
in paused Socket objects keeping the event loop alive.
Fixes#8200
Reviewed-by: Trevor Norris <trev.norris@gmail.com>
A streams1 stream will have its falsy values such as 0, false, or ""
eaten by the upgrade to streams2, even when objectMode is enabled.
Include test for said cases.
Reviewed-by: isaacs <i@izs.me>
Reviewed-by: Trevor Norris <trev.norris@gmail.com>
A ReadableStream with a base64 StringDecoder backed by only
one or two bytes would fail to output its partial data before
ending. This fix adds a check to see if the `read` was triggered
by an internal `flow`, and if so, empties any remaining data.
fixes#7914.
Signed-off-by: Fedor Indutny <fedor@indutny.com>
Switch condition order to check for null before calling isNaN().
Also remove two unnecessary calls to isNaN() that are already
covered by calls to isFinite(). This commit targets v0.10, as
opposed to #7891, which targets master (suggested by
@bnoordhuis). Closes#7840.
Signed-off-by: Fedor Indutny <fedor@indutny.com>
Default highWaterMark is now set properly when using stream Duplex's
writableObjectMode and readableObjectMode options.
Added condition to the already existing split objectMode test to ensure
the highWaterMark is being set to the correct default value on both the
ReadableState and WritableState for readableObjectMode and
writableObjectMode.
Signed-off-by: Fedor Indutny <fedor@indutny.com>
The [Stream documentation for .push](http://nodejs.org/api/stream.html#stream_readable_push_chunk_encoding)
explicitly states multiple times that null is a special cased value
that indicates the end of a stream. It is confusing and undocumented
that undefined *also* ends the stream, even though in object mode
there is a distinct and important difference.
The docs for Object-Mode also explicitly mention null as the *only*
special cased value, making no mention of undefined.
Signed-off-by: Fedor Indutny <fedor@indutny.com>
This commit introduces `readableObjectMode` and
`writableObjectMode` options for Duplex streams.
This can be used mostly to make parsers and
serializers with Transform streams.
Also the docs section about stream state objects
is removed, because it is not relevant anymore.
The example from the section is remade to show
new options.
fixes#6284
Signed-off-by: Timothy J Fontaine <tjfontaine@gmail.com>
Anna Henningsen
Myles Borins
Italo A. Casas
Calvin Metcalf
Rich Trott
David Halls
Brian White
James M Snell
Evan Lucas
Alexis Campailla
cjihrig
Jackson Tian
micnic
Sakthipriyan Vairamani
Jeremiah Senkpiel
Ben Noordhuis
Roman Reiss
Yosuke Furukawa
Yazhong Liu
Trevor Norris
Sam Newman
isaacs
Chris Dickinson
James Halliday
Andrew Oppenlander
Fedor Indutny
Jonathan Reem
Vladimir Kurchatkin