When a transform stream's callback is called more than once,
an error is emitted with a somewhat confusing message. This
commit hopes to improve the quality of the error message.
Fixes: https://github.com/nodejs/node/issues/12513
PR-URL: https://github.com/nodejs/node/pull/12520
Reviewed-By: Matteo Collina <matteo.collina@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Adds destroy() and _destroy() methods to Readable, Writable, Duplex
and Transform. It also standardizes the behavior and the implementation
of destroy(), which has been inconsistent in userland and core.
This PR also updates all the subsystems of core to use the new
destroy().
PR-URL: https://github.com/nodejs/node/pull/12925
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Calvin Metcalf <calvin.metcalf@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Improve message when tranform._transform() method is not implemented
Improve error message when Readable._read() is not implemented
Remove extra word in err msg when Writable._write() when not implemented
Remove extra word in err msg when Transform._transform() when not implemented
PR-URL: https://github.com/nodejs/node/pull/8801
Reviewed-By: Rich Trott <rtrott@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
Reviewed-By: Luigi Pinca <luigipinca@gmail.com>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Ilkka Myller <ilkka.myller@nodefield.com>
TransformState has the writeencoding property that gets set on the
first _write. It is not declared when the transform state is initially
constructed and can cause a deopt.
PR-URL: https://github.com/nodejs/node/pull/5032
Reviewed-By: Brian White <mscdex@mscdex.net>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: James M Snell <jasnell@gmail.com>
PR-URL: https://github.com/nodejs/node/pull/4617
Reviewed-By: Roman Reiss <me@silverwind.io>
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Brian White <mscdex@mscdex.net>
This commit fixes some error messages that are not consistent with
some general rules which most of the error messages follow.
PR-URL: https://github.com/nodejs/node/pull/3374
Reviewed-By: Roman Reiss <me@silverwind.io>
Many of the util.is*() methods used to check data types
simply compare against a single value or the result of
typeof. This commit replaces calls to these methods with
equivalent checks. This commit does not touch calls to the
more complex methods (isRegExp(), isDate(), etc.).
Fixes: https://github.com/iojs/io.js/issues/607
PR-URL: https://github.com/iojs/io.js/pull/647
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
This commit replaces a number of var statements throughout
the lib code with const statements.
PR-URL: https://github.com/iojs/io.js/pull/541
Reviewed-By: Ben Noordhuis <info@bnoordhuis.nl>
The copyright and license notice is already in the LICENSE file. There
is no justifiable reason to also require that it be included in every
file, since the individual files are not individually distributed except
as part of the entire package.
Turn on strict mode for the files in the lib/ directory. It helps
catch bugs and can have a positive effect on performance.
PR-URL: https://github.com/node-forward/node/pull/64
Reviewed-By: Colin Ihrig <cjihrig@gmail.com>
Reviewed-By: Fedor Indutny <fedor@indutny.com>
If a transform stream has objectMode = true, it should
allow falsey values other than (null) like 0, false, ''.
null is reserved to indicate stream eof but other falsey
values should flow through properly.
If there is an encoding, and we do 'stream.push(chunk, enc)', and the
encoding argument matches the stated encoding, then we're converting from
a string, to a buffer, and then back to a string. Of course, this is a
completely pointless bit of work, so it's best to avoid it when we know
that we can do so safely.
In synchronous Writable streams (where the _write cb is called on the
current tick), the 'finish' event (and thus the end() callback) can in
some cases be called before all the write() callbacks are called.
Use a counter, and have stream.Transform rely on the 'prefinish' event
instead of the 'finish' event.
This has zero effect on most streams, but it corrects an edge case and
makes it perform more deterministically, which is a Good Thing.
The stall is exposed in the test, though the test itself asserts before
it stalls.
The test is constructed to replicate the stalling state of a complex
Passthrough usecase since I was not able to reliable trigger the stall.
Some of the preconditions for triggering the stall are:
* rs.length >= rs.highWaterMark
* !rs.needReadable
* _transform() handler that can return empty transforms
* multiple sync write() calls
Combined this can trigger a case where rs.reading is not cleared when
further progress requires this. The fix is to always clear rs.reading.
Now that highWaterMark increases when there are large reads, this
greatly reduces the number of calls necessary to _read(size), assuming
that _read actually respects the size argument.
This makes it so that `stream.push(chunk)` is the only way to signal the
end of reading, removing the confusing disparity between the
callback-style _read method, and the fact that most real-world streams
do not have a 1:1 corollation between the "please give me data" event,
and the actual arrival of a chunk of data.
It is still possible, of course, to implement a `CallbackReadable` on
top of this. Simply provide a method like this as the callback:
function readCallback(er, chunk) {
if (er)
stream.emit('error', er);
else
stream.push(chunk);
}
However, *only* fs streams actually would behave in this way, so it
makes not a lot of sense to make TCP, TLS, HTTP, and all the rest have
to bend into this uncomfortable paradigm.
The Readable and Writable classes will nextTick certain things
if in sync mode. The sync flag gets unset after a call to _read
or _write. However, most of these behaviors should also be
deferred until nextTick if no reads have been made (for example,
the automatic '_read up to hwm' behavior on Readable.push(chunk))
Set the sync flag to true in the constructor, so that it will not
trigger an immediate 'readable' event, call to _read, before the
user has had a chance to set a _read method implementation.
It seems like a good idea on the face of it, but lowWaterMarks are
actually not useful, and in practice should always be set to zero.
It would be worthwhile for writers if we actually did some kind of
writev() type of thing, but actually this just delays calling write()
and the overhead of doing a bunch of Buffer copies is not worth the
slight benefit of calling write() fewer times.
The refactor in b43e544140 to use
stream.push() in Transform inadvertently caused it to immediately
consume all the written data, regardless of whether or not the readable
side was being consumed.
Only pull data through the _transform() process when the readable side
is being consumed.
Fix#4667
This also slightly changes the semantics, in that a 'readable'
event may be triggered by the first write() call, even if a
user has not yet called read().
This happens because the Transform _write() handler is calling
read(0) to start the flow of data. Technically, the new behavior
is more 'correct', since it is more in line with the semantics
of the 'readable' event in other streams.
sreepurnajasti
Brian White
cjihrig
Matteo Collina
James M Snell
Madhav Gharmalkar
Italo A. Casas
Jesús Leganés Combarro "piranna
Evan Lucas
zoubin
micnic
Sam Newman
isaacs
Ben Noordhuis
Jeff Barczewski
Rafael Garcia
Gil Pedersen