It seems like a good idea on the face of it, but lowWaterMarks are
actually not useful, and in practice should always be set to zero.
It would be worthwhile for writers if we actually did some kind of
writev() type of thing, but actually this just delays calling write()
and the overhead of doing a bunch of Buffer copies is not worth the
slight benefit of calling write() fewer times.
In zlibBuffer(), don't wait for the garbage collector to reclaim the zlib memory
but release it manually. Reduces memory consumption by a factor of 10 or more
with some workloads.
Test case:
function f() {
require('zlib').deflate('xxx', g);
}
function g() {
setTimeout(f, 5);
}
f();
Observe RSS memory usage with and without this commit. After 10,000 iterations,
RSS stabilizes at ~35 MB with this commit. Without, RSS is over 300 MB and keeps
growing.
Cause: whenever the JS object heap hits the high-water mark, the V8 GC sweeps
it clean, then tries to grow it in order to avoid more sweeps in the near
future. Rule of thumb: the bigger the JS heap, the lazier the GC can be.
A side effect of a bigger heap is that objects now live longer. This is harmless
in general but it affects zlib context objects because those are tied to large
buffers that live outside the JS heap, on the order of 16K per context object.
Ergo, don't wait for the GC to reclaim the memory - it may take a long time.
Fixes#4172.
In zlibBuffer(), don't wait for the garbage collector to reclaim the zlib memory
but release it manually. Reduces memory consumption by a factor of 10 or more
with some workloads.
Test case:
function f() {
require('zlib').deflate('xxx', g);
}
function g() {
setTimeout(f, 5);
}
f();
Observe RSS memory usage with and without this commit. After 10,000 iterations,
RSS stabilizes at ~35 MB with this commit. Without, RSS is over 300 MB and keeps
growing.
Cause: whenever the JS object heap hits the high-water mark, the V8 GC sweeps
it clean, then tries to grow it in order to avoid more sweeps in the near
future. Rule of thumb: the bigger the JS heap, the lazier the GC can be.
A side effect of a bigger heap is that objects now live longer. This is harmless
in general but it affects zlib context objects because those are tied to large
buffers that live outside the JS heap, on the order of 16K per context object.
Ergo, don't wait for the GC to reclaim the memory - it may take a long time.
Fixes#4172.
This is a combination of 20 commits. Their commit messages are preserved
below for the benefit of future generations.
* Adding a shortcut to easily compress/decompress a string of text.
* Making the API consistent. unzip should accept a Buffer for input as well.
* Adding docs.
* Oops, typo.
* Propagate error through the callback.
* Adding zlib from string tests.
* Typo in test.
* Remove 'end' listeners, and join buffers properly instead of joining them
as a string.
* Oops, needs to be rendered to a string.
* Updated test to include multi-byte characters.
* unzip should return a raw Buffer. Updated docs to reflect.
* And finally updating test.
* EventEmitter.destroy() is a bit more customary
* Revert "EventEmitter.destroy() is a bit more customary"
* Renaming internal methods to "buffer" instead of string.
* Remove the 'error' listeners as well.
* @isaacs: spacing/style, and compress duplicate functions into one
* @isaacs: Update docs
* @isaacs: doc style fix
isaacs
Ben Noordhuis
Dominic Tarr
Andreas Madsen
ssuda
Christian Ress
Nicolas LaCasse
Fedor Indutny
Matt Robenolt