You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.

715 lines
20 KiB

# Buffer
Stability: 3 - Stable
Pure JavaScript is Unicode friendly but not nice to binary data. When
dealing with TCP streams or the file system, it's necessary to handle octet
streams. Node has several strategies for manipulating, creating, and
consuming octet streams.
Raw data is stored in instances of the `Buffer` class. A `Buffer` is similar
to an array of integers but corresponds to a raw memory allocation outside
the V8 heap. A `Buffer` cannot be resized.
The `Buffer` class is a global, making it very rare that one would need
to ever `require('buffer')`.
Converting between Buffers and JavaScript string objects requires an explicit
encoding method. Here are the different string encodings.
* `'ascii'` - for 7 bit ASCII data only. This encoding method is very fast, and
will strip the high bit if set.
* `'utf8'` - Multibyte encoded Unicode characters. Many web pages and other
document formats use UTF-8.
* `'utf16le'` - 2 or 4 bytes, little endian encoded Unicode characters.
Surrogate pairs (U+10000 to U+10FFFF) are supported.
* `'ucs2'` - Alias of `'utf16le'`.
* `'base64'` - Base64 string encoding.
* `'binary'` - A way of encoding raw binary data into strings by using only
the first 8 bits of each character. This encoding method is deprecated and
should be avoided in favor of `Buffer` objects where possible. This encoding
will be removed in future versions of Node.
* `'hex'` - Encode each byte as two hexadecimal characters.
## Class: Buffer
The Buffer class is a global type for dealing with binary data directly.
It can be constructed in a variety of ways.
### new Buffer(size)
* `size` Number
Allocates a new buffer of `size` octets.
### new Buffer(array)
* `array` Array
Allocates a new buffer using an `array` of octets.
### new Buffer(str, [encoding])
* `str` String - string to encode.
* `encoding` String - encoding to use, Optional.
Allocates a new buffer containing the given `str`.
`encoding` defaults to `'utf8'`.
### Class Method: Buffer.isEncoding(encoding)
* `encoding` {String} The encoding string to test
Returns true if the `encoding` is a valid encoding argument, or false
otherwise.
buffer: use smalloc as backing data store Memory allocations are now done through smalloc. The Buffer cc class has been removed completely, but for backwards compatibility have left the namespace as Buffer. The .parent attribute is only set if the Buffer is a slice of an allocation. Which is then set to the alloc object (not a Buffer). The .offset attribute is now a ReadOnly set to 0, for backwards compatibility. I'd like to remove it in the future (pre v1.0). A few alterations have been made to how arguments are either coerced or thrown. All primitives will now be coerced to their respective values, and (most) all out of range index requests will throw. The indexes that are coerced were left for backwards compatibility. For example: Buffer slice operates more like Array slice, and coerces instead of throwing out of range indexes. This may change in the future. The reason for wanting to throw for out of range indexes is because giving js access to raw memory has high potential risk. To mitigate that it's easier to make sure the developer is always quickly alerted to the fact that their code is attempting to access beyond memory bounds. Because SlowBuffer will be deprecated, and simply returns a new Buffer instance, all tests on SlowBuffer have been removed. Heapdumps will now show usage under "smalloc" instead of "Buffer". ParseArrayIndex was added to node_internals to support proper uint argument checking/coercion for external array data indexes. SlabAllocator had to be updated since handle_ no longer exists.
12 years ago
### Class Method: Buffer.isBuffer(obj)
* `obj` Object
* Return: Boolean
Tests if `obj` is a `Buffer`.
### Class Method: Buffer.byteLength(string, [encoding])
* `string` String
* `encoding` String, Optional, Default: 'utf8'
* Return: Number
Gives the actual byte length of a string. `encoding` defaults to `'utf8'`.
This is not the same as `String.prototype.length` since that returns the
number of *characters* in a string.
Example:
str = '\u00bd + \u00bc = \u00be';
console.log(str + ": " + str.length + " characters, " +
Buffer.byteLength(str, 'utf8') + " bytes");
// ½ + ¼ = ¾: 9 characters, 12 bytes
### Class Method: Buffer.concat(list, [totalLength])
* `list` {Array} List of Buffer objects to concat
* `totalLength` {Number} Total length of the buffers when concatenated
Returns a buffer which is the result of concatenating all the buffers in
the list together.
If the list has no items, or if the totalLength is 0, then it returns a
zero-length buffer.
If the list has exactly one item, then the first item of the list is
returned.
If the list has more than one item, then a new Buffer is created.
If totalLength is not provided, it is read from the buffers in the list.
However, this adds an additional loop to the function, so it is faster
to provide the length explicitly.
### buf.length
* Number
The size of the buffer in bytes. Note that this is not necessarily the size
of the contents. `length` refers to the amount of memory allocated for the
buffer object. It does not change when the contents of the buffer are changed.
buf = new Buffer(1234);
console.log(buf.length);
buf.write("some string", 0, "ascii");
console.log(buf.length);
// 1234
// 1234
### buf.write(string, [offset], [length], [encoding])
* `string` String - data to be written to buffer
* `offset` Number, Optional, Default: 0
* `length` Number, Optional, Default: `buffer.length - offset`
* `encoding` String, Optional, Default: 'utf8'
Writes `string` to the buffer at `offset` using the given encoding.
`offset` defaults to `0`, `encoding` defaults to `'utf8'`. `length` is
the number of bytes to write. Returns number of octets written. If `buffer` did
not contain enough space to fit the entire string, it will write a partial
amount of the string. `length` defaults to `buffer.length - offset`.
The method will not write partial characters.
buf = new Buffer(256);
len = buf.write('\u00bd + \u00bc = \u00be', 0);
console.log(len + " bytes: " + buf.toString('utf8', 0, len));
### buf.toString([encoding], [start], [end])
* `encoding` String, Optional, Default: 'utf8'
* `start` Number, Optional, Default: 0
* `end` Number, Optional, Default: `buffer.length`
Decodes and returns a string from buffer data encoded with `encoding`
(defaults to `'utf8'`) beginning at `start` (defaults to `0`) and ending at
`end` (defaults to `buffer.length`).
See `buffer.write()` example, above.
### buf.toJSON()
Returns a JSON-representation of the Buffer instance. `JSON.stringify`
implicitly calls this function when stringifying a Buffer instance.
Example:
var buf = new Buffer('test');
var json = JSON.stringify(buf);
console.log(json);
// '{"type":"Buffer","data":[116,101,115,116]}'
var copy = JSON.parse(json, function(key, value) {
return value && value.type === 'Buffer'
? new Buffer(value.data)
: value;
});
console.log(copy);
// <Buffer 74 65 73 74>
### buf[index]
<!--type=property-->
<!--name=[index]-->
Get and set the octet at `index`. The values refer to individual bytes,
so the legal range is between `0x00` and `0xFF` hex or `0` and `255`.
Example: copy an ASCII string into a buffer, one byte at a time:
str = "node.js";
buf = new Buffer(str.length);
for (var i = 0; i < str.length ; i++) {
buf[i] = str.charCodeAt(i);
}
console.log(buf);
// node.js
### buf.copy(targetBuffer, [targetStart], [sourceStart], [sourceEnd])
* `targetBuffer` Buffer object - Buffer to copy into
* `targetStart` Number, Optional, Default: 0
* `sourceStart` Number, Optional, Default: 0
* `sourceEnd` Number, Optional, Default: `buffer.length`
Does copy between buffers. The source and target regions can be overlapped.
`targetStart` and `sourceStart` default to `0`.
`sourceEnd` defaults to `buffer.length`.
All values passed that are `undefined`/`NaN` or are out of bounds are set equal
to their respective defaults.
Example: build two Buffers, then copy `buf1` from byte 16 through byte 19
into `buf2`, starting at the 8th byte in `buf2`.
buf1 = new Buffer(26);
buf2 = new Buffer(26);
for (var i = 0 ; i < 26 ; i++) {
buf1[i] = i + 97; // 97 is ASCII a
buf2[i] = 33; // ASCII !
}
buf1.copy(buf2, 8, 16, 20);
console.log(buf2.toString('ascii', 0, 25));
// !!!!!!!!qrst!!!!!!!!!!!!!
### buf.slice([start], [end])
* `start` Number, Optional, Default: 0
* `end` Number, Optional, Default: `buffer.length`
Returns a new buffer which references the same memory as the old, but offset
and cropped by the `start` (defaults to `0`) and `end` (defaults to
`buffer.length`) indexes. Negative indexes start from the end of the buffer.
**Modifying the new buffer slice will modify memory in the original buffer!**
Example: build a Buffer with the ASCII alphabet, take a slice, then modify one
byte from the original Buffer.
var buf1 = new Buffer(26);
for (var i = 0 ; i < 26 ; i++) {
buf1[i] = i + 97; // 97 is ASCII a
}
var buf2 = buf1.slice(0, 3);
console.log(buf2.toString('ascii', 0, buf2.length));
buf1[0] = 33;
console.log(buf2.toString('ascii', 0, buf2.length));
// abc
// !bc
### buf.readUInt8(offset, [noAssert])
* `offset` Number
* `noAssert` Boolean, Optional, Default: false
* Return: Number
Reads an unsigned 8 bit integer from the buffer at the specified offset.
Set `noAssert` to true to skip validation of `offset`. This means that `offset`
may be beyond the end of the buffer. Defaults to `false`.
Example:
var buf = new Buffer(4);
buf[0] = 0x3;
buf[1] = 0x4;
buf[2] = 0x23;
buf[3] = 0x42;
for (ii = 0; ii < buf.length; ii++) {
console.log(buf.readUInt8(ii));
}
// 0x3
// 0x4
// 0x23
// 0x42
### buf.readUInt16LE(offset, [noAssert])
### buf.readUInt16BE(offset, [noAssert])
* `offset` Number
* `noAssert` Boolean, Optional, Default: false
* Return: Number
Reads an unsigned 16 bit integer from the buffer at the specified offset with
specified endian format.
Set `noAssert` to true to skip validation of `offset`. This means that `offset`
may be beyond the end of the buffer. Defaults to `false`.
Example:
var buf = new Buffer(4);
buf[0] = 0x3;
buf[1] = 0x4;
buf[2] = 0x23;
buf[3] = 0x42;
console.log(buf.readUInt16BE(0));
console.log(buf.readUInt16LE(0));
console.log(buf.readUInt16BE(1));
console.log(buf.readUInt16LE(1));
console.log(buf.readUInt16BE(2));
console.log(buf.readUInt16LE(2));
// 0x0304
// 0x0403
// 0x0423
// 0x2304
// 0x2342
// 0x4223
### buf.readUInt32LE(offset, [noAssert])
### buf.readUInt32BE(offset, [noAssert])
* `offset` Number
* `noAssert` Boolean, Optional, Default: false
* Return: Number
Reads an unsigned 32 bit integer from the buffer at the specified offset with
specified endian format.
Set `noAssert` to true to skip validation of `offset`. This means that `offset`
may be beyond the end of the buffer. Defaults to `false`.
Example:
var buf = new Buffer(4);
buf[0] = 0x3;
buf[1] = 0x4;
buf[2] = 0x23;
buf[3] = 0x42;
console.log(buf.readUInt32BE(0));
console.log(buf.readUInt32LE(0));
// 0x03042342
// 0x42230403
### buf.readInt8(offset, [noAssert])
* `offset` Number
* `noAssert` Boolean, Optional, Default: false
* Return: Number
Reads a signed 8 bit integer from the buffer at the specified offset.
Set `noAssert` to true to skip validation of `offset`. This means that `offset`
may be beyond the end of the buffer. Defaults to `false`.
Works as `buffer.readUInt8`, except buffer contents are treated as two's
complement signed values.
### buf.readInt16LE(offset, [noAssert])
### buf.readInt16BE(offset, [noAssert])
* `offset` Number
* `noAssert` Boolean, Optional, Default: false
* Return: Number
Reads a signed 16 bit integer from the buffer at the specified offset with
specified endian format.
Set `noAssert` to true to skip validation of `offset`. This means that `offset`
may be beyond the end of the buffer. Defaults to `false`.
Works as `buffer.readUInt16*`, except buffer contents are treated as two's
complement signed values.
### buf.readInt32LE(offset, [noAssert])
### buf.readInt32BE(offset, [noAssert])
* `offset` Number
* `noAssert` Boolean, Optional, Default: false
* Return: Number
Reads a signed 32 bit integer from the buffer at the specified offset with
specified endian format.
Set `noAssert` to true to skip validation of `offset`. This means that `offset`
may be beyond the end of the buffer. Defaults to `false`.
Works as `buffer.readUInt32*`, except buffer contents are treated as two's
complement signed values.
### buf.readFloatLE(offset, [noAssert])
### buf.readFloatBE(offset, [noAssert])
* `offset` Number
* `noAssert` Boolean, Optional, Default: false
* Return: Number
Reads a 32 bit float from the buffer at the specified offset with specified
endian format.
Set `noAssert` to true to skip validation of `offset`. This means that `offset`
may be beyond the end of the buffer. Defaults to `false`.
Example:
var buf = new Buffer(4);
buf[0] = 0x00;
buf[1] = 0x00;
buf[2] = 0x80;
buf[3] = 0x3f;
console.log(buf.readFloatLE(0));
// 0x01
### buf.readDoubleLE(offset, [noAssert])
### buf.readDoubleBE(offset, [noAssert])
* `offset` Number
* `noAssert` Boolean, Optional, Default: false
* Return: Number
Reads a 64 bit double from the buffer at the specified offset with specified
endian format.
Set `noAssert` to true to skip validation of `offset`. This means that `offset`
may be beyond the end of the buffer. Defaults to `false`.
Example:
var buf = new Buffer(8);
buf[0] = 0x55;
buf[1] = 0x55;
buf[2] = 0x55;
buf[3] = 0x55;
buf[4] = 0x55;
buf[5] = 0x55;
buf[6] = 0xd5;
buf[7] = 0x3f;
console.log(buf.readDoubleLE(0));
// 0.3333333333333333
### buf.writeUInt8(value, offset, [noAssert])
* `value` Number
* `offset` Number
* `noAssert` Boolean, Optional, Default: false
Writes `value` to the buffer at the specified offset. Note, `value` must be a
valid unsigned 8 bit integer.
Set `noAssert` to true to skip validation of `value` and `offset`. This means
that `value` may be too large for the specific function and `offset` may be
beyond the end of the buffer leading to the values being silently dropped. This
should not be used unless you are certain of correctness. Defaults to `false`.
Example:
var buf = new Buffer(4);
buf.writeUInt8(0x3, 0);
buf.writeUInt8(0x4, 1);
buf.writeUInt8(0x23, 2);
buf.writeUInt8(0x42, 3);
console.log(buf);
// <Buffer 03 04 23 42>
### buf.writeUInt16LE(value, offset, [noAssert])
### buf.writeUInt16BE(value, offset, [noAssert])
* `value` Number
* `offset` Number
* `noAssert` Boolean, Optional, Default: false
Writes `value` to the buffer at the specified offset with specified endian
format. Note, `value` must be a valid unsigned 16 bit integer.
Set `noAssert` to true to skip validation of `value` and `offset`. This means
that `value` may be too large for the specific function and `offset` may be
beyond the end of the buffer leading to the values being silently dropped. This
should not be used unless you are certain of correctness. Defaults to `false`.
Example:
var buf = new Buffer(4);
buf.writeUInt16BE(0xdead, 0);
buf.writeUInt16BE(0xbeef, 2);
console.log(buf);
buf.writeUInt16LE(0xdead, 0);
buf.writeUInt16LE(0xbeef, 2);
console.log(buf);
// <Buffer de ad be ef>
// <Buffer ad de ef be>
### buf.writeUInt32LE(value, offset, [noAssert])
### buf.writeUInt32BE(value, offset, [noAssert])
* `value` Number
* `offset` Number
* `noAssert` Boolean, Optional, Default: false
Writes `value` to the buffer at the specified offset with specified endian
format. Note, `value` must be a valid unsigned 32 bit integer.
Set `noAssert` to true to skip validation of `value` and `offset`. This means
that `value` may be too large for the specific function and `offset` may be
beyond the end of the buffer leading to the values being silently dropped. This
should not be used unless you are certain of correctness. Defaults to `false`.
Example:
var buf = new Buffer(4);
buf.writeUInt32BE(0xfeedface, 0);
console.log(buf);
buf.writeUInt32LE(0xfeedface, 0);
console.log(buf);
// <Buffer fe ed fa ce>
// <Buffer ce fa ed fe>
### buf.writeInt8(value, offset, [noAssert])
* `value` Number
* `offset` Number
* `noAssert` Boolean, Optional, Default: false
Writes `value` to the buffer at the specified offset. Note, `value` must be a
valid signed 8 bit integer.
Set `noAssert` to true to skip validation of `value` and `offset`. This means
that `value` may be too large for the specific function and `offset` may be
beyond the end of the buffer leading to the values being silently dropped. This
should not be used unless you are certain of correctness. Defaults to `false`.
Works as `buffer.writeUInt8`, except value is written out as a two's complement
signed integer into `buffer`.
### buf.writeInt16LE(value, offset, [noAssert])
### buf.writeInt16BE(value, offset, [noAssert])
* `value` Number
* `offset` Number
* `noAssert` Boolean, Optional, Default: false
Writes `value` to the buffer at the specified offset with specified endian
format. Note, `value` must be a valid signed 16 bit integer.
Set `noAssert` to true to skip validation of `value` and `offset`. This means
that `value` may be too large for the specific function and `offset` may be
beyond the end of the buffer leading to the values being silently dropped. This
should not be used unless you are certain of correctness. Defaults to `false`.
Works as `buffer.writeUInt16*`, except value is written out as a two's
complement signed integer into `buffer`.
### buf.writeInt32LE(value, offset, [noAssert])
### buf.writeInt32BE(value, offset, [noAssert])
* `value` Number
* `offset` Number
* `noAssert` Boolean, Optional, Default: false
Writes `value` to the buffer at the specified offset with specified endian
format. Note, `value` must be a valid signed 32 bit integer.
Set `noAssert` to true to skip validation of `value` and `offset`. This means
that `value` may be too large for the specific function and `offset` may be
beyond the end of the buffer leading to the values being silently dropped. This
should not be used unless you are certain of correctness. Defaults to `false`.
Works as `buffer.writeUInt32*`, except value is written out as a two's
complement signed integer into `buffer`.
### buf.writeFloatLE(value, offset, [noAssert])
### buf.writeFloatBE(value, offset, [noAssert])
* `value` Number
* `offset` Number
* `noAssert` Boolean, Optional, Default: false
Writes `value` to the buffer at the specified offset with specified endian
format. Note, behavior is unspecified if `value` is not a 32 bit float.
Set `noAssert` to true to skip validation of `value` and `offset`. This means
that `value` may be too large for the specific function and `offset` may be
beyond the end of the buffer leading to the values being silently dropped. This
should not be used unless you are certain of correctness. Defaults to `false`.
Example:
var buf = new Buffer(4);
buf.writeFloatBE(0xcafebabe, 0);
console.log(buf);
buf.writeFloatLE(0xcafebabe, 0);
console.log(buf);
// <Buffer 4f 4a fe bb>
// <Buffer bb fe 4a 4f>
### buf.writeDoubleLE(value, offset, [noAssert])
### buf.writeDoubleBE(value, offset, [noAssert])
* `value` Number
* `offset` Number
* `noAssert` Boolean, Optional, Default: false
Writes `value` to the buffer at the specified offset with specified endian
format. Note, `value` must be a valid 64 bit double.
Set `noAssert` to true to skip validation of `value` and `offset`. This means
that `value` may be too large for the specific function and `offset` may be
beyond the end of the buffer leading to the values being silently dropped. This
should not be used unless you are certain of correctness. Defaults to `false`.
Example:
var buf = new Buffer(8);
buf.writeDoubleBE(0xdeadbeefcafebabe, 0);
console.log(buf);
buf.writeDoubleLE(0xdeadbeefcafebabe, 0);
console.log(buf);
// <Buffer 43 eb d5 b7 dd f9 5f d7>
// <Buffer d7 5f f9 dd b7 d5 eb 43>
### buf.fill(value, [offset], [end])
* `value`
* `offset` Number, Optional
* `end` Number, Optional
Fills the buffer with the specified value. If the `offset` (defaults to `0`)
and `end` (defaults to `buffer.length`) are not given it will fill the entire
buffer.
var b = new Buffer(50);
b.fill("h");
### buf.toArrayBuffer()
Creates a new `ArrayBuffer` with the copied memory of the buffer instance.
## buffer.INSPECT_MAX_BYTES
* Number, Default: 50
How many bytes will be returned when `buffer.inspect()` is called. This can
be overridden by user modules.
Note that this is a property on the buffer module returned by
`require('buffer')`, not on the Buffer global, or a buffer instance.
## Class: SlowBuffer
Returns an un-pooled `Buffer`.
In order to avoid the garbage collection overhead of creating many individually
allocated Buffers, by default allocations under 4KB are sliced from a single
larger allocated object. This approach improves both performance and memory
usage since v8 does not need to track and cleanup as many `Persistent` objects.
In the case where a developer may need to retain a small chunk of memory from a
pool for an indeterminate amount of time it may be appropriate to create an
un-pooled Buffer instance using SlowBuffer and copy out the relevant bits.
// need to keep around a few small chunks of memory
var store = [];
socket.on('readable', function() {
var data = socket.read();
// allocate for retained data
var sb = new SlowBuffer(10);
// copy the data into the new allocation
data.copy(sb, 0, 0, 10);
store.push(sb);
});
Though this should used sparingly and only be a last resort *after* a developer
has actively observed undue memory retention in their applications.