Manipulate Bytes & Buffers in Node.js

In Node.js, a Buffer is a built-in object of fixed length that provides a way to store and work with data, similar to an array in JavaScript.

However, unlike arrays, buffers are more suitable for dealing with binary data for operations such as reading from or writing to non text-based files (e.g., images, audio files) and manipulating low-level network data (e.g., network packets).

Note that while the Buffer class is available within the global scope, it is still recommended to explicitly reference it via a require or import statement.

// CommonJS syntax
const { Buffer } = require('node:buffer');

// ECMAScript syntax
import { Buffer }  from 'node:buffer';

Character encodings

📚 Definition: In programming, character encoding is a system that maps characters to specific values, to ensure that text data is displayed correctly across different systems.

For example, in the UTF-8 encoding, the letter 'A' is encoded as the decimal value 65 (or 41 in hexadecimal).

In Node.js, the Buffer class is a subclass of the Uint8Array class, which means that it can only store 8-bit unsigned integers from 0 to 255.

By default, bytes are encoded in UTF-8 and represented in hexadecimal.

For example, the following buffer contains the hexadecimal values of the UTF-8 encoded string 'hello':

<Buffer 68 65 6c 6c 6f>

Note: Node.js buffers currently support the following strings character encodings:

• 'utf8'
• 'utf16le'
• 'latin1'

And the following binary-to-text encodings:

• 'base64'
• 'base64url'
• 'hex'

Create a new buffer

To create a new zero-filled fixed-size buffer, you can use the Buffer.alloc() static method:

Buffer.alloc(size)

Where size is an integer representing the length of the buffer in bytes.

Example

In this example, we're creating a new empty buffer of 3 bytes:

const { Buffer } = require('node:buffer');

const buf = Buffer.alloc(3);
console.log(buf);

Which will produce this output:

<Buffer 00 00 00>

Create a new Buffer with a pre-filled value

To create a new Buffer with a pre-filled value, you can specify the fill value as second argument of the Buffer.alloc() static method:

Buffer.alloc(size, fill, encoding?)

Where:

• fill is a string whose characters will be repeated in order until they fill the specified size.
• encoding is an optional string describing the character encoding. Defaults to 'utf8'.

Note: Since buffers are designed to store bytes, the fill characters will be automatically converted to their corresponding hexadecimal value.

Example

In this example, we're filling 4 different buffers of same size with 4 strings of different lengths:

const { Buffer } = require('node:buffer');

const buf1 = Buffer.alloc(4, 'a');
const buf2 = Buffer.alloc(4, 'ab');
const buf3 = Buffer.alloc(4, 'abc');
const buf4 = Buffer.alloc(4, 'abcd');

console.log(buf1);
console.log(buf2);
console.log(buf3);
console.log(buf4);

Which will produce this output:

<Buffer 61 61 61 61>
<Buffer 61 62 61 62>
<Buffer 61 62 63 61>
<Buffer 61 62 63 64>

Create a new buffer from a string

To create a new buffer from a string, you can use the Buffer.from() static method:

Buffer.from(string, encoding?)

Where:

• string is the string to encode.
• encoding is an optional string describing the character encoding. Defaults to 'utf8'.

Example

In this example, we're creating a new buffer from the UTF-8-encoded string 'hello':

const { Buffer } = require('node:buffer');

const str = Buffer.from('hello', 'utf8');
console.log(str);

Which will produce this output:

<Buffer 68 65 6c 6c 6f>

Access buffer bytes

Since buffers are a subclass of the Uint8Array class, you can use the square brackets syntax [] to access the value stored at a specific index:

buffer[index]

Where index is either a user-defined number, a variable containing a number, or a computed number.

Note: When trying to access a non-existent buffer index, the returned value will be undefined.

Example

In this example, we're outputting the value of each byte of the buffer named buf:

const { Buffer } = require('node:buffer');

const buf = Buffer.from('hello');

console.log(
  buf[0],
  buf[1],
  buf[2],
  buf[3],
  buf[4]
);

Which will produce this output:

104 101 108 108 111

Which correspond to these values in hexadecimal:

68 65 6c 6c 6f

Get the buffer length

To get the number of bytes in a buffer, you can use its length property:

buffer.length

Modify buffer bytes

To modify the value of a single buffer byte, you can specify its index in brackets [] and use the assignment operator:

buffer[index] = value;

To replace multiple bytes of a buffer at once, you can use the write() method of the buffer instance:

buffer.write(string, offset?, length?, encoding?);

Where:

• string is the string to write into the Buffer.
• offset is an optional integer representing the number of bytes to skip before starting to write. Defaults to 0.
• length is an optional integer representing the maximum number of bytes to write. Defaults to buffer.length - offset.
• encoding is an optional string describing the character encoding. Defaults to "utf8".

Example

In this example, we're replacing the first two bytes of the buf buffer by the hexadecimal values 48 and 61, which correspond to the letter H and a in UTF-8, forming the string 'Hallo' instead of 'Hello':

const { Buffer } = require('node:buffer');

const buf = Buffer.from('hello');

buf[0] = 0x48;
buf[1] = 0x61;

console.log(buf);

Which will produce this output:

<Buffer 48 61 6c 6c 6f>

Example

In this other example, we're using the write() method of the buffer instance to directly replace the first two bytes of the buf buffer by the string "Ha":

const { Buffer } = require('node:buffer');

const buf = Buffer.from('hello');

buf.write('Ha');

console.log(buf);

Which will produce this output:

<Buffer 48 61 6c 6c 6f>

Convert a buffer to a string

To convert a buffer into a string, you can use the toString() method of the buffer instance:

buffer.toString(encoding?)

Where encoding is an optional string describing the character encoding. Defaults to 'utf8'.

Example

In this example, we're creating a new buffer from an hexadecimal string, that we then convert into a latin1-encoded string:

const { Buffer } = require('node:buffer');

const buf = Buffer.from('a1486f6c6121', 'hex');

console.log(buf);
console.log(buf.toString('latin1'));

Which will produce this output:

<Buffer a1 48 6f 6c 61 21>
¡Hola!

Summary

Here's a summary of what you've learned in this lesson:

• A buffer is an instance of the Buffer class that represents an array of fixed length used to store 8-bit unsigned integers.
• The length property is used to get the number of bytes of a buffer.
• The alloc() static method is used to create new buffers of predetermined length.
• The from() static method is used to create new buffers from strings.
• The buffer[index] syntax is used to access specific buffer bytes.
• The write() method is used to rewrite the bytes of a buffer.
• The toString() method is used to convert a buffer to a string.