# ![unified][logo]
[![Build Status][travis-badge]][travis]
[![Coverage Status][codecov-badge]][codecov]
**unified** is an interface for processing text using syntax trees.
It’s what powers [**remark**][remark], [**retext**][retext], and
[**rehype**][rehype], but it also allows for processing between
multiple syntaxes.
[`unifiedjs.github.io`][site], the website for **unified** provides a less
technical, more practical, introduction to unified. Make sure to visit it
and try its introductionary [Guides][].
## Installation
[npm][]:
```bash
npm install unified
```
## Usage
```js
var unified = require('unified');
var markdown = require('remark-parse');
var remark2rehype = require('remark-rehype');
var doc = require('rehype-document');
var format = require('rehype-format');
var html = require('rehype-stringify');
var reporter = require('vfile-reporter');
unified()
.use(markdown)
.use(remark2rehype)
.use(doc)
.use(format)
.use(html)
.process('# Hello world!', function (err, file) {
console.error(reporter(err || file));
console.log(String(file));
});
```
Yields:
```html
no issues found
Hello world!
```
## Table of Contents
* [Description](#description)
* [API](#api)
* [processor()](#processor)
* [processor.use(plugin\[, options\])](#processoruseplugin-options)
* [processor.parse(file|value)](#processorparsefilevalue)
* [processor.stringify(node\[, file\])](#processorstringifynode-file)
* [processor.run(node\[, file\]\[, done\])](#processorrunnode-file-done)
* [processor.runSync(node\[, file\])](#processorrunsyncnode-file)
* [processor.process(file|value\[, done\])](#processorprocessfilevalue-done)
* [processor.processSync(file|value)](#processorprocesssyncfilevalue)
* [processor.data(key\[, value\])](#processordatakey-value)
* [processor.freeze()](#processorfreeze)
* [Plugin](#plugin)
* [function attacher(\[options\])](#function-attacheroptions)
* [function transformer(node, file\[, next\])](#function-transformernode-file-next)
* [Preset](#preset)
* [License](#license)
## Description
**unified** is an interface for processing text using syntax trees.
Syntax trees are a representation understandable to programs.
Those programs, called [**plug-in**][plugin]s, take these trees and
modify them, amongst other things. To get to the syntax tree from
input text, there’s a [**parser**][parser], and, to get from that
back to text, there’s a [**compiler**][compiler]. This is the
[**process**][process] of a **processor**.
```ascii
┌──────────────┐
┌─ │ Transformers │ ─┐
▲ └──────────────┘ ▼
└────────┐ ┌────────┘
│ │
┌────────┐ │ │ ┌──────────┐
Input ──▶ │ Parser │ ──▶ Tree ──▶ │ Compiler │ ──▶ Output
└────────┘ └──────────┘
```
###### Processors
Every processor implements another processor. To create a new
processor, invoke another processor. This creates a processor that is
configured to function the same as its ancestor. But, when
the descendant processor is configured in the future, that
configuration does not change the ancestral processor.
Often, when processors are exposed from a library (for example,
unified itself), they should not be configured directly, as that
would change their behaviour for all users. Those processors are
[**frozen**][freeze], and new processors should be made from them before
they are used, by invoking them.
###### Node
The syntax trees used in **unified** are [**Unist**][unist] nodes,
which are plain JavaScript objects with a `type` property. The
semantics of those `type`s are defined by other projects.
There are several [utilities][unist-utilities] for working with these
nodes.
###### List of Processors
The following projects process different syntax trees. They parse
text to their respective syntax tree, and they compile their syntax
trees back to text. These processors can be used as-is, or their
parsers and compilers can be mixed and matched with other plug-ins
to process between different syntaxes.
* [**rehype**][rehype] ([**HAST**][hast]) — HTML
* [**remark**][remark] ([**MDAST**][mdast]) — Markdown
* [**retext**][retext] ([**NLCST**][nlcst]) — Natural language
###### File
When processing documents, metadata is often gathered about that
document. [**VFile**][vfile] is a virtual file format which stores
data, and handles metadata and messages for **unified** and its
plug-ins.
There are several [utilities][vfile-utilities] for working with these
files.
###### Configuration
To configure a processor, invoke its [`use`][use] method, supply it a
[**plug-in**][plugin], and optionally settings.
###### Integrations
**unified** can integrate with the file-system through
[**unified-engine**][engine]. On top of that, CLI apps can be created
with [**unified-args**][args], Gulp plug-ins with
[**unified-engine-gulp**][gulp], and Atom Linters with
[**unified-engine-atom**][atom].
###### Programming interface
The API gives access to processing metadata (such as lint messages), and
supports multiple passed through files:
```js
var unified = require('unified');
var markdown = require('remark-parse');
var styleGuide = require('remark-preset-lint-markdown-style-guide');
var remark2retext = require('remark-retext');
var english = require('retext-english');
var equality = require('retext-equality');
var remark2rehype = require('remark-rehype');
var html = require('rehype-stringify');
var reporter = require('vfile-reporter');
unified()
.use(markdown)
.use(styleGuide)
.use(remark2retext, unified().use(english).use(equality))
.use(remark2rehype)
.use(html)
.process('*Emphasis* and _importance_, you guys!', function (err, file) {
console.error(reporter(err || file));
console.log(String(file));
});
```
Which yields:
```txt
1:16-1:28 warning Emphasis should use `*` as a marker emphasis-marker remark-lint
1:34-1:38 warning `guys` may be insensitive, use `people`, `persons`, `folks` instead gals-men retext-equality
⚠ 2 warnings
Emphasis and importance, you guys!
```
###### Processing between syntaxes
The processors can be combined in two modes.
**Bridge** mode transforms the syntax tree from one flavour (the origin)
to another (the destination). Then, transformations are applied on that
tree. Finally, the origin processor continues transforming the original
syntax tree.
**Mutate** mode also transforms the syntax tree from one flavour to
another. But then the origin processor continues transforming the
destination syntax tree.
In the previous example (“Programming interface”), `remark-retext` is
used in bridge mode: the origin syntax tree is kept after retext is
finished; whereas `remark-rehype` is used in mutate mode: it sets a
new syntax tree and discards the original.
* [**remark-retext**][remark-retext]
* [**remark-rehype**][remark-rehype]
* [**rehype-retext**][rehype-retext]
* [**rehype-remark**][rehype-remark]
## API
### `processor()`
Object describing how to process text.
###### Returns
`Function` — A new [**unfrozen**][freeze] processor which is
configured to function the same as its ancestor. But, when the
descendant processor is configured in the future, that configuration
does not change the ancestral processor.
###### Example
The following example shows how a new processor can be created (from
the remark processor) and linked to **stdin**(4) and **stdout**(4).
```js
var remark = require('remark');
var concat = require('concat-stream');
process.stdin.pipe(concat(function (buf) {
process.stdout.write(remark().processSync(buf).toString());
}));
```
### `processor.use(plugin[, options])`
Configure the processor to use a [**plug-in**][plugin], and configure
that plug-in with optional options.
###### Signatures
* `processor.use(plugin[, options])`
* `processor.use(preset)`
* `processor.use(list)`
###### Parameters
* `plugin` ([`Plugin`][plugin])
* `options` (`*`, optional) — Configuration for `plugin`
* `preset` (`Object`) — Object with an optional `plugins` (set to `list`),
and/or an optional `settings` object
* `list` (`Array`) — plugins, presets, and arguments (a plugin and options
in an array), in an array
###### Returns
`processor` — The processor on which `use` is invoked.
###### Note
`use` cannot be called on [frozen][freeze] processors. Invoke the processor
first to create a new unfrozen processor.
###### Example
There are many ways to pass plugins to `.use()`. The below example
gives an overview.
```js
var unified = require('unified');
unified()
// Plugin with options:
.use(plugin, {})
// Plugins:
.use([plugin, pluginB])
// Two plugins, the second with options:
.use([plugin, [pluginB, {}]])
// Preset with plugins and settings:
.use({plugins: [plugin, [pluginB, {}]], settings: {position: false}})
// Settings only:
.use({settings: {position: false}});
function plugin() {}
function pluginB() {}
```
### `processor.parse(file|value)`
Parse text to a syntax tree.
###### Parameters
* `file` ([**VFile**][file])
— Or anything which can be given to `vfile()`
###### Returns
[**Node**][node] — Syntax tree representation of input.
###### Note
`parse` [freezes][freeze] the processor, if not already frozen.
#### `processor.Parser`
Function handling the parsing of text to a syntax tree. Used in the
[**parse**][parse] phase in the process and invoked with a `string`
and [**VFile**][file] representation of the document to parse.
If `Parser` is a normal parser, it should return a [`Node`][node]: the syntax
tree representation of the given file.
`Parser` can also be a constructor function, in which case it’s invoked with
`new`. In that case, instances should have a `parse` method, which is invoked
(without arguments), and should return a [`Node`][node].
### `processor.stringify(node[, file])`
Compile a syntax tree to text.
###### Parameters
* `node` ([**Node**][node])
* `file` ([**VFile**][file], optional);
— Or anything which can be given to `vfile()`
###### Returns
`string` — String representation of the syntax tree file.
###### Note
`stringify` [freezes][freeze] the processor, if not already frozen.
#### `processor.Compiler`
Function handling the compilation of syntax tree to a text. Used in the
[**stringify**][stringify] phase in the process and invoked with a
[`Node`][node] and [**VFile**][file] representation of the document to
stringify.
If `Compiler` is a normal stringifier, it should return a `string`: the text
representation of the given syntax tree.
`Compiler` can also be a constructor function, in which case it’s invoked with
`new`. In that case, instances should have a `compile` method, which is invoked
(without arguments), and should return a `string`.
### `processor.run(node[, file][, done])`
Transform a syntax tree by applying [**plug-in**][plugin]s to it.
###### Parameters
* `node` ([**Node**][node])
* `file` ([**VFile**][file], optional)
— Or anything which can be given to `vfile()`
* `done` ([`Function`][run-done], optional)
###### Returns
[**Promise**][promise], if `done` is not given. Rejected with an error,
or resolved with the resulting syntax tree.
###### Note
`run` [freezes][freeze] the processor, if not already frozen.
##### `function done(err[, node, file])`
Invoked when transformation is complete. Either invoked with an
error, or a syntax tree and a file.
###### Parameters
* `err` (`Error`) — Fatal error
* `node` ([**Node**][node])
* `file` ([**VFile**][file])
### `processor.runSync(node[, file])`
Transform a syntax tree by applying [**plug-in**][plugin]s to it.
If asynchronous [**plug-in**][plugin]s are configured, an error is thrown.
###### Parameters
* `node` ([**Node**][node])
* `file` ([**VFile**][file], optional)
— Or anything which can be given to `vfile()`
###### Returns
[**Node**][node] — The given syntax tree.
###### Note
`runSync` [freezes][freeze] the processor, if not already frozen.
### `processor.process(file|value[, done])`
Process the given representation of a file as configured on the
processor. The process invokes `parse`, `run`, and `stringify`
internally.
###### Parameters
* `file` ([**VFile**][file])
* `value` (`string`) — String representation of a file
* `done` ([`Function`][process-done], optional)
###### Returns
[**Promise**][promise], if `done` is not given. Rejected with an error,
or resolved with the resulting file.
###### Note
`process` [freezes][freeze] the processor, if not already frozen.
#### `function done(err, file)`
Invoked when the process is complete. Invoked with a fatal error, if
any, and the [**VFile**][file].
###### Parameters
* `err` (`Error`, optional) — Fatal error
* `file` ([**VFile**][file])
###### Example
```js
var unified = require('unified');
var markdown = require('remark-parse');
var remark2rehype = require('remark-rehype');
var doc = require('rehype-document');
var format = require('rehype-format');
var html = require('rehype-stringify');
var reporter = require('vfile-reporter');
unified()
.use(markdown)
.use(remark2rehype)
.use(doc)
.use(format)
.use(html)
.process('# Hello world!')
.then(function (file) {
console.log(String(file));
}, function (err) {
console.error(String(err));
});
```
Yields:
```html
Hello world!
```
### `processor.processSync(file|value)`
Process the given representation of a file as configured on the
processor. The process invokes `parse`, `run`, and `stringify`
internally.
If asynchronous [**plug-in**][plugin]s are configured, an error is thrown.
###### Parameters
* `file` ([**VFile**][file])
* `value` (`string`) — String representation of a file
###### Returns
[**VFile**][file] — Virtual file with modified [`contents`][vfile-contents].
###### Note
`processSync` [freezes][freeze] the processor, if not already frozen.
###### Example
```js
var unified = require('unified');
var markdown = require('remark-parse');
var remark2rehype = require('remark-rehype');
var doc = require('rehype-document');
var format = require('rehype-format');
var html = require('rehype-stringify');
var reporter = require('vfile-reporter');
var processor = unified()
.use(markdown)
.use(remark2rehype)
.use(doc)
.use(format)
.use(html);
console.log(processor.processSync('# Hello world!').toString());
```
Yields:
```html
Hello world!
```
### `processor.data(key[, value])`
Get or set information in an in-memory key-value store accessible to
all phases of the process. An example is a list of HTML elements
which are self-closing (i.e., do not need a closing tag), which is
needed when parsing, transforming, and compiling HTML.
###### Parameters
* `key` (`string`) — Identifier
* `value` (`*`, optional) — Value to set. Omit if getting `key`
###### Returns
* `processor` — If setting, the processor on which `data` is invoked
* `*` — If getting, the value at `key`
###### Note
Setting information with `data` cannot occur on [frozen][freeze] processors.
Invoke the processor first to create a new unfrozen processor.
###### Example
The following example show how to get and set information:
```js
var unified = require('unified');
console.log(unified().data('alpha', 'bravo').data('alpha'))
```
Yields:
```txt
bravo
```
### `processor.freeze()`
Freeze a processor. Frozen processors are meant to be extended, and not to
be configured or processed directly.
Once a processor is frozen, it cannot be unfrozen. But, a new processor
functioning just like it can be created by invoking the processor.
It’s possible to freeze processors explicitly, by calling `.freeze()`, but
[`.parse()`][parse], [`.run()`][run], [`.stringify()`][stringify], and
[`.process()`][process] call `.freeze()` to freeze a processor too.
###### Returns
`Processor` — The processor on which `freeze` is invoked.
###### Example
The following example, `index.js`, shows how [**rehype**][rehype]
prevents extensions to itself:
```js
var unified = require('unified');
var parse = require('rehype-parse');
var stringify = require('rehype-stringify');
module.exports = unified().use(parse).use(stringify).freeze();
```
The below example, `a.js`, shows how that processor can be used and
configured.
```js
var rehype = require('rehype');
var format = require('rehype-format');
// ...
rehype()
.use(format)
// ...
```
The below example, `b.js`, shows a similar looking example which
operates on the frozen [**rehype**][rehype] interface. If this
behaviour was allowed it would result in unexpected behaviour, so
an error is thrown. **This is invalid**:
```js
var rehype = require('rehype');
var format = require('rehype-format');
// ...
rehype
.use(format)
// ...
```
Yields:
```txt
~/node_modules/unified/index.js:440
throw new Error(
^
Error: Cannot invoke `use` on a frozen processor.
Create a new processor first, by invoking it: use `processor()` instead of `processor`.
at assertUnfrozen (~/node_modules/unified/index.js:440:11)
at Function.use (~/node_modules/unified/index.js:172:5)
at Object. (~/b.js:6:4)
```
## `Plugin`
A **unified** plugin changes the way the applied-on processor works,
in the following ways:
* It modifies the [**processor**][processor]: such as changing the
parser, the compiler, or linking the processor to other processors
* It transforms the [**syntax tree**][node] representation of a file
* It modifies metadata of a file
Plug-in’s are a concept which materialise as [**attacher**][attacher]s.
###### Example
`move.js`:
```js
module.exports = move;
function move(options) {
var expected = (options || {}).extname;
if (!expected) {
throw new Error('Missing `extname` in options');
}
return transformer;
function transformer(tree, file) {
if (file.extname && file.extname !== expected) {
file.extname = expected;
}
}
}
```
`index.js`:
```js
var unified = require('unified');
var parse = require('remark-parse');
var remark2rehype = require('remark-rehype');
var stringify = require('rehype-stringify');
var vfile = require('to-vfile');
var reporter = require('vfile-reporter');
var move = require('./move');
unified()
.use(parse)
.use(remark2rehype)
.use(move, {extname: '.html'})
.use(stringify)
.process(vfile.readSync('index.md'), function (err, file) {
console.error(reporter(err || file));
if (file) {
vfile.writeSync(file); // Written to `index.html`.
}
});
```
### `function attacher([options])`
An attacher is the thing passed to [`use`][use]. It configures the
processor and in turn can receive options.
Attachers can configure processors, such as by interacting with parsers
and compilers, linking them to other processors, or by specifying how
the syntax tree is handled.
###### Context
The context object is set to the invoked on [`processor`][processor].
###### Parameters
* `options` (`*`, optional) — Configuration
###### Returns
[`transformer`][transformer] — Optional.
###### Note
Attachers are invoked when the processor is [frozen][freeze]: either when
`.freeze()` is called explicitly, or when [`.parse()`][parse], [`.run()`][run],
[`.stringify()`][stringify], or [`.process()`][process] is called for the first
time.
### `function transformer(node, file[, next])`
Transformers modify the syntax tree or metadata of a file.
A transformer is a function which is invoked each time a file is
passed through the transform phase. If an error occurs (either
because it’s thrown, returned, rejected, or passed to [`next`][next]),
the process stops.
The transformation process in **unified** is handled by [`trough`][trough],
see it’s documentation for the exact semantics of transformers.
###### Parameters
* `node` ([**Node**][node])
* `file` ([**VFile**][file])
* `next` ([`Function`][next], optional)
###### Returns
* `Error` — Can be returned to stop the process
* [**Node**][node] — Can be returned and results in further
transformations and `stringify`s to be performed on the new
tree
* `Promise` — If a promise is returned, the function is asynchronous,
and **must** be resolved (optionally with a [**Node**][node]) or
rejected (optionally with an `Error`)
#### `function next(err[, tree[, file]])`
If the signature of a transformer includes `next` (third argument),
the function **may** finish asynchronous, and **must** invoke `next()`.
###### Parameters
* `err` (`Error`, optional) — Stop the process
* `node` ([**Node**][node], optional) — New syntax tree
* `file` ([**VFile**][file], optional) — New virtual file
## `Preset`
A **unified** preset provides a potentially sharable way to configure
processors. They can contain multiple plugins and optionally settings as
well.
###### Example
`preset.js`:
```js
exports.settings = {bullet: '*', fences: true};
exports.plugins = [
require('remark-preset-lint-recommended'),
require('remark-comment-config'),
require('remark-preset-lint-markdown-style-guide'),
[require('remark-toc'), {maxDepth: 3, tight: true}],
require('remark-github')
];
```
`index.js`:
```js
var remark = require('remark');
var vfile = require('to-vfile');
var reporter = require('vfile-reporter');
var preset = require('./preset');
remark()
.use(preset)
.process(vfile.readSync('index.md'), function (err, file) {
console.error(reporter(err || file));
if (file) {
vfile.writeSync(file);
}
});
```
## License
[MIT][license] © [Titus Wormer][author]
[logo]: https://cdn.rawgit.com/unifiedjs/unified/84f55c8/logo.svg
[travis-badge]: https://img.shields.io/travis/unifiedjs/unified.svg
[travis]: https://travis-ci.org/unifiedjs/unified
[codecov-badge]: https://img.shields.io/codecov/c/github/unifiedjs/unified.svg
[codecov]: https://codecov.io/github/unifiedjs/unified
[npm]: https://docs.npmjs.com/cli/install
[license]: LICENSE
[author]: http://wooorm.com
[site]: https://unifiedjs.github.io
[guides]: https://unifiedjs.github.io/#guides
[rehype]: https://github.com/wooorm/rehype
[remark]: https://github.com/wooorm/remark
[retext]: https://github.com/wooorm/retext
[hast]: https://github.com/syntax-tree/hast
[mdast]: https://github.com/syntax-tree/mdast
[nlcst]: https://github.com/syntax-tree/nlcst
[unist]: https://github.com/syntax-tree/unist
[engine]: https://github.com/unifiedjs/unified-engine
[args]: https://github.com/unifiedjs/unified-args
[gulp]: https://github.com/unifiedjs/unified-engine-gulp
[atom]: https://github.com/unifiedjs/unified-engine-atom
[remark-rehype]: https://github.com/wooorm/remark-rehype
[remark-retext]: https://github.com/wooorm/remark-retext
[rehype-retext]: https://github.com/wooorm/rehype-retext
[rehype-remark]: https://github.com/wooorm/rehype-remark
[unist-utilities]: https://github.com/syntax-tree/unist#list-of-utilities
[vfile]: https://github.com/vfile/vfile
[vfile-contents]: https://github.com/vfile/vfile#vfilecontents
[vfile-utilities]: https://github.com/vfile/vfile#related-tools
[file]: #file
[node]: #node
[processor]: #processor
[process]: #processorprocessfilevalue-done
[parse]: #processorparsefilevalue
[parser]: #processorparser
[stringify]: #processorstringifynode-file
[run]: #processorrunnode-file-done
[compiler]: #processorcompiler
[use]: #processoruseplugin-options
[attacher]: #function-attacheroptions
[transformer]: #function-transformernode-file-next
[next]: #function-nexterr-tree-file
[freeze]: #processorfreeze
[plugin]: #plugin
[run-done]: #function-doneerr-node-file
[process-done]: #function-doneerr-file
[trough]: https://github.com/wooorm/trough#function-fninput-next
[promise]: https://developer.mozilla.org/Web/JavaScript/Reference/Global_Objects/Promise