@ably/delta-codec 中文文档教程
Vcdiff Codec Library for JavaScript
支持 Vcdiff delta 格式的 Vcdiff 编解码器库,定义如下 RFC 3284。 这个库支持需要使用 Ably 的增量流而不使用官方 JavaScript 客户端库的开发人员(例如 MQTT 和 上海证券交易所 应用)。
在本文档和代码本身中,我们将 Vcdiff 负载称为“增量”。 在其他地方,此类增量有效负载可能被称为补丁或差异,但为了在该存储库中保持一致,我们坚持使用术语“增量”和“增量”。
Installation from npm for Node.js
npm install @ably/delta-codec
并要求为:
var deltaCodec = require('@ably/delta-codec');
Basic Stream Decoder
VcdiffDecoder
构造函数提供了公共 API 的最基本入口点。 它提供了一种应用 Vcdiff 增量流的有状态方式,在将每个增量应用于先前值后生成新值。
首先提供基值,将使用“setBase”方法在其上应用第一个增量:
let decoder = new deltaCodec.VcdiffDecoder();
decoder.setBase(value);
解码器像这样初始化后,然后使用 applyDelta
方法应用每个后续增量:
let result = decoder.applyDelta(delta);
// TODO call method on result to get the value format you require
< code>decoder 保留当前值,此方法的结果封装了新值,因为 delta
已应用于先前的值。 此方法的 result
提供了允许您以所需格式访问新当前值的方法:
asUint8Array()
: aUint8Array
, being the new current value, as received (i.e. 'raw' bytes)asUtf8String()
: astring
, decoded from data using UTF-8asObject()
: a JavaScript object, decoded from data usingJSON.parse
setBase(value)
Instance Method
接受单个 value
参数,即“base”,它可能是ArrayBuffer
、Uint8Array
、Buffer
或 string
。 如果提供了 string
,那么它将在存储为当前值之前由库进行 UTF-8 编码。 这是应用程序指定的自由格式数据。
解码器还公开了另一种方法,setBase64Base(value)
,其中单个 value
参数必须是 string
并且在之前由库进行 Base64 解码被存储为当前值。
applyDelta(delta)
Instance Method
接受单个 delta
参数,它可以是 ArrayBuffer
、Uint8Array
或 Buffer
。 这是一个 Vcdiff 格式增量。
解码器还公开了一个替代方法,applyBase64Delta(delta)
,其中单个 delta
参数必须是 string
并且在之前由库进行 Base64 解码作为 Vcdiff 格式增量应用到当前值。
isDelta(data)
Static Method
接受单个 data
参数,它可以是 ArrayBuffer
、Uint8Array
或 Buffer
。 仅当 data
具有 Vcdiff 增量标头时才返回 true
。
此方法可用于接收二进制有效负载,以检测它是应解释为绝对值还是应解释为应用于先前值的增量。 如果接收到的有效载荷旁边有元数据可用以指示它们是否为增量,则应避免这种“嗅探”,例如通过 SSE 从 Ably 接收封装数据流时的情况(参见示例)。
Checked Stream Decoder
CheckedVcdiffDecoder
是 VcdiffDecoder
的变体,可以在值和应用于它们的增量具有唯一标识符时使用。 已检查解码器上的“set”和“apply”方法具有相同的名称,但这些标识符需要额外的参数:
applyDelta(delta, deltaId, baseId)
applyBase64Delta(delta, deltaId, baseId)
setBase(value, baseId)
setBase64Base(value, baseId)
提供给“set”方法的 baseId
参数和 deltaId 提供给“apply”方法的参数与当前值一起存储,然后与后续“apply”调用中提供的
baseId
参数进行比较。 如果不匹配,将抛出一个 Error
。
Example Use Cases
Node.js: Text stream from Ably via SSE (enveloped)
默认情况下,从 Ably 接收到的事件数据以 JSON 格式封装。 我们解码此数据并检查 Ably 格式化的消息内容,以确定此消息中的数据是绝对值还是应用于先前接收到的值的增量。
const deltaCodec = require('@ably/delta-codec');
const EventSource = require('eventsource');
const prefix = '[?delta=vcdiff]';
const url = `https://realtime.ably.io/event-stream?channels=${prefix}${CHANNEL_NAME}&v=1.2&key=${APP_KEY}`;
const eventSource = new EventSource(url);
const decoder = new deltaCodec.CheckedVcdiffDecoder();
eventSource.onmessage = function onEventSourceMessage(event) {
const message = JSON.parse(event.data);
let value;
const deltaExtras = (message.extras && message.extras.delta) ? message.extras.delta : null;
if (deltaExtras) {
if (deltaExtras.format !== 'vcdiff') {
throw new Error(`Delta format ${deltaExtras.format} not understood.`);
}
value = decoder.applyBase64Delta(message.data, message.id, deltaExtras.from).asUtf8String();
} else {
value = message.data;
decoder.setBase(value, message.id);
}
console.log(`received: ${value}`);
};
eventSource.onerror = function onEventSourceError(event) {
console.log(`error: ${event.data}`);
};
Node.js: Text stream from Ably via SSE (not enveloped)
对于这个例子,我们已经订阅了 Ably 作为我们的事件源,并指定我们不希望入站事件数据被 JSON 封装。 如果没有信封,事件将更小,占用更少的传输带宽,但这意味着我们需要“嗅探”每个入站事件的数据,以确定它是绝对值还是应用到先前接收到的值的增量。
绝对值作为字符串发送给我们,随时可以使用。 增量以 Base64 编码的二进制形式发送给我们。
const deltaCodec = require('@ably/delta-codec');
const EventSource = require('eventsource');
const prefix = '[?delta=vcdiff]';
const url = `https://realtime.ably.io/event-stream?channels=${prefix}${CHANNEL_NAME}&v=1.2&key=${APP_KEY}&enveloped=false`;
const eventSource = new EventSource(url);
const decoder = new deltaCodec.VcdiffDecoder();
eventSource.onmessage = function onEventSourceMessage(event) {
const stringData = event.data;
let value;
if (deltaCodec.VcdiffDecoder.isBase64Delta(stringData)) {
value = decoder.applyBase64Delta(stringData).asUtf8String();
} else {
value = stringData;
decoder.setBase(value);
}
console.log(`received: ${value}`);
};
eventSource.onerror = function onEventSourceError(event) {
console.log(`error: ${event.data}`);
};
Node.js: Binary stream from Ably via MQTT
通过 MQTT 接收的原始二进制数据没有编码或封装它的其他形式的信封。 我们需要“嗅探”每个入站有效载荷,以确定它是绝对值还是应用于先前接收到的值的增量。 在此示例中,我们正在传输 UTF-8 编码的字符串。
const deltaCodec = require('@ably/delta-codec');
const mqtt = require('mqtt');
const brokerUrl = `mqtts://mqtt.ably.io`;
const options = {
username: APP_KEY_NAME,
password: APP_KEY_SECRET,
};
const prefix = '[?delta=vcdiff]';
const client = mqtt.connect(brokerUrl, options);
client.on('connect', () => {
client.subscribe(`${prefix}${CHANNEL_NAME}`);
});
const decoder = new deltaCodec.VcdiffDecoder();
client.on('message', (topic, message) => {
let value;
if (deltaCodec.VcdiffDecoder.isDelta(message)) {
value = decoder.applyDelta(message).asUtf8String();
} else {
decoder.setBase(message);
value = message.toString();
}
console.log(`received: ${value}`);
});
Contributing
Building
您可以使用 Webpack 触发构建:
npm run grunt -- build
在 dist
文件夹中创建 delta-codec.js
和 delta-codec.min.js
。
Testing
在所有运行时(Node 和浏览器)
npm test
运行测试:在单个运行时运行测试:
- Node (very quick):
npm run grunt -- test:node
- Local browser (Firefox):
npm run grunt -- test:browser:local
- Remote browsers (Safari, Firefox, Chrome, IE, Edge, Chrome Mobile and Mobile Safari):
npm run grunt -- test:browser:remote
已知问题: 在本地浏览器中使用 npm run grunt -- test:browser:local
或通过在 macOS 上使用 npm test
间接进行测试时,您可能会看到“分段错误”。 启动控制台应用程序以查找相关的崩溃报告,其中也会有。 第 7 期中的更多信息。
的远程浏览器测试
支持 您需要为 BROWSERSTACK_USERNAME
和 BROWSERSTACK_ACCESSKEY
配置环境变量。
Release Procedure
在 master
分支上:
- Increment the version, regenerate from source (a.k.a. build / bundle) and make a tagged commit which includes the built output from the
/dist
folder by runningnpm run grunt -- release:patch
(or "major", "minor" or "prepatch" as appropriate - see grunt-bump Usage Examples) - Release the tagged commit to Github using
git push origin master --follow-tags
- Release to NPM using
npm publish . --access public
(this package is configured to require that 2FA is used by publishers) - Release to Ably's CDN using
npm run grunt -- publish-cdn
(operable by Ably staff only) - Visit tags and draft new release for the newly created tag
Vcdiff Codec Library for JavaScript
A Vcdiff codec library supporting the Vcdiff delta format, as defined by RFC 3284. This library supports developers who need to consume delta streams from Ably without using the official JavaScript client library (e.g. for MQTT and SSE applications).
Throughout this documentation, and within the code itself, we refer to a Vcdiff payload as a 'delta'. Elsewhere such delta payloads may be referred to as patches or diffs, but for consistency within this repository we stick to the terms 'delta' and 'deltas'.
Installation from npm for Node.js
npm install @ably/delta-codec
and require as:
var deltaCodec = require('@ably/delta-codec');
Basic Stream Decoder
The VcdiffDecoder
constructor provides the most basic entry point to the public API. It provides a stateful way of applying a stream of Vcdiff deltas, producing a new value after each delta has been applied to the previous value.
First provide the base value, upon which the first delta will be applied using the 'setBase' method:
let decoder = new deltaCodec.VcdiffDecoder();
decoder.setBase(value);
Once the decoder has been initialized like this, then each subsequent delta is applied using the applyDelta
method:
let result = decoder.applyDelta(delta);
// TODO call method on result to get the value format you require
The decoder
retains the current value, with the result of this method encapsulating that new value now that delta
has been applied to the previous value. The result
of this method offers methods to allow you to access the new current value in the format you require:
asUint8Array()
: aUint8Array
, being the new current value, as received (i.e. 'raw' bytes)asUtf8String()
: astring
, decoded from data using UTF-8asObject()
: a JavaScript object, decoded from data usingJSON.parse
setBase(value)
Instance Method
Accepts a single value
argument, the 'base', which may be ArrayBuffer
, Uint8Array
, Buffer
or string
. If a string
is supplied then it will be UTF-8 encoded by the library before being stored as the current value. This is freeform data, as specified by the application.
The decoder also exposes an alternative method, setBase64Base(value)
, where the single value
argument must be string
and is Base64 decoded by the library before being stored as the current value.
applyDelta(delta)
Instance Method
Accepts a single delta
argument which may be ArrayBuffer
, Uint8Array
or Buffer
. This is a Vcdiff format delta.
The decoder also exposes an alternative method, applyBase64Delta(delta)
, where the single delta
argument must be string
and is Base64 decoded by the library before being applied as a Vcdiff format delta to the current value.
isDelta(data)
Static Method
Accepts a single data
argument which may be ArrayBuffer
, Uint8Array
or Buffer
. Returns true
only if data
has a Vcdiff delta header.
This method can be used on receipt of a binary payload to detect whether it should be interpreted as an absolute value or as a delta to be applied to the previous value. Such 'sniffing' should be avoided where there is metadata available alongside received payloads to indicate whether they are deltas or not, as is the case when receiving a stream of enveloped data from Ably over SSE (see example).
Checked Stream Decoder
The CheckedVcdiffDecoder
is a variant of VcdiffDecoder
that can be used when the values and the deltas applied to them have unique identifiers. The 'set' and 'apply' methods on the checked decoder have the same names but require additional arguments for these identifiers:
applyDelta(delta, deltaId, baseId)
applyBase64Delta(delta, deltaId, baseId)
setBase(value, baseId)
setBase64Base(value, baseId)
The baseId
argument supplied to the 'set' methods and the deltaId
arguments supplied to the 'apply' methods are stored alongside the current value and then compared to the baseId
argument supplied in subsequent 'apply' calls. An Error
is thrown if there is a mismatch.
Example Use Cases
Node.js: Text stream from Ably via SSE (enveloped)
By default the event data received from Ably is enveloped in JSON format. We decode this data and inspect the Ably formatted message contents in order to establish whether the data in this message is an absolute value or a delta to be applied to the previously received value.
const deltaCodec = require('@ably/delta-codec');
const EventSource = require('eventsource');
const prefix = '[?delta=vcdiff]';
const url = `https://realtime.ably.io/event-stream?channels=${prefix}${CHANNEL_NAME}&v=1.2&key=${APP_KEY}`;
const eventSource = new EventSource(url);
const decoder = new deltaCodec.CheckedVcdiffDecoder();
eventSource.onmessage = function onEventSourceMessage(event) {
const message = JSON.parse(event.data);
let value;
const deltaExtras = (message.extras && message.extras.delta) ? message.extras.delta : null;
if (deltaExtras) {
if (deltaExtras.format !== 'vcdiff') {
throw new Error(`Delta format ${deltaExtras.format} not understood.`);
}
value = decoder.applyBase64Delta(message.data, message.id, deltaExtras.from).asUtf8String();
} else {
value = message.data;
decoder.setBase(value, message.id);
}
console.log(`received: ${value}`);
};
eventSource.onerror = function onEventSourceError(event) {
console.log(`error: ${event.data}`);
};
Node.js: Text stream from Ably via SSE (not enveloped)
For this example we have subscribed to Ably as our event source and specified that we do not want the inbound event data to be JSON enveloped. Without envelopes the events will be smaller, taking up less transmission bandwidth, however this then means we need 'sniff' each inbound event's data to identify whether it is an absolute value or a delta to be applied to the previously received value.
Absolute values are sent to us as strings, ready to use. Deltas are sent to us as Base64 encoded binary.
const deltaCodec = require('@ably/delta-codec');
const EventSource = require('eventsource');
const prefix = '[?delta=vcdiff]';
const url = `https://realtime.ably.io/event-stream?channels=${prefix}${CHANNEL_NAME}&v=1.2&key=${APP_KEY}&enveloped=false`;
const eventSource = new EventSource(url);
const decoder = new deltaCodec.VcdiffDecoder();
eventSource.onmessage = function onEventSourceMessage(event) {
const stringData = event.data;
let value;
if (deltaCodec.VcdiffDecoder.isBase64Delta(stringData)) {
value = decoder.applyBase64Delta(stringData).asUtf8String();
} else {
value = stringData;
decoder.setBase(value);
}
console.log(`received: ${value}`);
};
eventSource.onerror = function onEventSourceError(event) {
console.log(`error: ${event.data}`);
};
Node.js: Binary stream from Ably via MQTT
The raw binary data received over MQTT has no encoding or other form of envelope encapsulating it. We need to 'sniff' each inbound payload to identify whether it is an absolute value or a delta to be applied to the previously received value. In this example we are transporting UTF-8 encoded strings.
const deltaCodec = require('@ably/delta-codec');
const mqtt = require('mqtt');
const brokerUrl = `mqtts://mqtt.ably.io`;
const options = {
username: APP_KEY_NAME,
password: APP_KEY_SECRET,
};
const prefix = '[?delta=vcdiff]';
const client = mqtt.connect(brokerUrl, options);
client.on('connect', () => {
client.subscribe(`${prefix}${CHANNEL_NAME}`);
});
const decoder = new deltaCodec.VcdiffDecoder();
client.on('message', (topic, message) => {
let value;
if (deltaCodec.VcdiffDecoder.isDelta(message)) {
value = decoder.applyDelta(message).asUtf8String();
} else {
decoder.setBase(message);
value = message.toString();
}
console.log(`received: ${value}`);
});
Contributing
Building
You can trigger a build using Webpack with:
npm run grunt -- build
which creates delta-codec.js
and delta-codec.min.js
in the dist
folder.
Testing
To run tests in all runtimes (Node and browsers):
npm test
To run tests on a single runtime:
- Node (very quick):
npm run grunt -- test:node
- Local browser (Firefox):
npm run grunt -- test:browser:local
- Remote browsers (Safari, Firefox, Chrome, IE, Edge, Chrome Mobile and Mobile Safari):
npm run grunt -- test:browser:remote
Known Issue: When testing in a local browser either using npm run grunt -- test:browser:local
or indirectly by using npm test
on macOS, you may see a "segmentation fault". Launch the Console app to find the associated crash reports, of which there will be too. More information in issue #7.
Remote browser testing supported by
for which you will need to configure environment variables for BROWSERSTACK_USERNAME
and BROWSERSTACK_ACCESSKEY
.
Release Procedure
On the master
branch:
- Increment the version, regenerate from source (a.k.a. build / bundle) and make a tagged commit which includes the built output from the
/dist
folder by runningnpm run grunt -- release:patch
(or "major", "minor" or "prepatch" as appropriate - see grunt-bump Usage Examples) - Release the tagged commit to Github using
git push origin master --follow-tags
- Release to NPM using
npm publish . --access public
(this package is configured to require that 2FA is used by publishers) - Release to Ably's CDN using
npm run grunt -- publish-cdn
(operable by Ably staff only) - Visit tags and draft new release for the newly created tag