@ably/delta-codec 中文文档教程

发布于 3年前 浏览 24 项目主页 更新于 3年前

Vcdiff Codec Library for JavaScript

Build Status

支持 Vcdiff delta 格式的 Vcdiff 编解码器库,定义如下 RFC 3284。 这个库支持需要使用 Ably 的增量流而不使用官方 JavaScript 客户端库的开发人员(例如 MQTT上海证券交易所 应用)。

在本文档和代码本身中,我们将 Vcdiff 负载称为“增量”。 在其他地方,此类增量有效负载可能被称为补丁或差异,但为了在该存储库中保持一致,我们坚持使用术语“增量”和“增量”。

Installation from npm for Node.js

npm install @ably/delta-codec

并要求为:

var deltaCodec = require('@ably/delta-codec');

Basic Stream Decoder

VcdiffDecoder 构造函数提供了公共 API 的最基本入口点。 它提供了一种应用 Vcdiff 增量流的有状态方式,在将每个增量应用于先前值后生成新值。

首先提供基值,将使用“setBase”方法在其上应用第一个增量:

let decoder = new deltaCodec.VcdiffDecoder();
decoder.setBase(value);

解码器像这样初始化后,然后使用 applyDelta 方法应用每个后续增量:

let result = decoder.applyDelta(delta);
// TODO call method on result to get the value format you require

< code>decoder 保留当前值,此方法的结果封装了新值,因为 delta 已应用于先前的值。 此方法的 result 提供了允许您以所需格式访问新当前值的方法:

  • asUint8Array(): a Uint8Array, being the new current value, as received (i.e. 'raw' bytes)
  • asUtf8String(): a string, decoded from data using UTF-8
  • asObject(): a JavaScript object, decoded from data using JSON.parse

setBase(value) Instance Method

接受单个 value 参数,即“base”,它可能是ArrayBufferUint8ArrayBufferstring。 如果提供了 string,那么它将在存储为当前值之前由库进行 UTF-8 编码。 这是应用程序指定的自由格式数据。

解码器还公开了另一种方法,setBase64Base(value),其中单个 value 参数必须是 string 并且在之前由库进行 Base64 解码被存储为当前值。

applyDelta(delta) Instance Method

接受单个 delta 参数,它可以是 ArrayBufferUint8ArrayBuffer。 这是一个 Vcdiff 格式增量。

解码器还公开了一个替代方法,applyBase64Delta(delta),其中单个 delta 参数必须是 string 并且在之前由库进行 Base64 解码作为 Vcdiff 格式增量应用到当前值。

isDelta(data) Static Method

接受单个 data 参数,它可以是 ArrayBufferUint8ArrayBuffer。 仅当 data 具有 Vcdiff 增量标头时才返回 true

此方法可用于接收二进制有效负载,以检测它是应解释为绝对值还是应解释为应用于先前值的增量。 如果接收到的有效载荷旁边有元数据可用以指示它们是否为增量,则应避免这种“嗅探”,例如通过 SSE 从 Ably 接收封装数据流时的情况(参见示例)。

Checked Stream Decoder

CheckedVcdiffDecoderVcdiffDecoder 的变体,可以在值和应用于它们的增量具有唯一标识符时使用。 已检查解码器上的“set”和“apply”方法具有相同的名称,但这些标识符需要额外的参数:

  • applyDelta(delta, deltaId, baseId)
  • applyBase64Delta(delta, deltaId, baseId)
  • setBase(value, baseId)
  • setBase64Base(value, baseId)

提供给“set”方法的 baseId 参数和 deltaId 提供给“apply”方法的参数与当前值一起存储,然后与后续“apply”调用中提供的 baseId 参数进行比较。 如果不匹配,将抛出一个 Error

Example Use Cases

Node.js: Text stream from Ably via SSE (enveloped)

默认情况下,从 Ably 接收到的事件数据以 JSON 格式封装。 我们解码此数据并检查 Ably 格式化的消息内容,以确定此消息中的数据是绝对值还是应用于先前接收到的值的增量。

const deltaCodec = require('@ably/delta-codec');
const EventSource = require('eventsource');

const prefix = '[?delta=vcdiff]';
const url = `https://realtime.ably.io/event-stream?channels=${prefix}${CHANNEL_NAME}&v=1.2&key=${APP_KEY}`;
const eventSource = new EventSource(url);
const decoder = new deltaCodec.CheckedVcdiffDecoder();
eventSource.onmessage = function onEventSourceMessage(event) {
  const message = JSON.parse(event.data);
  let value;
  const deltaExtras = (message.extras && message.extras.delta) ? message.extras.delta : null;
  if (deltaExtras) {
    if (deltaExtras.format !== 'vcdiff') {
      throw new Error(`Delta format ${deltaExtras.format} not understood.`);
    }
    value = decoder.applyBase64Delta(message.data, message.id, deltaExtras.from).asUtf8String();
  } else {
    value = message.data;
    decoder.setBase(value, message.id);
  }
  console.log(`received: ${value}`);
};
eventSource.onerror = function onEventSourceError(event) {
  console.log(`error: ${event.data}`);
};

Node.js: Text stream from Ably via SSE (not enveloped)

对于这个例子,我们已经订阅了 Ably 作为我们的事件源,并指定我们不希望入站事件数据被 JSON 封装。 如果没有信封,事件将更小,占用更少的传输带宽,但这意味着我们需要“嗅探”每个入站事件的数据,以确定它是绝对值还是应用到先前接收到的值的增量。

绝对值作为字符串发送给我们,随时可以使用。 增量以 Base64 编码的二进制形式发送给我们。

const deltaCodec = require('@ably/delta-codec');
const EventSource = require('eventsource');

const prefix = '[?delta=vcdiff]';
const url = `https://realtime.ably.io/event-stream?channels=${prefix}${CHANNEL_NAME}&v=1.2&key=${APP_KEY}&enveloped=false`;
const eventSource = new EventSource(url);
const decoder = new deltaCodec.VcdiffDecoder();
eventSource.onmessage = function onEventSourceMessage(event) {
  const stringData = event.data;
  let value;
  if (deltaCodec.VcdiffDecoder.isBase64Delta(stringData)) {
    value = decoder.applyBase64Delta(stringData).asUtf8String();
  } else {
    value = stringData;
    decoder.setBase(value);
  }
  console.log(`received: ${value}`);
};
eventSource.onerror = function onEventSourceError(event) {
  console.log(`error: ${event.data}`);
};

Node.js: Binary stream from Ably via MQTT

通过 MQTT 接收的原始二进制数据没有编码或封装它的其他形式的信封。 我们需要“嗅探”每个入站有效载荷,以确定它是绝对值还是应用于先前接收到的值的增量。 在此示例中,我们正在传输 UTF-8 编码的字符串。

const deltaCodec = require('@ably/delta-codec');
const mqtt = require('mqtt');

const brokerUrl = `mqtts://mqtt.ably.io`;
const options = {
  username: APP_KEY_NAME,
  password: APP_KEY_SECRET,
};
const prefix = '[?delta=vcdiff]';
const client = mqtt.connect(brokerUrl, options);
client.on('connect', () => {
  client.subscribe(`${prefix}${CHANNEL_NAME}`);
});
const decoder = new deltaCodec.VcdiffDecoder();
client.on('message', (topic, message) => {
  let value;
  if (deltaCodec.VcdiffDecoder.isDelta(message)) {
    value = decoder.applyDelta(message).asUtf8String();
  } else {
    decoder.setBase(message);
    value = message.toString();
  }
  console.log(`received: ${value}`);
});

Contributing

Building

您可以使用 Webpack 触发构建:

npm run grunt -- build

dist 文件夹中创建 delta-codec.jsdelta-codec.min.js

Testing

在所有运行时(Node 和浏览器)

npm test

运行测试:在单个运行时运行测试:

  • Node (very quick): npm run grunt -- test:node
  • Local browser (Firefox): npm run grunt -- test:browser:local
  • Remote browsers (Safari, Firefox, Chrome, IE, Edge, Chrome Mobile and Mobile Safari): npm run grunt -- test:browser:remote

已知问题: 在本地浏览器中使用 npm run grunt -- test:browser:local 或通过在 macOS 上使用 npm test 间接进行测试时,您可能会看到“分段错误”。 启动控制台应用程序以查找相关的崩溃报告,其中也会有。 第 7 期中的更多信息。

的远程浏览器测试

支持 您需要为 BROWSERSTACK_USERNAMEBROWSERSTACK_ACCESSKEY 配置环境变量。

Release Procedure

master 分支上:

  1. Increment the version, regenerate from source (a.k.a. build / bundle) and make a tagged commit which includes the built output from the /dist folder by running npm run grunt -- release:patch (or "major", "minor" or "prepatch" as appropriate - see grunt-bump Usage Examples)
  2. Release the tagged commit to Github using git push origin master --follow-tags
  3. Release to NPM using npm publish . --access public (this package is configured to require that 2FA is used by publishers)
  4. Release to Ably's CDN using npm run grunt -- publish-cdn (operable by Ably staff only)
  5. Visit tags and draft new release for the newly created tag

Vcdiff Codec Library for JavaScript

Build Status

A Vcdiff codec library supporting the Vcdiff delta format, as defined by RFC 3284. This library supports developers who need to consume delta streams from Ably without using the official JavaScript client library (e.g. for MQTT and SSE applications).

Throughout this documentation, and within the code itself, we refer to a Vcdiff payload as a 'delta'. Elsewhere such delta payloads may be referred to as patches or diffs, but for consistency within this repository we stick to the terms 'delta' and 'deltas'.

Installation from npm for Node.js

npm install @ably/delta-codec

and require as:

var deltaCodec = require('@ably/delta-codec');

Basic Stream Decoder

The VcdiffDecoder constructor provides the most basic entry point to the public API. It provides a stateful way of applying a stream of Vcdiff deltas, producing a new value after each delta has been applied to the previous value.

First provide the base value, upon which the first delta will be applied using the 'setBase' method:

let decoder = new deltaCodec.VcdiffDecoder();
decoder.setBase(value);

Once the decoder has been initialized like this, then each subsequent delta is applied using the applyDelta method:

let result = decoder.applyDelta(delta);
// TODO call method on result to get the value format you require

The decoder retains the current value, with the result of this method encapsulating that new value now that delta has been applied to the previous value. The result of this method offers methods to allow you to access the new current value in the format you require:

  • asUint8Array(): a Uint8Array, being the new current value, as received (i.e. 'raw' bytes)
  • asUtf8String(): a string, decoded from data using UTF-8
  • asObject(): a JavaScript object, decoded from data using JSON.parse

setBase(value) Instance Method

Accepts a single value argument, the 'base', which may be ArrayBuffer, Uint8Array, Buffer or string. If a string is supplied then it will be UTF-8 encoded by the library before being stored as the current value. This is freeform data, as specified by the application.

The decoder also exposes an alternative method, setBase64Base(value), where the single value argument must be string and is Base64 decoded by the library before being stored as the current value.

applyDelta(delta) Instance Method

Accepts a single delta argument which may be ArrayBuffer, Uint8Array or Buffer. This is a Vcdiff format delta.

The decoder also exposes an alternative method, applyBase64Delta(delta), where the single delta argument must be string and is Base64 decoded by the library before being applied as a Vcdiff format delta to the current value.

isDelta(data) Static Method

Accepts a single data argument which may be ArrayBuffer, Uint8Array or Buffer. Returns true only if data has a Vcdiff delta header.

This method can be used on receipt of a binary payload to detect whether it should be interpreted as an absolute value or as a delta to be applied to the previous value. Such 'sniffing' should be avoided where there is metadata available alongside received payloads to indicate whether they are deltas or not, as is the case when receiving a stream of enveloped data from Ably over SSE (see example).

Checked Stream Decoder

The CheckedVcdiffDecoder is a variant of VcdiffDecoder that can be used when the values and the deltas applied to them have unique identifiers. The 'set' and 'apply' methods on the checked decoder have the same names but require additional arguments for these identifiers:

  • applyDelta(delta, deltaId, baseId)
  • applyBase64Delta(delta, deltaId, baseId)
  • setBase(value, baseId)
  • setBase64Base(value, baseId)

The baseId argument supplied to the 'set' methods and the deltaId arguments supplied to the 'apply' methods are stored alongside the current value and then compared to the baseId argument supplied in subsequent 'apply' calls. An Error is thrown if there is a mismatch.

Example Use Cases

Node.js: Text stream from Ably via SSE (enveloped)

By default the event data received from Ably is enveloped in JSON format. We decode this data and inspect the Ably formatted message contents in order to establish whether the data in this message is an absolute value or a delta to be applied to the previously received value.

const deltaCodec = require('@ably/delta-codec');
const EventSource = require('eventsource');

const prefix = '[?delta=vcdiff]';
const url = `https://realtime.ably.io/event-stream?channels=${prefix}${CHANNEL_NAME}&v=1.2&key=${APP_KEY}`;
const eventSource = new EventSource(url);
const decoder = new deltaCodec.CheckedVcdiffDecoder();
eventSource.onmessage = function onEventSourceMessage(event) {
  const message = JSON.parse(event.data);
  let value;
  const deltaExtras = (message.extras && message.extras.delta) ? message.extras.delta : null;
  if (deltaExtras) {
    if (deltaExtras.format !== 'vcdiff') {
      throw new Error(`Delta format ${deltaExtras.format} not understood.`);
    }
    value = decoder.applyBase64Delta(message.data, message.id, deltaExtras.from).asUtf8String();
  } else {
    value = message.data;
    decoder.setBase(value, message.id);
  }
  console.log(`received: ${value}`);
};
eventSource.onerror = function onEventSourceError(event) {
  console.log(`error: ${event.data}`);
};

Node.js: Text stream from Ably via SSE (not enveloped)

For this example we have subscribed to Ably as our event source and specified that we do not want the inbound event data to be JSON enveloped. Without envelopes the events will be smaller, taking up less transmission bandwidth, however this then means we need 'sniff' each inbound event's data to identify whether it is an absolute value or a delta to be applied to the previously received value.

Absolute values are sent to us as strings, ready to use. Deltas are sent to us as Base64 encoded binary.

const deltaCodec = require('@ably/delta-codec');
const EventSource = require('eventsource');

const prefix = '[?delta=vcdiff]';
const url = `https://realtime.ably.io/event-stream?channels=${prefix}${CHANNEL_NAME}&v=1.2&key=${APP_KEY}&enveloped=false`;
const eventSource = new EventSource(url);
const decoder = new deltaCodec.VcdiffDecoder();
eventSource.onmessage = function onEventSourceMessage(event) {
  const stringData = event.data;
  let value;
  if (deltaCodec.VcdiffDecoder.isBase64Delta(stringData)) {
    value = decoder.applyBase64Delta(stringData).asUtf8String();
  } else {
    value = stringData;
    decoder.setBase(value);
  }
  console.log(`received: ${value}`);
};
eventSource.onerror = function onEventSourceError(event) {
  console.log(`error: ${event.data}`);
};

Node.js: Binary stream from Ably via MQTT

The raw binary data received over MQTT has no encoding or other form of envelope encapsulating it. We need to 'sniff' each inbound payload to identify whether it is an absolute value or a delta to be applied to the previously received value. In this example we are transporting UTF-8 encoded strings.

const deltaCodec = require('@ably/delta-codec');
const mqtt = require('mqtt');

const brokerUrl = `mqtts://mqtt.ably.io`;
const options = {
  username: APP_KEY_NAME,
  password: APP_KEY_SECRET,
};
const prefix = '[?delta=vcdiff]';
const client = mqtt.connect(brokerUrl, options);
client.on('connect', () => {
  client.subscribe(`${prefix}${CHANNEL_NAME}`);
});
const decoder = new deltaCodec.VcdiffDecoder();
client.on('message', (topic, message) => {
  let value;
  if (deltaCodec.VcdiffDecoder.isDelta(message)) {
    value = decoder.applyDelta(message).asUtf8String();
  } else {
    decoder.setBase(message);
    value = message.toString();
  }
  console.log(`received: ${value}`);
});

Contributing

Building

You can trigger a build using Webpack with:

npm run grunt -- build

which creates delta-codec.js and delta-codec.min.js in the dist folder.

Testing

To run tests in all runtimes (Node and browsers):

npm test

To run tests on a single runtime:

  • Node (very quick): npm run grunt -- test:node
  • Local browser (Firefox): npm run grunt -- test:browser:local
  • Remote browsers (Safari, Firefox, Chrome, IE, Edge, Chrome Mobile and Mobile Safari): npm run grunt -- test:browser:remote

Known Issue: When testing in a local browser either using npm run grunt -- test:browser:local or indirectly by using npm test on macOS, you may see a "segmentation fault". Launch the Console app to find the associated crash reports, of which there will be too. More information in issue #7.

Remote browser testing supported by

for which you will need to configure environment variables for BROWSERSTACK_USERNAME and BROWSERSTACK_ACCESSKEY.

Release Procedure

On the master branch:

  1. Increment the version, regenerate from source (a.k.a. build / bundle) and make a tagged commit which includes the built output from the /dist folder by running npm run grunt -- release:patch (or "major", "minor" or "prepatch" as appropriate - see grunt-bump Usage Examples)
  2. Release the tagged commit to Github using git push origin master --follow-tags
  3. Release to NPM using npm publish . --access public (this package is configured to require that 2FA is used by publishers)
  4. Release to Ably's CDN using npm run grunt -- publish-cdn (operable by Ably staff only)
  5. Visit tags and draft new release for the newly created tag
    我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
    原文