@adguard/hostlist-compiler 中文文档教程

Question

Hostlist compiler

这是一个简单的工具，可以更轻松地编译 主机阻止列表 与 AdGuard Home 或任何其他具有 DNS 过滤 的 AdGuard 产品兼容。

• Usage
• Configuration
• Command-line
• API
• Transformations
• RemoveComments
• Compress
• RemoveModifiers
• Validate
• Deduplicate
• InvertAllow
• How to build

Usage

首先，安装 hostlist-compiler：

npm i -g @adguard/hostlist-compiler

准备列表配置（阅读下面的更多信息）并运行编译器：

hostlist-compiler -c configuration.json -o output.txt

Configuration

配置定义您的过滤器列表源，以及应用于源的转换。

以下是此配置的示例：

{
  "name": "List name",
  "description": "List description",
  "homepage": "https://example.org/",
  "license": "GPLv3",
  "sources": [
    {
      "name": "Local rules",
      "source": "rules.txt",
      "type": "adblock",
      "transformations": ["RemoveComments", "Compress"],
      "exclusions": ["excluded rule 1"],
      "exclusions_sources": ["exclusions.txt"],
      "inclusions": ["*"],
      "inclusions_sources": ["inclusions.txt"]
    },
    {
      "name": "Remote rules",
      "source": "https://example.org/rules",
      "type": "hosts",
      "exclusions": ["excluded rule 1"]
    }
  ],
  "transformations": ["Deduplicate", "Compress"],
  "exclusions": ["excluded rule 1", "excluded rule 2"],
  "exclusions_sources": ["global_exclusions.txt"],
  "inclusions": ["*"],
  "inclusions_sources": ["global_inclusions.txt"]
}

• name - (mandatory) the list name.
• description - (optional) the list description.
• homepage - (optional) URL to the list homepage.
• license - (optional) Filter list license.
• sources - (mandatory) array of the list sources.
• .source - (mandatory) path or URL of the source. It can be a traditional filter list or a hosts file.
• .name - (optional) name of the source.
• .type - (optional) type of the source. It could be adblock for Adblock-style lists or hosts for /etc/hosts style lists. If not specified, adblock is assumed.
• .transformations - (optional) a list of transformations to apply to the source rules. By default, no transformations are applied. Learn more about possible transformations here.
• .exclusions - (optional) a list of rules (or wildcards) to exclude from the source.
• .exclusions_sources - (optional) a list of files with exclusions.
• .inclusions - (optional) a list of wildcards to include from the source. All rules that don't match these wildcards won't be included.
• .inclusions_sources - (optional) a list of files with inclusions.
• transformations - (optional) a list of transformations to apply to the final list of rules. By default, no transformations are applied. Learn more about possible transformations here.
• exclusions - (optional) a list of rules (or wildcards) to exclude from the source.
• exclusions_sources - (optional) a list of files with exclusions.
• .inclusions - (optional) a list of wildcards to include from the source. All rules that don't match these wildcards won't be included.
• .inclusions_sources - (optional) a list of files with inclusions.

以下是最小配置的示例：

{
  "name": "test list",
  "sources": [
    {
      "source": "rules.txt"
    }
  ]
}

排除和包含规则

请注意，排除或包含规则可以是纯字符串、通配符或正则表达式。

• plainstring - every rule that contains plainstring will match the rule
• *.plainstring - every rule that matches this wildcard will match the rule
• /regex/ - every rule that matches this regular expression, will match the rule. By default, regular expressions are case-insensitive.
• ! comment - comments will be ignored.

Command-line

命令行参数。

Usage: hostlist-compiler [options]

Options:
  --version      Show version number                                   [boolean]
  --config, -c   Path to the compiler configuration file     [string] [required]
  --output, -o   Path to the output file                     [string] [required]
  --verbose, -v  Run with verbose logging                              [boolean]
  -h, --help     Show help                                             [boolean]

Examples:
  hostlist-compiler -c config.json -o       compile a blocklist and write the
  output.txt                                output to output.txt

API

npm i @adguard/hostlist-compiler

const compile = require("@adguard/hostlist-compiler");

const configuration = {
  name: "test list",
  sources: [
    {
      source:
        "https://adguardteam.github.io/AdGuardSDNSFilter/Filters/filter.txt",
    },
  ],
};

async function main() {
  const compiled = compile(configuration);
}

main();

Transformations

以下是可用转换的完整列表：

1. RemoveComments
2. Compress
3. RemoveModifiers
4. Validate
5. Deduplicate
6. InvertAllow

请注意，这些转换始终按照此处指定的顺序应用。

RemoveComments

这是一个非常简单的转换，它只是删除注释（例如所有以 ! 或 # 开头的规则）。

Compress

重要：此转换将 hosts 列表转换为 adblock 列表。

它的作用如下：

1. It converts all rules to adblock-style rules. For instance, 0.0.0.0 example.org will be converted to ||example.org^.
2. It discards the rules that are now redundant because of other existing rules. For instance, ||example.org blocks example.org and all it's subdomains, therefore additional rules for the subdomains are now redundant.

RemoveModifiers

默认情况下，AdGuard Home 将忽略带有不受支持的修饰符的规则，并且此处列出的所有修饰符均不受支持。 但是，带有这些修饰符的规则可能适用于 DNS 级别的阻止，这就是为什么您可能希望在从传统过滤器列表导入规则时删除它们。

以下是将被删除的修饰符列表：

• $third-party and $3p modifiers
• $document modifier
• $all modifier
• $popup modifier

重要提示：请对此保持谨慎。 从传统广告拦截规则中盲目删除 $third-party 会导致大量误报。 这正是为什么有一个选项可以排除规则的原因——您可能需要使用它。

Validate

如果您使用传统广告拦截器的过滤器列表作为来源，则此转换非常重要。

它从列表中删除危险或不兼容的规则。

所以这就是它的作用：

• Discards domain-specific rules (e.g. ||example.org^$domain=example.com). You don't want to have domain-specific rules working globally.
• Discards rules with unsupported modifiers. Click here to learn more about which modifiers are supported.
• Discards rules that are too short.

如果在无效规则之前有评论，它们也将被删除。

Deduplicate

此转换只是从指定源中删除重复项。

关于这个转换有两个重要的注意事项：

1. It keeps the original rules order.
2. It ignores comments. However, if the comments precede the rule that is being removed, the comments will be also removed.

例如：

! rule1 comment 1
rule1
! rule1 comment 2
rule1

下面是转换后将留下的内容：

! rule1 comment 2
rule1

InvertAllow

这个转换将阻止规则转换为“允许”规则。 请注意，它对 /etc/hosts 规则没有任何作用（除非它们之前通过不同的转换转换为 adblock 样式语法，例如 Compress）。

关于此转换有两个重要注意事项：

1. It keeps the original rules order.
2. It ignores comments, empty lines, /etc/hosts rules and existing "allow" rules.

示例：

原始列表：

! comment 1
rule1

# comment 2
192.168.11.11   test.local
@@rule2

这是应用此转换后的内容：

! comment 1
@@rule1

# comment 2
192.168.11.11   test.local
@@rule2

How to build

• yarn install - installs dependencies
• yarn lint - runs eslint
• yarn test - runs tests
• node src/cli.js -c examples/sdn/configuration.json -o filter.txt - runs compiler with the example configuration