Question

Consul Template 使用了 Go 的模板语法.如果你对他的语法不熟悉建议你读下文档.他的语法看起来与 Mustache, Handlebars, 或者 Liquid 类似。

在 Go 提供的模板函数之外,Consul Template 暴露了以下的函数:

API 函数

datacenters

查询目录中的所有数据中心.使用以下语法:

{{datacenters}}

file

读取并输出磁盘上的本地文件，如果无法读取产生一个错误.使用如下语法

{{file "/path/to/local/file"}}

这个例子将输出 /path/to/local/file 文件内容到模板。 注意:这不会在嵌套模板中被处理

key

查询 Consul 指定 key 的值，如果 key 的值不能转换为字符串，将产生错误.使用如下语法:

{{key "service/redis/maxconns@east-aws"}}

上面的例子查询了在 east-aws 数据中心的 service/redis/maxconns 的值.如果忽略数据中心参数，将会查询本地数据中心的值:

{{key "service/redis/maxconns"}}

Consul 键值结构的美妙在于，这完全取决于你!

key_or_default

查询 Consul 中指定的 key 的值，如果 key 不存在，则返回默认值.使用方式如下

{{key_or_default "service/redis/maxconns@east-aws" "5"}}

注意 Consul Template 使用了多个阶段的运算.在第一阶段的运算如果 Consul 没有返回值，则会一直使用默认值.后续模板解析中如果值存在了则会读取真实的值.这很重要，运维 Consul Templae 不会因为 key_or_default 没找到 key 而阻塞模板的的渲染.即使 key 存在如果 Consul 没有按时返回这个数据，也会使用默认值来进行替代。

ls

查看 Consul 的所有以指定前缀开头的 key-value 对.如果有值无法转换成字符串则会产生一个错误:

{{range ls "service/redis@east-aws"}}
{{.Key}} {{.Value}}{{end}}

如果 Consul 实例在 east-aws 数据中心存在这个结构 service/redis ,渲染后的模板应该类似这样:

minconns 2
maxconns 12

如果你忽略数据中心属性，则会返回本地数据中心的查询结果。

node

查询目录中的一个节点信息

{{node "node1"}}

如果未指定任何参数，则当前 agent 所在节点将会被返回:

{{node}}

你可以指定一个可选的参数来指定数据中心:

{{node "node1" "@east-aws"}}

如果指定的节点没有找到则会返回 nil .如果节点存在就会列出节点的信息和节点提供的服务。

{{with node}}{{.Node.Node}} ({{.Node.Address}}){{range .Services}}
  {{.Service}} {{.Port}} ({{.Tags | join ","}}){{end}}
{{end}}

nodes

查询目录中的全部节点，使用如下语法

{{nodes}}

这个例子会查询 Consul 的默认数据中心.你可以使用可选参数指定一个可选参数来指定数据中心:

{{nodes "@east-aws"}}

这个例子会查询 east-aws 数据中心的所有几点。

secret

查询 Vault 中指定路径的密匙.如果指定的路径不存在或者 Vault 的 Token 没有足够权限去读取指定的路径，将会产生一个错误.如果路径存在但是 key 不存在则返回 <no value> .

{{with secret "secret/passwords"}}{{.Data.password}}{{end}}

可以使用如下字段:

LeaseID - the unique lease identifier
LeaseDuration - the number of seconds the lease is valid
Renewable - if the secret is renewable
Data - the raw data - this is a map[string]interface{}, so it can be queried using Go's templating "dot notation"
If the map key has dots "." in it, you need to access the value using the index function:

{{index .Data "my.key.with.dots"}}
If additional arguments are passed to the function, then the operation is assumed to be a write operation instead of a read operation. The write operation must return data in order to be valid. This is especially useful for the PKI secret backend, for example.

{{ with secret "pki/issue/my-domain-dot-com" "common_name=foo.example.com" }}
{{ .Data.certificate }}
{{ end }}
The parameters must be key=value pairs, and each pair must be its own argument to the function:

{{ secret "path/" "a=b" "c=d" "e=f" }}
Please always consider the security implications of having the contents of a secret in plain-text on disk. If an attacker is able to get access to the file, they will have access to plain-text secrets.

Please note that Vault does not support blocking queries. As a result, Consul Template will not immediately reload in the event a secret is changed as it does with Consul's key-value store. Consul Template will fetch a new secret at half the lease duration of the original secret. For example, most items in Vault's generic secret backend have a default 30 day lease. This means Consul Template will renew the secret every 15 days. As such, it is recommended that a smaller lease duration be used when generating the initial secret to force Consul Template to renew more often.

secrets

Query Vault to list the secrets at the given path. Please note this requires Vault 0.5+ and the endpoint you want to list secrets must support listing. Not all endpoints support listing. The result is the list of secret names as strings.

{{range secrets "secret/"}}{{.}}{{end}}

The trailing slash is optional in the template, but the generated secret dependency will always have a trailing slash in log output.

To iterate and list over every secret in the generic secret backend in Vault, for example, you would need to do something like this:

{{range secrets "secret/"}}
{{with secret (printf "secret/%s" .)}}
{{range $k, $v := .Data}}
{{$k}}: {{$v}}
{{end}}
{{end}}
{{end}}

You should probably never do this. Please also note that Vault does not support blocking queries. To understand the implications, please read the note at the end of the secret function.

service

查询 Consul 中匹配表达式的服务.语法如下:

{{service "release.web@east-aws"}}

上面的例子查询 Consul 中，在 east-aws 数据中心存在的健康的 web 服务.tag 和数据中心参数是可选的.从当前数据中心查询所有节点的 web 服务而不管 tag,查询语法如下:

{{service "web"}}

这个函数返回 []*HealthService 结构.可按照如下方式应用到模板:

{{range service "web@data center"}}
server {{.Name}} {{.Address}}:{{.Port}}{{end}}

产生如下输出:

server nyc_web_01 123.456.789.10:8080
server nyc_web_02 456.789.101.213:8080

默认值会返回健康的服务，如果你想获取所有服务，可以增加 any 选项，如下:

{{service "web" "any"}}

这样就会返回注册过的所有服务，而不论他的状态如何。

如果你想过滤指定的一个或者多个健康状态，你可以通过逗号隔开多个健康状态:

{{service "web" "passing, warning"}}

这样将会返回被他们的节点和服务级别的检查定义标记为 "passing" 或者 "warning"的服务. 请注意逗号是 OR 而不是 AND 的意思。

指定了超过一个状态过滤，并包含 any 将会返回一个错误.因为 any 是比所有状态更高级的过滤。

后面这 2 种方式有些架构上的不同:

{{service "web"}}
{{service "web" "passing"}}

前者会返回 Consul 认为 healthy 和 passing 的所有服务.后者将返回所有已经在 Consul 注册的服务.然后会执行一个客户端的过滤.通常如果你想获取健康的服务，你应该不要使用 passing 参数，直接忽略第三个参数即可.然而第三个参数在你想查询 passing 或者 warning 的服务会比较有用，如下:

{{service "web" "passing, warning"}}

服务的状态也是可见的，如果你想自己做一些额外的过滤，语法如下:

{{range service "web" "any"}}
{{if eq .Status "critical"}}
// Critical state!{{end}}
{{if eq .Status "passing"}}
// Ok{{end}}

执行命令时，在 Consul 将服务设置为维护模式，只需要在你的命令上包上 Consul 的 maint 调用:

#!/bin/sh
set -e
consul maint -enable -service web -reason "Consul Template updated"
service nginx reload
consul maint -disable -service web

另外如果你没有安装 Consul agent,你可以直接调用 API 请求:

#!/bin/sh
set -e
curl -X PUT "http://$CONSUL_HTTP_ADDR/v1/agent/service/maintenance/web?enable=true&reason=Consul+Template+Updated"
service nginx reload
curl -X PUT "http://$CONSUL_HTTP_ADDR/v1/agent/service/maintenance/web?enable=false"

services

查询 Consul 目录中的所有服务，使用如下语法:

{{services}}

这个例子将查询 Consul 的默认数据中心，你可以指定一个可选参数来指定数据中心:

{{services "@east-aws"}}

请注意: services 函数与 service 是不同的, service 接受更多参数并且查询监控的服务列表.这个查询 Consul 目录并返回一个服务的 tag 的 Map,如下:

{{range services}}
{{.Name}}
{{range .Tags}}
  {{.}}{{end}}
{{end}}

tree

查询所有指定前缀的 key-value 值对，如果其中的值有无法转换为字符串的则引发错误:

{{range tree "service/redis@east-aws"}}
{{.Key}} {{.Value}}{{end}}

如果 Consul 实例在 east-aws 数据中心有一个 service/redis 结构，模板的渲染结果类似下面:

minconns 2
maxconns 12
nested/config/value "value"

和 ls 不同, tree 返回前缀下的所有 key.和 Unix 的 tree 命令比较像.如果忽略数据中心参数，则会使用本地数据中心

帮助函数

byKey

将 tree 返回的 key-value 值对结果创建一个 map,这个 map 根据他们的顶级目录进行组合.例如如果 Consul 的 kv 存储如下结构:

groups/elasticsearch/es1
groups/elasticsearch/es2
groups/elasticsearch/es3
services/elasticsearch/check_elasticsearch
services/elasticsearch/check_indexes

使用下面的模板:

{{range $key, $pairs := tree "groups" | byKey}}{{$key}}:
{{range $pair := $pairs}}  {{.Key}}={{.Value}}
{{end}}{{end}}

结果如下:

elasticsearch:
  es1=1
  es2=1
  es3=1

注意顶部的 key 会被从 key 的值中剥离出来.如果在剥离前缀后没有前缀，值会被从列表移除。

结果的对会被映射为 map,使用可以使用 key 来访问一个单独的值:

{{$weights := tree "weights"}}
{{range service "release.web"}}
  {{$weight := or (index $weights .Node) 100}}
  server {{.Node}} {{.Address}}:{{.Port}} weight {{$weight}}{{end}}

byTag

将被 service 或者 services 函数返回的列表，按照 tag 对服务创建 Map.

{{range $tag, $services := service "web" | byTag}}{{$tag}}
{{range $services}} server {{.Name}} {{.Address}}:{{.Port}}
{{end}}{{end}}

contains

检查目标是否包含在枚举的元素中

{{ if .Tags | contains "production" }}
# ...
{{ end }}
env

读取当前进程可以访问的环境变量。

{{env "CLUSTER_ID"}}

这个函数可以加入管道:

{{env "CLUSTER_ID" | toLower}}

explode

将 tree 或者 ls 的结果转化为深度嵌套的 map,用来进行解析和递归。

{{ tree "config" | explode }}

注意: 解开后，你将丢失所有的关于键值对的元数据。

你可以访问深度嵌套的值:

{{ with tree "config" | explode }}
{{.a.b.c}}{{ end }}

注意: 你需要在 Consul 中保存有一个合理的格式的数据.可以查看 Go 的 text/template 包获取更多信息。

in

检查目标十分在一个可枚举的元素中。

{ if in .Tags "production" }}
# ...
{{ end }}

loop

接受多个参数，行为受这些参数的影响。

如果给 loop 一个数字，他讲返回一个 goroutine ,开始于 0 循环直到等于参数的值:

{{range loop 5}}
# Comment{{end}}

如果给 2 个数字，则这个函数返回一个 goroutine 从第一个数字开始循环直到等于第二个参数的值。

{{range $i := loop 5 8}}
stanza-{{$i}}{{end}}

渲染结果为:

stanza-5
stanza-6
stanza-7

Note: It is not possible to get the index and the element since the function returns a goroutine, not a slice. In other words, the following is not valid:

# Will NOT work!
{{range $i, $e := loop 5 8}}
# ...{{end}}

join

将提供的列表作为管道与提供的字符串连接:

{{$items | join ","}}

trimSpace

对输入的内容移除掉空白,tab 和换行符

{ file "/etc/ec2_version"| trimSpace }}

parseBool

将给定的字符串解析为布尔值:

{{"true" | parseBool}}

这个可以与一个 key 检查和条件检查相结合.如下:

{{if key "feature/enabled" | parseBool}}{{end}}

parseFloat

将给定的字符串解析为 10 进制 float64 类型数字:

{{"1.2" | parseFloat}}

parseInt

将给定字符串解析为 10 禁止 int64 类型数字:

{{"1" | parseInt}}

这个可以与其他的帮助函数结合使用，例如:

{{range $i := loop key "config/pool_size" | parseInt}}
# ...{{end}}

parseJSON

将输入，通常是通过 key 获取的值，解析成 JSON

Takes the given input (usually the value from a key) and parses the result as JSON:

{{with $d := key "user/info" | parseJSON}}{{$d.name}}{{end}}

注意 : Consul Template 计算模板很多次.第一次计算时会是空，因为数据还未载入，这意味着我们需要检查空的响应.例如:

{{with $d := key "user/info" | parseJSON}}
{{if $d}}
...
{{end}}
{{end}}

它只适用简单的 key,但是如果你想遍历 key 或者使用 index 函数会失败.将要访问的代码包含在 {{ if $d }}...{{end}} 之中就够了。

Alternatively you can read data from a local JSON file:

{{with $d := file "/path/to/local/data.json" | parseJSON}}{{$d.some_key}}{{end}}

parseUint

Takes the given string and parses it as a base-10 int64:

{{"1" | parseUint}}

See parseInt for examples.

plugin

Takes the name of a plugin and optional payload and executes a Consul Template plugin.

{{plugin "my-plugin"}}

This is most commonly combined with a JSON filter for customization:

{{tree "foo" | explode | toJSON | plugin "my-plugin"}}

Please see the plugins section for more information about plugins.

regexMatch

Takes the argument as a regular expression and will return true if it matches on the given string, or false otherwise.

` {{"foo.bar" | regexMatch "foo([.a-z]+)"}}

regexReplaceAll

Takes the argument as a regular expression and replaces all occurrences of the regex with the given string. As in go, you can use variables like $1 to refer to subexpressions in the replacement string.

{{"foo.bar" | regexReplaceAll "foo([.a-z]+)" "$1"}}

replaceAll

Takes the argument as a string and replaces all occurrences of the given string with the given string.

{{"foo.bar" | replaceAll "." "_"}}

This function can be chained with other functions as well:

{{service "web"}}{{.Name | replaceAll ":" "_"}}{{end}}

split

Splits the given string on the provided separator:

{{"foo\nbar\n" | split "\n"}}

This can be combined with chained and piped with other functions:

{{key "foo" | toUpper | split "\n" | join ","}}

timestamp

Returns the current timestamp as a string (UTC). If no arguments are given, the result is the current RFC3339 timestamp:

{{timestamp}} // e.g. 1970-01-01T00:00:00Z

If the optional parameter is given, it is used to format the timestamp. The magic reference date Mon Jan 2 15:04:05 -0700 MST 2006 can be used to format the date as required:

{{timestamp "2006-01-02"}} // e.g. 1970-01-01

See Go's time.Format() for more information.

As a special case, if the optional parameter is "unix", the unix timestamp in seconds is returned as a string.

{{timestamp "unix"}} // e.g. 0

toJSON

Takes the result from a tree or ls call and converts it into a JSON object.

{{ tree "config" | explode | toJSON }} // e.g. {"admin":{"port":1234},"maxconns":5,"minconns":2}

Note: This functionality should be considered final. If you need to manipulate keys, combine values, or perform mutations, that should be done outside of Consul. In order to keep the API scope limited, we likely will not accept Pull Requests that focus on customizing the toJSON functionality.

toJSONPretty

Takes the result from a tree or ls call and converts it into a pretty-printed JSON object, indented by two spaces.

{{ tree "config" | explode | toJSONPretty }}

/*
{
  "admin": {
    "port": 1234
  },
  "maxconns": 5,
  "minconns": 2,
}
*/

Note: This functionality should be considered final. If you need to manipulate keys, combine values, or perform mutations, that should be done outside of Consul. In order to keep the API scope limited, we likely will not accept Pull Requests that focus on customizing the toJSONPretty functionality.

toLower

Takes the argument as a string and converts it to lowercase.

{{key "user/name" | toLower}}

See Go's strings.ToLower() for more information.

toTitle

Takes the argument as a string and converts it to titlecase.

{{key "user/name" | toTitle}}

See Go's strings.Title() for more information.

toUpper

Takes the argument as a string and converts it to uppercase.

{{key "user/name" | toUpper}}

See Go's strings.ToUpper() for more information.

toYAML

Takes the result from a tree or ls call and converts it into a pretty-printed YAML object, indented by two spaces.

{{ tree "config" | explode | toYAML }}
/*
admin:
  port: 1234
maxconns: 5
minconns: 2
*/

Note: This functionality should be considered final. If you need to manipulate keys, combine values, or perform mutations, that should be done outside of Consul. In order to keep the API scope limited, we likely will not accept Pull Requests that focus on customizing the toYAML functionality.

Math Functions

The following functions are available on floats and integer values.

add

Returns the sum of the two values.

{{ add 1 2 }} // 3

This can also be used with a pipe function.

{{ 1 | add 2 }} // 3

subtract

Returns the difference of the second value from the first.

{{ subtract 2 5 }} // 3

This can also be used with a pipe function.

{{ 5 | subtract 2 }}

Please take careful note of the order of arguments.

multiply

Returns the product of the two values.

{{ multiply 2 2 }} // 4

This can also be used with a pipe function.

{{ 2 | multiply 2 }} // 4

divide

Returns the division of the second value from the first.

{{ divide 2 10 }} // 5

This can also be used with a pipe function.

{{ 10 | divide 2 }} // 5

Please take careful note of the order or arguments.

Plugins

Authoring Plugins

For some use cases, it may be necessary to write a plugin that offloads work to another system. This is especially useful for things that may not fit in the "standard library" of Consul Template, but still need to be shared across multiple instances.

Consul Template plugins must have the following API:

$ NAME [INPUT...]

NAME - the name of the plugin - this is also the name of the binary, either a full path or just the program name. It will be executed in a shell with the inherited PATH so e.g. the plugin cat will run the first executable cat that is found on the PATH. INPUT - input from the template - this will always be JSON if provided Important Notes

Plugins execute user-provided scripts and pass in potentially sensitive data from Consul or Vault. Nothing is validated or protected by Consul Template, so all necessary precautions and considerations should be made by template authors Plugin output must be returned as a string on stdout. Only stdout will be parsed for output. Be sure to log all errors, debugging messages onto stderr to avoid errors when Consul Template returns the value.

Always exit 0 or Consul Template will assume the plugin failed to execute Ensure the empty input case is handled correctly (see Multi-phase execution) Data piped into the plugin is appended after any parameters given explicitly (eg {{ "sample-data" | plugin "my-plugin" "some-parameter"}} will call my-plugin some-parameter sample-data) Here is a sample plugin in a few different languages that removes any JSON keys that start with an underscore and returns the JSON string:

#! /usr/bin/env ruby
require "json"

if ARGV.empty?
  puts JSON.fast_generate({})
  Kernel.exit(0)
end

hash = JSON.parse(ARGV.first)
hash.reject! { |k, _| k.start_with?("_")  }
puts JSON.fast_generate(hash)
Kernel.exit(0)
func main() {
  arg := []byte(os.Args[1])

  var parsed map[string]interface{}
  if err := json.Unmarshal(arg, &parsed); err != nil {
    fmt.Fprintln(os.Stderr, fmt.Sprintf("err: %s", err))
    os.Exit(1)
  }

  for k, _ := range parsed {
    if string(k[0]) == "_" {
      delete(parsed, k)
    }
  }

  result, err := json.Marshal(parsed)
  if err != nil {
    fmt.Fprintln(os.Stderr, fmt.Sprintf("err: %s", err))
    os.Exit(1)
  }

  fmt.Fprintln(os.Stdout, fmt.Sprintf("%s", result))
  os.Exit(0)
}
Caveats