如何使用pyparsing根据行前缀将行列表分解为单个组

Question

我正在尝试解析命令的输出 IP Netns Exec VPN_NS IPSEC Stroke Statusall（示例粘贴在下面）。

该命令为每个服务（oof-＃n-＃i）终结器（#N）提供多行，并使用该终结器（#I）实例，因此

oof-2-1是terminator服务器oof-2实例1。

。

从示例中，我试图获得这样的内容：

results = {
    'connections':
        {
            'oof-1-1': [ 3 lines starting with oof-1-1 in section "Connections" ],
            'oof-1-2': [ 3 lines starting with oof-1-2 in section "Connections" ]
            'oof-2-1': [ 3 lines starting with oof-2-1 in section "Connections" ]
        },

    'sec_assocs':
        {
            'oof-1-1': [ 3 lines starting with oof-1-1 in section "Security Associations" ],
            'oof-1-2': [ 3 lines starting with oof-1-2 in section "Security Associations" ]
            'oof-2-1': [ 3 lines starting with oof-2-1 in section "Security Associations" ]
        }
}

每个ID都包含以其开头的行列表。

这是Strongswan命令的完整输出。

sample = """
Status of IKE charon daemon (strongSwan 5.9.1, Linux 4.15.0-162-generic, x86_64):
  uptime: 25 hours, since Mar 23 15:23:53 2022
  worker threads: 11 of 16 idle, 5/0/0/0 working, job queue: 0/0/0/0, scheduled: 10
  loaded plugins: charon aesni 
Listening IP addresses:
  169.254.123.2
  192.168.51.254
Connections:
     oof-1-1:  %any...10.1.0.242  IKEv2, dpddelay=30s
     oof-1-1:   remote: [server] uses public key authentication
     oof-1-1:   child:  dynamic === 0.0.0.0/0 TUNNEL, dpdaction=restart
     oof-1-2:  %any...10.1.0.242  IKEv2, dpddelay=30s
     oof-1-2:   remote: [server] uses public key authentication
     oof-1-2:   child:  dynamic === 0.0.0.0/0 TUNNEL, dpdaction=restart
     oof-2-1:  %any...10.1.0.242  IKEv2, dpddelay=30s
     oof-2-1:   remote: [server] uses public key authentication
     oof-2-1:   child:  dynamic === 0.0.0.0/0 TUNNEL, dpdaction=restartd
Security Associations:
     oof-1-1:  %any...10.1.0.242  IKEv2, dpddelay=30s
     oof-1-1:   remote: [server] uses public key authentication
     oof-1-1:   child:  dynamic === 0.0.0.0/0 TUNNEL, dpdaction=restart
     oof-1-2:  %any...10.1.0.242  IKEv2, dpddelay=30s
     oof-1-2:   remote: [server] uses public key authentication
     oof-1-2:   child:  dynamic === 0.0.0.0/0 TUNNEL, dpdaction=restart
     oof-2-1:  %any...10.1.0.242  IKEv2, dpddelay=30s
     oof-2-1:   remote: [server] uses public key authentication
     oof-2-1:   child:  dynamic === 0.0.0.0/0 TUNNEL, dpdaction=restartd
"""

这是解析解决方案中使用的样本：

sample = """
Connections:
     oof-1-1:  %any...10.1.0.242  IKEv2, dpddelay=30s
     oof-1-1:   remote: [server] uses public key authentication
     oof-1-1:   child:  dynamic === 0.0.0.0/0 TUNNEL, dpdaction=restart
     oof-1-2:  %any...10.1.0.242  IKEv2, dpddelay=30s
     oof-1-2:   remote: [server] uses public key authentication
     oof-1-2:   child:  dynamic === 0.0.0.0/0 TUNNEL, dpdaction=restart
     oof-2-1:  %any...10.1.0.242  IKEv2, dpddelay=30s
     oof-2-1:   remote: [server] uses public key authentication
     oof-2-1:   child:  dynamic === 0.0.0.0/0 TUNNEL, dpdaction=restartd
Security Associations:
     oof-1-1:  %any...10.1.0.242  IKEv2, dpddelay=30s
     oof-1-1:   remote: [server] uses public key authentication
     oof-1-1:   child:  dynamic === 0.0.0.0/0 TUNNEL, dpdaction=restart
     oof-1-2:  %any...10.1.0.242  IKEv2, dpddelay=30s
     oof-1-2:   remote: [server] uses public key authentication
     oof-1-2:   child:  dynamic === 0.0.0.0/0 TUNNEL, dpdaction=restart
     oof-2-1:  %any...10.1.0.242  IKEv2, dpddelay=30s
     oof-2-1:   remote: [server] uses public key authentication
     oof-2-1:   child:  dynamic === 0.0.0.0/0 TUNNEL, dpdaction=restartd
"""

Answer 1

后处理是使用这种解析数据的最直接方法。这是您试图解析的结构的BNF：

group ::= label ':' line...
label ::= word...
line ::= prefix ':' rest_of_line
prefix ::= word '-' int '-' int

word和int只是一个单词或数字，“ ...”表示重复。

这将pyparsing转换为：

import pyparsing as pp

COLON = pp.Suppress(":")
label = pp.Combine(
            pp.Word(pp.alphas)[1, ...], adjacent=False, joinString=" "
            )
prefix = pp.Combine(
            pp.Word(pp.alphas) + "-" + pp.Word(pp.nums) + "-" + pp.Word(pp.nums)
            )
post_prefix = COLON + pp.restOfLine
line = pp.Group(prefix("prefix") + post_prefix)
lines = pp.Group(line[...])
group = pp.Group(label("group_label") + COLON + lines("subgroups"))

pyparsing将为您生成此铁路图：

这可以解析您的文本，但是为了通过其前缀重新组合线，我们可以添加一个使用itertools.groupby的解析操作：

def regroup_lines(t):
    from itertools import groupby
    from operator import itemgetter

    ret = pp.ParseResults([])
    parsed_lines = t[0]
    for prefix, subgroup in groupby(parsed_lines, key=itemgetter("prefix")):
        # each line in subgroup has the prefix and the rest of the line after the ':'
        # repackage the multiple lines into a single group that is labeled with 
        # the common prefix, and contains the line contents
        ret.append(pp.ParseResults.from_dict(
            {
                'prefix': prefix,
                'lines': [line[1] for line in subgroup],
            }
        ))
    return ret

lines.add_parse_action(regroup_lines)

通过使用解析操作，在分析时进行重组，因此不需要额外的放置后处理。

现在，我们可以解析您的样本并获得重组结果：

results = group[...].parseString(sample)

这是一个简短的功能，可以打印出解析的组：

def print_groups(parsed):
    for group in parsed:
        print(group.group_label)
        for subgroup in group.subgroups:
            print(f"- {subgroup.prefix}")
            for line in subgroup.lines:
                print(f"  {line!r}")
        print()

print_groups(results)

它给出：

Connections
- oof-1-1
  '  %any...10.1.0.242  IKEv2, dpddelay=30s'
  '   remote: [server] uses public key authentication'
  '   child:  dynamic === 0.0.0.0/0 TUNNEL, dpdaction=restart'
- oof-1-2
  '  %any...10.1.0.242  IKEv2, dpddelay=30s'
  '   remote: [server] uses public key authentication'
  '   child:  dynamic === 0.0.0.0/0 TUNNEL, dpdaction=restart'
- oof-2-1
  '  %any...10.1.0.242  IKEv2, dpddelay=30s'
  '   remote: [server] uses public key authentication'
  '   child:  dynamic === 0.0.0.0/0 TUNNEL, dpdaction=restartd'

Security Associations
- oof-1-1
  '  %any...10.1.0.242  IKEv2, dpddelay=30s'
  '   remote: [server] uses public key authentication'
  '   child:  dynamic === 0.0.0.0/0 TUNNEL, dpdaction=restart'
- oof-1-2
  '  %any...10.1.0.242  IKEv2, dpddelay=30s'
  '   remote: [server] uses public key authentication'
  '   child:  dynamic === 0.0.0.0/0 TUNNEL, dpdaction=restart'
- oof-2-1
  '  %any...10.1.0.242  IKEv2, dpddelay=30s'
  '   remote: [server] uses public key authentication'
  '   child:  dynamic === 0.0.0.0/0 TUNNEL, dpdaction=restartd'

这是工作示例的完整来源：

import pyparsing as pp

COLON = pp.Suppress(":")
label = pp.Combine(pp.Word(pp.alphas)[1, ...], adjacent=False, joinString=" ")
label.setName("label")
prefix = pp.Combine(pp.Word(pp.alphas) + "-" + pp.Word(pp.nums) + "-" + pp.Word(pp.nums))
prefix.setName("prefix")
post_prefix = COLON + pp.restOfLine
line = pp.Group(prefix("prefix") + post_prefix)
lines = pp.Group(line[...])


def regroup_lines(t):
    from itertools import groupby
    from operator import itemgetter

    ret = pp.ParseResults([])
    for prefix, subgroup in groupby(t[0], key=itemgetter("prefix")):
        ret.append(pp.ParseResults.from_dict(
            {
                'prefix': prefix,
                'lines': [line[1] for line in subgroup],
            }
        ))
    return ret
lines.add_parse_action(regroup_lines)

group = pp.Group(label("group_label") + COLON + lines("subgroups"))
pp.autoname_elements()
group.create_diagram("groupby_1.html", show_results_names=True)
results = group[...].parseString(sample)


def print_groups(parsed):
    for group in parsed:
        print(group.group_label)
        for subgroup in group.subgroups:
            print(f"- {subgroup.prefix}")
            for line in subgroup.lines:
                print(f"  {line!r}")
        print()

print_groups(results)