对 Python 文件使用 ANTLR4 `parser.file_input()` 时出现 ParseCancellationException

发布于 2025-01-12 18:21:37 字数 1567 浏览 1 评论 0原文

我正在使用 ANTLR4 编写 Java 代码来解析 Python 文件。我使用的词法分析器和解析器是来自 antlr/grammars-v4 Github 的 Python3Lexer.g4 和 Python3Parser.g4。 java解析代码在大多数情况下工作正常，但有时会出现以下错误。

line 431:1 no viable alternative at input '<EOF>'
Parser Exception: org.antlr.v4.runtime.misc.ParseCancellationException
org.antlr.v4.runtime.misc.ParseCancellationException
        at org.antlr.v4.runtime.BailErrorStrategy.recover(BailErrorStrategy.java:51)
        at Python3Parser.simple_stmt(Python3Parser.java:1667)
        at Python3Parser.stmt(Python3Parser.java:1567)
        at Python3Parser.file_input(Python3Parser.java:348)
        at ConvertPython.serializeFile(ConvertPython.java:89)

这是 ConvertPython.java 的一部分：

      Python3Lexer lexer = new Python3Lexer(CharStreams.fromFileName(f));
      CommonTokenStream tokens = new CommonTokenStream(lexer);
      vocab = lexer.getVocabulary();

      Python3Parser parser = new Python3Parser(tokens);
      ParserRuleContext t = parser.file_input(); // the exception line

这是一个失败的 Python：

...
SYBYL2SYMB = {
    "Mo": "Mo",
    "Sn": "Sn",
}

当我测试它时，我发现这个 dict 不能是 Python 文件的最后一行。如果后面有新行，也不例外。

此外，我发现Python代码print resultmatrix_在输入'resultmatrix_'解析器异常：org.antlr.v4.runtime.misc.ParseCancellationException行231:7没有可行的替代方案代码>.我认为这是因为这段代码是Python2，但我使用的ANTLR语法是针对Python3的。

PS，我是 ANTLR 的新手。请告诉我应该发布什么内容才能得到您的理解。非常感谢！

原文

I am writing Java code using ANTLR4 to parse Python files. The lexer and parser I use are Python3Lexer.g4 and Python3Parser.g4 from antlr/grammars-v4 Github. The java parsing code works fine most of the time, but sometimes I get the following error.

line 431:1 no viable alternative at input '<EOF>'
Parser Exception: org.antlr.v4.runtime.misc.ParseCancellationException
org.antlr.v4.runtime.misc.ParseCancellationException
        at org.antlr.v4.runtime.BailErrorStrategy.recover(BailErrorStrategy.java:51)
        at Python3Parser.simple_stmt(Python3Parser.java:1667)
        at Python3Parser.stmt(Python3Parser.java:1567)
        at Python3Parser.file_input(Python3Parser.java:348)
        at ConvertPython.serializeFile(ConvertPython.java:89)

Here is part of the ConvertPython.java:

      Python3Lexer lexer = new Python3Lexer(CharStreams.fromFileName(f));
      CommonTokenStream tokens = new CommonTokenStream(lexer);
      vocab = lexer.getVocabulary();

      Python3Parser parser = new Python3Parser(tokens);
      ParserRuleContext t = parser.file_input(); // the exception line

Here is one failing Python:

...
SYBYL2SYMB = {
    "Mo": "Mo",
    "Sn": "Sn",
}

When I tested it, I found this dict cannot be the last line of the Python file. If there is a new line after it, there is no exception.

Besides, I found there would be line 231:7 no viable alternative at input 'resultmatrix_' Parser Exception: org.antlr.v4.runtime.misc.ParseCancellationException for Python code print resultmatrix_. I think it's because this code is Python2 but the ANTLR grammar I'm using is for Python3.

PS, I'm new to ANTLR. Please tell me what I should post for your understanding. Thank you a lot!

分享到QQ

分享到微博

如果你对这篇内容有疑问，欢迎到本站社区发帖提问参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。

发布评论

需要登录才能够评论，你可以免费注册一个本站的账号。

小清晰的声音 2025-01-19 18:21:37

语法期望“简单语句”末尾有一个 NEWLINE。

这有效：

String input = "SYBYL2SYMB = {\n" +
    "    \"Mo\": \"Mo\",\n" +
    "    \"Sn\": \"Sn\",\n" +
    "}\n";

Python3Lexer lexer = new Python3Lexer(CharStreams.fromString(input));
Python3Parser parser = new Python3Parser(new CommonTokenStream(lexer));

parser.file_input();

The grammar expects a NEWLINE at the end of the "simple statement".

This works:

String input = "SYBYL2SYMB = {\n" +
    "    \"Mo\": \"Mo\",\n" +
    "    \"Sn\": \"Sn\",\n" +
    "}\n";

Python3Lexer lexer = new Python3Lexer(CharStreams.fromString(input));
Python3Parser parser = new Python3Parser(new CommonTokenStream(lexer));

parser.file_input();

回复收藏 0 原文

~没有更多了~