在 Java 中通过 ANTLR API 创建 AST

发布于 2024-12-13 03:14:17 字数 2701 浏览 0 评论 0原文

我目前正在开发一个项目,该项目需要我动态生成 ANTLR 语法,因为生成的语言取决于用户输入。因此,我在代码中生成 ANTLR 语法,并从中生成词法分析器和解析器。

我的目标是拥有一个用生成语法的语言编写的输入程序(它实际上是通过遗传算法创建的,但这与这里无关),并最终拥有一个代表该程序的 AST。但是,目前我只能生成 ParseTree 对象,这对于我的程序来说还不够。

有人知道如何使用 ANTLR API 生成代表 AST 的对象吗? (例如 antlr.collections.AST 对象)。我将在此处附加一段代码,但测试它的最佳方法是运行位于 https://snowdrop.googlecode.com/svn/trunk/src/ANTLRTest/

public class GEQuorra extends GEModel {

    Grammar grammar;
    private org.antlr.tool.Grammar lexer;
    private org.antlr.tool.Grammar parser;
    private String startRule;
    private String ignoreTokens;

    public GEQuorra(IntegrationTest.Grammar g) {
        grammar = new Grammar(g.getBnfGrammar());

        setGrammar(grammar);

        try {
            ignoreTokens = "WS";
            startRule = "agentProgram";
            parser = new org.antlr.tool.Grammar(g.getAntlrGrammar());

            @SuppressWarnings("rawtypes")
            List leftRecursiveRules = parser.checkAllRulesForLeftRecursion();
            if (leftRecursiveRules.size() > 0) {
                throw new Exception("Grammar is left recursive");
            }

            String lexerGrammarText = parser.getLexerGrammar();
            lexer = new org.antlr.tool.Grammar();
            lexer.importTokenVocabulary(parser);
            lexer.setFileName(parser.getFileName());
            lexer.setGrammarContent(lexerGrammarText);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    @Override
    public double getFitness(CandidateProgram program) {
        try {

            GECandidateProgram gecp = (GECandidateProgram) program;
             System.out.println("Parsing:" + gecp.getSourceCode());

            CharStream input = new ANTLRStringStream(gecp.getSourceCode());
            Interpreter lexEngine = new Interpreter(lexer, input);
            FilteringTokenStream tokens = new FilteringTokenStream(lexEngine);
            StringTokenizer tk = new StringTokenizer(ignoreTokens, " ");
            while (tk.hasMoreTokens()) {
                String tokenName = tk.nextToken();
                tokens.setTokenTypeChannel(lexer.getTokenType(tokenName), 99);
            }

            Interpreter parseEngine = new Interpreter(parser, tokens);
            ParseTree t;
            t = parseEngine.parse(startRule);

            return 1.0 / t.toStringTree().length();
        } catch (Exception e) {
            // Something failed, return very big fitness, making it unfavorable
            return Double.MAX_VALUE;
        }
    }

其中 t.toStringTree() 包含 ParseTree。

I'm currently working on a project which that require me to generate an ANTLR grammar on the fly because the generated language depends on user input. Hence I generate the ANTLR grammar in code, and generate a lexer and parser from it.

My goal is to have an input program that is written in the language of the generated grammar (it is actually created through genetic algorithms, but that's not relevant here), and to ultimately have an AST representing the program. However, currently I'm only able to generate a ParseTree object, and this is not sufficient for my program.

Does anybody know how to use the ANTLR API to generate an object representing the AST? (For example an antlr.collections.AST object). I'll append a piece of code here, but the best way to test it is to run the Eclipse project that resides in https://snowdrop.googlecode.com/svn/trunk/src/ANTLRTest/

public class GEQuorra extends GEModel {

    Grammar grammar;
    private org.antlr.tool.Grammar lexer;
    private org.antlr.tool.Grammar parser;
    private String startRule;
    private String ignoreTokens;

    public GEQuorra(IntegrationTest.Grammar g) {
        grammar = new Grammar(g.getBnfGrammar());

        setGrammar(grammar);

        try {
            ignoreTokens = "WS";
            startRule = "agentProgram";
            parser = new org.antlr.tool.Grammar(g.getAntlrGrammar());

            @SuppressWarnings("rawtypes")
            List leftRecursiveRules = parser.checkAllRulesForLeftRecursion();
            if (leftRecursiveRules.size() > 0) {
                throw new Exception("Grammar is left recursive");
            }

            String lexerGrammarText = parser.getLexerGrammar();
            lexer = new org.antlr.tool.Grammar();
            lexer.importTokenVocabulary(parser);
            lexer.setFileName(parser.getFileName());
            lexer.setGrammarContent(lexerGrammarText);

        } catch (Exception e) {
            e.printStackTrace();
        }
    }

    @Override
    public double getFitness(CandidateProgram program) {
        try {

            GECandidateProgram gecp = (GECandidateProgram) program;
             System.out.println("Parsing:" + gecp.getSourceCode());

            CharStream input = new ANTLRStringStream(gecp.getSourceCode());
            Interpreter lexEngine = new Interpreter(lexer, input);
            FilteringTokenStream tokens = new FilteringTokenStream(lexEngine);
            StringTokenizer tk = new StringTokenizer(ignoreTokens, " ");
            while (tk.hasMoreTokens()) {
                String tokenName = tk.nextToken();
                tokens.setTokenTypeChannel(lexer.getTokenType(tokenName), 99);
            }

            Interpreter parseEngine = new Interpreter(parser, tokens);
            ParseTree t;
            t = parseEngine.parse(startRule);

            return 1.0 / t.toStringTree().length();
        } catch (Exception e) {
            // Something failed, return very big fitness, making it unfavorable
            return Double.MAX_VALUE;
        }
    }

Where t.toStringTree() contains the ParseTree.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文