在 Java 中通过 ANTLR API 创建 AST
我目前正在开发一个项目,该项目需要我动态生成 ANTLR 语法,因为生成的语言取决于用户输入。因此,我在代码中生成 ANTLR 语法,并从中生成词法分析器和解析器。
我的目标是拥有一个用生成语法的语言编写的输入程序(它实际上是通过遗传算法创建的,但这与这里无关),并最终拥有一个代表该程序的 AST。但是,目前我只能生成 ParseTree 对象,这对于我的程序来说还不够。
有人知道如何使用 ANTLR API 生成代表 AST 的对象吗? (例如 antlr.collections.AST 对象)。我将在此处附加一段代码,但测试它的最佳方法是运行位于 https://snowdrop.googlecode.com/svn/trunk/src/ANTLRTest/
public class GEQuorra extends GEModel {
Grammar grammar;
private org.antlr.tool.Grammar lexer;
private org.antlr.tool.Grammar parser;
private String startRule;
private String ignoreTokens;
public GEQuorra(IntegrationTest.Grammar g) {
grammar = new Grammar(g.getBnfGrammar());
setGrammar(grammar);
try {
ignoreTokens = "WS";
startRule = "agentProgram";
parser = new org.antlr.tool.Grammar(g.getAntlrGrammar());
@SuppressWarnings("rawtypes")
List leftRecursiveRules = parser.checkAllRulesForLeftRecursion();
if (leftRecursiveRules.size() > 0) {
throw new Exception("Grammar is left recursive");
}
String lexerGrammarText = parser.getLexerGrammar();
lexer = new org.antlr.tool.Grammar();
lexer.importTokenVocabulary(parser);
lexer.setFileName(parser.getFileName());
lexer.setGrammarContent(lexerGrammarText);
} catch (Exception e) {
e.printStackTrace();
}
}
@Override
public double getFitness(CandidateProgram program) {
try {
GECandidateProgram gecp = (GECandidateProgram) program;
System.out.println("Parsing:" + gecp.getSourceCode());
CharStream input = new ANTLRStringStream(gecp.getSourceCode());
Interpreter lexEngine = new Interpreter(lexer, input);
FilteringTokenStream tokens = new FilteringTokenStream(lexEngine);
StringTokenizer tk = new StringTokenizer(ignoreTokens, " ");
while (tk.hasMoreTokens()) {
String tokenName = tk.nextToken();
tokens.setTokenTypeChannel(lexer.getTokenType(tokenName), 99);
}
Interpreter parseEngine = new Interpreter(parser, tokens);
ParseTree t;
t = parseEngine.parse(startRule);
return 1.0 / t.toStringTree().length();
} catch (Exception e) {
// Something failed, return very big fitness, making it unfavorable
return Double.MAX_VALUE;
}
}
其中 t.toStringTree() 包含 ParseTree。
I'm currently working on a project which that require me to generate an ANTLR grammar on the fly because the generated language depends on user input. Hence I generate the ANTLR grammar in code, and generate a lexer and parser from it.
My goal is to have an input program that is written in the language of the generated grammar (it is actually created through genetic algorithms, but that's not relevant here), and to ultimately have an AST representing the program. However, currently I'm only able to generate a ParseTree object, and this is not sufficient for my program.
Does anybody know how to use the ANTLR API to generate an object representing the AST? (For example an antlr.collections.AST object). I'll append a piece of code here, but the best way to test it is to run the Eclipse project that resides in https://snowdrop.googlecode.com/svn/trunk/src/ANTLRTest/
public class GEQuorra extends GEModel {
Grammar grammar;
private org.antlr.tool.Grammar lexer;
private org.antlr.tool.Grammar parser;
private String startRule;
private String ignoreTokens;
public GEQuorra(IntegrationTest.Grammar g) {
grammar = new Grammar(g.getBnfGrammar());
setGrammar(grammar);
try {
ignoreTokens = "WS";
startRule = "agentProgram";
parser = new org.antlr.tool.Grammar(g.getAntlrGrammar());
@SuppressWarnings("rawtypes")
List leftRecursiveRules = parser.checkAllRulesForLeftRecursion();
if (leftRecursiveRules.size() > 0) {
throw new Exception("Grammar is left recursive");
}
String lexerGrammarText = parser.getLexerGrammar();
lexer = new org.antlr.tool.Grammar();
lexer.importTokenVocabulary(parser);
lexer.setFileName(parser.getFileName());
lexer.setGrammarContent(lexerGrammarText);
} catch (Exception e) {
e.printStackTrace();
}
}
@Override
public double getFitness(CandidateProgram program) {
try {
GECandidateProgram gecp = (GECandidateProgram) program;
System.out.println("Parsing:" + gecp.getSourceCode());
CharStream input = new ANTLRStringStream(gecp.getSourceCode());
Interpreter lexEngine = new Interpreter(lexer, input);
FilteringTokenStream tokens = new FilteringTokenStream(lexEngine);
StringTokenizer tk = new StringTokenizer(ignoreTokens, " ");
while (tk.hasMoreTokens()) {
String tokenName = tk.nextToken();
tokens.setTokenTypeChannel(lexer.getTokenType(tokenName), 99);
}
Interpreter parseEngine = new Interpreter(parser, tokens);
ParseTree t;
t = parseEngine.parse(startRule);
return 1.0 / t.toStringTree().length();
} catch (Exception e) {
// Something failed, return very big fitness, making it unfavorable
return Double.MAX_VALUE;
}
}
Where t.toStringTree() contains the ParseTree.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论