从 JavaCC 源代码生成 Python 语言的解析器？
我的意思是标题中的 ??? 因为我不太确定。让我解释一下情况。
我不是计算机科学专业的学生&我从未上过任何编译器课程。到目前为止，我曾经认为编译器编写者或学习编译器课程的学生非常出色，因为他们必须用他们编写编译器的任何语言来编写编译器的解析器组件。这不是一件容易的事吧？
我正在处理信息检索问题。我想要的编程语言是Python。
解析器性质： http://ir.iit.edu/~dagr/frDocs/fr940104。 0.txt是样本语料库。该文件包含大约 50 个带有一些 XML 样式标记的文档。 （您可以在上面的链接中看到它）。我需要记下其他一些值，例如 注释一些 &hyph;与空间； & 字符实体。我不知道为什么语料库有这样的东西，因为它知道它既不是由浏览器呈现的，也不是正确的 XML 文档。
我想到使用任何 Python XML 解析器并提取所需的文本。但经过一番搜索后，我发现 JavaCC 解析器源代码 (Parser.jj) 对于我在此处使用的同一语料库。快速查找JavaCC，然后是Compiler-compiler 揭示了毕竟编译器编写者并不像我想象的那么伟大。他们使用 Compiler-compiler 生成所需语言的解析器代码。 Wiki 说编译器-编译器的输入是语法（通常是 BNF）。这就是我迷路的地方。
- 是 Parser.jj 语法（输入到编译器-编译器称为 JavaCC）？这绝对不是BNF。这个语法叫什么？为什么Java语言有这个语法呢？没有通用语法语言吗？
- 我想要 python 解析器来解析语料库。有什么方法可以翻译 Parser.jj 以获得 python 等效项吗？如果是，那是什么？如果不是，我还有什么其他选择？
- 有谁知道这个语料库是什么吗？它的原始来源在哪里？我想看一些关于它的描述。它在互联网上发布，名称为
frDocs.tar.gz
I do mean the ??? in the title because I'm not exactly sure. Let me explain the situation.
I'm not a computer science student & I never did any compilers course. Till now I used to think that compiler writers or students who did compilers course are outstanding because they had to write Parser component of the compiler in whatever language they are writing the compiler. It's not an easy job right?
I'm dealing with Information Retrieval problem. My desired programming language is Python.
Parser Nature:
http://ir.iit.edu/~dagr/frDocs/fr940104.0.txt is the sample corpus. This file contains around 50 documents with some XML style markup. (You can see it in above link). I need to note down other some other values like <DOCNO> FR940104-2-00001 </DOCNO> & <PARENT> FR940104-2-00001 </PARENT> and I only need to index the <TEXT> </TEXT> portion of document which contains some varying tags which I need to strip down and a lot of <!-- --> comments that are to be neglected and some &hyph; &space; & character entities. I don't know why corpus has things like this when its know that it's neither meant to be rendered by browser nor a proper XML document.
I thought of using any Python XML parser and extract desired text. But after little searching I found JavaCC parser source code (Parser.jj) for the same corpus I'm using here. A quick look up on JavaCC followed by Compiler-compiler revealed that after all compiler writers aren't as great as I thought. They use Compiler-compiler to generate parser code in desired language. Wiki says input to compiler-compiler is input is a grammar (usually in BNF). This is where I'm lost.
- Is Parser.jj the grammar (Input to compiler-compiler called JavaCC)? It's definitely not BNF. What is this grammar called? Why is this grammar has Java language? Isn't there any universal grammar language?
- I want python parser for parsing the corpus. Is there any way I can translate Parser.jj to get python equivalent? If yes, what is it? If no, what are my other options?
- By any chance does any one know what is this corpus? Where is its original source? I would like to see some description for it. It is distributed on internet with name
frDocs.tar.gz
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
为什么称此为“XML 样式”标记？ - 对我来说，这看起来像是非常标准/基本的 XML。
尝试 elementTree 或 lxml。不用编写解析器，而是使用已经存在的稳定、强化的库之一。
Why do you call this "XML-style" markup? - this looks like pretty standard/basic XML to me.
Try elementTree or lxml. Instead of writing a parser, use one of the stable, well-hardened libraries that are already out there.
你无法从 (n E)BNF 语法构建一个解析器 - 更不用说整个编译器 - 它只是语法，即语法（有些语法，如 Python 的基于缩进的块规则，无法在其中建模根本），而不是语义。您可以针对这些方面使用单独的工具，或者使用更先进的框架（例如 C++ 中的 Boost::Spirit 或 Haskell 中的 Parsec）来统一两者。
JavaCC（如 yacc）负责生成解析器，即理解从源代码读取的标记的子程序。为此，他们将类似 (E)BNF 的表示法与用生成的解析器将使用的语言（例如构建解析树）编写的代码混合在一起 - 在本例中为 Java。当然，创造另一种语言是可能的——但由于现有的语言可以相对较好地处理这些任务，所以这是毫无意义的。而且由于编译器的其他部分可能是用相同的语言手工编写的，因此保留“我有 ze 令牌，我该如何处理它们？”是有意义的。部分给将编写这些其他部分的人；）
我从未听说过“PythonCC”，谷歌也没有听说过（好吧，谷歌代码上有一个“pythoncc”项目，但它的描述只是说“pythoncc是一个程序，尝试为 Python 脚本生成优化的机器代码。”并且自 3 月份以来没有提交）。你的意思是 任何这些 python 解析库/工具吗？ 但我不认为有自动将 javaCC 代码转换为 Python 等效代码的方法 - 但整个事情看起来相当简单，所以如果您深入了解并了解一些有关通过 javaCC 和 [您选择的 python 库/工具] 进行解析的知识，您也许能够翻译它...
You can't build a parser - let alone a whole compiler - from a(n E)BNF grammar - it's just the grammar, i.e. syntax (and some syntax, like Python's indentation-based block rules, can't be modeled in it at all), not the semantics. Either you use seperate tools for these aspects, or use a more advances framework (like Boost::Spirit in C++ or Parsec in Haskell) that unifies both.
JavaCC (like yacc) is responsible for generating a parser, i.e. the subprogram that makes sense of the tokens read from the source code. For this, they mix a (E)BNF-like notation with code written in the language the resulting parser will be in (for e.g. building a parse tree) - in this case, Java. Of course it would be possible to make up another language - but since the existing languages can handle those tasks relatively well, it would be rather pointless. And since other parts of the compiler might be written by hand in the same language, it makes sense to leave the "I got ze tokens, what do I do wit them?" part to the person who will write these other parts ;)
I never heard of "PythonCC", and google didn't either (well, theres a "pythoncc" project on google code, but it's describtion just says "pythoncc is a program that tries to generate optimized machine Code for Python scripts." and there was no commit since march). Do you mean any of these python parsing libraries/tools? But I don't think there's a way to automatically convert the javaCC code to a Python equivalent - but the whole thing looks rather simple, so if you dive in and learn a bit about parsing via javaCC and [python library/tool of your choice], you might be able to translate it...