在 Flex/Bison 中嵌入其他语言

发布于 2024-10-02 22:28:51 字数 1503 浏览 3 评论 0原文

底线:

如果您想使用 Flex/Bison 在 C++ 中添加一项非常小的功能,您会怎么做?例如,是否能够使用以下语法声明 void xxx() 函数:foo%%: xxx

整个故事:

一旦我编写了一个自定义着色器处理程序,该程序从多个块构建了即用型 Cg/GLSL 像素和顶点着色器。我添加了一些功能,主要与静态编译相关(类似于“更好的预处理器”)。

例如,看起来像

#if LIGHT_TYPE == POINT
    float lightStrength = dot(N, normalize(pos - lightPos));
#elif LIGHT_TYPE == DIRECTIONAL
    float lightStrength = dot(N, lightDir);
#endif

纯宏的东西,看起来像

[LightType = Point]
[Require: pos, lightPos]
float LightStrength()
{
    !let %N Normal
    return dot(%N, normalize(pos - lightPos));
}

在我的“语言”中。正如您所看到的,可以为可变的光/材质类型提供“函数”。还可以“调用”其他函数并标记特定着色器需要哪些统一/变化的属性。

为什么要这么努力?因为(特别是在像 SM 2.0 这样的早期卡中)属性存在许多限制,并且我的“编译器”生成了即用型着色器,其中包含所需属性/变量的列表、像素和顶点着色器之间的托管参数、优化了一些静态内容(只是为了可读性,因为 Cg 编译器稍后会对其进行优化)。

好吧,但我写这些并不是为了赞扬自己或其他什么。

我用 C# 编写了这个“编译器”,它最初是一个非常小的程序。但随着时间的推移,添加了许多功能,现在程序一团糟,没有重构的选项。另外,用 C# 编码意味着我无法将此编译器直接嵌入到 C++ 游戏中,这迫使我生成进程来编译着色器(这需要大量时间)。

我想使用 C++ 和 Flex/Bison 工具包重写我的语言/编译器。我已经在 Flex/Bison 中编写了高效的数学解析器,所以我在这方面有一些经验。然而,有一件事我自己无法解决,这是我问题的一个主题。

我怎样才能将 GLSL 嵌入到我的语言中?在 C# 编译器中,我只是逐行检查行是否以特殊字符(如 % 或 [)开头,然后使用 string_replace 和/或正则表达式做了许多技巧和技巧。

现在我想写下一个干净的 Bison 语法并正确地执行它。但包括 GLSL 的整个语法让我感到害怕。它是一种相当庞大且复杂的语言,并且不断发展。我最想做的就是传递所有这些 GLSL 代码。

如果您根本没有 GLSL/Cg/图形编程经验,那么这并不是很重要。这个问题可以改写为“底线”。

那么,如果您想使用 Flex/Bison 在 C++ 中添加一项非常小的功能,您会怎么做呢?例如,是否能够使用以下语法声明 void xxx() 函数:foo%%: xxx

The bottom line:

If you would like to add one, very small feature into C++ using Flex/Bison, how would you do that? For example, ability to declare void xxx() functions with syntax: foo%%: xxx?

The whole story:

Once I have coded a custom shader processing program that built ready-to-use Cg/GLSL pixel and vertex shaders from a number of blocks. I've added few features, mostly related to static compilation (something like a "better preprocessor").

For example, something that would look like

#if LIGHT_TYPE == POINT
    float lightStrength = dot(N, normalize(pos - lightPos));
#elif LIGHT_TYPE == DIRECTIONAL
    float lightStrength = dot(N, lightDir);
#endif

with pure macros, looks like

[LightType = Point]
[Require: pos, lightPos]
float LightStrength()
{
    !let %N Normal
    return dot(%N, normalize(pos - lightPos));
}

in my "language". As you can see, "functions" can be provided for variable light/material types. There is also a possibility to "call" other function and to mark what uniform/varying attributes are required for specific shader.

Why all this effort? Because (especially in early cards like SM 2.0) there were numerous limitations of attributes, and my "compiler" produced ready-to-use shader with list of required attributes/variables, managed parameters between pixel and vertex shaders, optimized some static stuff (just for readability, as Cg compiler would optimize it later anyway).

OK, but I am not writing all of this to praise myself or something.

I've written this "compiler" in C# and it initially was a very small program. But as time passed, many features were added and now the program is a complete mess with no options of refactoring. Also, being coded in C# means I cannot embed this compiler directly in C++ game, forcing me to spawn processes to compile shaders (this takes a lot of time).

I would like to rewrite my language/compiler using C++ and Flex/Bison toolkit. I've already coded efficient math parser in Flex/Bison so I am somewhat experienced in this matter. However there is one thing that I cannot resolve myself and this is a very topic of my question.

How can I possibly embed GLSL into my language? In C# compiler, I've just went line-by-line and checked if line starts with special characters (like % or [) and later did many tricks&hacks using string_replace and/or regexes.

Now I would like to write down a clean Bison grammar and do it properly. But including whole syntax of GLSL scares me. It is quite a big and complicated language, constantly evolving. And most I would do would be to pass-through all this GLSL code anyway.

If you are not experienced with GLSL/Cg/graphics programming at all, this is not quite important. The question can be rewritten into the "bottom line".

So, if you would like to add one, very small feature into C++ using Flex/Bison, how would you do that? For example, ability to declare void xxx() functions with syntax: foo%%: xxx?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

最后的乘客 2024-10-09 22:28:51

我曾经为一个学校项目向 pascal 添加了多线程机制,所以我不得不处理这个问题。我所做的是找到 pascal 的 BNF 定义并复制它,主要是使我的标记在 99% 的情况下等于文本,并在我添加的 1% 的新标记中添加新的中间语言代码。 Pascal 很简单,因此 BNF 定义也相对简单。 C++,没那么多。

Stroustroup 的 C++ 编程语言具有几乎可供解析器使用的语言语法,但需要手动复制大量代码。也许某个网站可以帮助您找到“标准”C++ 的 bison/lex 集,然后您就可以修改它。

编辑:
我找到了我的 yacc 文件,所以这是我的 pascal 代码的一个小示例,

procheader: PROCEDURE ID paramslist ';'{thread::inLocalDecl=true; $="procedure "+$2+$3+";\n";}
|            FUNCTION ID paramslist ':' ID ';'{thread::inLocalDecl=true; $="function "+$2+$3+":"+$5+";\n";}
|  THREAD {thread::inThreadDecl=true;
       thread::inArgDecl=true;}
   ID {thread::curName=$3;} paramslist { thread::inArgDecl=false;} ';'
{ 
   /*a lot of code to transform this custom construct into standard pascal*/
}

前两个元素只是标准 pascal:我将输入标记逐字复制到输出字符串中(即我实际上没有修改输入文件中的任何内容)。

显然,第三个元素(我的 THREAD 关键字)不是标准的 pascal。所以我将输出转换为我实际上可以用标准 pascal 编译的东西。

基本上,要编译我的“线程”pascal,我必须获取源文件,将其通过解析器传递,然后编译输出。

I added multithreading mechanisms to pascal for a school project once, so I've had to deal with this. What I did was I found a BNF definition of pascal and copied that, mainly making my tokens equal to the text in 99% of the cases, and adding the new intermediate language code in the 1% of new tokens I added. Pascal is simple, so the BNF definition is relatively simple. C++, not so much.

The C++ Programming Language by Stroustroup has the language grammar that is practically parser-ready, but that's a lot of code to copy by hand. Maybe a certain website can help you find a bison/lex set for "standard" C++, and then you can modify that.

EDIT:
I found my yacc files, so here's a small example from my pascal code

procheader: PROCEDURE ID paramslist ';'{thread::inLocalDecl=true; $="procedure "+$2+$3+";\n";}
|            FUNCTION ID paramslist ':' ID ';'{thread::inLocalDecl=true; $="function "+$2+$3+":"+$5+";\n";}
|  THREAD {thread::inThreadDecl=true;
       thread::inArgDecl=true;}
   ID {thread::curName=$3;} paramslist { thread::inArgDecl=false;} ';'
{ 
   /*a lot of code to transform this custom construct into standard pascal*/
}

the first two elements were just standard pascal: I copied the input tokens into the output strings verbatim (ie I didn't actually modify anything from the input file).

the third element(my THREAD keyword) was, obviously, not standard pascal. So I transformed the output into something that I could actually compile in standard pascal.

Basically to compile my "threaded" pascal, I had to take my source file, pass it through my parser, then compile the output.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文