在传递给非托管代码之前将委托固定在结构中

发布于 2024-07-16 15:20:34 字数 2987 浏览 5 评论 0原文

我正在尝试使用非托管 C dll 将图像数据加载到 C# 应用程序中。 该库有一个相当简单的接口,您可以在其中传递一个包含三个回调的结构,一个用于接收图像的大小,一个用于接收每一行像素,最后一个在加载完成时调用。 像这样(C#托管定义):

[System.Runtime.InteropServices.StructLayoutAttribute(System.Runtime.InteropServices.LayoutKind.Sequential)]
public struct st_ImageProtocol
{
   public st_ImageProtocol_done Done;    
   public st_ImageProtocol_setSize SetSize;    
   public st_ImageProtocol_sendLine SendLine;
}

开始st_ImageProtocol的类型是delgates:

public delegate int st_ImageProtocol_sendLine(System.IntPtr localData, int rowNumber, System.IntPtr pixelData);

在我使用的测试文件中,SetSize应该被调用一次,然后SendLine将被调用200次(图像中的每行像素一次) ,最后触发 Done 回调。 实际发生的情况是,SendLine 被调用 19 次,然后抛出 AccessViolationException,声称该库试图访问受保护的内存。

我可以访问 C 库的代码(尽管我无法更改功能),并且在调用 SendLine 方法的循环期间,它不会分配或释放任何新内存,所以我的假设是委托本身是问题,我需要在传递它之前固定它(目前我在委托本身内部没有代码,除了一个计数器来查看它被调用的频率,所以我怀疑我是否破坏了托管端的任何内容)。 问题是我不知道该怎么做; 我一直用来在非托管空间中声明结构的方法不适用于委托(Marshal.AllocHGlobal()),并且我找不到任何其他合适的方法。 委托本身是 Program 类中的静态字段,因此它们不应该被垃圾收集,但我猜运行时可能会移动它们。

Chris Brumme 的这篇博客文章 表示代表们不在传递到非托管代码之前不需要固定:

显然,非托管函数指针必须引用固定地址。 如果 GC 重新安置它,那将是一场灾难! 这导致许多应用程序为委托创建固定句柄。 这是完全没有必要的。 非托管函数指针实际上指的是我们动态生成的用于执行转换和转换的本机代码存根。 编组。 该存根存在于 GC 堆之外的固定内存中。

但我不知道当委托是结构的一部分时这是否成立。 它确实意味着可以手动固定它们,而且我对如何执行此操作或任何更好的建议感兴趣,了解为什么循环会运行 19 次然后突然失败。

谢谢。


编辑以回答 Johan 的问题...

分配结构的代码如下:

_sendLineFunc = new st_ImageProtocol_sendLine(protocolSendLineStub);

_imageProtocol = new st_ImageProtocol()
                     {
                          //Set some other properties...
                          SendLine = _sendLineFunc
                     };

int protocolSize = Marshal.SizeOf(_imageProtocol);
_imageProtocolPtr = Marshal.AllocHGlobal(protocolSize);
Marshal.StructureToPtr(_imageProtocol, _imageProtocolPtr, true);

其中 _sendLineFunc 和 _imageProtocol 变量都是 Program 类的静态字段。 如果我正确理解其内部原理,这意味着我将一个指向 _imageProtocol 变量的副本的非托管指针传递到 C 库中,但该副本包含对静态 _sendLineFunc 的引用。 这应该意味着副本不会被 GC 触及 - 因为它是非托管的 - 并且委托不会被收集,因为它仍然在范围内(静态)。

该结构实际​​上作为另一个回调的返回值传递给库,但作为指针:

private static IntPtr beginCallback(IntPtr localData, en_ImageType imageType)
{
    return _imageProtocolPtr;
}

基本上还有另一个结构类型保存图像文件名和指向该回调的函数指针,库确定存储的图像类型在文件中并使用此回调来请求给定类型的正确协议结构。 我的文件名结构的声明和管理方式与上面的协议相同,因此可能包含相同的错误,但由于该委托仅被调用一次并被快速调用,所以我还没有遇到任何问题。


编辑更新

感谢大家的回复,但在这个问题上又花了几天时间但没有取得任何进展后,我决定搁置它。 如果有人感兴趣,我正在尝试为 Lightwave 3D 渲染应用程序的用户编写一个工具,一个很好的功能是能够查看 Lightwave 支持的所有不同图像格式(其中一些相当奇特)。 我认为最好的方法是为 Lightwave 用于图像处理的插件架构编写一个 C# 包装器,这样我就可以使用他们的代码来实际加载文件。 不幸的是,在针对我的解决方案尝试了许多插件之后,我遇到了各种我无法理解或修复的错误,我的猜测是 Lightwave 没有以标准方式调用插件上的方法,可能是为了提高安全性运行外部代码(我承认是在黑暗中进行的疯狂刺杀)。 目前我将放弃图像功能,如果我决定恢复它,我将以不同的方式处理它。

再次感谢,尽管我没有得到我想要的结果,但通过这个过程我学到了很多东西。

I'm trying to use an unmanaged C dll for loading image data into a C# application. The library has a fairly simple interface where you pass in a struct that contains three callbacks, one to receive the size of the image, one that receives each row of the pixels and finally one called when the load is completed. Like this (C# managed definition):

[System.Runtime.InteropServices.StructLayoutAttribute(System.Runtime.InteropServices.LayoutKind.Sequential)]
public struct st_ImageProtocol
{
   public st_ImageProtocol_done Done;    
   public st_ImageProtocol_setSize SetSize;    
   public st_ImageProtocol_sendLine SendLine;
}

The types starting st_ImageProtocol are delgates:

public delegate int st_ImageProtocol_sendLine(System.IntPtr localData, int rowNumber, System.IntPtr pixelData);

With the test file that I'm using the SetSize should get called once, then the SendLine will get called 200 times (once for each row of pixels in the image), finally the Done callback gets triggered. What actually happens is that the SendLine is called 19 times and then a AccessViolationException is thrown claiming that the library tried to access protected memory.

I have access to the code of the C library (though I can't change the functionality) and during the loop where it calls the SendLine method it does not allocate or free any new memory, so my assumption is that the delegate itself is the issue and I need to pin it before I pass it in (I have no code inside the delegate itself currently, besides a counter to see how often it gets called, so I doubt I'm breaking anything on the managed side). The problem is that I don't know how to do this; the method I've been using to declare the structs in unmanaged space doesn't work with delegates (Marshal.AllocHGlobal()) and I can't find any other suitable method. The delegates themselves are static fields in the Program class so they shouldn't be being garbage collected, but I guess the runtime could be moving them.

This blog entry by Chris Brumme says that delegates don't need to be pinned before being passed into unmanaged code:

Clearly the unmanaged function pointer must refer to a fixed address. It would be a disaster if the GC were relocating that! This leads many applications to create a pinning handle for the delegate. This is completely unnecessary. The unmanaged function pointer actually refers to a native code stub that we dynamically generate to perform the transition & marshaling. This stub exists in fixed memory outside of the GC heap.

But I don't know if this holds true when the delegate is part of a struct. It does imply that it is possible to manually pin them though, and I'm interested in how to do this or any better suggestions as to why a loop would run 19 times then suddenly fail.

Thanks.


Edited to answer Johan's questions...

The code that allocates the struct is as follows:

_sendLineFunc = new st_ImageProtocol_sendLine(protocolSendLineStub);

_imageProtocol = new st_ImageProtocol()
                     {
                          //Set some other properties...
                          SendLine = _sendLineFunc
                     };

int protocolSize = Marshal.SizeOf(_imageProtocol);
_imageProtocolPtr = Marshal.AllocHGlobal(protocolSize);
Marshal.StructureToPtr(_imageProtocol, _imageProtocolPtr, true);

Where the _sendLineFunc and the _imageProtocol variables are both static fields of the Program class. If I understand the internals of this correctly, that means that I'm passing an unmanaged pointer to a copy of the _imageProtocol variable into the C library, but that copy contains a reference to the static _sendLineFunc. This should mean that the copy isn't touched by the GC - since it is unmanaged - and the delegate won't be collected since it is still in scope (static).

The struct actually gets passed to the library as a return value from another callback, but as a pointer:

private static IntPtr beginCallback(IntPtr localData, en_ImageType imageType)
{
    return _imageProtocolPtr;
}

Basically there is another struct type that holds the image filename and the function pointer to this callback, the library figures out what type of image is stored in the file and uses this callback to request the correct protocol struct for the given type. My filename struct is declared and managed in the same way as the protocol one above, so probably contains the same mistakes, but since this delegate is only called once and called quickly I haven't had any problems with it yet.


Edited to update

Thanks to everybody for their responses, but after spending another couple of days on the problem and making no progress I decided to shelve it. In case anyone is interested I was attempting write a tool for users of the Lightwave 3D rendering application and a nice feature would have been the ability to view all the different image formats that Lightwave supports (some of which are fairly exotic). I thought that the best way to do this would be to write a C# wrapper for the plugin architecture that Lightwave uses for image manipulation so I could use their code to actually load the files. Unfortunately after trying a number of the plugins against my solution I had a variety of errors that I couldn't understand or fix and my guess is that Lightwave doesn't call the methods on the plugins in a standard way, probably to improve the security of running external code (wild stab in the dark, I admit). For the time being I'm going to drop the image feature and if I do decide to reinstate it I'll approach it in a different way.

Thanks again, I learnt a lot through this process even though I didn't get the result I wanted.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(4

只怪假的太真实 2024-07-23 15:20:35

了解更多信息会很有趣:

  • 如何创建 ImageProtocol 结构? 它是局部变量还是类成员,还是使用 Marshal.AllocHGlobal 在非托管内存中分配它?

  • 它是如何发送到C函数的? 直接作为堆栈变量还是作为指针?


真是一个棘手的问题! 感觉委托数据被 GC 移动,导致访问冲突。 有趣的是,委托数据类型是一种引用数据类型,它将其数据存储在 GC 堆上。 该数据包含诸如要调用的函数的地址(函数指针)之类的内容,而且还包含对包含该函数的对象的引用。 这应该意味着,即使实际的函数代码存储在 GC 堆之外,保存函数指针的数据也存储在 GC 堆中,因此可以由 GC 移动。 昨晚我想了很多这个问题,但没有想到解决办法......

It would be interesting to know a little more:

  • How do you create the ImageProtocol struct? Is it a local variable or a class member or do you allocate it in unmanaged memory with Marshal.AllocHGlobal?

  • How is it sent to the C function? Directly as stack variable or as a pointer?


A really tricky problem! It feels like the delegate data is moved around by the GC which causes the access violation. The interesting thing is that the delegate data type is a reference data type, which stores its data on the GC heap. This data contains things like the address of the function to call (function pointer) but also a reference to the object that contains the function. This should mean that even though the actual function code is stored outside of the GC heap, the data that holds the function pointer is stored in the GC heap and can hence be moved by the GC. I thought about the problem a lot last night but haven't come up with a solution....

只想待在家 2024-07-23 15:20:35

您没有确切说明回调在 C 库中是如何声明的。 除非显式声明__stdcall,否则您将慢慢损坏堆栈。 您将看到您的方法被调用(可能参数相反),但在将来的某个时刻程序将崩溃。

据我所知,除了用 C 语言编写另一个回调函数(位于 C# 代码和需要 __cdecl 回调的库之间)之外,没有其他办法解决这个问题。

You don't say exactly how the callback is declared in the C library. Unless it is explictly declared __stdcall you'll slowly corrupt your stack. You'll see your method get called (probably with the parameters reversed) but at some point in the future the program will crash.

So far as I know there is no way around that, other than writing another callback function in C that sits between the C# code and the library that wants a __cdecl callback.

柳若烟 2024-07-23 15:20:35

如果 c 函数是 __cdecl 函数,那么您必须使用属性
[非托管函数指针(CallingConvention.Cdecl)]
在代表声明之前。

If the c function is a __cdecl function then you have to use the Attribut
[UnmanagedFunctionPointer(CallingConvention.Cdecl)]
before the delegate declaration.

洒一地阳光 2024-07-23 15:20:34

我在注册回调委托时遇到了类似的问题(它会被调用,然后噗!)。 我的问题是具有被委托方法的对象正在被 GC 处理。 我在一个更全局的地方创建了该对象,以防止它被 GC 回收。

如果类似的方法不起作用,请查看以下其他一些内容:

作为附加信息,请查看 GetFunctionPointerForDelegate 来自 Marshal 类。 这是您可以做到这一点的另一种方式。 只需确保代表未被 GC 处理即可。 然后,将它们声明为 IntPtr,而不是结构中的委托。

这可能无法解决固定问题,但请查看 < code>fixed 关键字,尽管这可能对您不起作用,因为您处理的生命周期比通常使用的生命周期更长。

最后,看看stackalloc< /a> 用于创建非 GC 内存。 这些方法需要使用unsafe,因此可能会对您的程序集施加一些其他限制。

I had a similar problem when registering a callback delegate (it would be called, then poof!). My problem was that the object with the method being delegated was getting GC'ed. I created the object in a more global place so as to keep it from being GC'ed.

If something like that doesn't work, here are some other things to look at:

As additional info, take a look at GetFunctionPointerForDelegate from the Marshal class. That is another way you could do this. Just make sure that the delegates are not GC'ed. Then, instead of delegates in your struct, declare them as IntPtr.

That may not solve the pinning, but take a look at fixed keyword, even though that may not work for you since you are dealing with a longer lifetime than for what that is typically used.

Finally, look at stackalloc for creating non-GC memory. These methods will require the use of unsafe, and might therefore put some other constraints on your Assemblies.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文