SWIG 生成的扩展造成内存泄漏

发布于 2024-09-12 17:45:25 字数 4757 浏览 10 评论 0 原文

我在使用 SWIG 在 PHP 中包装 C++ 库时遇到内存泄漏问题。当启用导向器时将包含复杂类型的 C++ 回调发送到 PHP 时,似乎会发生这种情况。下面是一个重现泄漏的独立示例:

Client.hpp:

#ifndef CLIENT_HPP_
#define CLIENT_HPP_

#include <vector>
#include "ProcedureCallback.hpp"

class Client {
public:
    void invoke(ProcedureCallback *callback) {
        callback->callback(std::vector<int>(0));
    }
};

#endif /* CLIENT_HPP_ */

ProcedureCallback.hpp:

#ifndef PROCEDURECALLBACK_HPP_
#define PROCEDURECALLBACK_HPP_

#include <vector>

class ProcedureCallback {
public:
    virtual void callback(std::vector<int>) = 0;
};

#endif /* PROCEDURECALLBACK_HPP_ */

因此要使用它,您创建一个 Client,将子类 ProcedureCallback 传递给 Client 的 invoke 方法,然后客户端调用您给它的callback 方法,并传递一个空的 int 向量。

这是 SWIG 接口文件:

%module(directors="1") debugasync
%feature("director");

%{
#include "Client.hpp"
#include "ProcedureCallback.hpp"
%}

%include "Client.hpp"
%include "ProcedureCallback.hpp"

它的输出非常大,因此我将其放在 Pastebin 上: debugasync_wrap.cpp 。对此文件感兴趣的可能是 SwigDirector_ProcedureCallback::callback (第 1319 行):

void SwigDirector_ProcedureCallback::callback(std::vector< int > arg0) {
  zval *args[1];
  zval *result, funcname;
  MAKE_STD_ZVAL(result);
  ZVAL_STRING(&funcname, (char *)"callback", 0);
  if (!swig_self) {
    SWIG_PHP_Error(E_ERROR, "this pointer is NULL");
  }

  zval obj0;
  args[0] = &obj0;
  {
    SWIG_SetPointerZval(&obj0, SWIG_as_voidptr(&arg0), SWIGTYPE_p_std__vectorT_int_t, 2);
  }
  call_user_function(EG(function_table), (zval**)&swig_self, &funcname,
    result, 1, args TSRMLS_CC);
  FREE_ZVAL(result);
  return;
fail:
  zend_error(SWIG_ErrorCode(),"%s",SWIG_ErrorMsg());
}

这也可能令人感兴趣(第 827 行):

static void
SWIG_ZTS_SetPointerZval(zval *z, void *ptr, swig_type_info *type, int newobject TSRMLS_DC) {
  swig_object_wrapper *value=NULL;
  /*
   * First test for Null pointers.  Return those as PHP native NULL
   */
  if (!ptr ) {
    ZVAL_NULL(z);
    return;
  }
  if (type->clientdata) {
    if (! (*(int *)(type->clientdata)))
      zend_error(E_ERROR, "Type: %s failed to register with zend",type->name);
    value=(swig_object_wrapper *)emalloc(sizeof(swig_object_wrapper));
    value->ptr=ptr;
    value->newobject=newobject;
    if (newobject <= 1) {
      /* Just register the pointer as a resource. */
      ZEND_REGISTER_RESOURCE(z, value, *(int *)(type->clientdata));
    } else {
      /*
       * Wrap the resource in an object, the resource will be accessible
       * via the "_cPtr" member. This is currently only used by
       * directorin typemaps.
       */
      value->newobject = 0;
      zval *resource;
      MAKE_STD_ZVAL(resource);
      ZEND_REGISTER_RESOURCE(resource, value, *(int *)(type->clientdata));
      zend_class_entry **ce = NULL;
      zval *classname;
      MAKE_STD_ZVAL(classname);
      /* _p_Foo -> Foo */
      ZVAL_STRING(classname, (char*)type->name+3, 1);
      /* class names are stored in lowercase */
      php_strtolower(Z_STRVAL_PP(&classname), Z_STRLEN_PP(&classname));
      if (zend_lookup_class(Z_STRVAL_P(classname), Z_STRLEN_P(classname), &ce TSRMLS_CC) != SUCCESS) {
        /* class does not exist */
        object_init(z);
      } else {
        object_init_ex(z, *ce);
      }
      Z_SET_REFCOUNT_P(z, 1);
      Z_SET_ISREF_P(z);
      zend_hash_update(HASH_OF(z), (char*)"_cPtr", sizeof("_cPtr"), (void*)&resource, sizeof(zval), NULL);
      FREE_ZVAL(classname);
    }
    return;
  }
  zend_error(E_ERROR, "Type: %s not registered with zend",type->name);
}

并演示 PHP 中的内存泄漏(debugasync.php 是一组由 SWIG 生成的代理类,我也将其上传到 Pastebin):

<?php

require('debugasync.php');

class MyCallback extends ProcedureCallback {
    public function callback($intVector) {}
}

$client = new Client();
$callback = new MyCallback();

while (true) {
    print(number_format(memory_get_usage()) . "\n");
    for ($j = 0; $j < 1000; $j++) {
        $client->invoke($callback);
    }
}

这会打印内存使用情况,执行 1k 次调用,然后重复。运行它会显示快速增长的内存空间:

$ php test.php 
692,664
1,605,488
2,583,232
3,634,776
4,538,784
5,737,760
6,641,768
7,545,816
^C

还值得注意的是,如果 C++ 回调传递基元(即 int)而不是复杂类型(即 std::vector), ),不存在内存泄漏。

造成这种内存泄漏的原因是什么?

更一般地说,我可以使用哪些工具来解决这个问题?即使在使用调试符号构建 PHP 之后,Valgrind 的 Massif 还没有真正能够缩小正在发生的事情的范围。

I'm having a memory leak problem wrapping a C++ library in PHP using SWIG. It seems to happen when callbacks from C++ containing complex types are sent to PHP while directors are enabled. Here is a standalone example to reproduce the leak:

Client.hpp:

#ifndef CLIENT_HPP_
#define CLIENT_HPP_

#include <vector>
#include "ProcedureCallback.hpp"

class Client {
public:
    void invoke(ProcedureCallback *callback) {
        callback->callback(std::vector<int>(0));
    }
};

#endif /* CLIENT_HPP_ */

ProcedureCallback.hpp:

#ifndef PROCEDURECALLBACK_HPP_
#define PROCEDURECALLBACK_HPP_

#include <vector>

class ProcedureCallback {
public:
    virtual void callback(std::vector<int>) = 0;
};

#endif /* PROCEDURECALLBACK_HPP_ */

So to use this, you create a Client, pass a subclassed ProcedureCallback to Client's invoke method, and Client then goes and calls the callback method of what you gave it, and passes an empty int vector.

This is the SWIG interface file:

%module(directors="1") debugasync
%feature("director");

%{
#include "Client.hpp"
#include "ProcedureCallback.hpp"
%}

%include "Client.hpp"
%include "ProcedureCallback.hpp"

Its output is very large, so I put it on pastebin instead: debugasync_wrap.cpp. Of interest in this file is probably SwigDirector_ProcedureCallback::callback (line 1319):

void SwigDirector_ProcedureCallback::callback(std::vector< int > arg0) {
  zval *args[1];
  zval *result, funcname;
  MAKE_STD_ZVAL(result);
  ZVAL_STRING(&funcname, (char *)"callback", 0);
  if (!swig_self) {
    SWIG_PHP_Error(E_ERROR, "this pointer is NULL");
  }

  zval obj0;
  args[0] = &obj0;
  {
    SWIG_SetPointerZval(&obj0, SWIG_as_voidptr(&arg0), SWIGTYPE_p_std__vectorT_int_t, 2);
  }
  call_user_function(EG(function_table), (zval**)&swig_self, &funcname,
    result, 1, args TSRMLS_CC);
  FREE_ZVAL(result);
  return;
fail:
  zend_error(SWIG_ErrorCode(),"%s",SWIG_ErrorMsg());
}

This may also be of interest (line 827):

static void
SWIG_ZTS_SetPointerZval(zval *z, void *ptr, swig_type_info *type, int newobject TSRMLS_DC) {
  swig_object_wrapper *value=NULL;
  /*
   * First test for Null pointers.  Return those as PHP native NULL
   */
  if (!ptr ) {
    ZVAL_NULL(z);
    return;
  }
  if (type->clientdata) {
    if (! (*(int *)(type->clientdata)))
      zend_error(E_ERROR, "Type: %s failed to register with zend",type->name);
    value=(swig_object_wrapper *)emalloc(sizeof(swig_object_wrapper));
    value->ptr=ptr;
    value->newobject=newobject;
    if (newobject <= 1) {
      /* Just register the pointer as a resource. */
      ZEND_REGISTER_RESOURCE(z, value, *(int *)(type->clientdata));
    } else {
      /*
       * Wrap the resource in an object, the resource will be accessible
       * via the "_cPtr" member. This is currently only used by
       * directorin typemaps.
       */
      value->newobject = 0;
      zval *resource;
      MAKE_STD_ZVAL(resource);
      ZEND_REGISTER_RESOURCE(resource, value, *(int *)(type->clientdata));
      zend_class_entry **ce = NULL;
      zval *classname;
      MAKE_STD_ZVAL(classname);
      /* _p_Foo -> Foo */
      ZVAL_STRING(classname, (char*)type->name+3, 1);
      /* class names are stored in lowercase */
      php_strtolower(Z_STRVAL_PP(&classname), Z_STRLEN_PP(&classname));
      if (zend_lookup_class(Z_STRVAL_P(classname), Z_STRLEN_P(classname), &ce TSRMLS_CC) != SUCCESS) {
        /* class does not exist */
        object_init(z);
      } else {
        object_init_ex(z, *ce);
      }
      Z_SET_REFCOUNT_P(z, 1);
      Z_SET_ISREF_P(z);
      zend_hash_update(HASH_OF(z), (char*)"_cPtr", sizeof("_cPtr"), (void*)&resource, sizeof(zval), NULL);
      FREE_ZVAL(classname);
    }
    return;
  }
  zend_error(E_ERROR, "Type: %s not registered with zend",type->name);
}

And to demonstrate the memory leak in PHP (debugasync.php is a set of proxy classes generated by SWIG which I have also uploaded to pastebin):

<?php

require('debugasync.php');

class MyCallback extends ProcedureCallback {
    public function callback($intVector) {}
}

$client = new Client();
$callback = new MyCallback();

while (true) {
    print(number_format(memory_get_usage()) . "\n");
    for ($j = 0; $j < 1000; $j++) {
        $client->invoke($callback);
    }
}

This prints memory usage, does 1k invocations, and repeats. Running it shows a quickly-growing memory space:

$ php test.php 
692,664
1,605,488
2,583,232
3,634,776
4,538,784
5,737,760
6,641,768
7,545,816
^C

Also of note is that if the C++ callback passes a primitive (i.e. int) instead of a complex type (i.e. std::vector<int>), there is no memory leak.

What is the cause of this memory leak?

And more generally, what tools can I use to solve this? Valgrind's massif hasn't really been able to narrow down what's going on, even after building PHP with debugging symbols.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

决绝 2024-09-19 17:45:25

我对 SWIG 一无所知,但如果内存使用情况是由 memory_get_usage 报告的,那么占用的内存将由 Zend Engine 内存管理器分配。

当脚本干净完成(没有 CTRL+C 或 die)时,内存管理器会告诉您它发现的内存泄漏,只要:

  • PHP 在调试模式下编译(--enable-debug)
  • 您的 php.ini 文件中有 report_memleaks = true

这将告诉您未释放的内存是在哪里分配的。

也就是说,你的片段没有什么特别有趣的地方;唯一的非堆栈分配变量被正确处理。

I know nothing about SWIG specifically, but if the memory usage is reported by memory_get_usage, then the taken memory is allocated with the Zend Engine memory manager.

When the script finishes cleanly (no CTRL+C or die), the memory manager will tell you about the memory leaks it's found as long as:

  • PHP is compiled in debug mode (--enable-debug)
  • You have report_memleaks = true in your php.ini file

This will tell you where the memory that wasn't freed was allocated.

That said, there isn't anything specially funny with your snippet; the only non-stack allocated variable is properly disposed of.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文