编写 Matlab C API 的替代品来编写 .mat 文件

发布于 2024-12-14 14:55:10 字数 5976 浏览 2 评论 0原文

我正在研究将结果输出为 matlab 的 .mat 文件格式的研究模型,并最初与 matlab 库链接以使用其文件输出功能。 最近,需求发生了变化(谁能猜到),以前仅适用于 Linux 的代码现在必须在 Windows 上编译,并且最好不需要 matlab 来构建 - 但仍然输出 .mat 文件。
所以我搜索并找到了libmatio(http://sourceforge.net/projects/matio/)。虽然这在 Linux 中很容易链接(你只需从存储库安装它),但在 Windows 上它很糟糕(基本上没有关于在 Windows 上构建它的信息)。事实上,Windows 支持实际上在 1.3.3 版本中(早在 2008 年)就被悄悄地删除了。
另外,APi 与 matlab 提供的完全不同,这需要我重写/重构大量代码。

所以我想出了这个疯狂的主意...... 我需要一个 Matlab API 的直接替代品,最好不使用库(以便非程序员可以轻松编译),所以我开始编写一个。
我只实现我需要的功能(编写双精度数组、字符串和复杂双精度数组,以及结构和结构嵌套)。所有这些都已经可以正常工作,除了一个:结构数组。

所有 matlab 数据都包含在一个名为“mxArray”的结构中,并且根据其类型,它包含指向 double、complex double 或一个或多个其他 mxArray 的指针。
将 mxArray 写入文件之前的最后一步是通过调用 calcArraySize() 计算其大小(及其子级的大小)(以字节为单位)。
这会在某些时候导致段错误,因为我试图访问空指针。为了找出原因,我通过 valgrind 运行了代码。与往常一样,我尝试按照问题出现的顺序处理问题,因为它们可能是以后发生的事情的原因。
所以 valgrind 告诉我的第一件事是:

==8405== Invalid write of size 8
==8405==    at 0x00404541: mxSetFieldByNumber (mxSetFieldByNumber.c:18) [A]
==8405==    by 0x00411679: calcAllRayInfo (calcAllRayInfo.c:156)
==8405==    by 0x0041dd42: main (cTraceo.c:111)
==8405==    Address 0x5500250 is 0 bytes inside a block of size 4 alloc'd
==8405==    at 0x04c28f9f: malloc (vg_replace_malloc.c:236)
==8405==    by 0x00401066: mallocChar (toolsMemory.c:69)
==8405==    by 0x00404314: mxCreateStructMatrix (mxCreateStructMatrix.c:43) [B]
==8405==    by 0x00411235: calcAllRayInfo (calcAllRayInfo.c:105)
==8405==    by 0x0041dd42: main (cTraceo.c:111)

注意:我在下面的代码中标记了 [A] 和 [B]。
结构定义(仅显示相关成员):

struct mxArray{
  bool           isStruct;  //determines if this mxArray is a structure (which contains other mxArrays)
  bool           isChild;   //determines wheter this mxArray is a Child of another (when set, its name will not be written to the matfile, as it is already defined in the parent's fieldnames
  uintptr_t      nFields;
  char           **fieldNames;  //something like: {"theta","r","z"};
  struct mxArray **field; //pointer to member mxArrays. only used when isStruct is set.
};
typedef struct mxArray mxArray;

我用来为 structMatrix 及其内容分配内存的函数:

mxArray* mxCreateStructMatrix(uintptr_t nRows, uintptr_t nCols, uintptr_t nFields, const char **fieldNames){
  /*
   * creates a 2D array of structures
   */
  mxArray*  outArray = NULL;

  /* do some input value validation */

  // allocate memory
  outArray  = malloc(nRows*nCols*sizeof(mxArray));
  if (outArray == NULL){
    fatal("mxCreateStructMatrix(): memory allocation error.");
  }

  // allocate memory for structure members (fields)
  for (uintptr_t iStruct=0; iStruct<nCols*nRows; iStruct++){
    outArray[iStruct].nFields       = nFields;
    outArray[iStruct].fieldNames        = malloc(nFields*sizeof(char*));

    //copy fieldnames into struct info
    for (uintptr_t iField=0; iField<nFields; iField++){
      //NOTE: strlen returns length of string not including the terminating NULL character
      outArray[iStruct].fieldNames[iField] = mallocChar(strlen(fieldNames[iField])+1);  // [B] <=======
      strncpy(outArray[iStruct].fieldNames[iField], fieldNames[iField], strlen(fieldNames[iField]));
    }

    outArray[iStruct].field     = NULL;
    outArray[iStruct].field     = malloc(nFields*sizeof(mxArray*));
    if (outArray[iStruct].field == NULL){
      fatal("mxCreateStructMatrix(): memory allocation error.\n");
    }
  }
return outArray;
}

mxArrays 还存在另外两个分配函数:

mxArray* mxCreateDoubleMatrix(uintptr_t nRows, uintptr_t nCols, uintptr_t numericType){
  /*
   * creates a 2D array of double precision floating point values.
   * can be real or complex.
   */
  [snip]
}
mxArray* mxCreateString(const char *inString)
  /*
   * creates an mxArray containing a string.
   */
  [snip]
}

该函数将一个 mxArray 指定为另一个 mxArray 的子级:

void    mxSetFieldByNumber(mxArray* mxStruct,       //pointer to the mxStruct
                           uint32_t index,      //linear index of the element 
                           uint32_t iField,     //index of the structure's field which we want to set.
                           mxArray* inArray){       //the mxArray we want to assign to the mxStruct
  /* 
   * Assigns an mxArray to one of the fields of a structArray
   */
  inArray->isChild = true;  //determines that this mxArray is a child of another one
  mxStruct[index].field[iField] = inArray;  // [A] <===============
}

用法是:

//create parent mxArray:
mxStruct = mxCreateStructMatrix(1, //number of rows
                                1, //number of columns
                                2, //number of fields in each element
                                fieldNames1);   //list of field names

//create children:
mxY = mxCreateDoubleMatrix(1 ,1, mxREAL);
mxZ = mxCreateDoubleMatrix(1 ,1, mxREAL);
mxSubStruct = mxCreateStructMatrix(1, //number of rows
                                   1, //number of columns
                                   3, //number of fields in each element
                                   fieldNames2); //list of field names

/* copy some values into the mxArrays */
[snip]

//link children to parents
mxSetFieldByNumber( mxStruct, //pointer to the parent mxArray
                    0,        //index of the element (linear)
                    0,        //position of the field (in this case, field 0 is "w"
                    mxY);     //the mxArray we want to add to the mxStruct

mxSetFieldByNumber( mxStruct,   0,  1,  mxZ);

mxSetFieldByNumber( mxSubStruct,    0,  0,  mxY);
mxSetFieldByNumber( mxSubStruct,    0,  1,  mxZ);

mxSetFieldByNumber( mxStruct,   0,  2,  mxSubStruct);

所以显然, mxStruct[index].field[iField] = inArray; 正在写入 mxStruct[index].fieldNames,从而留下mxStruct[index].field[iField] == NULL,当我尝试访问它时会导致段错误。
怎么会这样呢?调用mxCreateStructMatrix时两者都被正确分配,那么这些指针如何重叠呢?我所忽略的是什么?

I'm working on research model which outputs results to matlab's .mat file format, and initially linked with the matlab library to use its file outputting functions.
Recently, requirements changed (who'd have guessed) and the previously linux-only code now has to be compiled on windows, and preferably not require matlab for building -but still output .mat files.
So I searched and found libmatio (http://sourceforge.net/projects/matio/). Although this is easy to link with in linux (you just install it from repositories), its terrible on windows (there is basically no information on building it on windows). In fact it seems windows support was actually silently dropped in version 1.3.3 (back in 2008).
Also, the APi is completely different from the one provided by matlab, which would require me to rewrite/restructure way to much code.

So i came up with this crazy idea...
I needed a drop-in replacement for the Matlab API, preferably without using a library (to make it easy to compile for non-programmers), so i started writing one.
I'm only implementing the functionality I need (writing arrays of doubles, strings, and complex doubles, as well as structures, and structure nesting). All of those already work fine, except for one: arrays of structures.

All matlab data is contained in one struct called 'mxArray', and depending on it's type, it contains pointers to double, complex double, or one or more other mxArray.
The final step just before writing an mxArray to a file is calculating it's size (and that of it's children) in Bytes, by caling calcArraySize().
That causes a segfault at some point, because I'm tryng to access a null pointer. To track down the cause I ran the code through valgrind. As always, I try to take care of any issues in the order they arise, as they may be the cause of what's happening later.
So the first thing valgrind tells me about is:

==8405== Invalid write of size 8
==8405==    at 0x00404541: mxSetFieldByNumber (mxSetFieldByNumber.c:18) [A]
==8405==    by 0x00411679: calcAllRayInfo (calcAllRayInfo.c:156)
==8405==    by 0x0041dd42: main (cTraceo.c:111)
==8405==    Address 0x5500250 is 0 bytes inside a block of size 4 alloc'd
==8405==    at 0x04c28f9f: malloc (vg_replace_malloc.c:236)
==8405==    by 0x00401066: mallocChar (toolsMemory.c:69)
==8405==    by 0x00404314: mxCreateStructMatrix (mxCreateStructMatrix.c:43) [B]
==8405==    by 0x00411235: calcAllRayInfo (calcAllRayInfo.c:105)
==8405==    by 0x0041dd42: main (cTraceo.c:111)

NOTE: I marked [A] and [B] in the code below.
The Structure definition (showing only relevant members):

struct mxArray{
  bool           isStruct;  //determines if this mxArray is a structure (which contains other mxArrays)
  bool           isChild;   //determines wheter this mxArray is a Child of another (when set, its name will not be written to the matfile, as it is already defined in the parent's fieldnames
  uintptr_t      nFields;
  char           **fieldNames;  //something like: {"theta","r","z"};
  struct mxArray **field; //pointer to member mxArrays. only used when isStruct is set.
};
typedef struct mxArray mxArray;

The function I use to allocate memory for a structMatrix along with it's contents:

mxArray* mxCreateStructMatrix(uintptr_t nRows, uintptr_t nCols, uintptr_t nFields, const char **fieldNames){
  /*
   * creates a 2D array of structures
   */
  mxArray*  outArray = NULL;

  /* do some input value validation */

  // allocate memory
  outArray  = malloc(nRows*nCols*sizeof(mxArray));
  if (outArray == NULL){
    fatal("mxCreateStructMatrix(): memory allocation error.");
  }

  // allocate memory for structure members (fields)
  for (uintptr_t iStruct=0; iStruct<nCols*nRows; iStruct++){
    outArray[iStruct].nFields       = nFields;
    outArray[iStruct].fieldNames        = malloc(nFields*sizeof(char*));

    //copy fieldnames into struct info
    for (uintptr_t iField=0; iField<nFields; iField++){
      //NOTE: strlen returns length of string not including the terminating NULL character
      outArray[iStruct].fieldNames[iField] = mallocChar(strlen(fieldNames[iField])+1);  // [B] <=======
      strncpy(outArray[iStruct].fieldNames[iField], fieldNames[iField], strlen(fieldNames[iField]));
    }

    outArray[iStruct].field     = NULL;
    outArray[iStruct].field     = malloc(nFields*sizeof(mxArray*));
    if (outArray[iStruct].field == NULL){
      fatal("mxCreateStructMatrix(): memory allocation error.\n");
    }
  }
return outArray;
}

Two other allocation functions exist for mxArrays:

mxArray* mxCreateDoubleMatrix(uintptr_t nRows, uintptr_t nCols, uintptr_t numericType){
  /*
   * creates a 2D array of double precision floating point values.
   * can be real or complex.
   */
  [snip]
}
mxArray* mxCreateString(const char *inString)
  /*
   * creates an mxArray containing a string.
   */
  [snip]
}

This functions assigns one mxArray to be the child of another:

void    mxSetFieldByNumber(mxArray* mxStruct,       //pointer to the mxStruct
                           uint32_t index,      //linear index of the element 
                           uint32_t iField,     //index of the structure's field which we want to set.
                           mxArray* inArray){       //the mxArray we want to assign to the mxStruct
  /* 
   * Assigns an mxArray to one of the fields of a structArray
   */
  inArray->isChild = true;  //determines that this mxArray is a child of another one
  mxStruct[index].field[iField] = inArray;  // [A] <===============
}

Usage is:

//create parent mxArray:
mxStruct = mxCreateStructMatrix(1, //number of rows
                                1, //number of columns
                                2, //number of fields in each element
                                fieldNames1);   //list of field names

//create children:
mxY = mxCreateDoubleMatrix(1 ,1, mxREAL);
mxZ = mxCreateDoubleMatrix(1 ,1, mxREAL);
mxSubStruct = mxCreateStructMatrix(1, //number of rows
                                   1, //number of columns
                                   3, //number of fields in each element
                                   fieldNames2); //list of field names

/* copy some values into the mxArrays */
[snip]

//link children to parents
mxSetFieldByNumber( mxStruct, //pointer to the parent mxArray
                    0,        //index of the element (linear)
                    0,        //position of the field (in this case, field 0 is "w"
                    mxY);     //the mxArray we want to add to the mxStruct

mxSetFieldByNumber( mxStruct,   0,  1,  mxZ);

mxSetFieldByNumber( mxSubStruct,    0,  0,  mxY);
mxSetFieldByNumber( mxSubStruct,    0,  1,  mxZ);

mxSetFieldByNumber( mxStruct,   0,  2,  mxSubStruct);

So aparently, mxStruct[index].field[iField] = inArray; is writing into mxStruct[index].fieldNames, thus leaving mxStruct[index].field[iField] == NULL, which then causes a segfault when i try to access it.
How can this be? both are allocated correctly when calling mxCreateStructMatrix, so how can these pointers overlap? What is it I'm overlooking?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

玩世 2024-12-21 14:55:10

我认为问题出在您的最后一条语句上:

mxSetFieldByNumber( mxStruct,   0,  /* THIRD FIELD */ 3,  mxSubStruct);

您试图将 mxStruct 的第三个字段分配给另一个嵌套结构变量,问题是 mxStruct 仅用两个字段定义:

mxStruct = mxCreateStructMatrix(1, 1, /* TWO */ 2, fieldNames1);

与 MATLAB 不同,您的代码(据我所知)不支持动态添加结构字段:

%# -- MATLAB code --
s = struct('f1',[], 'f2',[]);
s.f3 = 99;       %# add a new field

这不会很难实现,您只需重新分配指针数组以容纳另一个字段并增加字段计数。

I think the problem is with your very last statement:

mxSetFieldByNumber( mxStruct,   0,  /* THIRD FIELD */ 3,  mxSubStruct);

you are trying to assign the third field of mxStruct another nested structure variable, the problem is mxStruct was defined with only two fields:

mxStruct = mxCreateStructMatrix(1, 1, /* TWO */ 2, fieldNames1);

Unlike MATLAB, your code (as far as I can tell) does not support adding structure fields on the fly:

%# -- MATLAB code --
s = struct('f1',[], 'f2',[]);
s.f3 = 99;       %# add a new field

This wouldn't be very hard to implement, you simply re-allocate the pointer arrays to accommodate one more field and increment the field count.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文