std::map 节点对齐
我正面临着一些非常奇妙的事情。 (场景:Win7 pro 64位,VC2008编译32位代码)
假设主程序实例化一个使用std::map的Class。
这是一个 std::map
class Props {
public:
/**
* value of property
* @var boost::any
*/
boost::any _a ;
/**
* expire time
* @var time_t
*/
time_t _tExpiry ;
/* Somethign more I'm not writing down here... */
}
现在...我构建一个 DLL,使用相同的类来完成自己的业务。
DLL 实例化该类并提供 std::map。
好吧...当主程序提供地图时,一切都正常,而 DLL 在第一个项目插入后崩溃。
更多(非常有趣)的事情。
如果我深入主程序完成的插入,我会到达 std::map
中单个 _Node 的构造函数 _Node 如下所示 (c:\Programmi\Microsoft Visual Studio 9.0\VC\include\xtree)
struct _Node
{ // tree node
_Node(_Nodeptr _Larg, _Nodeptr _Parg, _Nodeptr _Rarg,
const value_type& _Val, char _Carg)
: _Left(_Larg), _Parent(_Parg), _Right(_Rarg),
_Myval(_Val), _Color(_Carg), _Isnil(false)
{ // construct a node with value
}
_Nodeptr _Left; // left subtree, or smallest element if head
_Nodeptr _Parent; // parent, or root of tree if head
_Nodeptr _Right; // right subtree, or largest element if head
value_type _Myval; // the stored value, unused if head
char _Color; // _Red or _Black, _Black if head
char _Isnil; // true only if head (also nil) node
};
很好...所以我们的 _Node 结构有 _Left (4bytes), _Parent (4 bytes), _Right (4 bytes) _Myval (sizeof地图的键和值)、_Color(1 字节)和 _Isnil(1 字节)。
现在...我想要为地图提供的元素是一对
根据我的调试器,std::string 需要 0x20 字节,而 Props 只需要 8 个字节。
现在,当我询问调试器单个节点的大小时,我可以看到它是 0x38。 所以...0x4 + 0x4 + 0x4 + 0x20 + 0x8 + 0x1 + 0x1 = 0x36 + 填充 = 0x38。
这意味着 _Color 成员在 _Node 开始后的 0x34 字节处开始。
(好吧...我宁愿指定我的所有项目都使用 /Zp4,因此所有结构都是 4 字节打包的)。
让我们继续...当我跟踪 DLL 行为时,我可以看到一些非常惊人的东西。
std::map 的 _Buynode() 方法调用分配器来为我要插入的新元素提供新的大小。调用分配器而不是调用 _Node() 就地构造函数......并且它以另一种方式起作用!
在这种情况下,构造函数的行为就像 _Color 成员从 _Node 开头的 0x38 字节之后开始...就好像有不同的填充一样。
此后,在接下来尝试插入新值时,该过程失败,因为 _Color 和 _Isnil 的值...错误(0xcc,因为该部分内存未初始化)。
我确定我将所有项目中的 /Zp4 设置为解决方案,所以...
怎么了?
我“感觉”对齐有问题,但我无法说出什么问题......
提前致谢!
好的...我要添加更多内容。
这是来自 c:\Programmi\Microsoft Visual Studio 9.0\VC\include\xtree 的 _Node 结构
struct _Node
{ // tree node
_Node(_Nodeptr _Larg, _Nodeptr _Parg, _Nodeptr _Rarg,
const value_type& _Val, char _Carg)
: _Left(_Larg), _Parent(_Parg), _Right(_Rarg),
_Myval(_Val), _Color(_Carg), _Isnil(false)
{ // construct a node with value
}
_Nodeptr _Left; // left subtree, or smallest element if head
_Nodeptr _Parent; // parent, or root of tree if head
_Nodeptr _Right; // right subtree, or largest element if head
value_type _Myval; // the stored value, unused if head
char _Color; // _Red or _Black, _Black if head
char _Isnil; // true only if head (also nil) node
};
_Tree_nod(const key_compare& _Parg,
allocator_type _Al)
: _Traits(_Parg, _Al), _Alnod(_Al)
{ // construct traits from _Parg and allocator from _Al
}
typename allocator_type::template rebind<_Node>::other
_Alnod; // allocator object for nodes
};
如您所见,它的构造函数非常简单。
当我像之前所说的那样寻找 std::map 时,查看此结构使用的内存。
- _Left
4 个字节 - 4 个字节用于 _Parent
- _Right
4 个字节 - _Myval 为 0x28 字节(std::string 为 0x20,我自己的类为 8。我检查过,它是 8 字节)
- 1 个字节用于_Color
- _Isnil 的 1 个字节
当创建一个放置在树顶(我的意思是最左边)的新元素时,该构造函数将填充 _Left、_Parent、_Right、_Myval,而不是未初始化的 4 个字节(例如用 0xcc 填充)并填充什么它认为应该是 _Color 和 _Isnil 提前 4 个字节。
最荒谬的是 std::map 的其他方法不会“感觉到”这 4 个字节。
这些是导致断言的行。
if (_Isnil(_Ptr))
{
_Ptr = _Right(_Ptr); // end() ==> rightmost
if (_Isnil(_Ptr))
#if _HAS_ITERATOR_DEBUGGING
{
_DEBUG_ERROR("map/set iterator not decrementable");
_SCL_SECURE_OUT_OF_RANGE;
}
#elif _SECURE_SCL
{
_SCL_SECURE_OUT_OF_RANGE;
}
#else
return; // begin() shouldn't be incremented, don't move
#endif
}
发生这种情况是因为对 _IsNil(_Ptr) 的测试发现 _Isnil 是 0xcc,并给出了错误。发布版本可能不喜欢这不是一种乐趣。
有什么想法吗?
又迈出了一步!
我把整个解决方案(2个大项目,18个小项目,大约250000 C/C++代码行),并在Linux(gcc4.1.2)下编译。
一切都工作得很好。完全没有问题,std::map 工作正常。
我不得不说 Linux Makefile 让一切变得更加简单,同时也变得更加复杂。复杂是因为你必须自己做所有事情,简单是因为如果你不愿意,什么也不会发生。
这告诉我一件事:Visual Studio 2008 中有些问题,在某些特定条件下会跳到舞台上......问题是“导致此问题的条件是什么?”。
等待一个想法...
I'm facing something very amazing. (Scenario: Win7 pro 64 bit, VC2008 compiling 32 bit code)
Say a main program instantiate a Class that uses a std::map.
It's a std::map<std::string,Props> where class Props has only two members:
class Props {
public:
/**
* value of property
* @var boost::any
*/
boost::any _a ;
/**
* expire time
* @var time_t
*/
time_t _tExpiry ;
/* Somethign more I'm not writing down here... */
}
Now... I build a DLL, using the same class for it's own business.
The DLL instantiate that class and feed the std::map.
Well... When the main program feed the map everythig goes fine, while the DLL crashes after the first item insertion.
Something (very interesting) more.
If I go deep into insert done by the main program I reach the constructor of the single _Node into the std::map
_Node appear as follows (c:\Programmi\Microsoft Visual Studio 9.0\VC\include\xtree)
struct _Node
{ // tree node
_Node(_Nodeptr _Larg, _Nodeptr _Parg, _Nodeptr _Rarg,
const value_type& _Val, char _Carg)
: _Left(_Larg), _Parent(_Parg), _Right(_Rarg),
_Myval(_Val), _Color(_Carg), _Isnil(false)
{ // construct a node with value
}
_Nodeptr _Left; // left subtree, or smallest element if head
_Nodeptr _Parent; // parent, or root of tree if head
_Nodeptr _Right; // right subtree, or largest element if head
value_type _Myval; // the stored value, unused if head
char _Color; // _Red or _Black, _Black if head
char _Isnil; // true only if head (also nil) node
};
Very fine... so our _Node structure hase _Left (4bytes), _Parent (4 bytes), _Right (4 bytes) _Myval (sizeof key and value of the map), _Color (1byte) and _Isnil(1 byte).
Now... the element I want to feed the map with is a pair of <std::string, Props>.
According to my debugger, std::string requires 0x20 bytes, while Props only 8.
Now, when I ask my debugger the size of the single node, I can see it's 0x38.
So...0x4 + 0x4 + 0x4 + 0x20 + 0x8 + 0x1 + 0x1 = 0x36 + padding = 0x38.
This means that _Color member starts 0x34 bytes after the beginning of the _Node.
(Ok...I'd rather specify that all my projects use /Zp4 so all structures are 4 bytes packed).
Let's go on... When I follow the DLL behaviour I can se something very amazing.
The _Buynode() method of std::map calls the allocator to have new size for the new element I'm going to insert. The allocator is called than the _Node() in-place constructor is called.... and it acts another way!!
In this case, the constructor behaves as _Color member starts after 0x38 bytes from the beginning of the _Node... as if there's a different padding.
After this, on the following tryal to insert a new value, the procedure fails since the values of _Color and _Isnil are... wrong (0xcc since that part of memory isn't initialized).
I'm sure I set /Zp4 in all the projects into the solution so...
What's wrong?
I "feel" there's something wrong about aligment but I just can't say what...
Thanks in advance!
Ok... I'm going to add something more.
This is _Node structure from c:\Programmi\Microsoft Visual Studio 9.0\VC\include\xtree
struct _Node
{ // tree node
_Node(_Nodeptr _Larg, _Nodeptr _Parg, _Nodeptr _Rarg,
const value_type& _Val, char _Carg)
: _Left(_Larg), _Parent(_Parg), _Right(_Rarg),
_Myval(_Val), _Color(_Carg), _Isnil(false)
{ // construct a node with value
}
_Nodeptr _Left; // left subtree, or smallest element if head
_Nodeptr _Parent; // parent, or root of tree if head
_Nodeptr _Right; // right subtree, or largest element if head
value_type _Myval; // the stored value, unused if head
char _Color; // _Red or _Black, _Black if head
char _Isnil; // true only if head (also nil) node
};
_Tree_nod(const key_compare& _Parg,
allocator_type _Al)
: _Traits(_Parg, _Al), _Alnod(_Al)
{ // construct traits from _Parg and allocator from _Al
}
typename allocator_type::template rebind<_Node>::other
_Alnod; // allocator object for nodes
};
As you can see, it's costructor is pretty simple.
Looking at the memory used from this structure when I asf for a std::map as told before I have.
- 4 bytes for _Left
- 4 bytes for _Parent
- 4 bytes for _Right
- 0x28 bytes for _Myval (0x20 for the std::string and 8 for my own class. I checked, it's alqays 8 bytes)
- 1 byte for _Color
- 1 byte for _Isnil
When a new element placetd atop of the tree (I mean, the left most) is created, this costructor fills _Left, _Parent, _Right, _Myval, than lat 4 bytes uninitialized (say filled with 0xcc) and fills what it thinks should be _Color and _Isnil 4 bytes ahead.
The most absurde thing is that the other methods of std::map don't "feel" those 4 bytes.
Those are the lines that cause de assertion.
if (_Isnil(_Ptr))
{
_Ptr = _Right(_Ptr); // end() ==> rightmost
if (_Isnil(_Ptr))
#if _HAS_ITERATOR_DEBUGGING
{
_DEBUG_ERROR("map/set iterator not decrementable");
_SCL_SECURE_OUT_OF_RANGE;
}
#elif _SECURE_SCL
{
_SCL_SECURE_OUT_OF_RANGE;
}
#else
return; // begin() shouldn't be incremented, don't move
#endif
}
This happens since the test for _IsNil(_Ptr), finding that _Isnil is 0xcc, gives an error. Release version probably don't nut this is not a joy.
Any idea?
Another Step!
I took the whole solution (2 Big projects, 18 small ones, about 250000 C/C++ code lines), and compile it under Linux (gcc4.1.2).
Everything works veeeeeery fine. No problems at all and std::map works properly.
I have to say that Linux Makefile make everything more easy and complex at the same time. Complex because you have to do everything on your own, easy because nothing happens if you don't want to.
This tells me one thing: there's somethign wrong in Visual Studio 2008, something that jumps on the stage in some particular condition... the problem is "what's the condition that causes this?".
Waiting for an idea...
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(3)
我只是猜测,但我的直接猜测是您的问题并非源于对齐问题。
使用VC++,如果将这样的东西放入DLL中,则必须编译代码(主程序和DLL)才能使用DLL中的标准库。当您执行此操作时,主程序和 DLL 共享一个公共堆,因此它们可以分配/释放内存并且保持同步。
如果静态链接到标准库,则主程序和 DLL 将具有单独的堆。当/如果一个中的代码尝试删除在另一个中分配的项目(例如),您将遇到重大问题,因为它会尝试将其返回到并非来自其中的堆第一名。
I'm only guessing, but my immediate guess would be that your problem doesn't stem from alignment issues.
With VC++, if you put something like this in a DLL, you have to compile the code (both the main program and the DLL) to use the standard library in a DLL. When you do this, the main program and the DLL share a common heap, so they can allocate/free memory and things stay in synch.
If you link to the standard library statically, your main program and your DLL will have separate heaps. When/if code in one tries to delete an item that was allocated in the other (for one example), you'll run into major problems, because it'll try to return it to a heap that it didn't come from in the first place.
编译器选项为 /Zp4。您确定在所有情况下都使用了正确的选项吗?
The compiler option is /Zp4. Are you sure you used the correct option in all cases?
您的描述肯定指出了对齐/包装问题。
也许您可以在
sizeof(Props) == 8
上添加编译时检查。 Boost 有一个编译时检查模板,但在互联网上找到一个或创建自己的模板并不难。如果 sizeof 结果始终相同,请让我指向 检查迭代器以获取替代的检查内容。
在那之后,我就没有想法了。
Your description surely points in the direction of a alignment/packing problem.
Maybe you could add a compile-time check on the
sizeof(Props) == 8
. Boost has a compile-time check template, but it's not that hard to find one on internet or roll your own.If the sizeof turns out to be always the same, let me point at checked iterators for an alternative thing to check.
After that, I'm running out of ideas.