内存映射文件 std::allocator 实现冻结 WM6 设备
我有一个适用于 Windows Mobile 6.x 的 Visual Studio 2008 C++ 项目,其中需要的内存多于 32MB 进程插槽中的可用内存。所以,我正在考虑使用内存映射文件。我创建了一个标准分配器实现,用 CreateFileMapping 替换 new/delete和 MapViewOfFile。
预期用途是这样的:
struct Foo
{
char a[ 1024 ];
};
int _tmain( int argc, _TCHAR* argv[] )
{
std::vector< boost::shared_ptr< Foo > > v;
for( int i = 0; i < 40000; ++i )
{
v.push_back( boost::allocate_shared< Foo >( MappedFileAllocator< Foo >() ) );
}
return 0;
}
使用 std::allocator
,在该示例中我可以在得到 std::bad_alloc
异常之前获得 28197 次迭代。使用 MappedFileAllocator
,在设备完全冻结并必须重新启动之前,我进行了 32371 次迭代。由于我的设备有 512MB RAM,我希望能够从该循环中获得更多迭代。
我的 MappedFileAllocator 实现是:
template< class T >
class MappedFileAllocator
{
public:
typedef T value_type;
typedef size_t size_type;
typedef ptrdiff_t difference_type;
typedef T* pointer;
typedef const T* const_pointer;
typedef T& reference;
typedef const T& const_reference;
pointer address( reference r ) const { return &r; };
const_pointer address( const_reference r ) const { return &r; };
/// convert a MappedFileAllocator<T> to a MappedFileAllocator<U>
template< class U >
struct rebind { typedef MappedFileAllocator< U > other; };
MappedFileAllocator() throw() : mapped_file_( INVALID_HANDLE_VALUE ) { };
template< class U >
explicit MappedFileAllocator( const MappedFileAllocator< U >& other ) throw()
: mapped_file_( INVALID_HANDLE_VALUE )
{
if( other.mapped_file_ != this->mapped_file_ )
{
::DuplicateHandle( GetCurrentProcess(),
other.mapped_file_,
GetCurrentProcess(),
&this->mapped_file_,
0,
FALSE,
DUPLICATE_SAME_ACCESS );
}
};
pointer allocate( size_type n, const void* /*hint*/ = 0 )
{
if( n > max_size() )
throw std::bad_alloc();
if( n > 0 )
{
size_type buf_size = n * sizeof( value_type );
mapped_file_ = ::CreateFileMapping( INVALID_HANDLE_VALUE,
NULL,
PAGE_READWRITE,
0,
buf_size,
L"{45E4FA7B-7B1E-4939-8CBB-811276B5D4DE}" );
if( NULL == mapped_file_ )
throw std::bad_alloc();
LPVOID f = ::MapViewOfFile( mapped_file_,
FILE_MAP_READ | FILE_MAP_WRITE,
0,
0,
buf_size );
if( NULL == f )
{
::CloseHandle( mapped_file_ );
mapped_file_ = INVALID_HANDLE_VALUE;
throw std::bad_alloc();
}
return reinterpret_cast< T* >( f );
}
return 0;
};
void deallocate( pointer p, size_type n )
{
if( NULL != p )
{
::FlushViewOfFile( p, n * sizeof( T ) );
::UnmapViewOfFile( p );
}
if( INVALID_HANDLE_VALUE != mapped_file_ )
{
::CloseHandle( mapped_file_ );
mapped_file_ = INVALID_HANDLE_VALUE;
}
};
size_type max_size() const throw()
{
return std::numeric_limits< size_type >::max() / sizeof( T );
};
/// handle to the memory-mapped file
HANDLE mapped_file_;
private:
/// disallow assignment
void operator=( const MappedFileAllocator& );
}; // class MappedFileAllocator
任何人都可以建议我的 MappedFileAllocator 实现哪里可能出错吗?
谢谢, 保罗·H
I have a Visual Studio 2008 C++ project for Windows Mobile 6.x where I need more memory than is available to me in the 32MB process slot. So, I'm looking at using memory mapped files. I've created a standard allocator implementation that replaces new/delete with CreateFileMapping and MapViewOfFile.
The intended use is something like this:
struct Foo
{
char a[ 1024 ];
};
int _tmain( int argc, _TCHAR* argv[] )
{
std::vector< boost::shared_ptr< Foo > > v;
for( int i = 0; i < 40000; ++i )
{
v.push_back( boost::allocate_shared< Foo >( MappedFileAllocator< Foo >() ) );
}
return 0;
}
With the std::allocator
, I can get 28197 iterations in that example before I get a std::bad_alloc
exception. With the MappedFileAllocator
, I get 32371 iterations before the device completely freezes and has to be rebooted. Since my device has 512MB of RAM, I expected to be able to get far more iterations out of that loop.
My MappedFileAllocator
implementation is:
template< class T >
class MappedFileAllocator
{
public:
typedef T value_type;
typedef size_t size_type;
typedef ptrdiff_t difference_type;
typedef T* pointer;
typedef const T* const_pointer;
typedef T& reference;
typedef const T& const_reference;
pointer address( reference r ) const { return &r; };
const_pointer address( const_reference r ) const { return &r; };
/// convert a MappedFileAllocator<T> to a MappedFileAllocator<U>
template< class U >
struct rebind { typedef MappedFileAllocator< U > other; };
MappedFileAllocator() throw() : mapped_file_( INVALID_HANDLE_VALUE ) { };
template< class U >
explicit MappedFileAllocator( const MappedFileAllocator< U >& other ) throw()
: mapped_file_( INVALID_HANDLE_VALUE )
{
if( other.mapped_file_ != this->mapped_file_ )
{
::DuplicateHandle( GetCurrentProcess(),
other.mapped_file_,
GetCurrentProcess(),
&this->mapped_file_,
0,
FALSE,
DUPLICATE_SAME_ACCESS );
}
};
pointer allocate( size_type n, const void* /*hint*/ = 0 )
{
if( n > max_size() )
throw std::bad_alloc();
if( n > 0 )
{
size_type buf_size = n * sizeof( value_type );
mapped_file_ = ::CreateFileMapping( INVALID_HANDLE_VALUE,
NULL,
PAGE_READWRITE,
0,
buf_size,
L"{45E4FA7B-7B1E-4939-8CBB-811276B5D4DE}" );
if( NULL == mapped_file_ )
throw std::bad_alloc();
LPVOID f = ::MapViewOfFile( mapped_file_,
FILE_MAP_READ | FILE_MAP_WRITE,
0,
0,
buf_size );
if( NULL == f )
{
::CloseHandle( mapped_file_ );
mapped_file_ = INVALID_HANDLE_VALUE;
throw std::bad_alloc();
}
return reinterpret_cast< T* >( f );
}
return 0;
};
void deallocate( pointer p, size_type n )
{
if( NULL != p )
{
::FlushViewOfFile( p, n * sizeof( T ) );
::UnmapViewOfFile( p );
}
if( INVALID_HANDLE_VALUE != mapped_file_ )
{
::CloseHandle( mapped_file_ );
mapped_file_ = INVALID_HANDLE_VALUE;
}
};
size_type max_size() const throw()
{
return std::numeric_limits< size_type >::max() / sizeof( T );
};
/// handle to the memory-mapped file
HANDLE mapped_file_;
private:
/// disallow assignment
void operator=( const MappedFileAllocator& );
}; // class MappedFileAllocator
Can anybody suggest where I may be going wrong with my MappedFileAllocator
implementation?
Thanks,
PaulH
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(3)
检查 boost::interprocess 是否支持 Windows Mobile。它们具有创建内存映射文件并使用内存分配您想要的任何数据的工具:
http://www.boost.org/doc/libs/1_47_0/doc/html/interprocess/sharedmemory Betweenprocesses.html#interprocess.sharedmemory Betweenprocesses.mapped_file
Check if boost::interprocess supports Windows Mobile. They have facilities for creating memory-mapped files and using the memory to allocate any piece of data you want:
http://www.boost.org/doc/libs/1_47_0/doc/html/interprocess/sharedmemorybetweenprocesses.html#interprocess.sharedmemorybetweenprocesses.mapped_file
您正在使用匿名文件映射(没有实际文件支持的映射)。当您执行此操作时,映射由系统页面文件支持。移动操作系统实际上可能没有页面文件,因为“硬盘”可能是闪存设备。对闪存设备进行虚拟内存分页通常是一个坏主意,因为虚拟内存的性质意味着大量写入会很快烧毁闪存(尤其是较旧的类型)。
这似乎得到了您所获得的迭代次数的支持。看起来您最多可以映射设备上总内存的 60% 左右。
如果您打开一个真实文件(使用 OpenFile )并映射该文件,而不是在 CreateFileMapping 中使用 INVALID_FILE_HANDLE ,则可能可以使其适用于较大的部分。但如果您要大量写入文件映射,请小心您的闪存(和性能!)。
You're using an anonymous file mapping (one without an actual file backing it). When you do this, the mapping is backed by the system page file. It is likely that the Mobile OS does not actually have a Page File because the "hard disk" is probably a flash device. It's generally a bad idea to do virtual memory paging to flash devices because the nature of virtual memory means a high volume of writes which can quickly burn out flash (especially older types).
This seems to be backed up by the number of iterations you're getting. It looks like you're able to map up to around 60% of the total memory on your device.
You can probably get this to work with larger sections if you open a real file (using
OpenFile
) and map that rather than usingINVALID_FILE_HANDLE
inCreateFileMapping
. But be careful of your flash storage (and performance!) if you're going to be writing to your file mapping very much.我刚刚获得了这个问题的“热门问题”徽章,所以我想我应该(迟来的)发布答案。我已经没有可用的句柄了。 Windows Mobile 内核中有一个与溢出的句柄分配相关的无符号短计数器。一旦发生这种情况,设备就会冻结。
I just got the "popular question" badge for this one, so I thought I'd (belatedly) post the answer. I was running out of available handles. There is an unsigned short counter in the Windows Mobile kernel relating to handle allocation that was overflowing. Once that happened, the device would freeze.