mysql集群表已满，解决方法？
我们正在尝试将几个 Tb 放入 MySQL Cluster，不幸的是索引不适合内存。
有没有办法克服mysql的这个限制？
mysql进程范围操作有并行的方法吗？
我的数据在 MYisam 中有一个 3D 点：(id xyz idkey someblob)，有 128 个分区。由于内存限制，NDBCLuster 无法加载数据。
索引超过 idkey（这是预先计算的 peano-hilbert 键）。总行数约为 10^9。
谢谢阿曼。
编辑 我的设置是 2 个数据节点 2 个 mysqld 1 个 mdm。 每个 ndb 8Gb RAM，4 核。
整个系统有30Tb Raid6。
系统是linux Scientific LInux 6.0，集群是从源码编译的7.1。
We are trying to put several Tb in to MySQL Cluster, unfortunately the index does not fit in to memory.
Are there way to overcome this limitations of mysql?
are there way in mysql process range operations in parallel?
My data has a 3D points: (id x y z idkey someblob) inside the MYisam with 128 partitions. The NDBCLuster was unable load the data due to memory limits.
indexing goes over idkey(this is pre calculated peano-hilbert key).The total row count is about 10^9.
Thanks Arman.
EDIT
my setup is 2 datanodes 2 mysqld one mdm.
8Gb RAM per ndb with 4 cores.
The whole system has 30Tb Raid6.
The system is linux Scientific LInux 6.0, the cluster is 7.1 compiled from source.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（3）
听起来 MySQL 不适合这项任务（抱歉）。我会查看 Tokyo Tyrant，也许是 MongoDB 或任何其他分布式键值存储系统。还有专门的商业产品。
MongoDB 能够将其部分索引交换到 HD。我猜你的问题是 MySQL 无法做到这一点（尽管我不是 MySQL 专家）。
It sounds like MySQL is ill-suited for the task (sorry). I would check out Tokyo Tyrant, maybe MongoDB or any other distributed key-value storage system. There are also specialized commercial products.
MongoDB is able to swap out some of it's indexes to the HD. I guess your problem is that MySQL just can't do that (I'm not a MySQL-guy though).
也许你可以尝试修改你的config.ini文件。
但如果两个值太高，你会遇到这个错误：
由于位图页面未对齐而无法使用！
所以我还在努力解决它。祝你好运。
Maybe you can try to modify your config.ini file.
But if two values are too high, you will encounter this bug:
Unable to use due to bitmap pages missaligned!!
So I'm still trying to solve it. Good luck.
当我仅加载数据库表的结构时，我遇到了同样的问题。这意味着 DataMemory 或 IndexMemory 在这里没有帮助。
此外，表的数量没有达到 MaxNoOfTables 中的限制，因此这也不是问题。
我的解决方案是增加 MaxNoOfOrderedIndexes 和 MaxNoOfUniqueHashIndexes 的值，这反映了集群中可以拥有的最大索引数。因此，如果数据库中有很多索引，请尝试相应地增加这些变量。
当然，滚动重启必须在更改后进行才能生效！
I faced the same issue when I was loading only DB tables' structure. Which means
DataMemoryorIndexMemorywere not of help here.Also the number of tables didn't reach the limit in
MaxNoOfTablesso it is not the issue as well.The solution for me here was to increase the values for
MaxNoOfOrderedIndexesandMaxNoOfUniqueHashIndexeswhich reflect the max number of indexes you can have in the cluster. So if there are many indexes in your DB try to increase those variables accordingly.Of course, a rolling restart must be done after that change to take effect!