建立分布式计算网络
我构建的新科学应用程序需要大量计算能力。
由于我的预算有限，而且该应用程序需要大量 CPU，但内存很少（400MB），一位朋友建议我购买许多“旧”机器（Pentium 4、3Ghz、1GB 内存）并使用它们来运行我的实例应用程序（每个应用程序实例都是相互独立的，并且执行大量计算和 I\O）。
有一些商店似乎以便宜的价格提供此类机器（每台 100 美元）。
我的问题是，在这些机器上安装 ubuntu linux、将其全部设置在网络中并在其上安装 python / cython 之类的东西有多难？
安装Condor有多难，这样我就可以将作业“提交”到中央节点，并且计算将在各个节点/单元中进行？
I need a lot of computing power for a new scientific application I built.
As I am on a budget, and since the app requires lots of CPU but little memory (400MB), a friend recommended that I buy many "old" machines (Pentium 4, 3Ghz, 1GB memory) and use them to run instances of my application (every app instance is independent of one another, and does a lot of computations and I\O).
There are some stores that seem to offer such machines for cheap ($100 a piece).
My question is, how hard would it be to install ubuntu linux on these machines, set it all up in a network and install things like python / cython on it?
how hard would it be to install condor, so that I can just 'submit jobs' to a central node and the computations will be carried out in the various nodes / units?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（2）
Condor 正是完成这项工作的正确工具，并且在这样简单的配置中设置 Condor 相对容易。您可能需要查看类似rocksclusters.org 的内容来设置操作系统和机器集群本身。
现在，我肯定会首先对这些机器进行基准测试。您可能会惊讶于旧机器的速度有多慢，以及您的应用程序性能与时钟速度没有直接关系。当然，最快、最新、最高核心数的机器不会是最具成本效益的，但也不是最旧、最慢的单向机器。
Condor is exactly the right tool for this job, and it is relatively easy to set up Condor in straightforward configurations like this. You may want to look at something like rocksclusters.org for setting up the OS and the cluster of machines themselves.
Now, I would certainly benchmark these machines first. You may be surprised at how slow old machines are, and how your application performance is not directly related to clock speed. Certainly, the fastest, newest, highest core-count machines aren't going to be the most cost effective, but nether are the oldest, slowest one-way machines, either.
我认为您可以设置一台这样的机器并拍摄磁盘的映像。然后，您可以在集群的其余部分上使用该磁盘映像。
不过，我想知道如果你的机器有不同的配置，这会工作得如何。
I think you can setup one of such machine and take an image of the disk. Then, you can use that disk image on the rest of your cluster.
Though, I wonder how well this will work if your machines have different config.