为什么 hudson “mvn clean install”构建比在命令行上构建花费的时间长 3-6 倍？
我们发现 CI 服务器（hudson）上的构建时间相对较长，并且它们开始妨碍我们。我知道 hudson 所做的不仅仅是调用 maven，我很乐意为它多花 10-20% 的时间来完成这项工作，但数量级的放缓似乎太多了。
任何人都知道为什么会出现这种情况以及如何解决这个问题？我首先会说不原因是什么：
- 虚拟机 hudson 正在运行：在命令行上，它花费的时间与我的开发 PC 的
- 其他并发任务大致相同：我确定没有任何东西从构建任务中转移资源。maven
目标实际上是干净和安装的，没有像 javadoc、checkstyle 等花哨和资源密集型的东西。查看 hudson 构建任务控制台输出，当“检索以前的构建号”时，似乎存在延迟来自[我们的 Nexus 人工制品存储库]”，但我不知道衡量此步骤性能的简单方法，并且发布人工制品的操作似乎过于简单，无法证明速度上的总差异是合理的。
（问题也在此线程中进行了描述）
< strong>更新：
我们已将 Hudson/Jenkins 升级到最新版本，并且已经能够使用计时插件。简短版本：
- 好消息：我们现在知道 nexus 导致了问题
- 坏消息：我们仍然不知道为什么
更多详细信息
关于我们的一个实际 Maven 项目（maven 构建时间：3 分钟， Hudson 构建时间：9 分钟）我们可以看到 Hudson 也在 3 分钟内执行构建，但随后需要 6 分钟才能将工件上传到 Nexus。
使用 Nexus 的 Web UI 执行另一个人工制品的手动上传，我能够确认以下内容：
- 实际的人工制品上传是在一小部分时间内（即几秒钟）完成的，在
- 这几秒钟之后，人工制品显示为 < code>
/nexus/storage/test/test2/test2/1.0.0/test2-1.0.0.rpm
真正的谜题是为什么 nexus 需要一分钟多的时间才能完成创建这个文件：
据我所知，它只是计算 MD5 和 SHA1 签名并记录一般的工件信息，但是 75MB 文件的 md5sum 和 sha1sum 需要 <1 秒才能运行...
最后，它似乎不是某种网络超时，因为延迟似乎与工件大小大致成正比。
任何关于 Nexus 在收到文物后会做什么的想法都会受到赞赏。
更新2：
将nexus日志级别设置为调试，上传工件时nexus会记录以下内容：
<前><代码>... 2011-04-05 14:38:53 调试 [jpsc28za2RtYQ==] -osnpslfDefau~ - 复制流的缓冲区大小为： 4096
2011-04-05 14:39:55 调试 [ython-2.5.2.jar] - org.mortbay.log-响应 /nexus/content/groups/public/org/python/jython/2.5.2/jython-2.5.2.jar 200
2011-04-05 14:40:07 调试 [-2.5.2.jar.sha1] - org.mortbay.log-请求 /nexus/content/groups/public/org/python/jython/2.5.2/jython-2.5.2.jar.sha1 上
<前><代码>... 2011-04-05 14:40:12 调试 [-2.5.2.jar.sha1] - org.mortbay.log-响应 /nexus/content/groups/public/org/python/jython/2.5.2/jython-2.5.2.jar.sha1 200
2011-04-05 14:43:45 调试 [ndex.properties] - org.mortbay.log-请求 /nexus/content/groups/public/.index/nexus-maven-repository-index.properties 在 org.mortbay.jetty.HttpConnection@141a720
<前><代码>... 2011-04-05 14:44:04 调试 [ndex.properties] -osnpmmM2Group~ - 公共 检索项目（）：：找到 公共：/.index/nexus-maven-repository-index.properties
2011-04-05 14:44:04 调试 [ndex.properties] - org.mortbay.log-响应 /nexus/content/groups/public/.index/nexus-maven-repository-index.properties 200
2011-04-05 14:48:07 调试 [jpsc28za2RtYQ==] -osnpaDefaultAt~ - 将属性存储在 UID=test:/test/test/1.0.1/test-1.0.1.rpm
<前><代码>... 2011-04-05 14:48:07 调试 [w/icon-info.gif] - org.mortbay.log-servletholder=nexus
2011-04-05 14:48:08 调试 [w/icon-info.gif] - org.mortbay.log-响应 /nexus/ext-2.3/resources/images/default/window/icon-info.gif 200
2011-04-05 14:49:01 调试 [c=1302007326656] - org.mortbay.log- 请求 /nexus/service/local/log/config org.mortbay.jetty.HttpConnection@1dbd88f ....
它似乎只是坐在那里一分钟左右，然后继续工作。任何关于为什么 Nexus 这样做的想法都是值得赞赏的。
We are seeing relatively long build times on our CI server (hudson) and they're starting to get in our way. I am aware hudson does more than invoke maven and I would happily grant it 10-20% more time for the job, but an order of magnitude slowdown seems too much.
Anyone have an idea of why this might be and how to solve the problem? I will start by saying what is not the cause:
- the virtual machine hudson is running in: on the command line, it takes roughly the same amount of time as my development PC
- other concurrent tasks: I made sure there was nothing diverting resources from the build task
The maven goals are literally clean and install, nothing fancy and resource-intensive like javadoc, checkstyle etc. Looking at the hudson build task console output, there seem to be delays when "Retrieving previous build number from [our Nexus artefact repository]", but I don't know of a simple way to measure performance of this step and publishing an artefact seems too simple an operation to justify the total difference in speed.
(problem also described in this thread)
Update:
We have upgraded Hudson/Jenkins to the latest release and have been able to use the timing plugin. Short version:
- the good news: we now know nexus is causing the problem
- the bad news: we still don't know why
More details
On one of our actual maven projects (maven build time: 3 min, hudson build time: 9 min) we could see that hudson also performs the build in 3 min, but then takes 6 minutes to upload the artefact to nexus.
Performing a manual upload of another artefact using nexus' web UI, I was able to confirm the following:
- the actual artefact upload is done in a fraction of the time (i.e., in several seconds)
- after these several seconds, the artefact appears as
<nexusworkdir>/nexus/storage/test/test2/test2/1.0.0/test2-1.0.0.rpm
The real puzzler is why nexus takes over a minute to create this file:<nexusworkdir>/nexus/proxy/attributes/test/test2/test2/1.0.0/test2-1.0.0.rpm
As far as I can tell, it just calculates an MD5 and SHA1 signature and records general artefact information, but md5sum and sha1sum of a 75MB file take <1s to run...
Finally, it does not appear to be some sort of network time-out, because the delay seems to be roughly proportional to artefact size.
Any idea what nexus does after it receives an artefact is appreciated.
Update 2:
Setting the nexus log level to debug, nexus logs the following when an artefact is uploaded:
... 2011-04-05 14:38:53 DEBUG [jpsc28za2RtYQ==] -o.s.n.p.s.l.f.Defau~ -
Copying stream with buffer size of:
40962011-04-05 14:39:55 DEBUG [ython-2.5.2.jar] - org.mortbay.log- RESPONSE /nexus/content/groups/public/org/python/jython/2.5.2/jython-2.5.2.jar
2002011-04-05 14:40:07 DEBUG [-2.5.2.jar.sha1] - org.mortbay.log- REQUEST /nexus/content/groups/public/org/python/jython/2.5.2/jython-2.5.2.jar.sha1
on... 2011-04-05 14:40:12 DEBUG [-2.5.2.jar.sha1] - org.mortbay.log- RESPONSE /nexus/content/groups/public/org/python/jython/2.5.2/jython-2.5.2.jar.sha1
2002011-04-05 14:43:45 DEBUG [ndex.properties] - org.mortbay.log- REQUEST /nexus/content/groups/public/.index/nexus-maven-repository-index.properties
on
org.mortbay.jetty.HttpConnection@141a720... 2011-04-05 14:44:04 DEBUG [ndex.properties] -o.s.n.p.m.m.M2Group~ - public
retrieveItem() :: FOUND
public:/.index/nexus-maven-repository-index.properties2011-04-05 14:44:04 DEBUG [ndex.properties] - org.mortbay.log- RESPONSE /nexus/content/groups/public/.index/nexus-maven-repository-index.properties
2002011-04-05 14:48:07 DEBUG [jpsc28za2RtYQ==] -o.s.n.p.a.DefaultAt~ -
Storing attributes on
UID=test:/test/test/1.0.1/test-1.0.1.rpm... 2011-04-05 14:48:07 DEBUG [w/icon-info.gif] - org.mortbay.log- servlet holder=nexus
2011-04-05 14:48:08 DEBUG [w/icon-info.gif] - org.mortbay.log- RESPONSE /nexus/ext-2.3/resources/images/default/window/icon-info.gif
2002011-04-05 14:49:01 DEBUG [c=1302007326656] - org.mortbay.log- REQUEST /nexus/service/local/log/config on
org.mortbay.jetty.HttpConnection@1dbd88f
....
It appears to just be sitting there for a minute or so and than continues with its work. Any idea why nexus does this is appreciated.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
正如线程中所讨论的，我怀疑您的分叉 Maven 没有传递 JVM 参数。您可以使用 jconsole 检查允许的最大堆是您在 MAVEN_OPTS 中分配的吗？
如果将 Hudson 作为服务启动与从命令行启动 Hudson 有什么区别吗？
更新：
在 Nexus 上部署需要大量 RAM，比编译要多得多（根据我的经验）。低内存上的交换可能会减慢速度。
As discussed in the thread I suspect your forked Maven isn't getting the JVM parameters passed. Can you use jconsole to check the max heap allowed is what you have allocated in your MAVEN_OPTS?
Does it make any difference if you start Hudson as a service vs. starting Hudson from the command line?
Update:
Deploying on Nexus takes a load of RAM, much more than compiling (in my experience). The swapping on low memory might be what's slowing it down.