如何调试构建机器错误设置的性能?

发布于 2024-12-08 09:27:55 字数 1049 浏览 1 评论 0原文

我们必须定期设置新的构建环境,这个过程似乎并不那么简单。今天我有了一台新的构建机器,第一个 Maven 构建非常慢,我想澄清为什么性能如此糟糕。但如何做到这一点呢?

我们的背景是:

  • 我们使用多个构建机器,每个项目都有自己的。
  • 每台构建机器都有类似的设置,因此项目可以立即启动,而无需进行大量配置。
  • 我们预先配置了以下工具:
    • Hudson(当前为 2.1.1,但将会更改)
    • Artifactory 2.3.3.1
    • 声纳
    • Hudson、Artifactory 和 Sonar 配置了自己的 Tomcat
    • Maven 2.2.1 和 Maven 3.0.3(没有用户配置,只有安装有 settings.xml
    • Ant 1.7.1 和 Ant 1.8.2(此处不相关)
    • Subversion 1.6 客户端

所有工具应该协同工作,尤其是存储库链应该是:

  1. 构建机器 Maven 存储库
  2. 构建机器 Artifactory
  3. Central 公司 Artifactory(作为世界的镜像和缓存)
  4. Maven 中央(和其他存储库)

因此,当 Maven 构建需要依赖项时解决后,它将首先在本地 Maven 存储库中查找,然后在本地 Artifactory 存储库中查找,然后在中央 Artifactory 存储库中查找,然后才在互联网上查找。

我们通常必须使用代理来连接到互联网,我们的内联网中不需要它。

第一个构建(Maven Hello World)在大约 45 分钟内构建完成。那时,所有的引导都在发生,但我认为通过使用我们的存储库链(其中中央存储库已填满),构建会快得多。所以我认为调试的重点将是网络,本地构建不是问题。所以Maven和Artifactory的配置和交互正在考虑中。

如何调试这样的环境?我可以访问构建机器(如 sudo)和中央存储库,但我不知道如何开始、要证明什么、在哪里查找。那么您的经验是什么,您想分享的提示和技巧是什么?

We have to setup new build environments regularily, and the process seems not so simple. Today I have got a new build machine, and the first Maven build was so slow, that I wanted to clarify why the performance was so bad. But how to do that?

Our context is:

  • We use multiple build machines, each project gets its own.
  • Each build machine has a similar setup, so that projects can start immediately and don't have to configure a lot.
  • We have the following tools preconfigured:
    • Hudson (currently 2.1.1, but will change)
    • Artifactory 2.3.3.1
    • Sonar
    • Hudson, Artifactory and Sonar have their own Tomcat configured
    • Maven 2.2.1 and Maven 3.0.3 (with no user configuration, only the installation has a settings.xml)
    • Ant 1.7.1 and Ant 1.8.2 (not relevant here)
    • Subversion 1.6 client

All tools should work together, especially the repository chain should be:

  1. Build machine Maven repository
  2. Build machine Artifactory
  3. Central company Artifactory (is working as mirror and cache for the world)
  4. Maven central (and other repository)

So when the Maven build needs a dependency resolved, it will be first looked-up in the local Maven repo, from there in the local Artifactory repo, then in the central Artifactory repo and only then on the internet.

We normally have to use proxies to connect to the internet, we don't need it in our intranet.

The first build (Maven Hello World) was built in around 45 minutes. In that time, all bootstrapping was happening, but I would have thought by using our chain of repositories (where the central repository is well filled), the build would be much faster. So I think the focus of the debugging will be the network, the local build is not the problem. So configuration and interaction of Maven and Artifactory is under consideration.

How do you debug such an environment? I have access to the build machine (as sudo) and to the central repository, but I do not know how to start, what to prove, where to look. So what is your experience, what are the tips and tricks you would like to share?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

画离情绘悲伤 2024-12-15 09:27:55

以下是我到目前为止所做的一些事情。如果您还有其他建议,欢迎您!

我怀疑存储库链是邪恶的根源,所以我首先解决了这个问题。原因是:

  • 本地计算机上的实际构建(hello world 程序)可能会以毫秒为单位有所不同,但不会以分钟为单位。
  • 网络会带来影响,所以首先要解决这个问题。

如果在本地找不到某些内容,则存储库链很有趣。以下是确保这种情况的步骤:

  • 对于 Maven,删除本地缓存的内容。如果本地缓存已满,您不知道是否在本地缓存或其他地方找到了资源。 (如果其他一切都恢复正常,至少在最后执行此操作。)
  • 对于 Artifactory,也找到该缓存,并通过删除其内容来清理它。它只是一个缓存,所以它会被填充一个新的。
  • 如果您使用智能浏览器来测量查找,请确保您请求的内容不在浏览器的缓存中。
  • 或者使用像 wget 这样的工具来请求资源。
  • 尽量减少失败的根源。因此,请尝试将长距离的查找分成您可以控制的较小部分。
  • 不要使用 Maven 进行查找,首先(仅)从 Artifactory 存储库开始,然后再使用 Maven。

这导致了我想做的以下测试。每次我确保满足之前的先决条件:

  1. 请求https:///repo/。期望:

    • 本地查找将失败,因此必须在中央公司 Artifactory 的远程存储库中查找资源。
    • 可能的影响可能来自代理、人工查找。

    结果:查找一个简单的 POM 大约需要 30 秒。太多了。

  2. 删除代理。对于 wget,有一个选项 --no-proxy 可以做到这一点。期望:

    • 查找速度更快。

    结果:根本没有任何变化,因此代理不是原因。

  3. 请求https:///libs-snapshots-company/。所以将虚拟仓库改为真实的远程仓库。期望:

    • Artifactory 知道在哪里进行查找,因此速度会快得多。

    结果:POM立即被找到,所以这30秒是Artifactory在查找。但这可能是什么原因?

  4. 删除了 Artifactory 中的所有远程和虚拟存储库(只留下我们公司的存储库和缓存的 Maven 中心)。但再次使用 https:///repo/。期望:

    • Artifactory 会更快地找到存储库。

    结果:POM 瞬间到来,不可测量。

  5. 当时我很勇敢,刚刚开始构建(本地缓存为空)。构建需要 5 秒(而不是当天早上的 15 分钟)。

所以我想我现在已经更好地理解了可能出什么问题,还有很多问题。请添加您的想法作为答案,您将因此获得声誉!

Here are a few things I have done up to now. If you have additional advice, you are welcome!

I suspected the chain of repositories to be the source of evil, so I addressed that first. The reasons are:

  • The real build on the local machine (of a hello world program) may differ in milliseconds, but not minutes.
  • Network makes a difference, so attack that first.

The chain of repository is interesting, if something is not found locally. Here are the steps to ensure that that is the case:

  • For Maven, delete the contents of the local cache. If the local cache is filled, you don't know if a resource if found in the local cache or elsewhere. (Do that at least at the end, if everything else is working again.)
  • For Artifactory, find that cache as well, and clean it by deleting its contents. It is only a cache, so it will be filled a new.
  • If you use a clever browser for measuring the lookup, ensure that what you asked for is not in the cache of the browser.
  • Else use a tool like wget to ask for a resource.
  • Try to minimize the sources for failure. So try to divide the long distance of your lookup in smaller segments that you control.
  • Don't use Maven for doing the lookup, start first with the Artifactory repository (only), and later then with Maven.

This led to the following tests I wanted to do. Every time I ensured that the previous prerequisits were met:

  1. Ask for https://<my-project-artifactory>/repo/<my-pom>. Expectation:

    • Local lookup will fail, so has to find the resource in a remote repository in the central company Artifactory.
    • Possible effects could come from proxy, artifactory lookup.

    Result: Lookup for a simple POM needed ~ 30 seconds. That is too much.

  2. Remove the proxy. With wget, there is an option --no-proxy which does just that. Expection:

    • Faster lookup.

    Result: No change at all, so proxy was not the reason.

  3. Ask for https://<my-project-artifactory>/libs-snapshots-company/<my-pom>. So change the virtual repository to a real remote repository. Expectation:

    • Artifactory knows where to do the lookup, so it will be much faster.

    Result: POM was found immediately, so the 30 seconds are Artifactory doing lookup. But what could be the reason for that?

  4. Removed in Artifactory all remote and virtual repositories (only left our companies ones and the cached Maven central). But use again https://<my-project-artifactory>/repo/<my-pom>. Expectation:

    • Artifactory will find the repository much faster.

    Result: POM came in an instant, not measurable.

  5. I was then courageous and just started the build (with empty cache locally). The build needed then 5 seconds (instead of 15 minutes the same morning).

So I think I have now better understood what can go wrong, a lot of questions are remaining. Please add your ideas as answers, you will get reputation for them!

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文