持续集成中处理多个分支
我一直在处理公司中扩展 CI 的问题，同时试图找出在涉及 CI 和多个分支机构时应采取哪种方法。 stackoverflow 上有一个类似的问题，多个功能分支和持续集成。我开始了一个新的问题，因为我想得到更多的讨论并提供一些对问题的分析。
到目前为止，我发现我可以采取两种主要方法（或者也许还有其他方法？？？）。
- 每个分支有多组作业（这里讨论 Jenkins/Hudson）
- 编写工具来管理额外的工作
- 批量创建/修改/删除作业
- 每个分支的每个作业的自定义设置（SCM 网址、dep 管理存储库重复）
- 人们使用 shell 工具、ant 脚本和 Jenkins CLI 解决此问题的一些示例。看：
- http://jenkins.361315.n4。 nabble.com/Multiple-branches-best-practice-td2306578.html
- http://jenkins.361315.n4.nabble.com/Is-it-possible-to-handle-multiple-branches-where-some-jobs-should-run-on-each-one-无重复-td954729.html
- http://jenkins.361315.n4。 nabble.com/Parallel-development-with-branches-td1013013.html
- 自动配置或创建 hudson 作业
- 会给您的 CI 集群带来更多负载
- 开发人员的反馈周期变慢（如果基础架构无法处理新负载）
- 编写工具来管理额外的工作
- 每 2 个分支有多个作业集（dev 和 dev）稳定的）
- 手动管理这两个集合（如果您更改作业的配置，请务必在另一个分支中进行更改）
- PITA 但至少需要管理的很少
- 其他额外分支在推送到开发之前不会获得完整的测试套件
- 开发人员不满意。为什么开发人员应该关心 CI 扩展问题。他有一个简单的请求，当我分支时我想测试我的代码。简单。
- 手动管理这两个集合（如果您更改作业的配置，请务必在另一个分支中进行更改）
因此，如果我想为开发人员提供他们自己的自定义分支的 CI，我需要 Jenkins 的特殊工具（API 或 shellscripts 之类的？）并处理扩展。或者我可以告诉他们更频繁地合并到 DEV，并且在自定义分支上无需 CI。您会选择哪一种或者还有其他选择吗？
I've been dealing with the problem of scaling CI at my company and at the same time trying to figure out which approach to take when it comes to CI and multiple branches. There is a similar question at stackoverflow, Multiple feature branches and continuous integration. I've started a new one because I'd like to get more of discussion and provide some analysis in the question.
So far I've found that there are 2 main approaches that I can take (or maybe some others???).
- Multiple set of jobs (talking about Jenkins/Hudson here) per branch
- Write tooling to manage the extra jobs
- Create/modify/delete Jobs in bulk
- Custom settings for each job per branch (SCM url, dep management repos duplications)
- Some examples of people tackling this problem with shell tools, ant scripts and Jenkins CLI. See:
- http://jenkins.361315.n4.nabble.com/Multiple-branches-best-practice-td2306578.html
- http://jenkins.361315.n4.nabble.com/Is-it-possible-to-handle-multiple-branches-where-some-jobs-should-run-on-each-one-without-duplicatin-td954729.html
- http://jenkins.361315.n4.nabble.com/Parallel-development-with-branches-td1013013.html
- Configure or Create hudson job automatically
- Will cause more load on your CI cluster
- Feedback cycle for devs slows down (if the infrastructure cannot handle the new load)
- Write tooling to manage the extra jobs
- Multiple set of jobs per 2 branches (dev & stable)
- Manage the two sets manually (if you change the conf of a job then be sure to change in the other branch)
- PITA but at least so few to manage
- Other extra branches won't get a full test suite before they get pushed to dev
- Unsatisfied devs. Why should a dev care about CI scaling problems. He has a simple request, when I branch I would like to test my code. Simple.
- Manage the two sets manually (if you change the conf of a job then be sure to change in the other branch)
So it seems if I want to provide devs with CI for their own custom branches I need special tooling for Jenkins (API or shellscripts or something?) and handle scaling. Or I can tell them to merge more often to DEV and live without CI on custom branches. Which one would you take or are there other options?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（4）
当您谈论扩展 CI 时，您实际上是在谈论扩展 CI 服务器的使用来处理所有功能分支以及主线。最初，这看起来是一个很好的方法，因为分支中的开发人员可以获得 CI 作业所包含的自动化测试的所有优势。然而，您在管理 CI 服务器作业时遇到了问题（正如您所发现的），更重要的是，您并没有真正进行 CI。是的，您正在使用 CI 服务器，但您并没有持续集成所有开发人员的代码。
执行真正的 CI 意味着所有开发人员都定期向主线做出承诺。说起来容易，但困难的部分是在不破坏应用程序的情况下做到这一点。我强烈建议您查看持续交付，尤其是保持应用程序可发布部分 >第 13 章：管理组件和依赖项。要点是：
除了抽象分支之外，它们非常不言自明。这只是一个奇特的术语：
第 14 章：高级版本控制中的分支、流和持续集成部分中的以下段落总结了这些影响。
放弃功能分支需要相当大的思维转变，而且你总是会遇到阻力。根据我的经验，这种阻力是基于开发人员感觉在主线提交代码不安全，这是一个合理的担忧。这通常源于对上面列出的技术缺乏知识、信心或经验，也可能是对自动化测试缺乏信心。前者可以通过培训和开发人员支持来解决。后者是一个更难处理的问题，但是分支并不能提供任何额外的真正安全性，它只是将问题推迟到开发人员对他们的代码有足够的信心为止。
When you talk about scaling CI you're really talking about scaling the use of your CI server to handle all your feature branches along with your mainline. Initially this looks like a good approach as the developers in a branch get all the advantages of the automated testing that the CI jobs include. However, you run into problems managing the CI server jobs (like you have discovered) and more importantly, you aren't really doing CI. Yes, you are using a CI server, but you aren't continuously integrating the code from all of your developers.
Performing real CI means that all of your developers are committing regularly to the mainline. Easy to say, but the hard part is doing it without breaking your application. I highly recommend you look at Continuous Delivery, especially the Keeping Your Application Releasable section in Chapter 13: Managing Components and Dependencies. The main points are:
They are pretty self explanatory except branch by abstraction. This is just a fancy term for:
The following paragraph from the Branches, Streams, and Continuous Integration section in Chapter 14: Advanced Version Control summarises the impacts.
It takes quite a mind shift to give up feature branches and you will always get resistance. In my experience this resistance is based on developers not feeling safe committing code the the mainline and this is a reasonable concern. This in turn usually stems from a lack of knowledge, confidence or experience with the techniques listed above and possibly with the lack of confidence with your automated tests. The former can be solved with training and developer support. The latter is a far more difficult problem to deal with, however branching doesn't provide any extra real safety, it just defers the problem until the developers feel confident enough with their code.
我会为每个分支机构设置单独的工作。我之前已经这样做过，如果您正确设置了 Hudson/Jenkins，那么管理和设置并不难。创建多个作业的一种快速方法是从具有类似要求的现有作业中复制并根据需要进行修改。我不确定您是否希望允许每个开发人员为自己的分支设置自己的工作，但一个人（即构建经理）管理的工作量并不大。一旦自定义分支被合并到稳定分支中，当不再需要相应的作业时，就可以删除它们。
如果您担心 CI 服务器上的负载，您可以设置单独的 CI 实例，甚至单独的从属服务器，以帮助平衡多个服务器之间的负载。确保运行 Hudson/Jenkins 的服务器足够。我使用过 Apache Tomcat，只需确保它有足够的内存和处理能力来处理构建队列。
重要的是要清楚您想要使用 CI 实现什么，然后找到一种无需太多手动工作或重复即可实现它的方法。使用 CI 服务器执行的其他外部工具或脚本并没有什么问题，有助于简化整个构建管理过程。
I would set up separate jobs for each branch. I've done this before and it isn't hard to manage and set up if you've set up Hudson/Jenkins correctly. A quick way to create multiple jobs is to copy from an existing job that has similar requirements and modify them as needed. I'm not sure if you want to allow each developer to setup their own jobs for their own branches, but it isn't much work for one person (i.e. a build manager) to manage. Once the custom branches have been merged into stable branches, corresponding jobs can be removed when they are no longer necessary.
If you're worried about the load on the CI server, you could set up separate instances of the CI or even separate slaves to help balance the load across multiple servers. Make sure that the server you are running Hudson/Jenkins on is adequate. I've used Apache Tomcat and just had to ensure that it had enough memory and processing power to process the build queue.
It's important to be clear on what you want to achieve using CI and then figure out a way to implement it without much manual effort or duplication. There's nothing wrong with using other external tools or scripts that are executed by your CI server that help simplify your overall build management process.
我会选择 dev+stable 分支。如果您仍然想要自定义分支并担心负载，那么为什么不将这些自定义分支移至云端并让开发人员自行管理，例如 http://cloudbees.com/dev.cb
这就是Kohsuke现在所在的公司。
还有一个 Eclipse 工具，因此如果您使用 Eclipse，您将把它紧密集成到开发环境中。
I would choose dev+stable branches. And if you still want custom branches and afraid of the load, then why not move these custom ones to the cloud and let developers manage it themselves, e.g. http://cloudbees.com/dev.cb
This is the company where Kohsuke is now.
There is an Eclipse Tooling also, so if you are on Eclipse, you will have it tightly integrated right into dev env.
实际上，真正有问题的是使用功能分支构建隔离。在我们公司，我们有一组独立的 Maven 项目，它们都是更大发行版的一部分。这些项目由不同的团队维护，但对于每个发行版，所有项目都需要发布。现在，一个功能分支可能会从一个项目重叠到另一个项目，这就是构建隔离变得痛苦的时候。我们尝试过几种解决方案：
事实上，最后一个解决方案是最有希望的。所有其他解决方案都以某种方式存在缺陷。与 job-dsl 插件一起，可以轻松设置新的功能分支。只需复制并粘贴 groovy 脚本、调整分支并让种子作业创建新作业即可。确保种子作业删除非托管作业。然后，您可以轻松地在不同的 Maven 项目上使用功能分支进行扩展。
但正如汤姆上面所说，最好克服功能分支的必要性并教会开发人员干净地集成，但这是一个更长的过程，而且结果尚不清楚，因为许多遗留系统部分您将不再接触。
我的 2 美分
Actually what is really problematic is build isolation with feature branches. In our company we have a set of separate maven projects all be part of a larger distribution. These projects are maintained by different teams but for each distribution all projects need to be released. A featurebranch may now overlap from one project to another and thats when build isolation gets painfully. There are several solutions we've tried:
As a matter of fact, the last solution is the most promising. All other solutions lack in one or another way. Together with the job-dsl plugin it is easy to setup a new feature branch. simply copy and paste the groovy script, adapt branches and let the seed job create the new jobs. Make sure that the seed job removes nonmanaged jobs. Then you can easily scale with feature branches over different maven projects.
But as tom said well above, it would be nicer to overcome the necessity of feature branches and teach devs to integrate cleanly, but that is a longer process and the outcome is not clear with many legacy system parts you won't touch any more.
my 2 cents