创建新版本并将其发布到生产的过程是 SDLC 中的关键步骤,但它通常被作为事后的想法,并且每个公司之间的差异很大。
我希望人们能够分享他们在组织中对此流程所做的改进,以便我们都可以采取措施“减少痛苦”。
所以问题是,指定发布过程中一个痛苦/耗时的部分,以及您做了什么来改进它?
我的例子:在前一位雇主中,所有开发人员都在一个通用开发数据库上进行了数据库更改。然后到了发布时间,我们使用 Redgate 的 SQL Compare 根据 Dev 和 QA 数据库之间的差异生成一个巨大的脚本。
这种方法工作得相当好,但这种方法的问题是: -
- 包含开发数据库中的所有更改,其中一些可能仍然“正在进行中”。
- 有时开发人员会做出相互冲突的更改(直到发布投入生产时才注意到)
- 创建和验证脚本是一个耗时且手动的过程(我所说的验证是指尝试消除问题 1 和 2 等问题)。
- 当脚本出现问题时(例如,运行顺序,例如创建依赖于脚本中但尚未运行的外键记录的记录),需要花时间“调整”它,使其运行顺利。
- 这不是持续集成的理想场景。
因此,解决方案是: -
- 强制执行一项策略,即对数据库的所有更改都必须编写脚本。
- 命名约定对于确保脚本的正确运行顺序非常重要。
- 创建/使用工具在发布时运行脚本。
- 开发人员有自己的数据库副本进行开发(因此不再有“踩到彼此的脚趾”)
我们开始此过程后的下一个版本速度更快,问题更少,实际上发现的唯一问题是由于人“违反规则,例如不创建脚本。
一旦发布到 QA 的问题得到解决,发布到生产环境时就非常顺利了。
我们应用了一些其他更改(例如引入 CI),但这是最重要的,总体而言,我们将发布时间从大约 3 小时减少到最多 10-15 分钟。
The process of creating a new build and releasing it to production is a critical step in the SDLC but it is often left as an afterthought and varies greatly from one company to the next.
I'm hoping people will share improvements they have made to this process in their organisation so we can all takes steps to 'reduce the pain'.
So the question is, specify one painful/time consuming part of your release process and what did you do to improve it?
My example: at a previous employer all developers made database changes on one common development database. Then when it came to release time, we used Redgate's SQL Compare to generate a huge script from the differences between the Dev and QA databases.
This works reasonably well but the problems with this approach are:-
- ALL changes in the Dev database are included, some of which may still be 'works in progress'.
- Sometimes developers made conflicting changes (that were not noticed until the release was in production)
- It was a time consuming and manual process to create and validate the script (by validate I mean, try to weed out issues like problem 1 and 2).
- When there were problems with the script (eg the order in which things were run such as creating a record which relies on a foreign key record which is in the script but not yet run) it took time to 'tweak' it so it ran smoothly.
- It's not an ideal scenario for Continuous Integration.
So the solution was:-
- Enforce a policy of all changes to the database must be scripted.
- A naming convention was important for ensuring the correct running order of the scripts.
- Create/Use a tool to run the scripts at release time.
- Developers had their own copy of the database do develop against (so there was no more 'stepping on each others toes')
The next release after we started this process was much faster with fewer problems, indeed the only problems found were due to people 'breaking the rules', eg not creating a script.
Once the issues with releasing to QA were fixed, when it came time to release to production it was very smooth.
We applied a few other changes (like introducing CI) but this was the most significant, overall we reduced release time from around 3 hours down to a max of 10-15 minutes.
发布评论
评论(7)
在过去一年左右的时间里,我们做了一些事情来改进我们的构建流程。
完全自动化且完整的构建。我们总是每晚进行一次“构建”,但我们发现构建的构成有不同的定义。有些人会考虑它编译,通常人们包括单元测试,有时还包括其他东西。我们在内部澄清说,我们的自动化构建确实完成了从源代码控制到交付给客户所需的一切。我们对各个部分的自动化程度越高,流程就越好,并且在发布时我们需要手动执行的操作就越少(并且更少担心忘记某些东西)。例如,我们的构建版本用 svn 修订号标记所有内容,编译以几种不同语言完成的各种应用程序部分,运行单元测试,将编译输出复制到适当的目录以创建我们的安装程序,创建实际的安装程序,将安装程序复制到我们的测试网络,在测试机器上运行安装程序,并验证新版本是否正确安装。
代码完成和发布之间的延迟。随着时间的推移,我们逐渐增加了完成特定版本的编码与该版本交付给客户之间的延迟量。这为测试人员提供了更多的专用时间来测试变化不大的产品并生成更稳定的生产版本。源代码控制分支/合并在这里非常重要,因此开发团队可以在测试人员仍在处理上一个版本的同时开发下一个版本。
分公司老板。一旦我们对代码进行分支以创建发布分支,然后继续处理下一个版本的主干,我们就会分配一个轮换发布分支所有者,负责验证应用于该分支的所有修复。每次签入,无论规模大小,都必须由两名开发人员审核。
We've done a few things over the past year or so to improve our build process.
Fully automated and complete build. We've always had a nightly "build" but we found that there are different definitions for what constitutes a build. Some would consider it compiling, usually people include unit tests, and sometimes other things. We clarified internally that our automated build literally does everything required to go from source control to what we deliver to the customer. The more we automated various parts, the better the process is and less we have to do manually when it's time to release (and less worries about forgetting something). For example, our build version stamps everything with svn revision number, compiles the various application parts done in a few different languages, runs unit tests, copies the compile outputs to appropriate directories for creating our installer, creates the actual installer, copies the installer to our test network, runs the installer on the test machines, and verifies the new version was properly installed.
Delay between code complete and release. Over time we've gradually increased the amount of delay between when we finish coding for a particular release and when that release gets to customers. This provides more dedicated time for testers to test a product that isn't changing much and produces more stable production releases. Source control branch/merge is very important here so the dev team can work on the next version while testers are still working on the last release.
Branch owner. Once we've branched our code to create a release branch and then continued working on trunk for the following release, we assign a single rotating release branch owner that is responsible for verifying all fixes applied to the branch. Every single check-in, regardless of size, must be reviewed by two devs.
我们已经在使用 TeamCity(一个优秀的持续集成工具)来进行构建,其中包括单元测试。提到了三项重大改进:
1) 安装套件和一键式 UAT 部署
我们使用 NSIS(不是 MSI,这对于我们的需求来说更加复杂且不必要)。该安装工具包完成了所有必要的操作,例如停止 IIS、复制文件、将配置文件放在正确的位置、重新启动 IIS 等。然后,我们创建了一个 TeamCity 构建配置,该配置使用 psexec。
这使得我们的测试人员可以自己进行 UAT 部署,只要它们不包含数据库更改 - 但这些更改比代码更改要少得多。
当然,生产部署更加复杂,我们无法实现如此程度的自动化,但我们仍然使用相同的安装套件,这有助于确保 UAT 和生产之间的一致性。如果有任何内容丢失或未复制到正确的位置,通常会在 UAT 中找到它。
2) 自动化数据库部署
部署数据库更改也是一个大问题。我们已经编写了所有数据库更改的脚本,但在了解哪些脚本已经运行、哪些脚本仍需要运行以及按什么顺序运行方面仍然存在问题。我们为此研究了几种工具,但最终还是推出了我们自己的工具。
DB 脚本按版本号组织在目录结构中。除了脚本之外,开发人员还需要将脚本的文件名添加到文本文件中,每行一个文件名,并指定正确的顺序。我们编写了一个命令行工具来处理该文件并针对给定的数据库执行脚本。它还在数据库的一个特殊表中记录了它运行的脚本(以及何时),并且下次它不会再次运行这些脚本。这意味着开发人员只需添加一个数据库脚本,将其名称添加到文本文件中,然后针对 UAT 数据库运行该工具,而无需四处询问其他人上次运行的脚本。我们在生产中使用了相同的工具,但当然每个版本只运行一次。
真正使这项工作顺利进行的额外步骤是将数据库部署作为构建的一部分运行。我们的单元测试针对真实的数据库(非常小的数据库,数据最少)运行。构建脚本将从以前的版本恢复数据库的备份,然后运行当前版本的所有脚本并进行新的备份。 (实际上,情况有点复杂,因为我们也有补丁版本,并且仅针对完整版本进行备份,但该工具足够智能来处理这个问题。)这确保了数据库脚本一起测试 /em> 在每次构建时,如果开发人员进行了冲突的架构更改,它将很快被采纳。
唯一的手动步骤是在发布时:我们增加构建服务器上的版本号并复制“当前数据库”备份以使其成为“最后版本”备份。除此之外,我们不再需要担心构建所使用的数据库。 UAT 数据库偶尔仍然需要从备份中恢复(例如,因为系统无法撤消已删除的数据库脚本的更改),但这种情况相当罕见。
3)分支发布
这听起来很基础,几乎不值得一提,但我们一开始并没有这样做。合并回更改肯定会很痛苦,但不如为今天的版本和下个月的版本使用单一代码库那么痛苦!我们还找了对发布分支进行最多更改的人来进行合并,这有助于提醒每个人将发布分支的提交保持在绝对最低限度。
We were already using TeamCity (an excellent continuous integration tool) to do our builds, which included unit tests. There were three big improvements were mentioning:
1) Install kit and one-click UAT deployments
We packaged our app as an install kit using NSIS (not an MSI, which was so much more complicated and unnecessary for our needs). This install kit did everything necessary, like stop IIS, copy the files, put configuration files in the right places, restart IIS, etc. We then created a TeamCity build configuration which ran that install kit remotely on the test server using psexec.
This allowed our testers to do UAT deployments themselves, as long as they didn't contain database changes - but those were much rarer than code changes.
Production deployments were, of course, more involved and we couldn't automate them this much, but we still used the same install kit, which helped to ensure consistency between UAT and production. If anything was missing or not copied to the right place it was usually picked up in UAT.
2) Automating database deployments
Deploying database changes was a big problem as well. We were already scripting all DB changes, but there were still problems in knowing which scripts were already run and which still needed to be run and in what order. We looked at several tools for this, but ended up rolling our own.
DB scripts were organised in a directory structure by the release number. In addition to the scripts developers were required to add the filename of a script to a text file, one filename per line, which specified the correct order. We wrote a command-line tool which processed this file and executed the scripts against a given DB. It also recorded which scripts it had run (and when) in a special table in the DB and next time it did not run those again. This means that a developer could simply add a DB script, add its name to the text file and run the tool against the UAT DB without running around asking others what scripts they last ran. We used the same tool in production, but of course it was only run once per release.
The extra step that really made this work well is running the DB deployment as part of the build. Our unit tests ran against a real DB (a very small one, with minimal data). The build script would restore a backup of the DB from the previous release and then run all the scripts for the current release and take a new backup. (In practice it was a little more complicated, because we also had patch releases and the backup was only done for full releases, but the tool was smart enough to handle that.) This ensured that the DB scripts were tested together at every build and if developers made conflicting schema changes it would be picked up quickly.
The only manual steps were at release time: we incremented the release number on the build server and copied the "current DB" backup to make it the "last release" backup. Apart from that we no longer had to worry about the DB used by the build. The UAT database still occasionally had to be restored from backup (eg. since the system couldn't undo the changes for a deleted DB script), but that was fairly rare.
3) Branching for a release
It sounds basic and almost not worth mentioning, yet we weren't doing this to begin with. Merging back changes can certainly be a pain, but not as much of a pain as having a single codebase for today's release and next month's! We also got the person who made the most changes on the release branches to do the merge, which served to remind everyone to keep their release branch commits to an absolute minimum.
尽可能自动化您的发布过程。
正如其他人所暗示的,使用不同级别的构建“深度”。例如,开发人员构建可以直接从存储库生成用于在开发计算机上运行产品的所有二进制文件,而安装程序构建可以组装所有内容以在新计算机上安装。
这可能包括
等。安装程序构建可以将所有这些内容填充到可安装包(InstallShield、ZIP、RPM 或其他)中,甚至构建用于物理分发的 CD ISO。
安装程序构建的输出通常会移交给测试部门。安装包中未包含的任何内容(安装顶部的补丁......)都是一个错误。挑战您的开发人员提供无故障的安装过程。
Automate your release process whereever possible.
As others have hinted, use different levels of build "depth". For instance a developer build could make all binaries for runnning your product on the dev machine, directly from the repository while an installer build could assemble everything for installation on a new machine.
This could include
and so on. The installer build can stuff all this into an installable package (InstallShield, ZIP, RPM or whatever) and even build the CD ISOs for physical distribution.
The output of the installer build is what is typically handed over to the test department. Whatever is not included in the installation package (patch on top of the installation...) is a bug. Challenge your devs to deliver a fault free installation procedure.
自动单步构建。 ant 构建脚本编辑所有安装程序配置文件、需要更改(版本控制)的程序文件,然后构建。无需干预。
完成后仍然会运行一个脚本来生成安装程序,但我们将消除它。
CD 插图是手动版本化的;这也需要修复。
Automated single step build. The ant build script edits all the installer configuration files, program files that need changed ( versioning) and then builds. No intervention required.
There is still a script run to generate the installers when it's done, but we will eliminate that.
The CD artwork is versioned manually; that needs fixed too.
同意之前的评论。
这是我工作的地方发生的变化。当前的流程消除了您在问题中描述的“陷阱”。
我们使用 ant 从 svn 中提取代码(按标签版本)并提取依赖项并构建项目(有时也进行部署)。
每个环境(开发、集成、测试、生产)都使用相同的 ant 脚本(传递参数)。
项目流程
开发流程
测试过程
一个人管理发布过程,并确保每个人都遵守。此外,所有版本都会在发布前一周进行审核。仅在以下情况下才批准发布:
发布流程
Agree with previous comments.
Here is what has evolved where I work. This current process has eliminated the 'gotchas' that you've described in your question.
We use ant to pull code from svn (by tag version) and pull in dependencies and build the project (and at times, also to deploy).
Same ant script (passing params) is used for each env (dev, integration, test, prod).
Project process
Dev process
Test process
One person manages the release process, and ensures everyone complies. Additionally all releases are reviewed a week before launch. Releases are only approved if there are:
Release Process
我不了解或不实践 SDLC,但对我来说,这些工具对于实现顺利发布是不可或缺的:
I don't know or practice SDLC, but for me, these tools have been indispensible in achieving smooth releases:
在我工作的一个项目中,我们使用 Doctrine (PHP ORM) 迁移来升级和降级数据库。我们遇到了各种各样的问题,因为生成的模型不再与数据库模式匹配,导致迁移中途完全失败。
最后,我们决定编写我们自己的超级基本版本,同样的东西 - 没有什么花哨的,只是执行 SQL 的上行和下行。无论如何,效果很好(到目前为止 - 触摸木头)。尽管我们通过编写自己的代码稍微重新发明了轮子,但重点是保持简单这一事实意味着我们遇到的问题要少得多。现在发布是轻而易举的事情。
我想这个故事的寓意是,有时重新发明轮子是可以的,只要你这样做有充分的理由。
On a project where I work we were using Doctrine's (PHP ORM) migrations to upgrade and downgrade the database. We had all manner of problems as the generated models no longer matched with the database schema causing the migrations to completely fail half way.
In the end we decided to write our own super basic version of the same thing - nothing fancy, just up's and down's that execute SQL. Anyway it worked out great (so far - touch wood). Although we were reinventing the wheel slightly by writing our own, the fact that the focus was on keeping it simple meant that we have far less problems. Now a release is a cinch.
I guess the moral of the story here is that it is sometimes OK to reinvent the wheel some times as long as you are doing so for a good reason.