重新运行数据库开发脚本

发布于 2024-07-04 12:15:48 字数 467 浏览 7 评论 0原文

在我们当前的数据库开发环境中,我们有自动构建过程,检查 svn 创建数据库脚本中的所有 sql 代码,并将它们应用到各种开发/qa 数据库。

这一切都很好,并且比我们过去所做的有了巨大的进步,但是我们在重新运行脚本方面遇到了问题。 显然,这对于某些脚本(例如更改过程)来说不是问题,因为您可以一遍又一遍地运行它们,而不会对系统产生不利影响。 现在,为了添加元数据和运行诸如创建/更改表语句之类的语句,我们添加代码来检查并查看对象是否存在,如果存在,则不要运行它们。

我们的问题是,我们实际上只有一次机会来运行脚本,因为一旦运行脚本,对象就在环境中,系统将不会再次运行脚本。 如果部署后需要更改某些内容,我们将面临一个艰难的过程,即针对更新脚本运行更新脚本,并希望一切都按正确的顺序排列,并且所有 PK 在环境之间排列整齐(我们可以说数据库是, “特别的”)。

除了删除数据库并从头开始该过程(最后一个最新版本)之外,是否有人对此有更优雅的解决方案?

In our current database development evironment we have automated build procceses check all the sql code out of svn create database scripts and apply them to the various development/qa databases.

This is all well and good, and is a tremdous improvement over what we did in the past, but we have a problem with rerunning scripts. Obviously this isn't a problem with some scripts like altering procedures, because you can run them over and over without adversly affecting the system. Right now to add metadata and run statements like create/alter table statements we add code to check and see if the objects exists, and if they do, don't run them.

Our problem is that we really only get one shot to run the script, because once the script has been run, the objects are in the environment and system won't run the script again. If something needs to change once it's been deployed, we have a difficult process of running update scripts agaist the update scripts and hoping that everything falls in the correct order and all of the PKs line up between the environments (the databases are, shall we say, "special").

Short of dropping the database and starting the process from scratch (the last most current release), does anyone have a more elegant solution to this?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(6

不寐倦长更 2024-07-11 12:15:50

如果您想在脚本中具有可重新运行性,那么您不能将它们作为定义...我的意思是您需要专注于更改脚本而不是我的表脚本。

假设您有一个表 Customers:,

create table Customers (
   id int identity(1,1) primary key,
   first_name varchar(255) not null,
   last_name varchar(255) not null
)

稍后您想要添加一个状态列。 不要修改您的原始表脚本,该脚本已经运行(并且可以使用 if(!exists) 语法来防止它在再次运行时导致错误)。

相反,有一个名为 add_customer_status.sql 的新脚本,

在此脚本中您将得到类似以下内容:

alter table Customers
add column status varchar(50) null

update Customers set status = 'Silver' where status is null

alter table Customers
alter column status varchar(50) not null

您可以再次用 if(!exists) 块包装它以允许重新运行,但在这里我们利用了这样的概念:是一个更改脚本,我们相应地调整数据库。 如果客户表中已经有数据,那么我们仍然可以,因为我们添加了列,用数据作为种子,然后添加非空约束。

上面提到的两个迁移框架都很好,我在 MigratorDotNet 方面也获得了很好的体验。

if you want to have re-runnability in your scripts, then you can't have them as definitions... what I mean by this is that you need to focus on change scripts rather than here is my Table script.

let's say you have a table Customers:

create table Customers (
   id int identity(1,1) primary key,
   first_name varchar(255) not null,
   last_name varchar(255) not null
)

and later you want to add a status column. Don't modify your original table script, that one has already run (and can have the if(! exists) syntax to prevent it from causing errors while running again).

Instead, have a new script, called add_customer_status.sql

in this script you'll have something like:

alter table Customers
add column status varchar(50) null

update Customers set status = 'Silver' where status is null

alter table Customers
alter column status varchar(50) not null

Again you can wrap this with an if(! exists) block to allow re-running, but here we've leveraged the notion that this is a change script, and we adapt the database accordingly. If there is data already in the customers table then we're still okay, since we add the column, seed it with data, then add the not null constraint.

Both of the migration frameworks mentioned above are good, I've also had excellent experience with MigratorDotNet.

可是我不能没有你 2024-07-11 12:15:50

Scott 还提到了其他几个解决变更管理问题的 SQL 工具。 但我仍在自己滚动。

我想提出这个问题,并补充一下我的困惑,即仍然没有免费的、基于社区的工具来解决这个问题。 显然,脚本并不是维护数据库模式的令人满意的方式; 实例也不是。 那么,为什么我们不以单独的(并且与平台无关的)格式保存元数据呢?

这就是我现在正在做的事情。 我的主数据库架构是一个版本控制的 XML 文件,最初是通过一个简单的 Web 服务创建的。 一个简单的 JavaScript 程序将实例与它进行比较,一个简单的 XSL 转换会生成 CREATE 或 ALTER 语句。 它有限制,例如 RikMigrations; 例如,它并不总是正确地对相互依赖的对象进行排序。 (但是你猜怎么着 - Microsoft 的 SQL Server 数据库发布工具。)确实,它太简单了。 我只是没有包含我没有使用的对象(角色、用户等)。

所以,我的观点是,这个问题确实没有得到充分解决,迟早我们必须聚在一起解决这些棘手的细节。

Scott named a couple of other SQL tools that address the problem of change management. But I'm still rolling my own.

I would like to second this question, and add my puzzlement that there is still no free, community-based tool for this problem. Obviously, scripts are not a satisfactory way to maintain a database schema; neither are instances. So, why don't we keep metadata in a separate (and while we're at it, platform-neutral) format?

That's what I'm doing now. My master database schema is a version-controlled XML file, created initially from a simple web service. A simple javascript program compares instances against it, and a simple XSL transform yields the CREATE or ALTER statements. It has limits, like RikMigrations; for instance it doesn't always sequence inter-depdendent objects correctly. (But guess what — neither does Microsoft's SQL Server Database Publication tool.) Really, it's too simple. I simply didn't include objects (roles, users, etc.) that I wasn't using.

So, my view is that this problem is indeed inadequately addressed, and that sooner or later we'll have to get together and tackle the devilish details.

分開簡單 2024-07-11 12:15:50

我们采取了“删除并重新创建架构”的路线。 我们的 JUnit 测试包中有一些类,它们对脚本进行参数化,以便为执行代码的开发人员创建模式中的所有对象。 这允许所有开发人员共享一个测试数据库,每个人都可以同时创建/测试/删除他们的测试表,而不会发生冲突。

跑了很长时间吗? 是的。 起初,我们为此使用了设置方法,这意味着每次测试都会删除/创建表,并且花费的时间太长。 然后我们创建了一个 TestSuite,它可以在一个类的所有测试之前运行一次,然后在所有类测试完成后进行清理。 这仍然意味着当我们运行“AllTests”类(其中包括所有包中的所有测试)时,数据库设置运行了很多次。 我解决这个问题的方法是在 OracleTestSuite 代码中添加一个信号量,这样当第一个测试请求设置数据库时,它就会这样做,但任何后续调用只会增加一个计数器。 当调用每个tearDown() 方法时,计数器将递减计数器直到达到0,并且OracleTestSuite 代码将删除所有内容。 剩下的一个问题是测试是否假设数据库为空。 让数据库测试知道它们的运行顺序是很方便的,这样它们就可以利用数据库的状态,因为它可以减少数据库设置的重复。

我们使用 ObjectMothers 的概念来解决创建复杂域对象以进行测试的类似问题。 模拟对象可能是一个更好的答案,但我们当时还没有听说过它们。 经过这么长时间,我建议创建测试帮助器方法,可以为典型场景创建标准化数据集。 另外,这将有助于从数据角度记录重要的边缘情况。

We went the 'drop and recreate the schema' route. We had some classes in our JUnit test package which parameterized the scripts to create all the objects in the schema for the developer executing the code. This allowed all the developers to share one test database and everyone could simultaneously create/test/drop their test tables without conflicts.

Did it take a long time to run? Yes. At first we used the setup method for this which meant the tables were dropped/created for every test and that took way too long. Then we created a TestSuite which could be run once before all the tests for a class and then cleaned up when all the class tests were complete. This still meant that the db setup ran many times when we ran our 'AllTests' class which included all the tests in all our packages. How I solved it was adding a semaphore to the OracleTestSuite code so when the first test requested the database to be setup it would do that but any subsequent call would just increment a counter. As each tearDown() method was called, the counter would decrement the counter until it reached 0 and the OracleTestSuite code would drop everything. One issue this leaves is whether the tests assume that the database is empty. It can be convenient to let database tests know the order in which they run so they can take advantage of the state of the database because it can reduce the duplication of DB setup.

We used the concept of ObjectMothers to solve a similar problem with creating complex domain objects for testing purposes. Mock objects might be a better answer but we hadn't heard about them at the time. After all this time, I'd recommend creating test helper methods that could create standardized datasets for the typical scenarios. Plus that would help document the important edge cases from a data perspective.

も星光 2024-07-11 12:15:50

您是否将现有数据保存在数据库中? 如果没有,您可能需要查看类似于 Matt 为 .NET 提到的 RikMigrations 的内容

http://www.rikware.com/RikMigrations.html

我在我的项目中使用它来动态更新我的数据库,同时跟踪修订。 此外,它使得将数据库模式移动到不同的服务器等变得非常简单。

Do you keep your existing data in the database? If not, you may want to look at something similar to what Matt mentioned for .NET called RikMigrations

http://www.rikware.com/RikMigrations.html

I use that on my projects to update my database on the fly, while keeping track of revisions. Also, it makes it very simple to move database schema to different servers, etc.

静赏你的温柔 2024-07-11 12:15:50

我们解决这个问题 - 或者至少解决一个与此类似的问题 - 如下:

  1. 模式有一个版本号 - 它由一个表表示,每个版本有一行,该表以及版本号都包含诸如日期之类的无聊内容/该版本出现的时间戳。
  2. 通过让架构创建/修改 DDL 并封装在为我们执行更改的代码中。

在上面的上下文中,我们将构建架构更改代码作为构建过程的一部分,然后运行它,它只会应用尚未应用的架构更改。

根据我们的经验(这肯定不具有代表性),在大多数情况下,模式更改足够小/快,可以安全地在事务中运行,这意味着如果失败,我们会得到回滚,并且数据库是“安全的” -尽管在可行的情况下,我们总是建议在应用架构更新之前进行备份。

我是从令人讨厌的痛苦经历中演变而来的。 它不是一个完美的系统(或一个原创的想法),但通过这种方式工作,我们非常有信心,如果我们的数据库之一有两个具有相同版本的实例,那么这两个数据库的架构将在几乎所有方面都是相同的,并且我们可以安全地将任何数据库提升到该应用程序的当前模式,而不会产生不良影响。 (不幸的是,最后一个并不是 100% 正确 - 总是有例外 - 但它与事实相差不远!)

We address this - or at least a similar problem to this - as follows:

  1. The schema has a version number - this is represented by a table which has one row per version which, as well as the version number, carries boring things like a date/time stamp for when that version came into existence.
  2. By having the schema create/modify DDL wrapped in code that performs the changes for us.

In the context above one would build the schema change code as part of the build process then run it and it would only apply schema changes that haven't already been applied.

In our experience (which is bound not to be representative) in most cases the schema changes are sufficiently small/fast that they can safely be run in a transaction which means that if it fails we get a rollback and the db is "safe" - although one would always recommend taking backups before applying schema updates if practicable.

I evolved this out of nasty painful experience. Its not a perfect system (or an original idea) but as a result of working this way we have a high degree of confidence that if there are two instances of one of our databases with the same version that then the schema for those two databases will be the same in almost all respects and that we can safely bring any db up to the current schema for that application without ill effects. (That last isn't 100% true unfortunately - there's always an exception - but its not too far from the truth!)

毁梦 2024-07-11 12:15:48

我不确定如何在您的特定环境中最好地解决该问题,但我建议您阅读 Rail 的迁移功能,以获得一些关于如何开始的灵感。

http://wiki.rubyonrails.org/rails/pages/UnderstandMigrations

I'm not sure how best to approach the problem in your specific environment, but I'd suggest reading up on Rail's migrations feature for some inspiration on how to get started.

http://wiki.rubyonrails.org/rails/pages/UnderstandingMigrations

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文