我正在考虑制作一个新的轻量级数据库填充框架。我绝对讨厌 dbunit。在此之前,我想知道是否有人已经这样做了。
我不喜欢 dbunit 的事情:
1) 不推荐使用最简单的编写和入门格式。他们希望你使用臃肿的格式。有些甚至需要 xml 模式。是的,无论如何。
2) 它们不是按照您编写行的顺序填充行,而是按照 xml 文件中定义表的顺序填充行。这确实很糟糕,因为您无法以外键约束不会导致问题的方式对数据进行排序。这只会迫使您经历完全关闭它们的麻烦。
这也会浪费时间并让您的 junit 基类变得臃肿,以包含禁用外键约束的代码。您可能必须测试数据库类型(hsqldb 等)并以特定于数据库的方式禁用它们。这太糟糕了。
如果 dbunit 帮助自动禁用外键约束作为其框架的一部分,那就更好了,但他们不这样做。他们确实记录了方言……那么为什么不使用它们呢?最终,所有这些都会迫使程序员浪费时间,而不能快速起床进行测试。
3) XML 写起来很痛苦。这一点我不需要多说。他们还提供了很多方法来做到这一点,我认为这只会让事情变得复杂。只需提供一种真正可靠的方法即可完成。
4) 当你的数据变得很大时,跟踪 id 及其一致/正确的关系是一件非常痛苦的事情。
另外,如果您一个月没有在某个项目上工作,您如何记住 user_id 1 是管理员,user_id 2 是业务用户,user_id 3 是工程师,而 user_id 4 是其他用户?回去检查这会浪费更多时间。除了任意数字之外,应该有一种有意义的方式来检索它。
5)速度很慢。我发现除非使用 hsqldb,否则它的速度慢得令人痛苦。不一定是这样。还有很多方法可以搞乱其配置,因为“开箱即用”并不容易。为了让它正常工作,你必须经历一个坎坷。所有这些都是鼓励人们不要使用它,或者当他们开始使用它时感到生气。
6) 有些值往往会重复很多,比如日期。最好指定默认值,甚至让框架自动添加默认值,即使您没有告诉它将默认值放在那里。这样你就可以只用你想要的值创建对象,而忽略其余的。如果不需要的话,这肯定比指定列的每个角落和缝隙要好。
7)可能最烦人的事情是第一个条目必须包含所有值 - 甚至空占位符 - 或未来的行将不会选择您实际指定的列。
DBunit 也没有将 [NULL] 转换为真正的空值的合理默认值。您必须手动添加它。告诉我,谁没有用 dbunit 做过这样的事?每个人都有。不应该是这样的!
这意味着,如果您有一个多态对象,则必须在第一行中声明每个子类的连接表的所有外键,即使它们为空。如果您为所有子类模式创建一个表格,您仍然必须在第一行指定所有字段。这太糟糕了。
有什么可以让我满意的吗?或者我应该成为下一个更好的数据库测试框架的框架开发人员?
I was thinking of making a new, light-weight database population framework. I absolutely hate dbunit. Before I do, I want to know if someone already did it.
Things i dislike about dbunit:
1) The simplest format to write and get started is deprecated. They want you to use formats that are bloated. Some even require xml schemas. Yeah, whatever.
2) They populate rows not in the order you write them, but in the order tables are defined in the xml file. This is really bad because you can't order your data in such a way that foreign key constraints won't cause problems. This just forces you to go through the hassle of turning them off altogether.
This also wastes time and bloats up your junit base classes to include code to disable the foreign key constraints. You will probably have to test for the database type (hsqldb, etc.) and disable them in database-specific ways. This is way bad.
It could be better if dbunit helped in disabling foreign key constraints as part of their framework automatically, but they don't do this. They do keep track of dialects... so why not use them for this? Ultimately, all of this does is force the programmer to waste time and not get up and testing quickly.
3) XML is a pain to write. I don't need to say more about this. They also offer so many ways to do it, that I think it just complicates matters. Just offer one really solid way and be done with it.
4) When your data gets large, keeping track of the ids and their consistent/correct relationships is a royal pain.
Also, if you don't work on a project for a month, how are you to remember that user_id 1 was an admin, user_id 2 was a business user, user_id 3 was an engineer and user_id 4 was something else? Going back to check this is wasting more time. There should be a meaningful way to retrieve it other than an arbitrary number.
5) It's slow. I've found that unless hsqldb is used, it is painfully slow. It doesn't have to be. There are also numerous ways to mess up its configuration as it is not easy to do "out of the box". There is a hump that you must go through to get it working right. All this does is encourage people to not use it, or be pissed of when they do start to use it.
6) Some values tend to repeat a lot, likes dates. It'd be nice to specify defaults, or even have the framework put defaults in automatically, even without you telling it to put defaults in there. That way you can create objects just with the values you want, and leave the rest off. This sure beats specifying every nook and cranny of a column if it's not required.
7) Probably the most annoying thing is that the first entry must include ALL the values - even null placeholders - or future rows won't pick the columns that you actually specified.
DBunit doesn't have a sensible default for translating [NULL] to a real null value either. You have to manually add it. Tell me, who hasn't done this with dbunit? Everyone has. It shouldn't be like this!
What this means is that if you have a polymorphic object, you must declare all the foreign keys to the joining tables of each subclass in the first row, even though they are null. If you do a table for all subclasses pattern, you still have to specify all the fields on the first row. This is just awful.
Anything out there to satisfy me, or should I become the next framework developer of a much better database testing framework?
发布评论
评论(14)
我们正在编写 Daleq 作为 DbUnit 的包装器,以解决上述的一些问题。它允许在单元测试中填充数据库,而不是依赖于编辑 XML 文件。
We are writing Daleq as a wrapper around DbUnit to address some of the mentioned concerns. It allows populating a DB just within your unit test rather than relying on editing XML files.
我对 DBUnit 也有类似的问题。特别是使用它来填充本地开发数据以及从真实数据库导出数据。我遇到过几种情况,它会导出一个无法导入的数据集。
这启发我为它编写一个新的库: https://github.com/jeffskj/phonydata
这使用 Groovy DSL 来定义数据集,这使得数据的表示非常紧凑,并且可以做一些很酷的事情,例如生成随机数据,因为它只是 Groovy 代码。
I too had similar issues with DBUnit. Especially for using it to populate local development data and exporting data from a real database. I ran into several cases where it would export a dataset that it couldn't then import.
This inspired me to write a new library for it: https://github.com/jeffskj/phonydata
This uses a groovy DSL to define the datasets which makes for a very compact representation of the data and makes it possible to do cool things like generate random data since it's just groovy code.
DBUnit的情况有时确实令人沮丧。一些问题是从 Marc Philipp 与 dbunit-datasetbuilder,特别是如果您将其与 validator,处于非常早期的阶段。您可以在 SZE< 中查看它的实际情况/a>.
免责声明:所有引用的 github 资源均由我维护。
The situation of DBUnit is indeed sometimes frustrating. Some of the problem are solved from Marc Philipp with dbunit-datasetbuilder, specially if you combine it with the validator, which is in a very early stage. You can see it in action at SZE.
Disclaimer: All referenced github-resources are maintained by me.
可以找到使用 Spring 配置和 Specs2 测试的替代方案
An alternative using Spring configuration and Specs2 testing can be found here
我刚刚发布了一个基于 DSL 的 groovy 框架,名为pedal-loader,可通过 github 获取。文档此处。
它允许您直接使用 JPA 实体级抽象。由于它是一个 groovy 脚本,因此您可以使用所有 groovy 结构。
要将行插入到由名为 Student 的 JPA 实体支持的表中,其中包含名为 id、name 和grade 的字段(不是数据库列,而是映射字段),您可以执行如下操作:
Grade 是映射的 Student 类中的枚举到数据库列(可能使用 JPA 2.1 @Convert 注释)。 allStudents 是一个包含行的列表,rowOfInterest 是对特定行的引用。这些属性(allStudents 和 rowOfInterest)可用于您的单元测试。
I just released a groovy DSL based framework called pedal-loader available via github. Documentation here.
It allows you to work with JPA entity level abstraction directly. Since it is a groovy script, you can use all of the groovy constructs.
To insert rows into a table backed by a JPA entity called Student, with fields (not database columns, but mapped fields) called id, name and grade, you would do something like this:
Grade is an enum in the Student class that is mapped to the database column (perhaps using JPA 2.1 @Convert annotation). allStudents is a list that will hold the rows and rowOfInterest is a reference to a particular row. These properties (allStudents and rowOfInterest) become available to your unit test.
我不知道 DbUnit 有任何真正的替代品,并且 @Joe 在我眼中:
这么说,我个人已经在小型和大型项目中成功使用过 DbUnit 多次,而且我发现它非常有用,特别是在使用 Unitils 及其 DbUnit 模块时。这并不意味着它是完美的并且无法改进,但是有了合适的工具(无论是定制的还是像 Unitils 这样的工具),使用它都是一种不错的体验。
那么我来回答一下你的一些观点:
DbUnit 支持平面或结构化 XML、XLS、CSV。您想使用什么革命性的格式?顺便说一句,使用 XML 时,DTD 或模式不是必需的。但它给了你一些不错的东西,比如验证和自动完成,这有什么不好呢? Unitils 可以轻松为您生成它,请参阅生成数据库结构的 XSD 或 DTD。
他们正在等待您的补丁。
同时,Unitils 提供透明地处理约束的支持,请参阅禁用约束并更新序列。
我想痛苦是主观的,但我并不觉得痛苦,特别是在使用模式和自动完成时。您建议的灵丹妙药是什么?
保持它们很小,这是众所周知的最佳实践。您违反了已知的最佳实践,然后抱怨......
是的,任务切换会适得其反。但由于您正在处理低级数据,因此您必须知道它们是如何表示的,当然,除非您使用更高级别的 API,否则没有神奇的解决方案(但这不是 DbUnit 的目的)。
这是数据库和 JDBC 所固有的,而不是 DbUnit 所固有的。如果您希望事情尽可能快,请使用像 H2 这样的快速数据库(如果您有更好的不可知论方式来做事,我很高兴了解它)。
当使用 Unitils - Home - JavaPolis 2008< 等演示文稿中提到的 Unitils 时,情况并非如此/a> 或 单元测试:unitils &数据库维护。
如果您认为可以让事情变得更好,也许可以为现有的解决方案做出贡献。如果这是不可能的,并且如果您认为您可以创建杀手级数据库测试框架,那么我能说什么,那就去做吧。但不要忘记,咆哮很容易,使用自己的解决方案提出解决方案就不那么容易了。
I'm not aware of any real alternative to DbUnit and none of the tools mentioned by @Joe are in my eyes:
That being said, I've personally used DbUnit successfully several times, on small and huge projects, and I find it pretty usable, especially when using Unitils and its DbUnit module. This doesn't mean it's perfect and can't be improved but with decent tooling (either custom made or something like Unitils), using it has been a decent experience.
So let me answer some of your points:
DbUnit supports flat or structured XML, XLS, CSV. What revolutionary format would you like to use? By the way, a DTD or schema is not mandatory when using XML. But it gives you nice things like validation and auto-completion, how is that bad? And Unitils can generate it easily for you, see Generate an XSD or DTD of the database structure.
They are waiting for your patch.
Meanwhile, Unitils provides support to handle constraints transparently, see Disabling constraints and updating sequences.
I guess pain is subjective but I don't find it painful, especially when using a schema and autocompletion. What is the silver bullet you're suggesting?
Keep them small, that's a know best practice. You're going against a known best practice and then complain...
Yes, task switching is counter productive. But since you're working with low level data, you have to know how they are represented, there is no magic solution unless you use a higher level API of course (but that's not the purpose of DbUnit).
That's inherent to databases and JDBC, not DbUnit. Use a fast database like H2 if you want things to be as fast as possible (if you have a better agnostic way to do things, I'd be glad to learn about it).
Not when using Unitils as mentioned in presentations like Unitils - Home - JavaPolis 2008 or Unit testing: unitils & dbmaintain.
If you think you can make things better, maybe contribute to existing solutions. If that's not possible and if you think you can create the killer database testing framework, what can I say, do it. But don't forget, ranting is easy, coming up with solutions using your own solutions is less so.
作为一名 DbUnit 开发人员,我很感谢您的批评,并且我必须部分同意您的观点。我们目前正在开始设计下一个 DbUnit 主要版本,我希望邀请您参与讨论和开发。
我不会回答您的观点,因为您的问题与 DbUnit 并不真正相关,而是与 DbUnit 替代品相关。不管怎样,我只是想强调你的第 7 点是完全错误的:你不再需要指定第一行的所有列,该功能称为列感应。我不会告诉你为什么默认情况下它没有启用,因为你肯定足够聪明,可以自己理解它。
我会对scaladbtest进行深入的研究,希望我们能够整合他们的想法。
As a DbUnit developer I'm grateful for criticism and I must partially agree with you. We are currently starting the design of the next DbUnit major release and I wish to invite you to participate both in the discussion and development.
I'm not going to answer your points as your question is not really related to DbUnit, but to DbUnit alternatives. Anyway, I just want to highlight your point 7 is completely false: you do not need to specify all the columns on first row any more, the feature is called column sensing. I'm not going to tell you why it's not enabled by default as you are surely smart enough to understand it by yourself.
I'll give scaladbtest a deep examination in the hope we can integrate their ideas.
面对使用 DBUnit 的类似问题,我发现了这一点: http://dbsetup.ninja-squad.com/ index.html 可能会解决问题。例如,所有 DB 内容都包含在 java 类本身中,而不是在单独的文件中表示测试数据。
Faced with similar concerns using DBUnit I have found this : http://dbsetup.ninja-squad.com/index.html which may address concerns. Such as instead of representing test data in separate files all DB content is contained within the java class itself.
如果您使用 Spring 框架(或者不介意至少使用它进行测试),那么 Spring DBUnit 目前是我所知道和使用的普通 DBUnit 的最佳(维护)替代品。引用他们的网站:
Spring DBUnit 似乎是用于 DB 单元测试的“有点官方”的 Spring 解决方案(使用 DBUnit);至少该库的作者/维护者 Phil Webb 正在 SpringSource/Pivotal 工作。
If you use the Spring Framework (or don’t mind using it at least for testing), then Spring DBUnit is currently the best (maintained) alternative to plain DBUnit that I know and use. Quoting their website:
Spring DBUnit appears to be the ‘somewhat official’ Spring solution for DB unit testing (with DBUnit); at least the author/maintainer of the library, Phil Webb, is working at SpringSource/Pivotal.
我使用 DBUnit,并使用一些包装器来平滑粗糙的边缘。 Jailer 是一个可以补充或重叠功能的好工具。它可以从参考数据库中提取数据子集,并将其存储为 DBUnit 兼容的 XML 文件,或存储为“拓扑排序的 DML 文件”,这些文件遵循外键约束。
I use DBUnit, with a few wrappers to smooth over the rough edges. A nice tool that can either complement or overlap the functionality is Jailer. It can extract subsets of data from a reference database, and store this as either DBUnit compatible XML files, or as "topologically sorted DML files", which respect the foreign key constraints.
我刚刚发布了一个名为 JDBDT(Java 数据库增量测试)的库
您可以在软件测试中用于数据库设置和验证。
看看 http://jdbdt.org
最好,
爱德华多
I just released a library called JDBDT (Java Database Delta Testing) that
you may use for database setup and validation in software tests.
Have a look at http://jdbdt.org
Best,
Eduardo
另一次投票赞成用现代库包装 DBUnit,以提高可用性和简洁性。我的选择是 database-rider,这使得 DBUnit 使用起来变得轻而易举,甚至支持 JUnit 5,如以下示例所示:
请注意,这如何允许无缝地利用(简洁)YAML 测试数据集(YAML 不是 XML,尽管我相信 DBUnit 实际上本身就支持这些数据集)。
Another vote for wrapping DBUnit with a modern library to improve usability and conciseness. My choice is database-rider, which makes DBUnit a breeze to use and even supports JUnit 5 as demonstrated in the following example:
Notice how this allows leveraging (concise) YAML testing data sets seamlessly (YAML not XML, though I'm lead to believe that DBUnit actually supports those natively).
你说得很好。
在过去的几年里,我一直在为很多门户网站工作,主要使用 PHP,但有时也使用一些 Java。
和你一样,我不明白,这么多年之后,框架和单元测试开发人员似乎没有意识到过去十年存储处理发生了多大的变化。
仅仅将创建/插入/截断语句发送到某个数据库是不够的!
如果您大规模运营,您最终会使用各种存储后端,分层组织以快速推出热门内容。另外,在数据库方面还存在数据分区的问题。如果您没有适当的外键抽象,那么当您的存储设置发生更改时,您肯定会发疯。当我们这样做时:按外键优先级排序的固定装置有很多陷阱,而且我还没有看到 DBUnit 的真正解决方案。
无论如何,关键是仅仅有一个用于单元测试的基本数据库存储对于复杂的存储设置来说是不够的,因为它们通常无法在实时环境中重现问题并且维护起来很痛苦。
不想让自己听起来像个狂热爱好者:
ruby on Rails
是一个一切都还不错的地方。它有一个持久的模型概念,人们似乎实际上已经对此进行了一些思考。如果您正在处理
PHP
,Symfony
就是您该去的地方。它受到默认包含 Doctrine 的限制,而且也相当以数据库为中心,但它具有干净的接口和良好的可扩展性,并完全复制了 Rails 夹具系统。从专业角度来说,我现在需要坚持使用自制解决方案,但它们工作正常。You're making excellent point.
I've been working for a lot of web portals over the last years, mostly with PHP, but also some Java now and then.
And like you I don't get that after all these years framework and unittesting developers don't seem to realize how much storage handling has changed in the last decade.
It's not enough to just send create/insert/truncate statements to some database!
If you're operating at large scale you end up employing all sorts of storage backends, organized in layers to push hot content out fast. Plus on the Database front there's the issue of data partitioning. If you don't have a proper foreign key abstraction provided you will certainly go nuts when your storage setup changes. And while we're at it: fixture ordering by foreign key precedence has many pitfalls and I have yet to see a real solution for that with
DBUnit
.Anyway, the point is having just a basic database storage in place for unittesting is not enough for complex storage setups, since they often fail to reproduce problems in the live environment and are a pain in the ass to maintain.
Without wanting to sound like a fanboy: one place where things are okay is
ruby on rails
.That has a persistent model concept that people seem to have actually put some thought into. If you're dealing in
PHP
,Symfony
is the place to go. It is limited via the default inclusion ofDoctrine
, with is also quite DB-centric, but it has clean interfaces and great extensibility and copied the rails fixture system completely. Professionally I need to stick to homebrew solutions for now, but they work okay.以下是我特别喜欢或感兴趣的一些类似的工具(除了 DBunit 之外)的简短列表。至少他们可以提供一些灵感:
请注意,就范围或功能集而言,这些都不是 DBunit 的真正竞争对手。然而,其中有一些有趣的想法可能值得一看。祝你好运!
Here's a short list of a few tools in this vein (besides DBunit) that I particularly like, or find interesting. At the very least they may offer some inspiration:
Note that none of these are really competitors to DBunit in terms of scope or feature sets. However, there are some interesting ideas there that might be worth taking a look at. Good luck!