对于开发者来说,值得学习/使用SSIS吗?

发布于 2024-08-16 09:50:59 字数 184 浏览 1 评论 0原文

我开始在目前的工作中参与相当多的 ETL 工作,而且每个人似乎都非常偏爱 SSIS。我正在努力尝试通过 BI studio 进行最琐碎的转换,这些转换通常相当于几个 foreach 循环和一些 LINQ。我不确定这个工具对哪些用例或用户有用,并且可能会说服管理层我可以不用它。阐述我的观点的最佳方式是什么,或者如果有任何更好工具的建议,我们也将不胜感激。

I'm starting to get involved in quite a bit of ETL work a my current job, and everyone seems to be pretty partial to SSIS. I'm struggling trying to do the most trivial transformations through BI studio that would usually equate to a couple foreach loops with a pinch of LINQ. Im not sure of the use cases or users this tool would be useful for, and will probably convince management that I can do without. What's the best way to make my case, or if there are any recommendations for better tooling, that would be appreciated as well.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(3

第七度阳光i 2024-08-23 09:50:59

除了阅读一些基础知识之外,我不了解 SSIS。和您一样,我对 LINQ 非常熟悉,并且可以快速编写代码来推送和转换数据。

但是如果表中有 4 GB 数据怎么办?您认为您的 LINQ 代码能够优雅地处理这个问题吗?它可以处理,但并不容易,特别是当有很多表相互交叉引用时。一个简单的 LINQ 实现会尝试将所有内容加载到内存中并崩溃。

另一个理由是因为您的同事对 SSIS 很熟悉。您为公司提供的任何基于 SSIS 构建的解决方案对公司来说都比 LINQ 更有价值,因为其他人可以使用它并修复或扩展它。

I don't know SSIS, besides reading up some of the basics. Also like you I'm very comfortable with LINQ, and could quickly write code to push around and transform data.

But what if the table has 4 gigs of data? Do you think your LINQ code is going to handle that gracefully? It could be handled, but not easily, especially if there are a lot of tables cross-referencing each other. A naive LINQ implementation would try to load everything into memory and crash.

Another justification is because your coworkers are strong with SSIS. Any solution you give the company built on SSIS will be more valuable to the company than LINQ, since other people can pick it up and fix or extend it.

聊慰 2024-08-23 09:50:59

与所有事物一样,SSIS 有其优点和缺点。

学习曲线很陡峭,而且通常感觉应该很简单的事情其实并不简单。有时这是 SSIS 及其工具的错误,有时这只是没有以 SSIS 方式思考的情况。

它的一个好处是它是一个服务器产品,因此错误处理和日志记录之类的事情已经(至少部分)为您处理好了。

如果您正在跨不同系统构建复杂的 ETL 包,那么它是一个非常合适且功能强大的工具,并且可以很好地增强您的技能,特别是当您需要参与解决方案的初始设计过程时。如果您只知道 C# 代码,那么这就是您所能建议的,而 SSIS 包可能正是正确的解决方案。

目前,SSIS 如何与 SQL 以外的其他数据库产品集成存在一些问题(例如,它的 Oracle 支持很糟糕),但我相信 Microsoft 致力于 SSIS 并将不断改进它。如果您在微软商店工作,那么现在投资学习它可能是值得的。


缺点是,这是 Ayende 的 帖子 谈论 SSIS 的一些关键痛点。

我并不同意所有这些,或者不认为它们会破坏交易,但了解以后可能会伤害您的内容是件好事。

  • 缺乏良好的日志扩展性让我很恼火 - 除了 SSIS 之外,很难以其他方式登录
  • 配置模式可能是一个真正的痛苦

我确实认为 SSIS 的下一个版本将解决所有这些问题...这只是在微软世界的一部分,事情正在逐渐变得更好,但它们确实往往会变得更好。

As with everything, SSIS has its pros and its cons.

There is a steep learning curve, and often things that feel like they should be simple are not. Sometimes that is that fault of SSIS and its tooling, sometimes it is just a case of not thinking in the SSIS way.

One nice thing about it is that it is a server product so things like error handling and logging have been (at least partially) taken care of for you.

If you are building complicated ETL packages across different systems, then it is a very appropriate and powerful tool, and can be a great thing to add to your skill set, particularly when you need to be involved in the initial design process for a solution. If all you know is C# code, then that is all you can suggest, when an SSIS package may be just the right solution.

Currently there is a bit of pain around how SSIS integrates with other DB products than SQL (its Oracle support is horrible for example) but I believe that Microsoft is committed to SSIS and will keep improving it. If you work in a Microsoft shop then investing in learning it now is probably worthwhile.


And on the cons side, here is a post by Ayende talking about some of the key pain points with SSIS.

I don't agree with all of them, or don't think they are a deal breaker, but it is good to know what may hurt you later on.

  • The lack of good logging extensibility has annoyed me - it is quite hard to log in a way other than SSIS's
  • The configuration schema can be a real pain

I do think that the next version of SSIS will address all of those though... that's just part of the Microsoft world, things get better incrementally, but they do tend to get better.

一张白纸 2024-08-23 09:50:59

SSIS 是一个很棒的工具,但您必须首先掌握基本概念。它允许您专注于 ETL 过程,而不是编程和调试代码。有一个学习曲线。

查看 Microsoft Project Real 示例,看看 foreach 循环对于类似的东西来说是等价的。

您可能还想看看 Pentaho-Kettle 和 Oracle Data Integrator,但我猜您也会有同样的感觉。

SSIS is a great tool, but you have to grasp underlying concepts first. It allows you to focus on the ETL process, instead of programming and debugging your code. There is a learning curve.

Take a look at Microsoft Project Real examples and see what would be foreach loop equivalent for something like that.

You may also want to take a look at Pentaho-Kettle and Oracle Data Integrator, but my guess is you'll have the same feeling.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文