ASP.NET 自动化与手动ETL过程

发布于 2024-09-29 19:21:35 字数 478 浏览 0 评论 0原文

这是一个有趣的问题。我有一个用 C# 编写的 ETL 脚本,我一直定期手动运行该脚本。它用于更新我的网络应用程序的数据库。

我想要自动化 ETL 流程,并为 Web 应用程序管理员创建一个界面来手动启动 ETL 流程。

我可以让 sql server 按计划启动 ETL 过程,并在网页上实现一个按钮或其他东西来执行相同的操作,但我不想将我的代码放在两个不同的地方,因为我不想当它发生变化时,在 2 个地方更新它。但我不知道如何让我的网络应用程序告诉 SQL Server 手动启动计划的进程。这可以做到吗?

或者

我可以以某种方式在网络应用程序本身中实现调度,但现在大多数人都熟悉尝试时遇到的问题(应用程序可能无法在某些时间运行,必须等待请求才能启动进程(没有一些诡计))。另外,由于 ETL 过程需要一段时间,我不想让一些可怜的最终用户等待响应,所以它肯定必须使用新线程。

我还能做什么?你会怎么办?

Here's an interesting problem. I have an ETL script written in c# that I have been running manually on a somewhat regular basis. It is used to update my web app's database.

I want to automate the ETL process AND create an interface for the web app admins to manually start the ETL process.

I could have sql server kick off the ETL process on a schedule and implement a button or something on a web page that will do the same thing, but I don't want to put my code in 2 different places because I don't want to update it in 2 places when it changes. But I don't know how to make my web app tell SQL server to manually start a scheduled process. Can this be done?

OR

I could somehow implement the scheduling in the web app itself, but by now most people are familiar with the problems that are faced when trying that (app may not be running at certain times, must wait on request to start a process (without some trickery)). Also, since the ETL process takes a while, I don't want to make some poor end user wait on a response, so it would definitely have to use a new thread.

What else could I do? What would you do?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

压抑⊿情绪 2024-10-06 19:21:35

您提到您正在使用 SQL Server。它是什么版本? SQL Server Integration Services(该平台的成熟 ETL 工具)适合您吗?如果您想要计划的提取、转换和加载作业,SSIS 是 Microsoft 平台上的一个很棒的工具,并且包含在大多数 SQL Server 许可证中。

或者,查看 Quartz

You mentioned you are using SQL Server. What version is it? Is SQL Server Integration Services (which is the full-blown ETL tool for the platform) an option for you? If you want scheduled Extraction, Transformation and Load jobs, SSIS is a great tool on the Microsoft platform and is included in most SQL Server licenses.

Or, check out Quartz.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文