更新来自多个不同来源的数据
我正在建立一个包含客户信息的数据库。该数据库将处理客户数据（客户 ID、地址、电话等）以及一些有关特定客户收到的广告类型以及他们对此有何反应的基本信息。
数据将通过中央数据仓库进行维护，但有关客户和广告的其他信息也将从其他来源进行更新。例如，如果外部广告机构开展一项活动，我希望他们能够反馈有关 OptOut、电子邮件退回等的数据。我想我需要的是一个可以轻松分发给任意数量的机构的 API 。
我的第一个想法是为所有外部源设置一个 Web 服务 API，但由于我们可能会讨论大量数据（每批数百万条记录），我不确定 Web 服务是最佳选择。
所以我的问题是，这里的最佳实践是什么？我需要一个足够简单的解决方案，供广告机构（可能拥有中等技能的 IT 人员）使用。简单性是本质——在这种情况下我的意思是“简单性胜过性能”。如果设置太复杂，它就不起作用。
该系统很可能基于微软技术。
有什么建议吗？
I'm in the process of setting up a database with customer information. The database will handle customer data (customer id, address, phonenr etc.) as well as some basic information about which kind of advertisement a specific customer has been subjected to, and how they reacted to it.
The data will be maintained both from a central data-warehouse, but additional information about customers and the advertisement will also be updated from other sources. For example, if an external advertisement agency runs a campaign, I want them to be able to feed back data about OptOuts, e-mail bounces etc. I guess what I need is an API which can be easily handed out to any number of agencies.
My first thought was to set up a web service API for all external sources, but since we'll probably be talking large amounts of data (millions of records per batch) I'm not sure a web service is the best option.
So my question is, what's the best practice here? I need a solution simple enough for advertisement agencies (likely with moderately skilled IT-people) to make use of. Simplicity is of the essence – by which I mean “simplicity over performance” in this case. If the set up gets too complex, it won't work.
The system will very likely be based on Microsoft technology.
Any suggestions?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
您描述的过程通常称为使用 ETL 过程的数据集成。 ETL 代表提取-转换-加载。这个想法是通过从许多不同的数据源提取信息、转换信息然后将其加载到数据仓库中来构建中央数据仓库。
存在多种（也包括图形）工具来实现这样的过程。既然您说您可能会运行 Microsoft 堆栈，我建议您查看一下 Sql Server Integration Services (SSIS)。
关于您使用网络服务实现集成的建议，我认为这也不是一个好主意。同样，我认为将数据集成的负担转移给客户也不是一个好主意。您应该与您的客户就某种形式的数据交换格式达成一致，它可以像 CSV 文件一样简单，也可以是 XML、Excel 工作表、Access 数据库，使用任何适合您需求的格式。
任何现代 ETL 工具（例如 SSIS）都能够处理这些不同的数据源。
The process you're describing is commonly referred to as Data Integration using ETL processes. ETL stands for Extract-Transform-Load. The idea is to build up your central data warehouse by extracting information from a lot of different data-sources, transform it and then load it into your data warehouse.
A variety of (also graphical) tools exist to implement such a process. Since you said you'll probably running a Microsoft stack, I suggest having a look at Sql Server Integration Services (SSIS).
Regarding your suggestion to implement integration using a web-service, I don't think that's a good idea too. Similarily, I don't think shifting the burden of data integration to your customers is a good idea either. You should agree with your customers on some form of a data exchange format, it could be as simple as a CSV file, or XML, Excel sheets, Access databases, use whatever suits your needs.
Any modern ETL tool like SSIS is capable of working with those different data sources.