如何从 OData API 提取数据到 SQL
我想知道社区对于从 OData API 提取数据到 SQL 2008 R2 有何建议。我需要创建一个将数据导入 SQL 的夜间作业。我是否应该创建一个简单的控制台应用程序来迭代 OData API 并导入到 SQL？或者我可以创建某种类型的 SQL Server BI 应用程序吗？或者有更好的方法来做到这一点吗？
I'm wondering what the community suggests for extracting data from an OData API to SQL 2008 R2. I need to create a nightly job that imports the data to SQL. Should I create a simple console app that iterates through the OData API and imports to SQL? Or can I create some type of SQL Server BI app? Or is there a better way to do this?
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（1）
这会非常慢。 OData 不是用于批量操作的 API。它旨在让客户端访问各个实体并导航它们之间的关系，最多对一些过滤列表进行分页。
通过 OData 提取整个转储不会让任何人满意。 OData API 所有者必须调查是谁在他的 API 上进行所有这些夜间爬行，并发现是您，并且可能会切断您的联系。另一方面，您会发现 OData 不是一种有效的批量传输格式，并且来回封送 HTTP 编码实体并不是消耗带宽的最佳方式。每次抓取整个数据库，而不是仅仅发现上次抓取的增量图，只有在数据库达到临界大小 S 时才会起作用，此时更新所需的时间比您的频率要长重新汇集！
此外，如果不是您的数据，OData API 的使用条款很可能明确阻止此类批量爬网。
获取数据转储、存档并使用 FTP 进行复制。
This is going to be sooo slow. OData is not an API for bulk operations. It is designed for clients to access individual entities and navigate relations between them, at most paginate across some filtered lists.
Extracting a entire dump via OData is not going to make anybody happy. The OData API owner will have to investigate who is doing all these nightly crawls over his API and discover it is you and likely cut you off. You on the other hand will discover that OData is not an efficient bulk transport format and marshaling HTTP encoded entities back and forth is not exactly the best way to spend your bandwidth. And crawling the entire database every time, as opposed to just discovering the deltagrams from last crawl, is only going to work until the database reaches that critical size S at which the update takes longer than the frequency you're pooling!
Besides, if is not your data, it is extremely likely that the use terms for the OData API explicitly prevent such bulk crawls.
Get a dump of the data, archive it, and copy it using FTP.