如何在.NET 中高效、轻松地导入/导出大型业务数据？
当我们想要高效、轻松地将所有数据加载（导入）到 .NET 中的业务对象中时，对于大型业务数据来说，最佳的数据格式（XML、数据库等）是什么？
我想一次性加载所有数据并创建所有业务对象。 数据必须有一种“模式”，以便其他人能够创建它。
另请注意，我希望简化从特定数据模式（可能是 XSD 模式、数据库模式等）创建业务类的过程。 我尝试了两种方法，但没有一种方法完全令人满意：
XML、xsd.exe 或等效工具（如 Xsd2Code）来生成业务类和映射。对于大数据，XML 存储的大小成为一个问题。
数据库 (SQL Server CE)，利用实体框架创建业务类和映射：在第一次测试期间，为了增加数据库大小，EF 处于严重的麻烦（性能问题和内存不足异常）。
我想要实现的目标（一次性将大型业务数据加载到内存中）似乎是“标准”（或者真的是吗？），但我找不到一种高效且干净的方法来实现它。
What would be the best data format (XML, Database, etc.) for large business data when we want to efficiently and easily load all of it (import) into business objects in .NET ?
I want to load all the data in one go and create all business objects.
The data must have a kind of a "schema" to enable other people to create it.
Also note that I want to ease the process of creating the business classes from the specific schema of the data (could be XSD schema, Database schema, etc.).
I have tried two approaches, neither of which proved entirely satisfactory :
XML, xsd.exe or equivalent tool (like Xsd2Code) to generate the business classes and the mapping. For large data, size of the XML store becomes an issue.
Database (SQL Server CE), leveraging the Entity Framework to create the business classes and the mapping : during the first tests, for increasing database size, EF was in serious trouble (performance problems and out of memory exceptions).
What I want to achieve (loading large business data into memory in one go) seems "standard" (or is it really ?), yet I can't find an efficient and clean way to do it.
如果你对这篇内容有疑问，欢迎到本站社区发帖提问 参与讨论，获取更多帮助，或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱，如果其他用户或者作者回复了您的评论，将不能在第一时间通知您！
发布评论
评论（3）
你的问题其实有点模糊，所以我只能凭自己的感觉回答。存储数据是针对数据库的。您正在谈论业务对象，因此使用 OR/M 工具是一个合乎逻辑的选择。由于您使用的是 .NET，因此实体框架是一个合乎逻辑的选择。
在我看来，实体框架不应该引起任何其他解决方案不会给您带来的内存问题。我认为您的OOM问题是由于工具使用不当或者可能是其他问题引起的。如果您仍然遇到问题，请在此处询问有关此问题的具体问题。
祝你好运。
Your question is actually a bit vague, so I can only answer with my gut feeling. Storing data is for databases. You are talking about business objects, so using an OR/M tool is a logical pick. Since you're in .NET, Entity Framework is a logical pick.
It doesn’t seem to me that Entity Framework should cause any problems around memory that other solutions won't give you. I think your OOM problems are caused by improper use of the tool or perhaps other problems. If you're keep having problems with it, please ask a specific question about this problem, here at SO.
Good luck.
查看 .NET 框架中的 XML 序列化。我将其用于数据交换项目，.NET 支持使用很少的代码将信息序列化到对象或从对象中反序列化信息。这可能是你想要走的路。
编辑：我发现您也使用过 xsd.exe，您可以使用数据库将 XML 存储在表的列中以便于检索。实际上，我们在当前的项目中也在这样做。我们从第三方源接收一些 XML，因此我们将其存储在表中，根据架构对其进行验证，然后使用 xsd.exe 方法将其读入从 .xsd 文件生成的 .NET 对象中。我不确定您的数据有多大，但这种方法对我们来说效果很好。
Take a look at XML Serialization in the .NET framework. I'm using this with a data interchange project and .NET supports serializing and deserializing the information into and from objects with very little code. It may be the path you want to take.
Edit: I see you've played with xsd.exe as well, you could use a db to store the XML in a column in a table for easy retrieval. We're actually doing this as well in the current project. We're receiving some XML from a 3rd party source, so we store it in a table, validate it against the schema, then read it out into .NET objects generated from a .xsd file using the xsd.exe approach. I'm not sure how large your data is, but this approach works well for us.
如果您有 XML 和 SQL 数据库，则可以尝试 LINQ。使用 SQLMetal 程序自动生成与 SQLDatabase 表相对应的类。如果数据库中有关系，它们也会保留在 SQLMetal 生成的对象模型中。
If you have XML and SQLDatabases, you can try LINQ. Use SQLMetal program to automatically generate Classes that correspond to your SQLDatabase tables. If you have relationships in you DB they are also preserved in object model generated by SQLMetal.