ORM 的使用可能涉及数十亿条记录
前几天我在想这个问题,像 Twitter 这样的应用程序要处理数百万用户。我在想“跟随”的功能如何工作,数据库中的最大用户数可以跟随数据库中的最大用户数减一(他自己)。
如果这是一个多对多双向映射,它将创建一个可能包含数十亿条记录的关系表。 ORM 如何检索此类记录?例如,如果用户 A 关注 20,000 个用户,ORM 是否会为该用户加载 20,000 条记录,还是会使用分页方法?我可以处理带有小型记录集(例如小于 10 条记录)的 JPA/ORM,但除此之外,如何编写支持大型记录集的软件让我大吃一惊。抱歉,这个问题并不具体,但我正在尝试就此获取架构想法。在业余时间,我将对数十亿条记录进行一些测试,但希望首先从社区获得一些意见。
I was thinking of this the other day, apps like Twitter deal with millions of users. I was thinking how the functionality of 'following' would work, where the maximum amount of users in the database can follow the maximum amount of users in the database less one, (himself).
If this was a ManyToMany bidirectional mapping, it would create a relationship table with potentially billions of records. and how could an ORM retrieve such records? for example, if user A is following 20,000 users, would the ORM load 20,000 records for that one user, or would a pagination approach be used? I can deal with JPA / ORM with small recordsets (< 10 records, say), but beyond that blows my mind on how to write software to support large recordsets. Sorry this question is not specific, but I'm trying to get architectual ideas on this. In my spare time, I will run some tests against billions of records, but wanted to get some input from the community first.
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论
评论(1)
解决方案不是使用关系数据库:)而是使用 NoSQL。您可能会发现这很有用: http://nosql .mypopescu.com/post/407159447/cassandra-twitter-an-interview-with-ryan-king
the solution is not use a relational database :) and use NoSQL instead. you might find this useful: http://nosql.mypopescu.com/post/407159447/cassandra-twitter-an-interview-with-ryan-king