哪一个 NoSQL 数据库(如果有)可以为查询结果集提供*更改*流?

发布于 2024-10-21 12:11:18 字数 823 浏览 7 评论 0原文

哪一个 NoSQL 数据库(如果有)可以为查询结果集提供更改流

有人能给我举一些例子吗?

首先,我相信没有一个 SQL 数据库提供此功能 - 我是对的吗?

我需要能够指定任意、简单的查询,其在 SQL 中的等效项可以写为:

SELECT * FROM accounts WHERE balance < 0 and balance > -1000;

我想要一个初始结果集:

id: 100, name: Fred, balance: -10
id: 103, name: Mary, balance: -200

但随后我想要一个更改流永远跟随,直到我停止它们:

meta: remove, id: 100
meta: add,    id: 104, name: Alice, balance: -300
meta: remove, id: 103
meta: modify, id: 104, name: Alice, balance: -400
meta: modify, id: 104, name: Alison, balance: -400
meta: add,    id: 101, name: Clive, balance: -200
meta: modify, id: 104, name: Alison, balance: -100
...

注意:我'我不是谈论流式传输大型结果集。我正在寻找软实时的变化流。

此外,如果可能的话,它需要横向扩展。

谢谢,

克里斯。

Which, if any, of the NoSQL databases can provide stream of changes to a query result set?

Could anyone point me at some examples?

Firstly, I believe that none of the SQL databases provide this functionality - am I correct?

I need to be able to specify arbitrary, simple queries, whose equivalent in SQL might be written:

SELECT * FROM accounts WHERE balance < 0 and balance > -1000;

I want an an initial result set:

id: 100, name: Fred, balance: -10
id: 103, name: Mary, balance: -200

but then I want a stream of changes to follow, forever, until I stop them:

meta: remove, id: 100
meta: add,    id: 104, name: Alice, balance: -300
meta: remove, id: 103
meta: modify, id: 104, name: Alice, balance: -400
meta: modify, id: 104, name: Alison, balance: -400
meta: add,    id: 101, name: Clive, balance: -200
meta: modify, id: 104, name: Alison, balance: -100
...

Note: I'm not talking about streaming large result sets. I'm looking for a soft-realtime stream of changes.

Also, it needs to scale out, if possible.

Thanks,

Chris.

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(8

じ违心 2024-10-28 12:11:18

CouchDB 有一个更改源。基本上它是一个区块链,或者说是数据库自诞生以来每次变化的历史记录。您可以通过 JSON、JSONP、长轮询或作为连续流获取提要,并编写响应数据库中更改的应用程序。

这是我博客中的更改提要

要了解更多信息,请查看CouchDB 指南的此部分

CouchDB has a changes feed. Basically it's a block chain, or a history of every change in the database since inception. You can get the feed via JSON, JSONP, long polling or as a continuous stream and write applications that respond to changes in the database.

Here's the changes feed from my blog

To learn more check out this section of the CouchDB guide

樱娆 2024-10-28 12:11:18

尽管答案已被接受,但还有另一个答案触及了您的问题背后的假设的核心。

您与获取数据更改列表相关的业务问题是什么?如果您不只是获取数据更改列表,而是收到一组事件来告诉您数据更改的原因和方式,会怎样?

这个概念是“CQRS”作为架构背后的根本原因之一。基本上,您存储导致数据更改的所有事件,例如 FundsDeposited、FundsWithdrawn 等,并且您能够“重播”这些事件,不仅可以发现数据如何随时间变化,还可以发现原因。

一旦您沿着这条路走下去,您就能够将事件存储为流,并且不再局限于少数存储引擎。相反,您实际上可以使用任何存储引擎,它就能完成工作。

Although an answer has been accepted, there is another answer that gets to the heart of the assumptions underneath your question.

What is the business concern that you have related to getting a list of changes to the data? What if, instead of merely getting the list of changes to the data, you received a set of events that told you why and how the data changed.

This concept is one of the fundamental reasons behind "CQRS" as an architecture. Basically you store all events that caused a change to your data, e.g. FundsDeposited, FundsWithdrawn, etc. and you gain the ability to "replay" those events and discover not just how your data changed over time, but why.

Once you go down that road, you gain the ability to store events as a stream and you are no longer limited to a small handful of storage engines. Instead you could literally use any storage engine and it would get the job done.

若相惜即相离 2024-10-28 12:11:18

不确定这是否正是您正在寻找的东西,但认为它可能足够相关,值得一提!

如果在 MongoDB 中使用复制,所有写操作都存储在 oplog(操作日志)中。因此,每个插入/更新/删除都会记录在那里,以便可以在辅助节点上重放。它是一个有上限的集合,因此会循环并覆盖自身(您可以设置它的大小)。但从理论上讲,这个 oplog 可以用作检索更改流的方法 - 我自己还没有尝试过,但您可能可以轮询该 oplog。

Not sure if this is exactly the kind of thing you are looking for, but thought it possibly relevant enough to warrant a mention!

If you use replication in MongoDB, all write operations are stored in an oplog (operation log). So every insert/update/delete is recorded in there so that they can be replayed on the secondary nodes. It's a capped collection so cycles round and overwrites itself (you can set it's size). But in theory, this oplog could be used as a way to retrieve a stream of changes - I haven't tried it myself, but possibly you could poll that oplog.

风筝有风,海豚有海 2024-10-28 12:11:18

只是一个集思广益的答案:

让我们以 MongoDB 为例,并且不想像上面描述的那样访问更改源。 是的,与其他答案相比,这听起来很糟糕,但这是我在写作时出现这些答案之前的第一个想法......

当前与此问题相关的功能是上限集合(http://www.mongodb.org/display/DOCS/Capped+Collections),也许服务器 -端代码执行 (http://www.mongodb.org/display/DOCS /服务器端+代码+执行)。

使用上限集合,可以更轻松地写入大量数据,但读取较少数据(如日志文件) - 这种集合类型就是为这种情况而设计的。
服务器端脚本可用于外包大量处理(减少应用程序代码),但如果您想将逻辑完全集成到应用程序中,则可以忽略这一点。

不知道是否有带有“钩子”的NoSQL DB。我知道这在 postgres (SQL) 中是可能的。

目前,流逻辑必须在应用程序代码 AFAIK 中实现。

在 CouchDB 中,可以使用 MongoDB 中未实现的“视图”(如果这不正确,请给我一个链接,这也是一个有趣的主题!)。

不知道这是否有帮助。这是我第一次尝试在这里回答。

Only a brainstorming answer:

Let's take for example a MongoDB AND do not want to access the changes feed like described above. Yes, it sounds crappy compared to the other answers, but was my first idea before these answers popped up while writing ...

Current features -related to this question- are Capped Collections (http://www.mongodb.org/display/DOCS/Capped+Collections) and maybe Server-side Code Execution (http://www.mongodb.org/display/DOCS/Server-side+Code+Execution).

With capped collections it would be easier to write a lot of data but read less (like log files) - this collection type is made for such cases.
The server-side scripts can be used for outsourcing a lot of processing (less app code), but you can leave away this point if you want to completely integrate the logic in your app.

Don't know if there NoSQL DBs with "hooks". I know that's possible in postgres (SQL).

Currently the streaming logic has to be implemented in the app code AFAIK.

In CouchDB it could be possible with "Views" which are not implemented in MongoDB (if this isn't correct, please give me a link, this is a interesting topic, too!).

Don't know if this is helpful. It's my first try of an answer here on SO.

旧时模样 2024-10-28 12:11:18

此类事情应该在应用程序中完成,而不是在数据库中。

意思是,每次进行更改时,都应该记录为新记录。不是对记录的修改。
如果您这样做,您可以为您的应用程序添加更多智能

this type of thing should be done in the app, not the database.

Meaning, every time you make a change, it should be recorded as a new record. Not a modification to the record.
There's a whole lot more intelligence you can add to your app if you do it this way

凉城 2024-10-28 12:11:18

从 v.3.6 开始,MongoDB 使用 Change Streams 来允许应用程序订阅实时列表变化:

更改流允许应用程序访问实时数据更改,而无需担心操作日志的复杂性和风险。应用程序可以使用更改流来订阅集合上的所有数据更改并立即对其做出反应。

变更流可以使具有依赖业务系统的架构受益,一旦数据变更持久,就会通知下游系统。例如,变更流可以为开发人员在实施提取、转换和加载 (ETL) 服务、跨平台同步、协作功能和通知服务时节省时间。

默认情况下,流返回对集合中所有文档的更改,但您可以添加聚合管道以仅筛选与查询结果集匹配的文档。

As of v.3.6, MongoDB uses Change Streams to allow applications to subscribe to a realtime list of changes:

Change streams allow applications to access real-time data changes without the complexity and risk of tailing the oplog. Applications can use change streams to subscribe to all data changes on a collection and immediately react to them.

Change streams can benefit architectures with reliant business systems, informing downstream systems once data changes are durable. For example, change streams can save time for developers when implementing Extract, Transform, and Load (ETL) services, cross-platform synchronization, collaboration functionality, and notification services.

By default, a stream returns changes to all documents in a collection, but you can add an agregation pipeline to filter to only the documents which match your query result set.

这样的小城市 2024-10-28 12:11:18

如果接收所有更改(不仅仅是查询结果集的更改)是可以接受的,那么您可以创建 mongodb 复制从属服务器,并从主服务器接收所有更改。我见过 mongodb 复制从站甚至是用 php 编写的,所以实现它应该不会太难。

If recieving all changes (not only changes to a query result set) is accepteble, then you can create mongodb replication slave, and recieve all changes from master. I've seen mongodb replication slave written even in php, so it should not be too hard to implement that.

冷默言语 2024-10-28 12:11:18

mongoDB 实现了 tailable-cursor,但仅适用于上限集合。请参阅文档。根据您的具体要求,它可能有用。

mongoDB implements a tailable-cursor, but for capped collections only. See the docs. It may be of use depending on your specific requirements.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文