多个记录更新验证,并具有整个数据集
当前,我有一个终点,允许更新多个记录。在保存数据库更改之前,我需要验证从前端发送的数据。
我正在使用某些验证问题,这些验证需要对数据库表中的所有其他记录进行检查。 (例如:ATE间隔重叠/间隙,唯一对检查)。
在这些情况下,当我尝试进行验证时,我有两组数据集中
- 从前端发送的数据存储在内存/变量中。
- 数据库上的数据。
为了使验证正确运行,我需要一种方法将数据(更新的记录)与数据库中的数据合并(原始记录 +其他目前未更新的数据)中的数据。
是否有一个很好的方法可以使内存上的所有内容都加载并在此处合并两个数据集?
我想的另一个想法是,打开数据库事务将新数据设置为数据库,然后在执行差距/重叠检查查询时使用肮脏的读取。我不知道这是否是一个好方法。
额外的注释:
我将Oracle用作数据库和Dapper与它进行通信。 需要验证的表通常保存数百万记录。 同一问题对于创建端点。
另一个示例
我正在尝试创建实体。创建端点是调用的,它在主体上具有这些数据(日期格式DD/mm/yyy):
StartDate | EndDate |
---|---|
01/01/2022 | 05/01/2022 |
10/01/2022 | 11/01/2022 |
12/01/2022 12 | /01/2022222222222222222222222222222222222222/20222222222222222年12月12日15/01/2022 |
在数据库中我保存了这些记录:
ID | StartDate | Enddate |
---|---|---|
1 | 06/01/2022 | 09/01/2022 |
2 | 16/01/2022 | 20/01/2022 |
我需要检查日期之间是否有任何差距。如果我需要向用户发送警告(数据库中的数据可能无效 - 应用程序具有旧数据,目前我对此无能为力)。
我现在检查此操作的方式就像
WITH CTE_INNERDATA AS(
SELECT s.STARTDATE, s.ENDDATE
FROM mytable s
WHERE FK = somefkvalue
UNION ALL
SELECT :StartDt AS STARTDATE, :EndDt AS ENDDATE FROM DUAL -- this row contains the data from one of the rows that came form the front-end
),
CTE_DATA AS (
SELECT ctid.STARTDATE, ctid.ENDDATE, LAG(ctid.ENDDATE, 1) OVER (ORDER BY ctid.STARTDATE) AS PREV_ENDDATE FROM CTE_INNERDATA ctid
)
SELECT COUNT(1) FROM cte_data ctd
WHERE PREV_ENDDATE IS NOT NULL
AND PREV_ENDDATE < STARTDATE
在验证第三行(12/01/2022-2022-15/01/2022)时使用以下SQL使用以下SQL使用此SQL查询。 12/01/2022。 如果不使用单行联合,将其与前端发送的所有行一起使用,但我无法找到一种这样做的方法。
#update
i通过前端发送的记录进行迭代,并致电此方法检查空白。
private async Task ValidateIntervalGaps(int someFkValue, DateTime startdate, DateTime endDate)
{
var connection = _connectionProvider.GetOpenedConnection();
var gapsCount = await connection.QueryFirstAsync<int>(@"<<Query from above>>",
new { StartDt = startdate, EndDt = endDate, somefkvalue= someFkValue });
if (gapsCount > 0)
{
// Add warning message here
}
}
Currently, I have an endpoint that allows the update of multiple records. Before saving the changes to the database I need to validate the data that is being sent from the front end.
I am having an issue with some kinds of validations that require checks against all the other records in the database table. (Ex: ate intervals overlaps/gaps, unique pairs checks).
In these cases when I try to do the validations I have two sets of data
- The data sent from the front end that are stored in memory/variable.
- The data on the database.
For the validations to be run correctly I need a way to merge the data in memory(the updated records) with the data in a database(original records + other data that is not currently being updated).
Is there a good way of doing this that does not require loading everything on the memory and merging both datasets there?
Another idea I am thinking of is to open a database transaction set the new data to the database and then when executing the gaps/overlap check queries use dirty read. I don't know if this is a good approach though.
Extra notes:
I am using Oracle as a database and Dapper to communicate with it.
The tables that need validation usually hold millions of records.
The same issue is for the create endpoint.
Another example
I am trying to create entities. The create endpoint is called and it has these data on the body (date format dd/mm/yyy):
StartDate | EndDate |
---|---|
01/01/2022 | 05/01/2022 |
10/01/2022 | 11/01/2022 |
12/01/2022 | 15/01/2022 |
In database I have these records saved:
Id | StartDate | EndDate |
---|---|---|
1 | 06/01/2022 | 09/01/2022 |
2 | 16/01/2022 | 20/01/2022 |
I need to check if there are any gaps between the dates. If there are I need to send a warning to the user(data in the database can be invalid - the application has old data and I can't do anything about that at the moment).
The way I check for this right now is like by using the SQL below
WITH CTE_INNERDATA AS(
SELECT s.STARTDATE, s.ENDDATE
FROM mytable s
WHERE FK = somefkvalue
UNION ALL
SELECT :StartDt AS STARTDATE, :EndDt AS ENDDATE FROM DUAL -- this row contains the data from one of the rows that came form the front-end
),
CTE_DATA AS (
SELECT ctid.STARTDATE, ctid.ENDDATE, LAG(ctid.ENDDATE, 1) OVER (ORDER BY ctid.STARTDATE) AS PREV_ENDDATE FROM CTE_INNERDATA ctid
)
SELECT COUNT(1) FROM cte_data ctd
WHERE PREV_ENDDATE IS NOT NULL
AND PREV_ENDDATE < STARTDATE
Using this SQL query when validating the third row (12/01/2022 - 15/01/2022) there will be a gap between dates 09/01/2022 and 12/01/2022.
This issue would be fixed if instead of using union with a single row, to use it with all the rows send from the front-end but I can't figure out a way to do something like that.
#Update
I iterate through the records the frontend sent and call this method to check for gaps.
private async Task ValidateIntervalGaps(int someFkValue, DateTime startdate, DateTime endDate)
{
var connection = _connectionProvider.GetOpenedConnection();
var gapsCount = await connection.QueryFirstAsync<int>(@"<<Query from above>>",
new { StartDt = startdate, EndDt = endDate, somefkvalue= someFkValue });
if (gapsCount > 0)
{
// Add warning message here
}
}
如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。
绑定邮箱获取回复消息
由于您还没有绑定你的真实邮箱,如果其他用户或者作者回复了您的评论,将不能在第一时间通知您!
发布评论