导出BigQuery记录:选择哪些资源类型(它们之间有什么区别?)

发布于 2025-01-25 09:46:26 字数 1100 浏览 0 评论 0 原文

我想将相关的BigQuery(审核)记录导出到BQ数据集。

因此,我必须为此创建一个日志接收器。

我找到了一个博客,该博客仅使用 bigquery_project bigquery_dataset
https://dataform.co/blog/exporting-bigquery-usage-usage-usage-logs

我注意到还有更多资源。要查询:

  • bigquery_project
  • bigquery_resource
  • bigquery_dts_dts_config
  • bigquery_dataset bigquery_table
  • bigquery_dts_run
  • bigquery_biengengine_biengine_biengine_model
  • i

i i在此处找到该列表:

因此,最后三个与数据传输和生物发动机有关。

但是我想知道 project ,<代码>资源和 dataset 之间有什么区别?

这三种不同的日志中有哪些信息? 和:我应该使用三种(或七种资源类型中的哪一个)?

I want to export relevant BigQuery (audit) logging to a BQ dataset.

So I have to create a log sink for that.

I found a blog that sets up the BQ logging only using bigquery_project or bigquery_dataset:
https://dataform.co/blog/exporting-bigquery-usage-logs

I noticed that there are several more resource.types to query on:

  • bigquery_project
  • bigquery_resource
  • bigquery_dataset
  • bigquery_table
  • bigquery_dts_config
  • bigquery_dts_run
  • bigquery_biengine_model

I found that list here:
https://cloud.google.com/logging/docs/api/v2/resource-list#resource-types

So the last three have to do with data transfer and biengine.

But I wonder what is the difference between project, resource and dataset?

What info is in those 3 different kinds of logs?
And: which of the three (or which of the seven resource types) should I use?

log sink for exporting BQ logs

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

只是偏爱你 2025-02-01 09:46:26

auditdata payload 到bigquery_resource,而不是bigquery_dataset。在BigQueryAuditMetadata消息中,resource.type设置为以下值之一:
bigquery_dataset 用于操作到数据集的操作,例如 google.cloud.bigquery.v2.datasetservice。代码>对于所有其他称为方法的方法,例如作业资源。labels.location。包含作业的位置。

resource.type =“ bigquery_project”和logName:“ cloudaudit.googleapis.com” 用于获取项目的bigquery审核日志。

resource.type =“ bigquery_dataset”和logName:“ cloudaudit.googleapis.com”
用于获取数据集的BigQuery审核日志。

resource.type =“ bigquery_dts_config”和logName:“ cloudaudit.googleapis.com”
用于获取数据传输服务配置的BigQuery审核日志。

resource.type =“ bigquery_dts_run”和logName:“ cloudaudit.googleapis.com” 用于获取数据传输服务的BigQuery Audit Logs。

resource.type =“ bigquery_biengine_model”和logName:“ cloudaudit.googleapis.com” 用于获取BIQUERY AUDIT LOGS用于BI Engine Model。

示例:

要在查询构建器中显示给定传输config_id的日志条目,添加以下过滤器:

resource.type="bigquery_dts_config"
labels.run_id="transfer_config_id"

有关更多信息,您可以参考此 document

AuditData payload will return resource.type set to bigquery_resource, not bigquery_dataset. In BigQueryAuditMetadata messages, resource.type is set to one of the following values:
bigquery_dataset for operations to datasets such as google.cloud.bigquery.v2.DatasetService.* such as resource.labels.dataset_id contains the encapsulating dataset and bigquery_project for all other called methods, such as jobs resource.labels.location contains the location of the job.

resource.type="bigquery_project" AND logName:"cloudaudit.googleapis.com" is used to get BigQuery audit logs for a project.

resource.type="bigquery_dataset" AND logName:"cloudaudit.googleapis.com"
is used to get BigQuery audit logs for a dataset.

resource.type="bigquery_dts_config" AND logName:"cloudaudit.googleapis.com"
is used to get BigQuery audit logs for a Data Transfer Service configuration.

resource.type="bigquery_dts_run" AND logName:"cloudaudit.googleapis.com" is used to get BigQuery audit logs for a Data Transfer Service.

resource.type="bigquery_biengine_model" AND logName:"cloudaudit.googleapis.com" is used to get BigQuery audit logs for BI Engine Model.

Example:

To show log entries from a given transfer config_id, in the Query builder, add the following filter:

resource.type="bigquery_dts_config"
labels.run_id="transfer_config_id"

For more information you can refer to this document.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文