infuxdb:我的测量条目中只有两个最终到达数据库

发布于 2025-02-06 12:15:50 字数 2289 浏览 2 评论 0原文

我有以下代码:

if __name__ == '__main__':
     measurements = [
       {
            "name": "name_of_my_measurement",
            "tags": {
                "customer_code": "FOO"
            },
            "fields": {
                "data_path": "s3://foo/2022-05-22",
                "count": "40690526"
            }
        },
        {
            "name": "name_of_my_measurement",
            "tags": {
                "customer_code": "FOO"
            },
            "fields": {
                "data_path": "s3://foo/2022-05-21",
                "count": "10"
            }
        },
        ....
    ]

    for m in measurements:
        serialized_measurement = json_metric_serializer_format_to_influxdb_info(m)
        influxdb_utils.write_value_into_influxDB(client=i_client,
                                         tags=serialized_measurement.tags,
                                         fields=serialized_measurement.fields,
                                         measurement=serialized_measurement.measurement,
                                         time=serialized_measurement.time)




def json_metric_serializer_format_to_influxdb_info(meas):
    import collections
    influxdb_info = collections.namedtuple("influxdb_info", ["measurement", "fields", "tags", "time"])

    measurement = str(meas["name"])
    time = meas["fields"].get("time", "")
    fields = {x: meas["fields"][x] for x in meas["fields"] if not x == "time"}
    tags = meas["tags"]

    return influxdb_info(measurement=measurement, fields=fields, tags=tags, time=time)


def write_value_into_influxDB(client, tags, fields, measurement, time):
    import datetime

    if time == '':
        time = datetime.datetime.today().strftime('%Y-%m-%dT%H:%M:%SZ')

    json_body = [
        {
            "measurement": "{measurement}".format(measurement=measurement),
            "tags": tags,
            "fields": fields,
            "time": time
        }
    ]

    client.write_points(json_body)

测量数组中可能有100个条目,但是当代码运行两个条目时,这些条目最终以name_of_my_measurement测量表中。这些条目看起来完全随机。例如,这些data_path字段有一个日期,并且在测量数组中有20个连续的日期路径,每当我运行脚本时,我检查了涌入中的测量值,这两个测量值中的另一组是编写的。到数据库。这对我来说是完全随机的。

这里怎么了?

I have the following code:

if __name__ == '__main__':
     measurements = [
       {
            "name": "name_of_my_measurement",
            "tags": {
                "customer_code": "FOO"
            },
            "fields": {
                "data_path": "s3://foo/2022-05-22",
                "count": "40690526"
            }
        },
        {
            "name": "name_of_my_measurement",
            "tags": {
                "customer_code": "FOO"
            },
            "fields": {
                "data_path": "s3://foo/2022-05-21",
                "count": "10"
            }
        },
        ....
    ]

    for m in measurements:
        serialized_measurement = json_metric_serializer_format_to_influxdb_info(m)
        influxdb_utils.write_value_into_influxDB(client=i_client,
                                         tags=serialized_measurement.tags,
                                         fields=serialized_measurement.fields,
                                         measurement=serialized_measurement.measurement,
                                         time=serialized_measurement.time)




def json_metric_serializer_format_to_influxdb_info(meas):
    import collections
    influxdb_info = collections.namedtuple("influxdb_info", ["measurement", "fields", "tags", "time"])

    measurement = str(meas["name"])
    time = meas["fields"].get("time", "")
    fields = {x: meas["fields"][x] for x in meas["fields"] if not x == "time"}
    tags = meas["tags"]

    return influxdb_info(measurement=measurement, fields=fields, tags=tags, time=time)


def write_value_into_influxDB(client, tags, fields, measurement, time):
    import datetime

    if time == '':
        time = datetime.datetime.today().strftime('%Y-%m-%dT%H:%M:%SZ')

    json_body = [
        {
            "measurement": "{measurement}".format(measurement=measurement),
            "tags": tags,
            "fields": fields,
            "time": time
        }
    ]

    client.write_points(json_body)

There might be 100 entries in the measurements array but when the code runs only TWO of all these entries end up in the name_of_my_measurement measurement table. These entries look completely random. For example these data_path field has a date, and there are 20 consecutive date paths in the measurements array and whenever I run the script and I check the measurement in Influx, another set of two of these measurements are written to the database. This looks completely random to me.

What can it be going wrong here?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

千と千尋 2025-02-13 12:15:50

这是一个综合关键问题。当我更改这样的测量值时:

{
            "name": "name_of_my_measurement",
            "tags": {
                "customer_code": "FOO",
                "data_path": "s3://foo/2022-05-21"
            },
            "fields": {
                "count": "10"
            }
}

所有条目都已成功持续。

这也解释了这也解释您应该如何明智地选择标签和字段

This was a composite key problem. When I changed the measurements like this:

{
            "name": "name_of_my_measurement",
            "tags": {
                "customer_code": "FOO",
                "data_path": "s3://foo/2022-05-21"
            },
            "fields": {
                "count": "10"
            }
}

all the entries were successfully persisted.

This also explains how you should choose your tags and fields more wisely than I did.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文