在Elasticsearch上保存数据的最佳模型是什么?

发布于 2025-02-06 12:12:08 字数 3248 浏览 2 评论 0原文

我有铁轨应用程序,并将弹性搜索用作我的Rails应用程序中的搜索引擎。该应用程序从移动应用程序中收集数据,并且可以从任何类型的移动应用程序中收集。移动应用发送两种类型的数据用户资料详细信息和用户操作详细信息。我的应用程序管理员可以通过多种条件和操作搜索此数据,并获取特定的结果,哪些是用户配置文件的详细信息。之后,我的应用程序管理员可以与此个人资料进行通信,例如,发送电子邮件,SMS甚至在线聊天。就我而言,我有两个选择来保存用户数据。首先,我想将用户配置文件和用户操作详细信息保存在带有此结构配置文件的单独文档中:

POST profilee-2022-06-09/_doc
{
  "profile": {
    "app_id": "abbccddeeff",
    "profile_id": "2faae1d6-5875-4b36-b119-74a14589c841",
    "whatsapp_number": "whatsapp:+61478421940",
    "phone": "+61478421940",
    "email": "[email protected]",
    "first_name": "john",
    "last_name": "doe"
  }
}

用户操作详细信息:

POST events_app_id_2022-05-17/_doc
{
    "app_id": "9vlgwrr6rg",
    "event": "Email_Sign_Up",
    "profile_id": "2faae1d6-5875-4b36-b119-74a14589c840",
    "media": "x1z1",
    "date_time": "2022-05-17T11:48:02.511Z",
    "device_id": "2faae1d6-5875-4b36-b119-74a14589c840",
    "lib": "android",
    "lib_version": "1.0.0",
    "os": "Android",
    "os_version": "12",
    "manufacturer": "Google",
    "brand": "google",
    "model": "sdk_gphone64_arm64",
    "google_play_services": "available",
    "screen_dpi": 440,
    "screen_height": 2296,
    "screen_width": 1080,
    "app_version_string": "1.0",
    "app_build_number": 1,
    "has_nfc": false,
    "has_telephone": true,
    "carrier": "T-Mobile",
    "wifi": true,
    "bluetooth_version": "ble",
    "session_id": "b1ad31ab-d440-435f-ac12-3d03c30ac44f",
    "insert_id": "1e285b51-abcf-46ae-8359-9a9d58970cdf"
}

正如我在App Adpins搜索本文档之前所说的那样,以获取特定的配置文件并使用该结果与在这种情况下,他们的问题是移动用户可以创建一个配置文件,几天或几个月后创建一些操作,因此用户配置文件的详细信息和用户操作详细信息将在不同的日期生成,因此,如果App Admins想获取特定的结果从这些数据中写了一些复杂的查询,我在应用程序中的弹性搜索中至少有两个查询,这是不可能的必须在某种程度上添加我需要实现加入查询,该查询是基于弹性搜索文档的成本,因此在第二种情况下是不可能的,我决定在这样的文档中保存用户配置文件和操作:

POST profilee-2022-06-09/_doc
{
  "profile": {
    "app_id": "abbccddeeff",
    "profile_id": "urm-2faae1d6-5875-4b36-b119-74a14589c841",
    "whatsapp_number": "whatsapp:+61478421940",
    "phone": "+61478421940",
    "email": "[email protected]",
    "first_name": "john",
    "last_name": "doe",
    "events": [
      {
        "app_id": "abbccddeeff",
        "event": "sign_in",
        "profile_id": "urm-2faae1d6-5875-4b36-b119-74a14589c841",
        "media": "x1z1",
        "date_time": "2022-06-06T11:52:02.511Z"
      },
      {
        "app_id": "abbccddeeff",
        "event": "course_begin",
        "profile_id": "urm-2faae1d6-5875-4b36-b119-74a14589c841",
        "media": "x1z1",
        "date_time": "2022-06-06T11:56:02.511Z"
      },
      {
        "app_id": "abbccddeeff",
        "event": "payment",
        "profile_id": "urm-2faae1d6-5875-4b36-b119-74a14589c841",
        "media": "x1z1",
        "date_time": "2022-06-06T11:58:02.511Z"
      }
    ]
  }
}

在这种情况下,在这种情况下,同样的状态,我必须与以前一样,必须每天生成一个配置文件索引并将用户操作附加到它,所以这意味着我必须每天连续更新,假设我有100,000个配置文件有50个操作,意味着每天100,000 * 50更新在我的服务器上具有严重性,因此仍然是不可能的。那么,您能帮助我根据我的描述在弹性搜索中保存我的数据的最佳模型是什么?

更新:弹性搜索对我的要求有用吗?如果我切换到MongoDB等其他数据库或添加Hadoop,就我而言,它更有用吗?

I have a rails application and use elastic search as a search engine in my rails app. this app collects data from the mobile application and could collect from any kind of mobile app. mobile app sends two types of data user profile details and user actions details. my app admins could search over this data with multiple conditions and operations and fetch the specific results and which are user profile details. after that my app admins could communicate with this profile, for example, send an email, SMS, or even chat online. In my case I have two options to save user data; first of all, I want to save user profiles details and user action details in a separate document with this structure profile doc:

POST profilee-2022-06-09/_doc
{
  "profile": {
    "app_id": "abbccddeeff",
    "profile_id": "2faae1d6-5875-4b36-b119-74a14589c841",
    "whatsapp_number": "whatsapp:+61478421940",
    "phone": "+61478421940",
    "email": "[email protected]",
    "first_name": "john",
    "last_name": "doe"
  }
}

user actions details:

POST events_app_id_2022-05-17/_doc
{
    "app_id": "9vlgwrr6rg",
    "event": "Email_Sign_Up",
    "profile_id": "2faae1d6-5875-4b36-b119-74a14589c840",
    "media": "x1z1",
    "date_time": "2022-05-17T11:48:02.511Z",
    "device_id": "2faae1d6-5875-4b36-b119-74a14589c840",
    "lib": "android",
    "lib_version": "1.0.0",
    "os": "Android",
    "os_version": "12",
    "manufacturer": "Google",
    "brand": "google",
    "model": "sdk_gphone64_arm64",
    "google_play_services": "available",
    "screen_dpi": 440,
    "screen_height": 2296,
    "screen_width": 1080,
    "app_version_string": "1.0",
    "app_build_number": 1,
    "has_nfc": false,
    "has_telephone": true,
    "carrier": "T-Mobile",
    "wifi": true,
    "bluetooth_version": "ble",
    "session_id": "b1ad31ab-d440-435f-ac12-3d03c30ac44f",
    "insert_id": "1e285b51-abcf-46ae-8359-9a9d58970cdf"
}

As I said before app admins search over this document to fetch specific profiles and use that result to communicate with them, in this case, the problem is the mobile user could create a profile and a few days or a few months later create some actions so user profile details and user action details are generated in different days so if app admins want to fetch specific result from this data and wrote some complex query I have at least two queries by application on my elastic search in my app it's impossible because each query must save for later use by admin, so As a result of business logic it's impossible to me, and I have to add in some case I need to implement join query that based on elastic search documentation It has cost so it's impossible In the second scenario I decided to save both user profile and action in one docs somethings like this:

POST profilee-2022-06-09/_doc
{
  "profile": {
    "app_id": "abbccddeeff",
    "profile_id": "urm-2faae1d6-5875-4b36-b119-74a14589c841",
    "whatsapp_number": "whatsapp:+61478421940",
    "phone": "+61478421940",
    "email": "[email protected]",
    "first_name": "john",
    "last_name": "doe",
    "events": [
      {
        "app_id": "abbccddeeff",
        "event": "sign_in",
        "profile_id": "urm-2faae1d6-5875-4b36-b119-74a14589c841",
        "media": "x1z1",
        "date_time": "2022-06-06T11:52:02.511Z"
      },
      {
        "app_id": "abbccddeeff",
        "event": "course_begin",
        "profile_id": "urm-2faae1d6-5875-4b36-b119-74a14589c841",
        "media": "x1z1",
        "date_time": "2022-06-06T11:56:02.511Z"
      },
      {
        "app_id": "abbccddeeff",
        "event": "payment",
        "profile_id": "urm-2faae1d6-5875-4b36-b119-74a14589c841",
        "media": "x1z1",
        "date_time": "2022-06-06T11:58:02.511Z"
      }
    ]
  }
}

In this case, In the same state, I have to do as same as I do in before and I have to generate a profile index per day and append user action to it, so It means I have to update continuously each day, assume I have 100,000 profile and each one have 50 actions it means 100,000 * 50 per day update that have severity on my server so still it's impossible. So Could you please help me what is the best model to save my data in elastic search based on my descriptions?

Update: Does elastic search useful for my requirements? If I switch to other databases like MongoDB or add Hadoop it be more useful in my case?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。
列表为空,暂无数据
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文