从弹性v1.7到弹性7.x问题的建议者的重新索引问题

发布于 2025-01-31 08:26:21 字数 5147 浏览 2 评论 0 原文

我对reindex有问题,我不确定我发现的解决方案是最好的解决方案,因为我们遇到了Java客户端的索引问题。因此,我想确认重新索引步骤是否正确:

问题描述: 我在旧的v1.7上有一个索引,该索引需要迁移到使用上下文建议器的Elasticsearch v7.x。

GET /autocompleteterms_v20/_search?pretty 
{
 "took" : 2,
 "timed_out" : false,
 "_shards" : {
 "total" : 1,
 "successful" : 1,
 "failed" : 0
},
 "hits" : {
 "total" : 4130,
 "max_score" : 1.0,
 "hits" : [ {
 "_index" : "autocompleteterms_v20",
 "_type" : "autoitem",
 "_id" : "en_incorporated_by_reference",
 "_score" : 1.0,
 "_source":{ "name": "incorporated by reference", "name_suggest": { "input": "incorporated by reference", "context": { "lang": "en" }, "weight": 393 } }
 }, {
 "_index" : "autocompleteterms_v20",
 "_type" : "autoitem",
 "_id" : "en_double_gaming",
 "_score" : 1.0,
 "_source":{ "name": "double gaming", "name_suggest": { "input": "double gaming", "context": { 
 "lang": "en" }, "weight": 371 } }
 }, {
 "_index" : "autocompleteterms_v20",
"_type" : "autoitem",
"_id" : "en_checking_for_players",
"_score" : 1.0,
"_source":{ "name": "checking for players", "name_suggest": { "input": "checking for players", 
 "context": { "lang": "en" }, "weight": 2 } }
},

GET /autocompleteterms_v20/_mapping?pretty
{
"autocompleteterms_v20" : {
"mappings" : {
"autoitem" : {
"properties" : {
"name" : {
"type" : "string"
},
"name_suggest" : {
"type" : "completion",
"analyzer" : "standard",
"payloads" : true,
"preserve_separators" : false,
"preserve_position_increments" : false,
"max_input_length" : 50,
 "context" : {
 "lang" : {
 "type" : "category",
 "path" : "lang_field",
 "default" : [ "en" ]
  }
 }
 }
 }
 }
 }
 }
 }

为了归因于新群集,我发现建议者的V5有一些破坏的变化,因此当尝试在Reindex上转换索引的字段

  1. 首先,我创建映射:

      put autoCompletEterms_v20 {
     “设置”: {
     “指数”: {
     “ number_of_shards”:2,
     “ number_of_replicas”:0
     }
    },,
    “映射”:{
     “特性”: {
     “姓名”: {
      “类型”:“文本”
    },,
     “ name_suggest”:{
    “类型”:“完成”,
    “分析仪”:“标准”,
    “ preserve_separators”:false,
    “ preserve_position_increments”:false,
    “ max_input_length”:50,
    “上下文”:[
    {
     “名称”:“ lang”,
     “类型”:“类别”,
     “路径”:“ lang_field”
      }
    这是给出的
    }
    }
    }
    }
    {
     “认可”:是的,
     “ shards_acknowded”:是的,
     “索引”:“ autoCompleteTerms_v20”
    }
     
  2. 之后,我将字段上下文更改为上下文:

      post _reindex {
     “来源”: {
     “偏僻的”: {
      “主机”:http:// xxxx:9200,
      “用户名”:“用户”,
      “密码”:“密码”
     },,
     “索引”:“ autoCompleteTerms_v20”
     },,
     “ dest”:{
     “索引”:“ autoCompleteTerms_v20”
    },,
    “脚本”: {
    “来源”: ””” 
    ctx._source.name_suggest.contexts = ctx._source.name_suggest.context;
    ctx._source ['name_suggest']。删除('context');
    ”“”,
    “朗”:“无痛”
    }
    }
    {
    “ take”:366,
    “ timed_out”:false,
    “总计”:4130,
    “更新”:0,
    “创建”:4130,
    “删除”:0,
    “批次”:5,
    “ version_conflicts”:0,
    “ NOOP”:0,
    “ retries”:{
    “散装”:0,
    “搜索”:0
    },,
    “ thtottled_millis”:0,
    “ requests_per_second”:-1.0,
    “ thtottled_until_millis”:0,
    “失败”:[]
    }
     
  3. 使用显式上下文进行查询时,我会得到结果:

      post /autoCompletems_v20 /_search {
     “建议”: {
     “文字”:“精神”,
     “完成”:{
     “字段”:“ name_suggest”,
     “上下文”:{
     “ lang”:[[ 
        {“ context”:“ en”}
       这是给出的
       }
      }
      }
      }
      }
      {
      “ take”:5,
      “ timed_out”:false,
      “ _shards”:{
      “总计”:2,
      “成功”:2,
      “跳过”:0,
      “失败”:0
      },,
      “命中”:{
      “全部的” : {
      “值”:0,
      “关系”:“ eq”
      },,
      “ max_score”:null,
      “命中”:[]
      },,
      “建议” : {
      “完成”:[
      {
      “文字”:“精神”,
      “偏移”:0,
      “长度”:6,
      “选项” : [
      {
       “文字”:“精神”,
       “ _index”:“ autoCompleteTerms_v20”,
       “ _ type”:“ _doc”,
       “ _id”:“ en_spirit”,
       “ _score”:290.0,
       “_来源” : {
       “ name_suggest”:{
       “输入”:“精神”,
       “重量”:290,
       “上下文”:{
       “ lang”:“ en”
        }
       },,
       “名称”:“精神”
       },,
       “上下文”:{
       “ lang”:[[
       “ en”
      这是给出的
      }
      },,
      {
     “文字”:“游戏精神”,
     “ _index”:“ autoCompleteTerms_v20”,
     “ _ type”:“ _doc”,
     “ _id”:“ en_spirit_of_the_game”,
     “ _score”:34.0,
     “_来源” : {
     “ name_suggest”:{
      “输入”:“游戏精神”,
     “重量”:34,
     “上下文”:{
     “ lang”:“ en”
     }
    },,
    “名称”:“游戏精神”
     },,
    “上下文”:{
     “ lang”:[[
        “ en”
       这是给出的
       }
     }
     
  4. ,但是当我在没有上下文的情况下进行查询时,我会收到错误:

      POST /AUTOCOLETETETERMS_V20 /_SEARZ
     {
     “建议”: {
      “完成”:{
      “文字”:“精神”,
      “完成”:{
      “字段”:“ name_suggest”
      }
     }
    }
    }
     {
      “错误” : {
       “ root_cause”:[
      {
       “ type”:“ illegal_argument_exception”,
       “原因”:“上下文查询中缺少强制性上下文”
      }
      ],,
      “ type”:“ search_phase_execution_exception”,
     “原因”:“所有碎片都失败了”,
     “阶段”:“查询”,
     “分组”:是的,
     “ failed_shards”:[
     {
     “碎片”:0,
     “ index”:“ autoCompleteTerms_v20”,
     “ node”:“ b7epkwrvrzgklcnl7czapq”,
     “原因” : {
     “ type”:“ illegal_argument_exception”,
     “原因”:“上下文查询中缺少强制性上下文”
     }
     }
      ],,
     “ coade_by”:{
    “ type”:“ illegal_argument_exception”,
    “原因”:“上下文查询中缺少强制性上下文”,
    “ coade_by”:{
    “ type”:“ illegal_argument_exception”,
    “原因”:“上下文查询中缺少强制性上下文”
     }
     }
      },,
      “状态”:400
    }
     

但是在从v5.0的破裂变化中,我们有https://www.elastic.co/guide/en/elasticsearch/reference/5.0/breaking_50_suggester.html#_completion_mapping_with_multiple_contexts

it is mentioned: “没有上下文启用上下文的完成字段的新查询会产生所有索引建议的结果”

在这种情况下,我有两个主要问题:

  1. 是否正确完成了旧索引的重新索引?
  2. 我如何在不针对该索引选择上下文的情况下进行查询?

I have problem with the reindex, I‘m not sure 100% that the solution I found is the best as we are having issues to work with the index from the java client. So I want to confirm if the re-indexing steps are correct:

Problem Description:
I have an index on an old v1.7 that need to migrate to elasticsearch v7.x that uses context suggesters.

GET /autocompleteterms_v20/_search?pretty 
{
 "took" : 2,
 "timed_out" : false,
 "_shards" : {
 "total" : 1,
 "successful" : 1,
 "failed" : 0
},
 "hits" : {
 "total" : 4130,
 "max_score" : 1.0,
 "hits" : [ {
 "_index" : "autocompleteterms_v20",
 "_type" : "autoitem",
 "_id" : "en_incorporated_by_reference",
 "_score" : 1.0,
 "_source":{ "name": "incorporated by reference", "name_suggest": { "input": "incorporated by reference", "context": { "lang": "en" }, "weight": 393 } }
 }, {
 "_index" : "autocompleteterms_v20",
 "_type" : "autoitem",
 "_id" : "en_double_gaming",
 "_score" : 1.0,
 "_source":{ "name": "double gaming", "name_suggest": { "input": "double gaming", "context": { 
 "lang": "en" }, "weight": 371 } }
 }, {
 "_index" : "autocompleteterms_v20",
"_type" : "autoitem",
"_id" : "en_checking_for_players",
"_score" : 1.0,
"_source":{ "name": "checking for players", "name_suggest": { "input": "checking for players", 
 "context": { "lang": "en" }, "weight": 2 } }
},

GET /autocompleteterms_v20/_mapping?pretty
{
"autocompleteterms_v20" : {
"mappings" : {
"autoitem" : {
"properties" : {
"name" : {
"type" : "string"
},
"name_suggest" : {
"type" : "completion",
"analyzer" : "standard",
"payloads" : true,
"preserve_separators" : false,
"preserve_position_increments" : false,
"max_input_length" : 50,
 "context" : {
 "lang" : {
 "type" : "category",
 "path" : "lang_field",
 "default" : [ "en" ]
  }
 }
 }
 }
 }
 }
 }
 }

To reindex to the new cluster I have found that there were some breaking changes in v5 for the suggesters so when tried to transform the fields of the index on the reindex

  1. First I create the mapping:

    PUT autocompleteterms_v20 {
     "settings": {
     "index": {
     "number_of_shards": 2,
     "number_of_replicas": 0
     }
    },
    "mappings": {
     "properties": {
     "name": {
      "type": "text"
    },
     "name_suggest": {
    "type": "completion",
    "analyzer": "standard",
    "preserve_separators": false,
    "preserve_position_increments": false,
    "max_input_length": 50,
    "contexts": [
    {
     "name": "lang",
     "type": "category",
     "path": "lang_field"
      }
    ]
    }
    }
    }
    }
    {
     "acknowledged" : true,
     "shards_acknowledged" : true,
     "index" : "autocompleteterms_v20"
    }
    
  2. After that I reindex changing the field context to contexts:

     POST _reindex {
     "source": {
     "remote": {
      "host": http://x.x.x.x:9200,
      "username": "user",
      "password": "password"
     },
     "index": "autocompleteterms_v20"
     },
     "dest": {
     "index": "autocompleteterms_v20"
    },
    "script": {
    "source": """ 
    ctx._source.name_suggest.contexts = ctx._source.name_suggest.context;
    ctx._source['name_suggest'].remove('context');
    """,
    "lang": "painless"
    }
    }
    {
    "took" : 366,
    "timed_out" : false,
    "total" : 4130,
    "updated" : 0,
    "created" : 4130,
    "deleted" : 0,
    "batches" : 5,
    "version_conflicts" : 0,
    "noops" : 0,
    "retries" : {
    "bulk" : 0,
    "search" : 0
    },
    "throttled_millis" : 0,
    "requests_per_second" : -1.0,
    "throttled_until_millis" : 0,
    "failures" : [ ]
    }
    
  3. When I make a query using explicit context I get results:

     POST /autocompleteterms_v20/_search {
     "suggest": {
     "text": "spirit",
     "completion": {
     "field": "name_suggest",
     "contexts": {
     "lang": [ 
        { "context": "en" }
       ]
       }
      }
      }
      }
      }
      {
      "took" : 5,
      "timed_out" : false,
      "_shards" : {
      "total" : 2,
      "successful" : 2,
      "skipped" : 0,
      "failed" : 0
      },
      "hits" : {
      "total" : {
      "value" : 0,
      "relation" : "eq"
      },
      "max_score" : null,
      "hits" : [ ]
      },
      "suggest" : {
      "completeMe" : [
      {
      "text" : "spirit",
      "offset" : 0,
      "length" : 6,
      "options" : [
      {
       "text" : "spirit",
       "_index" : "autocompleteterms_v20",
       "_type" : "_doc",
       "_id" : "en_spirit",
       "_score" : 290.0,
       "_source" : {
       "name_suggest" : {
       "input" : "spirit",
       "weight" : 290,
       "contexts" : {
       "lang" : "en"
        }
       },
       "name" : "spirit"
       },
       "contexts" : {
       "lang" : [
       "en"
      ]
      }
      },
      {
     "text" : "spirit of the game",
     "_index" : "autocompleteterms_v20",
     "_type" : "_doc",
     "_id" : "en_spirit_of_the_game",
     "_score" : 34.0,
     "_source" : {
     "name_suggest" : {
      "input" : "spirit of the game",
     "weight" : 34,
     "contexts" : {
     "lang" : "en"
     }
    },
    "name" : "spirit of the game"
     },
    "contexts" : {
     "lang" : [
        "en"
       ]
       }
     }
    
  4. But when I make a query without context I get an error:

     POST /autocompleteterms_v20/_search
     {
     "suggest": {
      "completeMe": {
      "text": "spirit",
      "completion": {
      "field": "name_suggest"
      }
     }
    }
    }
     {
      "error" : {
       "root_cause" : [
      {
       "type" : "illegal_argument_exception",
       "reason" : "Missing mandatory contexts in context query"
      }
      ],
      "type" : "search_phase_execution_exception",
     "reason" : "all shards failed",
     "phase" : "query",
     "grouped" : true,
     "failed_shards" : [
     {
     "shard" : 0,
     "index" : "autocompleteterms_v20",
     "node" : "B7EpKWRVRzGkLCnl7CZapQ",
     "reason" : {
     "type" : "illegal_argument_exception",
     "reason" : "Missing mandatory contexts in context query"
     }
     }
      ],
     "caused_by" : {
    "type" : "illegal_argument_exception",
    "reason" : "Missing mandatory contexts in context query",
    "caused_by" : {
    "type" : "illegal_argument_exception",
    "reason" : "Missing mandatory contexts in context query"
     }
     }
      },
      "status" : 400
    }
    

But in the breaking change from v5.0 we have https://www.elastic.co/guide/en/elasticsearch/reference/5.0/breaking_50_suggester.html#_completion_mapping_with_multiple_contexts

it is mentioned:
"New queries with no context against a context-enabled completion field yields results from all indexed suggestions"

In this context I have two main questions:

  1. Is the reindexing of the old index is done correctly?
  2. How can I do queries without choosing context against that index?

如果你对这篇内容有疑问,欢迎到本站社区发帖提问 参与讨论,获取更多帮助,或者扫码二维码加入 Web 技术交流群。

扫码二维码加入Web技术交流群

发布评论

需要 登录 才能够评论, 你可以免费 注册 一个本站的账号。

评论(1

み格子的夏天 2025-02-07 08:26:21
  1. 是的,您做了正确的一切!

  2. 您不能因为还有另一个建议更改在ES 7.0中,强制提供上下文,因为否则性能过于降级

上下文完成建议

已删除了在6.x中删除的无上下文的启用和索引上下文的能力。 启用上下文的建议查询没有上下文必须访问每个建议,这会大大降低搜索性能。

  1. Yes you did everything correctly!

  2. You can't as there was another suggester change in ES 7.0 that made the provision of context mandatory because otherwise the performance was too degraded

Context Completion Suggester

The ability to query and index context enabled suggestions without context, deprecated in 6.x, has been removed. Context enabled suggestion queries without contexts have to visit every suggestion, which degrades the search performance considerably.

~没有更多了~
我们使用 Cookies 和其他技术来定制您的体验包括您的登录状态等。通过阅读我们的 隐私政策 了解更多相关信息。 单击 接受 或继续使用网站,即表示您同意使用 Cookies 和您的相关数据。
原文