如何在 Kubernetes 集群上部署 Elasticsearch？

Question

如何在 Kubernetes 集群上部署 Elasticsearch？我需要在集群外部使用 Elasticsearch。为此，我应该使用哪种服务？谁能发布部署和服务 yaml 文件吗？

Answer 1

如果您想在生产中运行elasticsearch，您可以使用helm图表来部署elasticsearch。

Helm 图表：https://github.com/elastic/helm-charts

如果您只是您可以在 YAML 文件下面进行部署以进行开发和测试：

apiVersion: apps/v1
kind: StatefulSet
metadata:
  labels:
    app : elasticsearch
    component: elasticsearch
    release: elasticsearch
  name: elasticsearch
spec:
  podManagementPolicy: Parallel
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app : elasticsearch
      component: elasticsearch
      release: elasticsearch
  serviceName: elasticsearch
  template:
    metadata:
      creationTimestamp: null
      labels:
        app : elasticsearch
        component: elasticsearch
        release: elasticsearch
    spec:
      containers:
      - env:
        - name: cluster.name
          value: <SET THIS>
        - name: discovery.type
          value: single-node
        - name: ES_JAVA_OPTS
          value: -Xms512m -Xmx512m
        - name: bootstrap.memory_lock
          value: "false"
        image: elasticsearch:6.5.0
        imagePullPolicy: IfNotPresent
        name: elasticsearch
        ports:
        - containerPort: 9200
          name: http
          protocol: TCP
        - containerPort: 9300
          name: transport
          protocol: TCP
        resources:
          limits:
            cpu: 250m
            memory: 1Gi
          requests:
            cpu: 150m
            memory: 512Mi
        securityContext:
          privileged: true
          runAsUser: 1000
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /usr/share/elasticsearch/data
          name: elasticsearch-data
      dnsPolicy: ClusterFirst
      initContainers:
      - command:
        - sh
        - -c
        - chown -R 1000:1000 /usr/share/elasticsearch/data
        - sysctl -w vm.max_map_count=262144
        - chmod 777 /usr/share/elasticsearch/data
        - chomod 777 /usr/share/elasticsearch/data/node
        - chmod g+rwx /usr/share/elasticsearch/data
        - chgrp 1000 /usr/share/elasticsearch/data
        image: busybox:1.29.2
        imagePullPolicy: IfNotPresent
        name: set-dir-owner
        resources: {}
        securityContext:
          privileged: true
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        volumeMounts:
        - mountPath: /usr/share/elasticsearch/data
          name: elasticsearch-data
      restartPolicy: Always
      schedulerName: default-scheduler
      securityContext: {}
      terminationGracePeriodSeconds: 10
  updateStrategy:
    type: OnDelete
  volumeClaimTemplates:
  - metadata:
      creationTimestamp: null
      name: elasticsearch-data
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 10Gi

参考要点：https://gist.github.com/harsh4870/ccd6ef71eaac2f09d7e136307e3ecda6

我应该使用哪种服务

您可以使用 LoadBalancer 类型公开 Elasticsearch 服务，并将其公开到互联网并使用它。

service.yaml

---
apiVersion: v1
kind: Service
metadata:
  name: eks-srv
spec:
  selector:
    app: elasticsearch
    component: elasticsearch
  ports:
    - name: db
      protocol: TCP
      port: 9200
      targetPort: 9200
    - name: monitoring
      protocol: TCP
      port: 9300
      targetPort: 9300
  type: LoadBalancer

注意：

您需要使用NodePort或LoadBalancer作为ClusterIp<的服务类型/strong> 除非您使用某些代理设置或入口，否则您将无法公开服务。

Answer 2

正如其他答案所指出的，您可以使用 helm 图表，但是 Elastic 也发布了其 自己的操作符，这是一个比部署裸 StatefulSet 更强大的选项

要安装操作符：

kubectl create -f https://download.elastic.co/downloads/eck/2.0.0/crds.yaml
kubectl apply -f https://download.elastic.co/downloads/eck/2.0.0/operator.yaml

并部署集群

cat <<EOF | kubectl apply -f -
apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: quickstart
spec:
  version: 8.0.0
  nodeSets:
  - name: default
    count: 1
    config:
      node.store.allow_mmap: false
EOF

如果您想准备好此生产，您可能需要做一些进一步的调整您都可以在文档中找到

Answer 3

好吧，以下 yaml 对我有用
elasticsearch-deploy.yaml：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: elasticsearch
  labels:
    app: elasticsearch
spec:
  selector:
    matchLabels:
      app: elasticsearch
  replicas: 1 
  template: 
    metadata:
      labels:
        app: elasticsearch
    spec:
      containers:
      - name: elasticsearch
        image: elasticsearch:7.9.0
        resources:
          requests:
            memory: 4Gi
          limits:
            memory: 5Gi
        ports:
        - containerPort: 9200
        - containerPort: 9300
        env:
        - name: discovery.type
          value: single-node

现在，我们想要从集群外部访问这个elastic-search。默认情况下，部署将分配 clusterip 服务，用于访问同一集群内的 pod。这里我们使用 NodePort 服务来访问集群外部。
elasticsearch-service.yaml：

apiVersion: v1
kind: Service
metadata:
  name: esservice
spec:
  selector: 
    app: elasticsearch
  type: NodePort  
  ports:
    - port: 9200
      targetPort: 9200
      nodePort: 31200

这使您的服务可以通过浏览器访问：

HTTP://<nodeip>:<nodeport>

例如：HTTP://192.168.18.90:31200/
输出如下：

name    "elasticsearch-5974b56749-fj8n7"
cluster_name    "docker-cluster"
cluster_uuid    "4o-NidnuSSiBI-RwEPWEVQ"
version 
number  "7.9.0"
build_flavor    "default"
build_type  "docker"
build_hash  "a479a2a7fce0389512d6a9361301708b92dff667"
build_date  "2020-08-11T21:36:48.204330Z"
build_snapshot  false
lucene_version  "8.6.0"
minimum_wire_compatibility_version  "6.8.0"
minimum_index_compatibility_version "6.0.0-beta1"
tagline "You Know, for Search"

Answer 4

您可以尝试使用 YAML 文件，如下所示：

kind: StatefulSet
apiVersion: apps/v1
metadata:
  name: elasticsearch
  labels:
    com/es: es
spec:
  replicas: 1
  selector:
    matchLabels:
          com/es: es
  template:
     metadata:
      labels:
        com/es: es
    spec:
      containers:
        - resources: {}
          terminationMessagePath: /dev/termination-log
          name: elasticsearch
          env:
            - name: discovery.type
              value: single-node
          ports:
            - name: rest
              containerPort: 9200
              protocol: TCP
            - name: inter-node
              containerPort: 9300
              protocol: TCP
          imagePullPolicy: IfNotPresent
          volumeMounts:
            - name: <volume-name>
              mountPath: <mount-path>
          terminationMessagePolicy: File
          image: 'docker.elastic.co/elasticsearch/elasticsearch:7.10.2'
      restartPolicy: Always
      terminationGracePeriodSeconds: 10
      dnsPolicy: ClusterFirst
      securityContext: {}
      imagePullSecrets:
        - name: <image-pullsecret>
      schedulerName: default-scheduler
  volumeClaimTemplates:
    - kind: PersistentVolumeClaim
      apiVersion: v1
      metadata:
        name: es-data
      spec:
        accessModes:
          - ReadWriteOnce
        resources:
          requests:
            storage: 15Gi
        volumeMode: Filesystem
  serviceName: es-service
  podManagementPolicy: OrderedReady

您可以使用此 yaml 创建 statefulset，statefullset 将
在内部创建elaticsearch pod。如果你想要卷挂载你
可以添加此 yaml 中提到的卷挂载数据。如果你
不要从规格中删除音量部分
VolumeClaimTemplates。

要使用外部的elasticsearch进行集群，请尝试以下操作：

kind: Service
apiVersion: v1
metadata:
  name: es-service
  labels:
    com/es: es
spec:
  ports:
    - name: rest
      protocol: TCP
      port: 9200
      targetPort: 9200
    - name: inter-node
      protocol: TCP
      port: 9300
      targetPort: 9300
  selector:
    com/es: es
  clusterIP: <ip of your cluster>
  clusterIPs:
    -  <ip of your cluster>
  type: ClusterIP

  ipFamilies:
    - IPv4


      

Answer 5

这个清单 (deployment.yaml) 在 GCP Kubernetes Engine 中适用于我，
您使用以下命令运行：

kubectl apply -f 部署.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: elasticsearch
  labels:
    app: elasticsearch
spec:
  selector:
    matchLabels:
      app: elasticsearch
  replicas: 1 
  template: 
    metadata:
      labels:
        app: elasticsearch
    spec:
      containers:
      - name: elasticsearch
        image: elasticsearch:7.9.0
        resources:
          requests:
            memory: 2Gi
          limits:
            memory: 4Gi
        ports:
        - containerPort: 9200
        - containerPort: 9300
        env:
        - name: discovery.type
          value: single-node

并使用此服务，您可以检查外部 IP (http://serviceIP:9200)，运行相同的命令：

kubectl apply -f service.yaml

apiVersion: v1
kind: Service
metadata:
  name: esservice
spec:
  type: LoadBalancer
  selector:
    app: elasticsearch
  ports:
  - protocol: TCP
    port: 9200
    targetPort: 9200