日志收集插件使用阿里云开源log-pilot

公众号：yunops

一、原理：

使用 log-pilot 作为日志收集客户端（使用 ack 集群的话 log-pilot 可作为组件在控制台安装），制定收集规则，将容器日志（落盘文件/控制台）收集到 elastic，使用 kibana 进行查阅

二、部署：

处于兼容性考虑，阿里云 log-pilot 建议使用 elastic 版本低于 7.0，这里使用 6.8.0。

1、部署 eck 工具

在 k8s 里部署 elastic 和 kibana，官方提供了便捷的部署工具 ECK：部署 eck 工具说明

2、部署 elastic

apiVersion: elasticsearch.k8s.elastic.co/v1
kind: Elasticsearch
metadata:
  name: unpro-k8s
  namespace: demo-ops
spec:
  version: 6.8.0
  volumeClaimDeletePolicy: DeleteOnScaledownOnly
  http:
    tls:
      selfSignedCertificate:
        disabled: true
  nodeSets:
    - name: lx-log
      count: 1
      podTemplate:
        spec:
          initContainers:
            - name: sysctl
              securityContext:
                privileged: true
              command: ["sh", "-c", "sysctl -w vm.max_map_count=262144"]
      volumeClaimTemplates:
        - metadata:
            name: elasticsearch-data
          spec:
            accessModes:
              - ReadWriteOnce
            resources:
              requests:
                storage: 200Gi
            storageClassName: alicloud-disk-essd

3、部署 Kibana

apiVersion: kibana.k8s.elastic.co/v1
kind: Kibana
metadata:
  name: unpro-k8s
  namespace: demo-ops
spec:
  version: 6.8.0
  count: 1
  elasticsearchRef:
    name: unpro-k8s
  http:
    tls:
      selfSignedCertificate:
        disabled: true
  config:
    i18n.locale: "zh-CN"
---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: unpro-k8s-kibana
  namespace: demo-ops
spec:
  rules:
    - host: unprolog.imwang.top
      http:
        paths:
          - backend:
              serviceName: unpro-k8s-kb-http
              servicePort: 5601
            path: /

4、部署 log-pilot

【可选】如果是 java 应用，为了聚合错误日志（如堆栈），需进行行首正则匹配：修改 log-pilot 镜像文件中的 filebeat.tpl 文件重新 build 镜像，配置详情可参考

apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: log-pilot
  labels:
    app: log-pilot
  namespace: demo-ops
spec:
  selector:
    matchLabels:
      app: log-pilot
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      labels:
        app: log-pilot
      annotations:
        scheduler.alpha.kubernetes.io/critical-pod: ""
    spec:
      tolerations:
        - key: node-role.kubernetes.io/master
          effect: NoSchedule
      containers:
        - name: log-pilot
          # 版本请参考https://github.com/AliyunContainerService/log-pilot/releases。
          image: registry-vpc.cn-hangzhou.aliyuncs.com/demo-public/log-pilot:v0.9.7-filebeat-multiline
          resources:
            limits:
              memory: 500Mi
            requests:
              cpu: 200m
              memory: 200Mi
          env:
            - name: "NODE_NAME"
              valueFrom:
                fieldRef:
                  fieldPath: spec.nodeName
            - name: "PILOT_LOG_PREFIX"
              value: "demo"
            - name: "LOGGING_OUTPUT"
              value: "elasticsearch"
            # 请确保集群到ES网络可达。
            - name: "ELASTICSEARCH_HOSTS"
              value: "unpro-k8s-es-http.demo-ops.svc.cluster.local:9200"
            # 配置ES访问权限。
            - name: "ELASTICSEARCH_USER"
              value: "elastic"
            - name: "ELASTICSEARCH_PASSWORD"
              value: "xxxxx"
          volumeMounts:
            - name: sock
              mountPath: /var/run/docker.sock
            - name: root
              mountPath: /host
              readOnly: true
            - name: varlib
              mountPath: /var/lib/filebeat
            - name: varlog
              mountPath: /var/log/filebeat
            - name: localtime
              mountPath: /etc/localtime
              readOnly: true
          livenessProbe:
            failureThreshold: 3
            exec:
              command:
                - /pilot/healthz
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 2
          securityContext:
            capabilities:
              add:
                - SYS_ADMIN
      terminationGracePeriodSeconds: 30
      volumes:
        - name: sock
          hostPath:
            path: /var/run/docker.sock
        - name: root
          hostPath:
            path: /
        - name: varlib
          hostPath:
            path: /var/lib/filebeat
            type: DirectoryOrCreate
        - name: varlog
          hostPath:
            path: /var/log/filebeat
            type: DirectoryOrCreate
        - name: localtime
          hostPath:
            path: /etc/localtime

二、采集日志：

三、Kibana 配置：

不展开详述，仅说明一些注意事项/常用的配置项

参考资料：