elastic stack 结合 search guard 配置日志服务器全过程

一、架构规划

   1. 操作系统:CentOS 6.8

   2. elastic stack套件 5.0.1 

   3. Search Guard 5.0.1

   4. 主机地址:192.168.5.251(node1) 192.168.5.252(node2) 192.168.5.253(node3)

二、软件下载地址:

   Elastic Stack:https://www.elastic.co/downloads

   Search Guard:https://github.com/floragunncom/search-guard/tree/es-5.0.1

   Search Guard SSL:https://github.com/floragunncom/search-guard-ssl/tree/5.1.1

三、搭建elasticsearch集群

一、同步时间 三个结点都需要执行
[root@localhost ~]# ntpdate time.windows.com
 8 Jan 10:43:19 ntpdate[1458]: step time server 52.169.179.91 offset -28798.105134 sec

二、安装软件包 三个结点都需要执行
[root@localhost elk]# ls
elasticsearch-5.0.1.rpm  filebeat-5.0.1-x86_64.rpm  jdk-8u101-linux-x64.rpm  kibana-5.0.1-x86_64.rpm  logstash-5.0.1.rpm

[root@localhost elk]# rpm -ivh *.rpm
warning: elasticsearch-5.0.1.rpm: Header V4 RSA/SHA512 Signature, key ID d88e42b4: NOKEY
Preparing...                ########################################### [100%]
   1:logstash               ########################################### [ 20%]
Could not find any executable java binary. Please install java in your PATH or set JAVA_HOME.
warning: %post(logstash-1:5.0.1-1.noarch) scriptlet failed, exit status 1
   2:kibana                 ########################################### [ 40%]
   3:jdk1.8.0_101           ########################################### [ 60%]
Unpacking JAR files...
	tools.jar...
	plugin.jar...
	javaws.jar...
	deploy.jar...
	rt.jar...
	jsse.jar...
	charsets.jar...
	localedata.jar...
   4:filebeat               ########################################### [ 80%]
Creating elasticsearch group... OK
Creating elasticsearch user... OK
   5:elasticsearch          ########################################### [100%]
### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using chkconfig
 sudo chkconfig --add elasticsearch
### You can start elasticsearch service by executing
 sudo service elasticsearch start

三、修改 90-nproc.conf nproc 值为 2048以上 三个结点都需要执行
[root@localhost elk]# vim /etc/security/limits.d/90-nproc.conf 
*          soft    nproc     2048

四、修改elasticsearch参数,搭建集群 三个结点都需要执行 

[root@localhost elk]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:ba:3a:b9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.5.253/24 brd 192.168.5.255 scope global eth1
    inet6 fe80::20c:29ff:feba:3ab9/64 scope link 
       valid_lft forever preferred_lft forever

[root@localhost elk]# vim /etc/elasticsearch/elasticsearch.yml 
cluster.name: elastic-cluster
node.name: node-3
network.host: 192.168.5.253
http.port: 9200
discovery.zen.ping.unicast.hosts: ["192.168.5.251", "192.168.5.252", "192.168.5.253"]
discovery.zen.minimum_master_nodes: 2
gateway.recover_after_nodes: 3



五、将当前主机修改好的elasticsearch参数 复制到另外的两台结点     

[root@localhost elk]# scp /etc/elasticsearch/elasticsearch.yml 192.168.5.251:/etc/elasticsearch/

[root@localhost elk]# scp /etc/elasticsearch/elasticsearch.yml 192.168.5.252:/etc/elasticsearch/


六、修改 192.168.5.251 192.168.5.252 node.name  network.host 值需要和当前主机一致

[root@localhost elk]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:07:50:12 brd ff:ff:ff:ff:ff:ff
    inet 192.168.5.251/24 brd 192.168.5.255 scope global eth1
    inet6 fe80::20c:29ff:fe07:5012/64 scope link 
       valid_lft forever preferred_lft forever

[root@localhost elk]# vim /etc/elasticsearch/elasticsearch.yml 
node.name: node-1
network.host: 192.168.5.251

[root@localhost elk]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN 
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
    link/ether 00:0c:29:94:a0:d9 brd ff:ff:ff:ff:ff:ff
    inet 192.168.5.252/24 brd 192.168.5.255 scope global eth1
    inet6 fe80::20c:29ff:fe94:a0d9/64 scope link 
       valid_lft forever preferred_lft forever

[root@localhost elk]# vim /etc/elasticsearch/elasticsearch.yml 
node.name: node-2
network.host: 192.168.5.252

七、验证集群是否搭建成功,如下图所示,表示成功

浏览器打开:http://192.168.5.253:9200/_cat/nodes

192.168.5.252 4 94 3 0.43 0.29 0.18 mdi * node-2
192.168.5.251 3 93 2 0.74 0.53 0.32 mdi - node-1
192.168.5.253 3 93 4 0.54 0.44 0.29 mdi - node-3



三、配置 elasticsearch plugin search-guard 

一、安装 search-guard 三个结点都需要执行
[root@localhost elk]# cd /usr/share/elasticsearch/
[root@localhost elasticsearch]# bin/elasticsearch-plugin install -b com.floragunn:search-guard-5:5.0.1-9

二、下载 search-guard-ssl 任意一台结点配置,本次以node-3结点
[root@localhost ~]# git clone https://github.com/floragunncom/search-guard-ssl.git
Initialized empty Git repository in /root/search-guard-ssl/.git/
^[[A^[[Aremote: Counting objects: 4870, done.
Receiving objects: 100% (4870/4870), 998.69 KiB | 63 KiB/s, done.
remote: Total 4870 (delta 0), reused 0 (delta 0), pack-reused 4870
Resolving deltas: 100% (2306/2306), done.

[root@localhost ~]# cd search-guard-ssl/example-pki-scripts/

/root/search-guard-ssl/example-pki-scripts
[root@localhost example-pki-scripts]# ls
clean.sh  etc  example.sh  gen_client_node_cert.sh  gen_node_cert.sh  gen_root_ca.sh

三、修改第三行的 0 为 3 生成 node-1 node-2 node-3 三张证书  任意一台结点配置,本次以node-3结点
[root@localhost example-pki-scripts]# cat example.sh 
#!/bin/bash
set -e
./clean.sh
./gen_root_ca.sh capass changeit
./gen_node_cert.sh 3 changeit capass && ./gen_node_cert.sh 1 changeit capass &&  ./gen_node_cert.sh 2 changeit capass
./gen_client_node_cert.sh spock changeit capass
./gen_client_node_cert.sh kirk changeit capass

[root@localhost example-pki-scripts]# ./example.sh 

[root@localhost example-pki-scripts]# ls
ca        etc                      gen_root_ca.sh    kirk.csr           kirk-signed.pem      node-1-signed.pem    node-2-signed.pem    node-3-signed.pem  spock.csr           spock-signed.pem
certs     example.sh               kirk.all.pem      kirk.key.pem       node-1.csr           node-2.csr           node-3.csr           spock.all.pem      spock.key.pem       truststore.jks
clean.sh  gen_client_node_cert.sh  kirk.crtfull.pem  kirk-keystore.jks  node-1-keystore.jks  node-2-keystore.jks  node-3-keystore.jks  spock.crtfull.pem  spock-keystore.jks
crl       gen_node_cert.sh         kirk.crt.pem      kirk-keystore.p12  node-1-keystore.p12  node-2-keystore.p12  node-3-keystore.p12  spock.crt.pem      spock-keystore.p12


三、复制 truststore.jks node-x-keystore.jks 证书到各结点,注意,node-x-keystore.jks x 表示对应主机的结点名
[root@localhost example-pki-scripts]# cp node-3-keystore.jks  truststore.jks /etc/elasticsearch/
[root@localhost example-pki-scripts]# scp node-2-keystore.jks  truststore.jks 192.168.5.252:/etc/elasticsearch/
[root@localhost example-pki-scripts]# scp node-1-keystore.jks  truststore.jks 192.168.5.251:/etc/elasticsearch/

四、配置elasticsearch 各结点基于tls加密通讯,并重启 注意 node-x-keystore.jks x 表示对应主机的结点名 三台都需要配置

[root@localhost example-pki-scripts]# vim /etc/elasticsearch/elasticsearch.yml 
searchguard.ssl.transport.keystore_filepath: node-3-keystore.jks
searchguard.ssl.transport.keystore_password: changeit
searchguard.ssl.transport.truststore_filepath: truststore.jks
searchguard.ssl.transport.truststore_password: changeit
searchguard.ssl.transport.enforce_hostname_verification: false
  
重启后,elasticsearch 之间的连接已经是加密的了,但是有如下报错,此问题是没有初始化 Search Guard 索引。

[root@localhost example-pki-scripts]# tail -f /var/log/elasticsearch/elastic-cluster.log 
[2017-01-08T12:21:29,918][ERROR][c.f.s.a.BackendRegistry  ] Not yet initialized (you may need to run sgadmin)



五、初始化  Search Guard 索引,配置帐号。 任意结点,本次配置以node-3结点。

复制kirk-keystore.jks证书
[root@localhost example-pki-scripts]# pwd
/root/search-guard-ssl/example-pki-scripts

[root@localhost example-pki-scripts]# cp kirk-keystore.jks /usr/share/elasticsearch/plugins/search-guard-5/sgconfig/

新增以下配置参数
[root@localhost example-pki-scripts]# vim /etc/elasticsearch/elasticsearch.yml 
searchguard.authcz.admin_dn:
  - "CN=kirk, OU=client, O=client, L=Test, C=DE"

重启服务
[root@localhost example-pki-scripts]# service elasticsearch restart
service elasticsearch restart elasticsearch restart
Stopping elasticsearch:                                    [  OK  ]
Starting elasticsearch:                                    [  OK  ]

初始化  Search Guard 索引

[root@localhost search-guard-5]# cd  /usr/share/elasticsearch/plugins/search-guard-5

[root@localhost search-guard-5]# tools/sgadmin.sh -ts /etc/elasticsearch/truststore.jks -tspass changeit -ks sgconfig/kirk-keystore.jks -kspass changeit -cd sgconfig/ -icl -nhnv -h 192.168.5.253
Search Guard Admin v5
Will connect to 192.168.5.253:9300 ... done
Contacting elasticsearch cluster 'elasticsearch' and wait for YELLOW clusterstate ...
Clustername: elastic-cluster
Clusterstate: GREEN
Number of nodes: 3
Number of data nodes: 3
searchguard index does not exists, attempt to create it ... done (auto expand replicas is on)
Populate config from /usr/share/elasticsearch/plugins/search-guard-5/sgconfig
Will update 'config' with sgconfig/sg_config.yml
   SUCC: Configuration for 'config' created or updated
Will update 'roles' with sgconfig/sg_roles.yml
   SUCC: Configuration for 'roles' created or updated
Will update 'rolesmapping' with sgconfig/sg_roles_mapping.yml
   SUCC: Configuration for 'rolesmapping' created or updated
Will update 'internalusers' with sgconfig/sg_internal_users.yml
   SUCC: Configuration for 'internalusers' created or updated
Will update 'actiongroups' with sgconfig/sg_action_groups.yml
   SUCC: Configuration for 'actiongroups' created or updated
Done with success

其中 sg_internal_users.yml 保存着默认的用户和密码
[root@localhost sgconfig]# head -n 5  sg_internal_users.yml 
# This is the internal user database
# The hash value is a bcrypt hash and can be generated with plugin/tools/hash.sh
admin:
  hash: $2a$12$VcCDgh2NDk07JGN0rjGbM.Ad41qVR/YFJcgHp0UGns5JDymv..TOG
  #password is: admin
[root@localhost sgconfig]# pwd
/usr/share/elasticsearch/plugins/search-guard-5/sgconfig

使用浏览器访问:http://192.168.5.253:9200 提示输入密码,输入默认用户: admin admin ,可登陆表示正常。


六、配置REST-API 基于https连接
[root@localhost elk]# vim /etc/elasticsearch/elasticsearch.yml,注意 node-x-keystore.jks x 表示对应主机的结点名 三台都需要配置

searchguard.ssl.http.enabled: true
searchguard.ssl.http.keystore_filepath: node-3-keystore.jks
searchguard.ssl.http.keystore_password: changeit
searchguard.ssl.http.truststore_filepath: truststore.jks
searchguard.ssl.http.truststore_password: changeit

使用浏览器访问:https://192.168.5.253:9200 提示输入密码,输入默认用户: admin admin ,可登陆表示正常
http://192.168.5.253:9200  无加密拒绝访问。

 

爱编程-编程爱好者经验分享平台

文章评论

  

版权所有 爱编程 © Copyright 2012. w2bc.com. All Rights Reserved.
闽ICP备12017094号-3