频道栏目
首页 > 系统 > Linux > 正文
linux下ElasticSearch.6.2.2集群安装与head、Kibana、X-Pack..插件的配置安装
2018-02-26 15:21:25 ???? ?? 来源:九师兄??
收藏?? 我要投稿

简介:主要在三台linux服务器上,集群安装ElasticSearch.6.2.1,及其es的插件,各种管理软件

1.集群安装es

1.1 环境

域名          ip
biluos.com      192.168.10.173
biluos1.com     192.168.10.174
biluos2.com     192.168.10.175

1.2 三台机器都安装jdk最新版本

[root@biluos es]# java -version
openjdk version "1.8.0_161"
OpenJDK Runtime Environment (build 1.8.0_161-b14)
OpenJDK 64-Bit Server VM (build 25.161-b14, mixed mode)
[root@biluos es]# 

[root@biluos1 ~]# java -version
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
[root@biluos1 ~]# 

[root@biluos2 ~]# java -version
java version "1.8.0_121"
Java(TM) SE Runtime Environment (build 1.8.0_121-b13)
Java HotSpot(TM) 64-Bit Server VM (build 25.121-b13, mixed mode)
[root@biluos2 ~]# 

1.3 三台机器都统一用户为es

[root@biluos ~]# useradd es
You have new mail in /var/spool/mail/root
[root@biluos ~]# passwd es                                 # 密码为 qwaszx
Changing password for user es.
New password: 
BAD PASSWORD: it is based on a dictionary word
BAD PASSWORD: is too simple
Retype new password: 
passwd: all authentication tokens updated successfully.
[root@biluos ~]#  mkdir /home/es
mkdir: cannot create directory `/home/es': File exists
[root@biluos ~]#  ll /home/                                    # 注意是不是es用户和用户组 
total 4
drwx------ 3 es es 4096 Feb 25 03:51 es
[root@biluos ~]# 

[root@biluos1 ~]# useradd es
[root@biluos1 ~]# passwd es                                    # 密码为 qwaszx
[root@biluos1 ~]#  mkdir /home/es
[root@biluos1 ~]#  ll /home/                               # 注意是不是es用户和用户组 


[root@biluos2 ~]# useradd es
[root@biluos2 ~]# passwd es                                    # 密码为 qwaszx
[root@biluos2 ~]#  mkdir /home/es
[root@biluos2 ~]#  ll /home/                               # 注意是不是es用户和用户组 

1.4 三台机器都建立/home/es/elasticsearch目录,用来存放es软件包和数据存储,使用es用户

[root@biluos1 ~]# su es
[es@biluos1 root]$ cd /home/es/
[es@biluos1 ~]$ mkdir elasticsearch
[es@biluos1 ~]$ ll
drwxrwxr-x 2 es es 4096 Feb 25 03:59 elasticsearch
[es@biluos1 ~]$ 

其余两台此处省略

1.5 三台机器都解压安装包到/home/es/elasticsearch

下载:https://www.elastic.co/cn/downloads/elasticsearch
包:elasticsearch-6.2.2.tar.gz
解压:tar -zxvf /home/es/elasticsearch/elasticsearch-6.2.2.tar.gz -C /home/es/elasticsearch

[root@biluos elasticsearch-6.2.0]# pwd
/home/es/elasticsearch/elasticsearch-6.2.0

[es@biluos1 elasticsearch-6.2.0]$ pwd
/home/es/elasticsearch/elasticsearch-6.2.0

[es@biluos2 elasticsearch-6.2.0]$ pwd
/home/es/elasticsearch/elasticsearch-6.2.0

1.6 三台机器都修改es软件包的权限为es用户

因为我是root解压的

[es@biluos2 elasticsearch]$ ll
total 4
drwxr-xr-x 9 root root 4096 Feb 25 04:04 elasticsearch-6.2.0
[es@biluos2 elasticsearch]$ 

使用root用户修改权限

[es@biluos2 ~]$ su root
Password: 
[root@biluos2 es]# chown -R es:es /home/es/

其余两台此处省略

1.7 三台机器都创建data数据目录和日志目录,使用es用户

[root@biluos2 es]# su es
[es@biluos2 ~]$ mkdir -p /home/es/elasticsearch/elasticsearch-6.2.0_data/data/
[es@biluos2 ~]$ mkdir -p /home/es/elasticsearch/elasticsearch-6.2.0_data/logs/

其余两台此处省略

1.8 三台机器都修改配置

191.168.10.173机器配置
vim /home/es/elasticsearch/elasticsearch-6.2.2/config/elasticsearch.yml

cluster.name: lcc-application                                                           # 必须一样
node.name: node-191-168-10-173                                                          # 必须不一样
path.data: /home/es/elasticsearch/elasticsearch-6.2.0_data/data/
path.logs: /home/es/elasticsearch/elasticsearch-6.2.0_data/logs/

bootstrap.memory_lock: false
bootstrap.system_call_filter: false


network.host: 0.0.0.0  
http.port: 9200

# 集群发现
#集群节点ip或者主机
discovery.zen.ping.unicast.hosts: ["192.168.10.173", "192.168.10.174","192.168.10.175"]     
#设置这个参数来保证集群中的节点可以知道其它N个有master资格的节点。默认为1,对于大的集群来说,可以设置大一点的值(2-4)                                                                         
discovery.zen.minimum_master_nodes: 3  

# 下面两行配置为haad插件配置,三台服务器一致。      
http.cors.enabled: true                                                                                                                                                                                                   
http.cors.allow-origin: "*"      

191.168.10.174机器配置
vim /home/es/elasticsearch/elasticsearch-6.2.2/config/elasticsearch.yml

cluster.name: lcc-application                                                           # 必须一样
node.name: node-191-168-10-174                                                          # 必须不一样
path.data: /home/es/elasticsearch/elasticsearch-6.2.0_data/data/
path.logs: /home/es/elasticsearch/elasticsearch-6.2.0_data/logs/

bootstrap.memory_lock: false
bootstrap.system_call_filter: false


network.host: 0.0.0.0  
http.port: 9200

# 集群发现
#集群节点ip或者主机
discovery.zen.ping.unicast.hosts: ["192.168.10.173", "192.168.10.174","192.168.10.175"]     
#设置这个参数来保证集群中的节点可以知道其它N个有master资格的节点。默认为1,对于大的集群来说,可以设置大一点的值(2-4)                                                                         
discovery.zen.minimum_master_nodes: 3  

# 下面两行配置为haad插件配置,三台服务器一致。      
http.cors.enabled: true                                                                                                                                                                                                   
http.cors.allow-origin: "*"      

191.168.10.175机器配置
vim /home/es/elasticsearch/elasticsearch-6.2.2/config/elasticsearch.yml

cluster.name: lcc-application                                                           # 必须一样
node.name: node-191-168-10-175                                                          # 必须不一样
path.data: /home/es/elasticsearch/elasticsearch-6.2.0_data/data/
path.logs: /home/es/elasticsearch/elasticsearch-6.2.0_data/logs/

bootstrap.memory_lock: false
bootstrap.system_call_filter: false


network.host: 0.0.0.0  
http.port: 9200

# 集群发现
#集群节点ip或者主机
discovery.zen.ping.unicast.hosts: ["192.168.10.173", "192.168.10.174","192.168.10.175"]     
#设置这个参数来保证集群中的节点可以知道其它N个有master资格的节点。默认为1,对于大的集群来说,可以设置大一点的值(2-4)                                                                         
discovery.zen.minimum_master_nodes: 3  

# 下面两行配置为haad插件配置,三台服务器一致。      
http.cors.enabled: true                                                                                                                                                                                                   
http.cors.allow-origin: "*"      

1.9 三台机器都修改 Linux下/etc/security/limits.conf文件设置

更改linux的最大文件描述限制要求
添加或修改如下:
* soft nofile 262144
* hard nofile 262144 更改linux的锁内存限制要求
添加或修改如下:
用户(elasticsearch) soft memlock unlimited
用户(elasticsearch) hard memlock unlimited

最后配置如下

vim /etc/security/limits.conf

* soft nofile 262144                                                                                                                                              
* hard nofile 262144                                                                                                                                              
es soft memlock unlimited                                                                                                                                         
es hard memlock unlimited 

这里es是es用户

1.10 三台机器都修改配置 Linux下/etc/security/limits.d/90-nproc.conf文件设置

更改linux的的最大线程数,添加或修改如下:
* soft nproc unlimited

vim /etc/security/limits.d/90-nproc.conf
*          soft    nproc     unlimited
root       soft    nproc     unlimited

1.11 三台机器都修改配置 Linux下/etc/sysctl.conf文件设置

更改linux一个进行能拥有的最多的内存区域要求,添加或修改如下:
vm.max_map_count = 262144 更改linux禁用swapping,添加或修改如下:
vm.swappiness = 1
vim /etc/sysctl.conf

vm.max_map_count = 262144
vm.swappiness = 1

1.12 三台机器都启动

[es@biluos elasticsearch-6.2.2]$  bin/elasticsearch

[es@biluos1 elasticsearch-6.2.2]$  bin/elasticsearch

[es@biluos2 elasticsearch-6.2.2]$  bin/elasticsearch

刚刚报错,

[2018-02-25T13:49:33,967][WARN ][o.e.d.z.ZenDiscovery     ] [node-191-168-10-173] not enough master nodes discovered during pinging (found [[Candidate{node={node-191-168-10-173}{Wquj4gYuTteo7M5uXzSBiQ}{19L-SAvFRvqvYyjsKtBP_g}{192.168.1.194}{192.168.1.194:9300}, clusterStateVersion=-1}]], but needed [3]), pinging again

我以为是因为没配置ssh,最后发现上面ip写错了,然后验证不配置ssh也没事。

这上面第一个启动的打印日志最多,其他的相同,日志比较少

现在有个问题,我访问
https://192.168.10.173:9200/
https://192.168.10.174:9200/
https://192.168.10.175:9200/
都能打印出版本号,证明都能正确运行,我尝试关闭掉一个174,发现https://192.168.10.174:9200/打不开了,然后,我觉得应该有一个是主master,但是谁是主呢?不好看到,下面我们安装图形化界面head试试

2 安装elasticsearch-head

三台机器只需要一台安装head就可以了

2.1 采用nodejs安装

6.X中,elasticsearch-head

不能放在elasticsearch的 plugins、modules 目录下 不能使用 elasticsearch-plugin install

直接启动elasticsearch即可

2.2 安装 elasticsearch-head

1.修改 elasticsearch/config/elasticsearch.yml
2.添加

http.cors.enabled: true
http.cors.allow-origin: "*"

3.下载 elasticsearch-head 或者 git clone 到随便一个文件夹

https://github.com/mobz/elasticsearch-head

我是直接下载到

[es@biluos elasticsearch-head-master]$ pwd
/home/es/elasticsearch-head-master

目录的

4.安装nodejs
https://blog.csdn.net/qq_21383435/article/details/79367366

5.cd /home/es/elasticsearch-head-master
6.npm install -g grunt-cli
grunt是一个很方便的构建工具,可以进行打包压缩、测试、执行等等的工作

[es@biluos elasticsearch-head-master]$ npm install -g grunt-cli 
npm WARN checkPermissions Missing write access to /opt/moudles/node-v8.9.4-linux-x64/lib/node_modules
npm ERR! path /opt/moudles/node-v8.9.4-linux-x64/lib/node_modules
npm ERR! code EACCES
npm ERR! errno -13
npm ERR! syscall access
npm ERR! Error: EACCES: permission denied, access '/opt/moudles/node-v8.9.4-linux-x64/lib/node_modules'
npm ERR!  { Error: EACCES: permission denied, access '/opt/moudles/node-v8.9.4-linux-x64/lib/node_modules'
npm ERR!   stack: 'Error: EACCES: permission denied, access \'/opt/moudles/node-v8.9.4-linux-x64/lib/node_modules\'',
npm ERR!   errno: -13,
npm ERR!   code: 'EACCES',
npm ERR!   syscall: 'access',
npm ERR!   path: '/opt/moudles/node-v8.9.4-linux-x64/lib/node_modules' }
npm ERR! 
npm ERR! Please try running this command again as root/Administrator.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/es/.npm/_logs/2018-02-25T02_49_37_372Z-debug.log

一看就知道是权限问题,因为我的nodejs是root安装的,这里我是es用户

[es@biluos elasticsearch-head-master]$ su root
Password: 
[root@biluos elasticsearch-head-master]#  npm install -g grunt-cli 
/opt/moudles/node-v8.9.4-linux-x64/bin/grunt -> /opt/moudles/node-v8.9.4-linux-x64/lib/node_modules/grunt-cli/bin/grunt
+ grunt-cli@1.2.0
added 16 packages in 21.663s

7.npm install

[root@biluos elasticsearch-head-master]# npm install 
npm WARN deprecated http2@3.3.7: Use the built-in module in node 9.0.0 or newer, instead
npm WARN deprecated coffee-script@1.10.0: CoffeeScript on NPM has moved to "coffeescript" (no hyphen)

> phantomjs-prebuilt@2.1.16 install /home/es/elasticsearch-head-master/node_modules/phantomjs-prebuilt
> node install.js

module.js:540
    throw err;
    ^

Error: Cannot find module '/home/es/elasticsearch-head-master/node_modules/phantomjs-prebuilt/install.js'
    at Function.Module._resolveFilename (module.js:538:15)
    at Function.Module._load (module.js:468:25)
    at Function.Module.runMain (module.js:684:10)
    at startup (bootstrap_node.js:187:16)
    at bootstrap_node.js:608:3
npm WARN elasticsearch-head@0.0.0 license should be a valid SPDX license expression
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.1.3 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! phantomjs-prebuilt@2.1.16 install: `node install.js`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the phantomjs-prebuilt@2.1.16 install script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2018-02-25T03_01_11_887Z-debug.log

这一点是因为我的elasticsearch是es用户,而elasticsearch-head-master文件是root用户

[root@biluos es]# ll
total 12
drwxr-xr-x 4 es   es   4096 Feb  9 14:55 elasticsearch

[root@biluos elasticsearch-head-master]# ll /home/es/elasticsearch-head-master/node_modules
total 1028
drwxr-xr-x  2 root root 4096 Feb 25 11:01 abbrev
drwxr-xr-x  2 root root 4096 Feb 25 11:01 accepts
drwxr-xr-x  3 root root 4096 Feb 25 11:01 after
drwxr-xr-x  2 root root 4096 Feb 25 11:01 ansi-regex
drwxr-xr-x  2 root root 4096 Feb 25 11:01 ansi-styles

这里我统一用户为es

[root@biluos es]# chown -R es:es /home/es/elasticsearch-head-master/

然后使用es用户,安装成功

[root@biluos elasticsearch-head-master]# su es 
[es@biluos elasticsearch-head-master]$ npm install 

> phantomjs-prebuilt@2.1.16 install /home/es/elasticsearch-head-master/node_modules/phantomjs-prebuilt
> node install.js

PhantomJS not found on PATH
Downloading https://github.com/Medium/phantomjs/releases/download/v2.1.1/phantomjs-2.1.1-linux-x86_64.tar.bz2
Saving to /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2
Receiving...
                                                                                                                                                                                               [=============---------------------------] 33                                                                                                                                                   [===================================-----] 87%
Received 22866K total.
Extracting tar contents (via spawned process)
Removing /home/es/elasticsearch-head-master/node_modules/phantomjs-prebuilt/lib/phantom
Copying extracted folder /tmp/phantomjs/phantomjs-2.1.1-linux-x86_64.tar.bz2-extract-1519527928056/phantomjs-2.1.1-linux-x86_64 -> /home/es/elasticsearch-head-master/node_modules/phantomjs-prebuilt/lib/phantom
Writing location.js file
Done. Phantomjs binary available at /home/es/elasticsearch-head-master/node_modules/phantomjs-prebuilt/lib/phantom/bin/phantomjs
npm notice created a lockfile as package-lock.json. You should commit this file.
npm WARN elasticsearch-head@0.0.0 license should be a valid SPDX license expression
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@1.1.3 (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.1.3: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

added 70 packages in 84.93s
[es@biluos elasticsearch-head-master]$ 

8.grunt server

[es@biluos elasticsearch-head-master]$ grunt server
(node:6612) ExperimentalWarning: The http2 module is an experimental API.
Running "connect:server" (connect) task
Waiting forever...
Started connect web server on https://localhost:9100

9.https://192.168.10.173:9100/ 成功

10.启动我们的elasticsearch

[es@biluos elasticsearch-6.2.2]$  bin/elasticsearch

[es@biluos1 elasticsearch-6.2.2]$  bin/elasticsearch

[es@biluos2 elasticsearch-6.2.2]$  bin/elasticsearch

11 .https://192.168.10.173:9100/ 查看

这里写图片描述

2.3 测试ctrl+c掉一个

我直接ctrl+c一个174会是什么结果呢?,发现173和175都报错

[2018-02-25T14:54:08,997][WARN ][o.e.d.z.ZenDiscovery     ] [node-191-168-10-173] not enough master nodes discovered during pinging (found [[Candidate{node={node-191-168-10-175}{dCejCwUFTnqOrYyrmfD4AQ}{y2SixREcR5STrzmlswaxcA}{192.168.10.175}{192.168.10.175:9300}, clusterStateVersion=8}, Candidate{node={node-191-168-10-173}{Wquj4gYuTteo7M5uXzSBiQ}{xzN6saH5Rp2CPQH5iunHZg}{192.168.1.194}{192.168.1.194:9300}, clusterStateVersion=8}]], but needed [3]), pinging again

而且https://192.168.10.173:9100/无法连接,连接也是红色的

感觉:应该和这个参数有关

## 设置这个参数来保证集群中的节点可以知道其它N个有master资格的节点。默认为1,对于大的集群来说,可以设置大一点的值(2-4)                                                                         
discovery.zen.minimum_master_nodes: 2

这个的含义是不是 我假设集群有3个节点 ,如果死掉了一个我的集群还能用?如果在死掉一个,节点还剩下一个了,小于2 是不是就不能用了?

discovery.zen.minimum_master_nodes: 5
是不是我集群可用的最少节点数限制


参数是最小可用的候选master节点数
一般设置为2,集群里留3个节点作为候选master

3 安装kibana(不需要安装x-pack)

集群的时候,我们只需要在一台机器上安装就可以了,
我开始安装kibana-6.2.1,但是报错如下,后来安装同样版本的kibana-6.2.2,就不报错了

log   [07:15:33.047] [warning] You're running Kibana 6.2.1 with some different versions of Elasticsearch. Update Kibana or Elasticsearch to the same version to prevent compatibility issues: v6.2.2 @ 192.168.10.174:9200 (192.168.10.174), v6.2.2 @ 192.168.1.194:9200 (192.168.1.194), v6.2.2 @ 192.168.10.175:9200 (192.168.10.175)                                                                                                                                                                                            

1.下载https://www.elastic.co/cn/downloads/kibana 版本要和es版本相同
2.直接解压到 /home/es/kibana/kibana-6.2.2-linux-x86_64
3.配置

# vim config/kibana.yml
elasticsearch.url: "https://192.168.10.173:9200"      # kibana监控哪台es机器
server.host: "192.168.10.173"                # kibana运行在哪台机器

4.运行

[root@biluos kibana-6.2.2-linux-x86_64]# bin/kibana            
  log   [07:54:52.807] [info][status][plugin:kibana@6.2.2] Status changed from uninitialized to green - Ready
  log   [07:54:52.882] [info][status][plugin:elasticsearch@6.2.2] Status changed from uninitialized to yellow - Waiting for Elasticsearch
  log   [07:54:53.091] [info][status][plugin:timelion@6.2.2] Status changed from uninitialized to green - Ready
  log   [07:54:53.107] [info][status][plugin:console@6.2.2] Status changed from uninitialized to green - Ready
  log   [07:54:53.116] [info][status][plugin:metrics@6.2.2] Status changed from uninitialized to green - Ready
  log   [07:54:53.152] [info][listening] Server running at https://192.168.10.173:5601
  log   [07:54:53.197] [info][status][plugin:elasticsearch@6.2.2] Status changed from yellow to green - Ready

5.查看界面https://192.168.10.173:5601 可以直接访问

点击复制链接 与好友分享!回本站首页
上一篇:linux下ElasticSearch.6.2.1与head、Kibana、X-Pack、SQL、IK、PINYIN插件的配置安装教程
下一篇:浅谈MES与SAP PP模块的集成应用
相关文章
图文推荐
点击排行

关于我们 | 联系我们 | 广告服务 | 投资合作 | 版权申明 | 在线帮助 | 网站地图 | 作品发布 | Vip技术培训 | 举报中心

版权所有: 红黑联盟--致力于做实用的IT技术学习网站