es文件夹

⼀次ES从5.2版本升级到7.13测试⽣产完全实操记录

（Elasticarchupgrade）

⼀次ES从5.2版本升级到7.13测试⽣产完全实操记录

⽬前正在使⽤的elasticarch版本是5.2，使⽤JavaTransportClient+Springbooot构建的项⽬,

升级es引擎到最新版本7.13，需要代码层⾯的改动。由于项⽬Springboot版本是1.4.2,即选⽤JavaRESTClient[7.13]

因为是跨⼤版本升级，给出升级⽅案

Ifyouarerunningaversionpriorto6.0,upgradeto6.8andreindexyouroldindicesorbringupanew7.13.4clusterandreindex

fromremote.

数据迁移⽅式：新建集群重建索引(bringupanew7.13.4clusterand)

测试环境ES实践

1.部署ES集群

测试环境2台节点的集群，启动2台虚拟机，

ip分别位192.169.10.167、192.169.10.168

，并下载相应版本的kibana

解压Es到/usr/local/es,

tar-zxvfelasticarch-7.13.0-linux-x86_-C/usr/local/es

但是由于安全的考虑，elasticarch不允许使⽤root⽤户来启动，所以需要创建⼀个新的⽤户，并为这个账户赋予相应的权限来启动

elasticarch集群。

uraddes

passwdes

#创建⽂件夹⽤于存放数据和⽇志

mkdir-p/var/data/elasticarch

mkdir-p/var/log/elasticarch

#改变⽂件的拥有者为es

chown-Res/usr/local/es/

chown-Res/var/log/elasticarch

chown-Res/var/data/elasticarch

配置

JDK配置

新版本⾃带适应的JDK，这⾥直接采⽤修改启动⽂件./bin/elasticarch

#vim./bin/elasticarch添加如下配置

exportJAVA_HOME=/usr/local/es/elasticarch-7.13.0/jdk

exportPATH=$JAVA_HOME/bin:$PATH

ES配置⽂件主要位于$ES_HOME/config⽬录下

最新版本Elasticarch主关于梦想的名人素材 要有三个配置⽂件：

的配置，more

sESJVM配置，more

tiesES⽇志配置，more

复制5.2版本的配置

配置如下

#集群名

:microants-es-004

#节点名

:node-01

#数据⽬录

:/var/data/elasticarch

#⽇志⽬录

:/var/log/elasticarch

#⽹络配置

:192.168.10.167

:9200

#迁移地址⽩名单

ist:["192.168.10.154:9200","192.168.10.155:9200"]

_hosts:["192.168.10.167","192.168.10.168"]

l_master_nodes:["node-01","node-02"]

s

-Xms2g

-Xmx2g

其他可能需要的配置

1.修改⽂件描述符数⽬2.最⼤映射数量MMP

#编辑/etc/curity/，追加以下内容；

*softnofile65536

*hardnofile65536

#/etc/⽂件最后添加⼀⾏

_map_count=262144

#执⾏/sbin/sysctl-p⽴即⽣效

启动

#切换es启动否则会失败

sues

./bin/elasticarch-d

验证启动

{

"name":"node-01",

"cluster_name":"microants-es-004",

"cluster_uuid":"_na_",

"version":{

"number":"7.13.0",

"build_flavor":"default",

"build_type":"tar",

"build_hash":"5ca8591c6fcdb1260ce95b08a8e023559635c6f3",

"build_date":"2021-05-19T22:22:26.081971330Z",

"build_snapshot":fal当幸福来敲门电影 ,

"lucene_version":"8.8.2",

"minimum_wire_compatibility_version":"6.8.0",

"minimum_index_compatibility_version":"6.0.0-beta1"

},

"tagline":"YouKnow,forSearch"

}

192.168.10.1681945150.430.190.11cdfhilmrstw-node-02

192.168.10.167234900.240.110.07cdfhilmrstw*node-01

部署kibana

下载找到对应版本

解压

tar-zxvfkibana-7.13.0-linux-x86_-C/opt/soft

配置vim./config/

:5601

:"192.168.10.167"

:["192.168.10.167:9200"]

:"zh-CN"

启动(最好添加kibana⽤户)

nohup./bin/kibana&

集群安全设置

1.在每个节点⽂件下加⼊

d:cation_mode:_authentication:required

::elastic-certificates.p12

2.启动

./bin/elasticarch

3.⽣成⽤户和密码采⽤⾛动⽣成

./bin/elasticarch-tup-passwordsauto

结果会打印再控制台保存如：

hangedpasswordforurapm_system

PASSWORDapm_system=IywjBT4YDU86NDw7ox

Changedpasswordforurkibana_system

PASSWO此外的近义词 RDkibana_system=SQmORp23LcZyPZU48l

Changedpasswordforurkibana

PASSWORDkibana=SQmORp2Nb3LcPZU48l

Changedpasswordforurlogstash_system

PASSWORDlogstash_system=cEtJPTbzktxc7aPuQx

Changedpasswordforurbeats_system

PASSWORDbeats_sys植树节图片 tem=5wDk7jQNu4iP5J7eLg

Changedpasswordforurremote_monitoring_ur

PASSWORDremote_monitoring_ur=p1HkCWzYVtt8SPGEtN

Changedpasswordforurelastic

PASSWORDelastic=PRmId87ogKerJboyLw

4.任何节点⽣成ca证书

./bin/elasticarch-certutilca

会⽣成⼀个elastic-stack-ca.p12⽂件

./bin/elasticarch-certutilcert--caelastic-stack-ca.p12

⽣成elastic-certificates.p12⽂件

5.复制⽂件elastic-certificates.p12到所有节点的conf⽬录下

6.如果证书有密码执⾏./bin/_password

7.重启es,验证查看节点是否可以通信

2.数据迁移

插件适配

查看⽼版本使⽤的插件

192.168.10.154:9200/_金牛男水瓶女 cat/plugins

返回：

node-0001analysis-icu5.2.0

node-0001analysis-ik5.2.0

node-0001analysis-kuromoji5.2.0

node-0001analysis-pinyin5.2.1

node-0002analysis-icu5.2.0

node-0002analysis-ik5.2.0

node-0002analysis-pinyin5.2.1

先官⽹查看[插件⽂档](/guide/en/elasticarch/plugins/7.13/)

#安装插件格式

sudobin/elasticarch-plugininstall[plugin_name]

安装icu

[root@localhostelasticarch-7.13.0]#sudobin/elasticarch-plugininstallanalysis-icu

->Installinganalysis-icu

->Downloadinganalysis-icufromelastic

[=================================================]100%

->Installedanalysis-icu

->PlearestartElasticarchtoactivateanypluginsinstalled

安装ik、pinyin

./bin/elasticarch-plugininstall/medcl/elasticarch-analysis-ik/releas/download/v6.3.0/

#⽹速慢下载了⼏次失败

#github⼿动下载download/medcl/elasticarch-analysis-ik/releas

#createpluginfoldercdyour-es-root/plugins/&&mkdirik

yuminstall-yunzipzip

#unzipplugintofolderyour-es-root/plugins/ik

#解压⼆者到plugins下

-d/usr/local/es/elasticarch-7.13.0/plugins/ik/

-d/usr/local/es/elasticarch-7.13.0/plugins/pinyin/

#重启es

ps-ef|grepelastic

kill94093

sues

./bin/elasticarch-d

node-01analysis-icu7.13.0

node-01analysis-ik7.13.0

node-01analysis-pinyin7.13.0

node-02analysis-ik7.13.0

node-02analysis-pinyin7.13.0

开始数据迁移

Tomanuallyreindexyouroldindicesinplace:

tiblemappings.

refresh_intervalto-1andthenumber_of_replicasto0forefficientreindexing.

uascripttoperformanynecessa愉快英语 ry

modificationstothedocumentdataandmetadataduringreindexing.

herefresh_intervalandnumber_of_replicastothevaluesudintheoldindex.

rtheindexstatustochangetogreen.

gleupdatealiasrequest:

Deletetheoldindex.

Addanaliaswiththeoldindexnametothenewindex.

Addanyaliasthatexistedontheoldindextothenewindex.

⽼数据5.2数据需要迁移的索引

greenopenller_subject_index_v1efIwSm31QAiNwEdus0BVNg516002233mb1.5mb

greenopenarch_keyword_index_v1BTalZTHlRrCTaxgFrcH-jA51341502.1mb1mb

greenopenplatform_coupon_index_v1lD15Hyl6TtSWV2GP79aX8Q51570192.4kb96.2kb

.

.

.

.

greenopensyscate_index_v1dEWSZp1sSq-lA9TINGyt0A514660159mb4.5mb

greenopencarsgSqMCkN-SSa3EhH6Vm1nCw518047.3kb23.6kb

greenopenproduct_keyword_hint_v1ubQgLk4FRVaAciEHP7imQw112800324.5kb162.2kb

greenopensubject_index_v1u-M6le0JSxmuxLxGLkNCUQ5114457917.1mb8.5mb

此处以ller_subject_index_v1索引例⼦演⽰

tiblemappings.

es.5.2的ting、mapping

点击查看代码

-es7.13tings、mapping

点击查看代码

最后别忘了别名映射

##迁移数据ller_subject_index_v1再kibana⼯具执⾏

##Post_reindex

{

"source":{

"remote":{

"host":"192.168.10.154:9200",

"socket_timeout":"1m",

"connect_timeout":"10s"

},

"index":"ller_subject_index_v1"

},

"dest":{

"index":"ller_subject_index_v1"

}

}

put/ller_subject_index_v1/_ttings

{

"index":{

"refresh_interval":"1s"

}

}

继续执⾏其他索引

⼩结及重要的点

1.出现{"statusCode":502,"error":"BadGateway","message":"Clientrequesttimeout"}，加上参数wait_for_completion=fal

2.⼀个索引有多个doc时需要拆分加type参数

应⽤层代码修改

ThisctiondescribeshowtomigrateexistingcodefromtheTransportClienttotheJavaHighLevelRESTClientreleadwith

theversion5.6.0ofElasticarch.

如何迁移

AdaptingexistingcodetoutheRestHighLevelClientinsteadoftheTransportClientrequiresthefollowingsteps:

dependencies

clientinitialization

applicationcode

调整现有代码以使⽤RestHighLevelClient代替TransportClient需要以下步骤：

-更新依赖

-更新客户端初始化

-更新应⽤代码

SincetheJavaHighLevelRESTClientdoesnotsupportrequestbuilders,applicationsthatuthemmustbechangedtou

requestsconstructorsinstead

**代码层⾯的改**变就是TransportClient的请求构建器不⽀持使⽤如：

IndexRequestBuilderindexRequestBuilder=eIndex();

DeleteRequestBuilderdeleteRequestBuilder=eDelete();

SearchRequestBuilderarchRequestBuilder=eSearch();

⽤JavaHighLevelREST构造器替换如：

IndexRequestrequest=newIndexRequest("index").id("id");

("{"field":"value"}",);