[Elasticsearch] AWS EC2에 Elasticsearch 설치

Elasticsearch

https://www.elastic.co/kr/downloads/elasticsearch

Download Elasticsearch

OSS 버전 다운로드 (elasticsearch-oss-7.4.0-linux-x86_64.tar.gz)

 

$ cd /service
$ wget https://download.oracle.com/otn/java/jdk/8u231-b11/5b13a193868b4bf28bcb45c792fce896/jdk-8u231-linux-x64.tar.gz
$ tar -xzvpf jdk-8u231-linux-x64.tar.gz

$ sudo vi /etc/profile
export JAVA_HOME=/services/jdk1.8.0_231/
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=$CLASSPATH:$JAVA_HOME/jre/lib/ext:$JAVA_HOME/lib/tools.jar

$ source /etc/profile

 

폴더 생성 및 사용자 변경

### 폴더 생성 ###
$ cd /
$ sudo mkdir services

### 사용자 변경 ###
$ sudo chown ec2-user:ec2-user services

 

설치 및 환경 설정

### 설치 ###
$ tar -xzvpf jdk-8u231-linux-x64.tar.gz

### 환경설정 ###
$ sudo vi /etc/profile
export JAVA_HOME=/services/jdk1.8.0_231/
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=$CLASSPATH:$JAVA_HOME/jre/lib/ext:$JAVA_HOME/lib/tools.jar

$ source /etc/profile

### Tip : profile 수정 후 명령어가 실행이 안될 경우 ###
$ PATH="/usr/local/bin:/usr/bin:/bin:/usr/bin/X11:/usr/games:"
$ export PATH

 

설치 및 환경 설정

$ mkdir elasticsearch

### AWS에서 9000 - 9300, 11200 포트 오픈 ###

### 환경설정 ###
$ vi /services/elasticsearch/conf/elasticsearch.yml
cluster.name: search-cluster
node.name: master
path.data: /services/elasticsearch/data
path.logs: /services/elasticsearch/logs
bootstrap.memory_lock: false
network.host: 0.0.0.0
http.port: 11200
discovery.seed_hosts: ["master:9300", "data1:9300", "data2:9300"]
cluster.initial_master_nodes: ["master"]
node.master: true
node.data: false

cluster.name: search-cluster
node.name: data1
path.data: /services/elasticsearch/data
path.logs: /services/elasticsearch/logs
bootstrap.memory_lock: false
network.host: 0.0.0.0
http.port: 11200
discovery.seed_hosts: ["master:9300", "data1:9300", "data2:9300"]
cluster.initial_master_nodes: ["master"]
node.master: false
node.data: true

cluster.name: search-cluster
node.name: data2
path.data: /services/elasticsearch/data
path.logs: /services/elasticsearch/logs
bootstrap.memory_lock: false
network.host: 0.0.0.0
http.port: 11200
discovery.seed_hosts: ["master:9300", "data1:9300", "data2:9300"]
cluster.initial_master_nodes: ["master"]
node.master: false
node.data: true

$ vi /services/elasticsearch/conf/jvm.options
1g -> 4g로 변경.
-Xms4g
-Xmx4g
-XX:+UseConcMarkSweepGC (삭제)

### 실행 ###
$ cd /services/elasticsearch-7.4.0/bin
$ ./elasticsearch

 

Tip

Tip : 아래 오류 발생 시
future versions of Elasticsearch will require Java 11; your Java version from [/services/jdk1.8.0_231/jre] does not meet this requirement
[ec2-user@ip-10-0-32-207 bin]$ ps ERROR: [3] bootstrap checks failed
[1]: max file descriptors [4096] for elasticsearch process is too low, increase to at least [65535]
[2]: memory locking requested for elasticsearch process but memory is not locked
[3]: max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144]

$ ulimit -a
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 62462
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 10240
cpu time (seconds, -t) unlimited
max user processes (-u) 1024
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited

$ sudo su -
$ ulimit -n 65536
$ echo 262144 > /proc/sys/vm/max_map_count
$ vi /etc/security/limits.conf
ec2-user hard nofile 65536
ec2-user soft nofile 65536
ec2-user hard nproc 65536
ec2-user soft nproc 65536
root hard nofile 65536
root soft nofile 65536
root hard nproc 65536
root soft nproc 65536

 

Cerebro 설치

사이트 : https://github.com/lmenezes/cerebro

 

Tip : .special는 관리자용 인덱스

$ vi conf/application.conf
hosts = [
  {
    host = "http://localhost:9200"
    name = "Localhost cluster"
  }
]

 

플러그인 설치

### icu 설치 ###
$ cd /services/elasticsearch-7.4.0
$ bin/elasticsearch-plugin install analysis-icu

# Tip : 삭제시 bin/elasticsearch-plugin remove analysis-icu

### nori 설치 ###
$ cd /services/elasticsearch-7.4.0
$ bin/elasticsearch-plugin install analysis-nori

# 동의어, 명사, 불용어 사전 설치
# 파일 : search-dic-sync-master.zip

# 파일 수정 :
# es-conn.js : hostName 부분 수정 (포트를 11200사용)
# http://localhost:9200 --> http://localhost:11200

# nori-dic.js :
# const dicPath = '/services/elasticsearch-7.4.0/config/userdic_ko.txt';


### FTP를 통해 src 부분 파일 업로드 ###
# node 설치
$ sudo su -
$ curl -sL https://rpm.nodesource.com/setup_10.x | bash -
$ yum install nodejs -y

### 실행 ###
$ node index.js