Realtime Sync MongoDB to Elasticsearch Using Monstache

Halo, kalau kamu merasa tulisan saya ngebantu kamu, kamu bisa ucapkan terima kasih lewat saweria .

If you feel this website help you, you can donate at saweria .

What I Used in This Tutorial

1. Monstache v6.7.12 docker version

2. Docker version 24.0.6, build ed223bc

3. Ubuntu 22.04.3 LTS

How To

1. Create docker-compose.yml

version: "3.8"

services:
  monstache:
    image: rwynn/monstache:latest
    hostname: monstache
    container_name: monstache-ctr
    restart: always
    volumes:
      - ./config.toml:/app/config.toml
      - ./logs-prod/:/app/logs-prod
    command: "-f /app/config.toml"
    network_mode: host

2. Create config.toml to put your own config

mongo-url = "mongodb://username:password@192.168.1.3:27017/your_db_name?authSource=admin"
elasticsearch-urls = ["http://192.168.1.2:9200]
elasticsearch-user = "elasticAccount"
elasticsearch-password = "notRealPassword"
elasticsearch-max-conns = 2
#elasticsearch-max-docs = 1
elasticsearch-max-seconds = 1
elasticsearch-retry = true
direct-read-namespaces = ["your_db_name.your_collection1,your_db_name.your_collection2]
#no need to change namespaces to make it easier to debug/find
change-stream-namespaces = ["your_db_name.your_collection1,your_db_name.your_collection2]
exit-after-direct-reads = false
verbose = false
dropped-collections = false
dropped-databases = false
delete-strategy = 1
replay = false
resume = false
resume-strategy = 0
index-as-update = true
enable-oplog = false
resume-name = "resume-your_db_name"
stats = true
index-stats = false
[logs]
info = "/app/logs-prod/info.log"
warn = "/app/logs-prod/warn.log"
error = "app/logs-prod/error.log"
trace = "app/logs-prod/trace.log" 

3. Run with docker-compose up -d

4. Please wait until MongoDB sync with elastic

5. To check if data sync, you can use curl or you can check your elastic dashboard

curl -u "elasticAccount:notRealPassword" -X GET "192.168.1.2:9200/_search?pretty" -H 'Content-Type: application/json' -d'
{
  "query": { 
    "bool": { 
      "must": [
        { "match": { "title": "kucing" }}
      ]
    }
  }
}
  '
#your should change title with what you search

Reference:

1. Monstache Docs

2. Alibaba Cloud - Use Monstache to synchronize data