Remote Elasticsearch Configuration for NetVizura

By default, NetVizura uses a locally installed Elasticsearch instance as its primary datastore. One of the key features of Elasticsearch is its almost infinitely scalable horizontal architecture, which can be leveraged as application requirements grow.

In this short guide, we will walk through the steps required to install and configure a single remote Elasticsearch node.

Installation

We will install a single-node Elasticsearch cluster with authentication enabled. The installation will be demonstrated on Ubuntu 24.04, but the same steps are applicable to most Linux distributions.

First, we add the Elasticsearch repository:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
sudo apt-get install apt-transport-https -y
echo "deb https://artifacts.elastic.co/packages/8.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-8.x.list
sudo apt update
sudo apt-get -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install elasticsearch -y

This is the basic Elasticsearch installation, to which we need to add a few additional steps. By default, once the installation is complete, you will see output similar to the following:

--------------------------- Security autoconfiguration information ------------------------------

Authentication and authorization are enabled.
TLS for the transport and HTTP layers is enabled and configured.

The generated password for the elastic built-in superuser is : +_znnrd9HkdV4=wK7znf

If this node should join an existing cluster, you can reconfigure this with
'/usr/share/elasticsearch/bin/elasticsearch-reconfigure-node --enrollment-token '
after creating an enrollment token on your existing cluster.

You can complete the following actions at any time:

Reset the password of the elastic built-in superuser with
'/usr/share/elasticsearch/bin/elasticsearch-reset-password -u elastic'.

Generate an enrollment token for Kibana instances with
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s kibana'.

Generate an enrollment token for Elasticsearch nodes with
'/usr/share/elasticsearch/bin/elasticsearch-create-enrollment-token -s node'.

-------------------------------------------------------------------------------------------------
### NOT starting on installation, please execute the following statements to configure elasticsearch service to start automatically using systemd
sudo systemctl daemon-reload
sudo systemctl enable elasticsearch.service
### You can start elasticsearch service by executing
sudo systemctl start elasticsearch.service
---

User and Role Management

We can either use the default elastic user and password mentioned above, or create and use a specific account.

First, let's create a new role:

POST /_security/role/netvizura
{
  "cluster": [
    "manage",
    "manage_index_templates"
  ],
  "indices": [
    {
      "names": ["raw_data*"],
      "privileges": [
        "read",
        "write",
        "delete_index",
        "view_index_metadata",
        "create_index"
      ]
    }
  ]
}

Next, we assign the role to a user:

POST /_security/user/nenad.spasic
{
  "password": "Kladovo",
  "roles": ["netvizura"],
  "full_name": "Nenad Spasic",
  "email": "nenad.spasic@netvizura.com"
}

Don’t skip the test to see whether the communication is working:

By default, we add some additional optimizations to ensure Elasticsearch performs well.

1) RAM Usage

RAM usage is set to 30% of available memory, but you can increase it up to 32GB. Here’s a quick script to handle that automatically:

total=$(free -m | grep '^Mem:' | awk '{print $2}')
if (( $total < 96000 ))
then
  elastic=$(( $total/3))m
else
  elastic=32000m
fi

# JVM optimization
sed -i '/Xms/d' /etc/elasticsearch/jvm.options
sed -i '/Xmx/d' /etc/elasticsearch/jvm.options
echo "-Xms$elastic" >> /etc/elasticsearch/jvm.options
echo "-Xmx$elastic" >> /etc/elasticsearch/jvm.options

This ensures Elasticsearch has enough RAM for both intensive read and write operations.

2) Basic Configuration

By default, you should only need to change a few settings in /etc/elasticsearch/elasticsearch.yml:

3) System Recommended Optimizations

# Limits optimization
sed -i '/^elasticsearch/d' /etc/security/limits.conf
cat >> /etc/security/limits.conf << EOF
elasticsearch   hard    nproc 4096
elasticsearch   soft    nproc 4096
elasticsearch   -       nofile 65535
EOF

# System optimization
grep -q '^vm.swappiness' /etc/sysctl.conf && sed -i 's/^vm.swappiness.*/vm.swappiness=1/' /etc/sysctl.conf || echo 'vm.swappiness=1' >> /etc/sysctl.conf
grep -q '^vm.max_map_count' /etc/sysctl.conf && sed -i 's/^vm.max_map_count.*/vm.max_map_count=262144/' /etc/sysctl.conf || echo 'vm.max_map_count=262144' >> /etc/sysctl.conf
sysctl -p
  

⚠️ Note: After applying all these settings, don’t forget to restart Elasticsearch: