Elasticsearch Operations and Performance Tuning

Most of Elasticsearch operations is memory, disk, segments, and retention policy rather than search algorithms.

What operations actually manages

Elasticsearch is a distributed search and analytics engine built on Apache Lucene. In production, response time is decided by resource allocation more than by search features. The balance between JVM heap and file system cache, the shard count, and the index lifecycle policy are what drive performance.

Leaving a cluster on default settings works until volume grows. Frequent OOM (out of memory) terminations, pauses from garbage collection (GC), and indexing lag are the typical symptoms. Operations therefore has to inspect heap, cache, segments, and retention together.

JVM heap and the file system cache

Heap memory causes problems when it is too small and when it is too large. The search daemon runs on the JVM, so heap size is the starting point for stability.

The first guideline is the 50% rule. Allocate no more than half of physical RAM to the heap (-Xms, -Xmx). Leave the remaining half for the operating system's file system cache. This cache affects how quickly Lucene reads segments from disk and therefore affects search latency.

The second is the 32GB boundary. Growing the heap to around 32GB or beyond costs the JVM its compressed object pointers and forces 64-bit pointers. Object memory overhead then rises by roughly 1.5x, so a larger heap can actually hold fewer objects (per the official Elasticsearch guidance).

Elasticsearch 7 and later use G1GC (Garbage First garbage collector) by default for short pause times. Changing heap size dynamically during operation can cause long pauses. Set the minimum heap (-Xms) and maximum heap (-Xmx) to the same value.

# jvm.options
-Xms16g
-Xmx16g

As heap usage approaches its limit, Elasticsearch rejects risky requests through circuit breakers. This is a safeguard that chooses a few failed requests over a node dying with OOM.

Indexing performance and the refresh interval

Bulk indexing can trade immediate search visibility for write throughput. A refresh makes new documents searchable. By default, self-managed Elastic Stack refreshes an index every second if it received a search during the previous 30 seconds. See the Elastic indexing guide.

Frequent refreshes create small segments and increase background merge costs. During a bulk load, increase or disable the refresh interval. Reduce the replica count to zero to decrease write load.

The following example uses the Elasticsearch Console HTTP request format.

PUT /logs-2026.06/_settings
{
  "index": {
    "refresh_interval": "30s",
    "number_of_replicas": 0
  }
}

After the load finishes, restore the production refresh interval and replica count. This restores search visibility and availability. Heavy bulk loading and real-time search on the same index can degrade both workloads. Separating them by time window or index is safer.

Search performance and filter context

Query context computes relevance scores. Filter context only determines whether a document matches. This distinction affects search latency.

Filter context skips relevance scoring and suits conditions that need only a match decision. Frequently repeated filters become cache candidates only when they meet eligibility rules such as segment size (Elastic query cache documentation). Put status and date range conditions in the filter clause of a bool query.

This search also uses the Elasticsearch Console HTTP request format.

GET /products/_search
{
  "query": {
    "bool": {
      "must":   [ { "match": { "title": "wireless earbuds" } } ],
      "filter": [
        { "term":  { "status": "active" } },
        { "range": { "price": { "lte": 100000 } } }
      ]
    }
  }
}

In this query the must clause computes text relevance, while the status and price conditions are handled by the filter clause with caching. Putting score-free conditions into must adds unnecessary computation to every search, so keep them separated.

Index lifecycle management (ILM)

Time series data such as logs and audit records accumulates daily and has to be retained for a long time. Index lifecycle management (ILM) changes the hardware tier according to the age of the data, balancing cost against search speed.

flowchart LR
    Hot["Hot tier<br/>writes and frequent searches"] --> Warm["Warm tier<br/>search oriented, lower cost"]
    Warm --> Cold["Cold tier<br/>rare access, long-term retention"]
    Cold --> Delete["Delete<br/>removed after the retention period"]

• Hot: takes writes and frequent searches on fast disks
• Warm: stops taking writes and sits on cheaper nodes for search
• Cold: long-term retention with rare access
• Delete: removed automatically once the retention period ends

Manual data movement is not practical. Assign node attributes, such as node.attr.box_type: hot, and automate transitions with ILM. For example, roll over writes to a new hot index when an index exceeds 30GB or reaches 7 days. Delete it after 90 days. These values are examples; choose values based on retention requirements and ingest volume.

Operating system kernel tuning

Elasticsearch runs on the JVM, but its stability also depends on kernel settings. The values below are what the official Elasticsearch guide recommends for production.

Elastic recommends disabling swap entirely. If the environment must retain swap, consider vm.swappiness=1 or memory locking (Elastic swap documentation). Verify through the node API that memory locking took effect.

# elasticsearch.yml
bootstrap.memory_lock: true

Setting vm.swappiness to 0 is not recommended. It can provoke the Linux OOM killer, which makes 1 the safe lower bound.

Failure signals and what to check

Degradation usually arrives in the same patterns. When the signals below appear, narrow down the cause.

• Mapping explosion: unbounded field growth from dynamic mapping bloats cluster state metadata
• Excessive shards: more shards and segments mean higher metadata management cost
• Uncontrolled aggregations: aggregating on high-cardinality fields burns through heap quickly
• Refresh, replica, and ingest cost: when indexing is slow, examine these three together

When aggregations or indexing suddenly slow down, start with heap utilization, GC frequency, and segment count. Piling heavy bulk loading and real-time search onto a single index degrades both, so split conflicting workloads across indices.

Summary

Elasticsearch operations covers heap, indexing, search, and retention. Keep the heap at or below half of RAM and respect the 32GB boundary. Set fixed -Xms and -Xmx values. During bulk loads, adjust the refresh interval and replica count to reduce write load.

Put conditions that do not need scores in filter clauses so they can use the bitset cache. Use ILM to manage time series data from Hot through Delete.

The 50% rule, 32GB boundary, and kernel recommendations here come from the official Elasticsearch guide. Rollover values such as 30GB, 7 days, and 90 days are examples. Adjust them to ingest volume and retention requirements.