Chapter 40

Elastic Search

Presentation & objectives

In this part of the course, we will discover and apply Elasticsearch, a powerful distributed search and analytics engine built on Apache Lucene. In particular, we will cover the following concepts:

  • Understanding the ELK Stack (Elasticsearch, Logstash, Kibana) architecture and data pipeline
  • Exploring document structure and field types in Elasticsearch
  • Mastering Query DSL (Domain Specific Language) for searching and filtering data
  • Understanding mapping and the difference between text and keyword field types
  • Working with aggregations for data analytics and insights
  • Querying arrays of objects and understanding object vs nested types

By the end of these activities, you will be able to:

  • Deploy and manage an Elasticsearch cluster with the ELK stack
  • Index, search, and analyze large volumes of data efficiently
  • Write complex queries using boolean logic, ranges, and filters
  • Understand field mapping and choose appropriate data types
  • Perform data aggregations to extract business insights

Very useful documentations can be found here:

Activities

  1. Setup & Prerequisites
  2. Architecture & Document Structure
  3. Queries & Filtering
  4. Mapping & Field Types
Warning

Before attending the sessions, you need to have the following tools installed on your machine:

  • Docker Desktop (includes Docker Engine + Docker Compose)
  • uv package manager (manages Python automatically)
  • curl (for testing HTTP endpoints)

Knowledge Check

Answer these questions once you have finished this course module.

--- primary_color: steelblue secondary_color: lightgray text_color: black shuffle_questions: false shuffle_answers: true --- # What is the primary purpose of Elasticsearch? 1. [X] A distributed search and analytics engine for storing, searching, and analyzing large volumes of data 2. [ ] A relational database management system for transactional data 3. [ ] A message broker for asynchronous communication between services 4. [ ] A web server for hosting static websites # In the ELK stack, what is the role of Logstash? 1. [ ] To provide a web interface for visualizing data 2. [X] To ingest, transform, and forward data to Elasticsearch 3. [ ] To store and index documents 4. [ ] To manage cluster health and shard allocation # What is the difference between text and keyword field types in Elasticsearch? 1. [ ] Text fields store numbers, keyword fields store strings 2. [X] Text fields are analyzed for full-text search, keyword fields are stored as-is for exact matching 3. [ ] Text fields can be sorted, keyword fields cannot 4. [ ] There is no difference between text and keyword fields # What type of query should you use for exact matching on a status field? 1. [ ] match query on the text field 2. [X] term query on the keyword field 3. [ ] range query on the text field 4. [ ] bool query on the numeric field # What is dynamic mapping in Elasticsearch? 1. [ ] A feature that automatically creates backups of indices 2. [ ] A process that redistributes shards across nodes 3. [X] Automatic detection and mapping of field types when indexing documents 4. [ ] A method for dynamically scaling cluster resources # What is the main limitation of the object type for arrays of objects? 1. [ ] Object arrays cannot be indexed in Elasticsearch 2. [ ] Object arrays use too much storage space 3. [X] Object arrays lose correlation between field values within the same array element 4. [ ] Object arrays cannot contain more than 10 elements