AI & ML Search with OpenSearch (elasticsearch + AI/ML)

Find the meaning in your data with OpenSearch & AI

What you'll learn
Understand and implement traditional search, neural search, hybrid search using Amazon's OpenSearch, apache-licensed open-source platform
Implement semantic search, retrieval augmented generation (RAG) using locally hosted models or external LLM providers like OpenAI
Implement real-time projects entirely on a local machine or a cloud VM using VS code, shell scripts, python and yaml templates
Implement reporting, alerting , dashboards, observability log patterns while understanding integration points with cloud
Complete multiple case studies, including migration of production data from elasticsearch to opensearch
Understand and implement agentic workflows involving RAG architectures on local and external LLMs

Requirements
Basics of running docker container, python programming basics, and eagerness to understand and unpack how search works
Local laptop with at least 4GB RAM (8GB preferable) and 2 CPU cores (4 preferable). Be ready to spend about $5 or lesser using a public LLM service e.g. Open AI

Description
Elasticsearch is a well-known search platform adopted in enterprises, SMBs and startups. Elasticsearch excels at lexical search use cases using BM25 algorithm , that is built on top of Lucene. However, with the advent of AI and large language models, Semantic Search, Hybrid Search, Neural Search, Multi-modal search etc. have become more of a norm than rarity. OpenSearch (originally a fork of Elasticsearch started in 2021) has gained immense popularity and adoption in open source, and enterprise communities with its Apache open source license and a Linux foundation project. While providing parity with all the lexical search capabilities of elasticsearch, OpenSearch integrates with LLM models (e.g. sentence transformers) , providers like OpenAI, Cohere, Anthropic and defines agentic workflows. As a win, Oracle switched to OpenSearch for its PeopleSoft search capabilities. AWS provides Opensearch-as-a-service on its cloud and that already speaks to the production readiness.AI & ML Search with OpenSearch course provides end-end training on installing, configuring and understanding OpenSearch , while implementing real search use cases like retrieval-augmented-generation (RAG), agentic workflows and migrating from Elasticsearch to OpenSearch. Emphasis has been laid on AI/ML use cases more than the traditional/lexical concepts, though the latter is covered for historical context. To compare Elasticsearch (ELK stack) & OpenSearch, we can roughly equate the below:Elasticsearch ~ OpenSearchLogstash ~ Data PrepperKibana ~ OpenSearch Dashboards OpenSearch is a fast moving platform in terms of its releases and features. We will be using version 2.17 which is production-ready as of September 2024. Docker has been extensively used in the course to ensure execution reproducibility of the entire course code. I am excited to be your instructor and hoping you resonate the same excitement !

Who this course is for
Undergrad with no-real-world project experience
Real-world experienced professionals from non-search domains
Software Developer
Devops Engineer / SysOps admin / Site Reliability Engineer
Data Scientist / Analyst / Engineer
Test Engineer planning to switch careers laterally
Polyglot engineers eager to save costs , improve performance of existing search platforms