Generated Standalone Python Script (Exact Canvas Pipeline)
This self-contained Python script reproduces the exact boxes and connections on your canvas in dependency order. It makes real API calls (OpenAI, Ollama, Gemini, Groq) or runs self-contained simulations if offline.
# Code generating...
API Keys & Connections
OpenAI / Azure
Enter your OpenAI or Azure OpenAI API key, or leave blank for smart simulation.
Optional: Base URL for OpenAI or Azure endpoint.
Optional: The specific model to use.
Google Gemini
Get from Google AI Studio
HuggingFace
Get from HF Settings
Groq (Ultra-Fast)
Get from Groq Console
OpenRouter (Multi-Model Gateway)
Access 100+ models via OpenRouter
Tools Calling & Web Search
Enables real-time web retrieval for autonomous tool calling agents via Tavily API
Load Workflow
Select a previously saved workflow from your account below.
Loading workflows...
AI FlowLab Account & Access
Administrator Approval Console
Registered Users & Model Permissions
Review new user registrations. Approving a user grants access to live AI models and sends an activation email.
| User | Role | Registered | Status | Actions |
|---|
User Keys Inspector
Inspecting personal keys for user
Only administrators and this account owner can view these keys. If empty, the user automatically falls back to system admin keys during live execution.
User Access & Assignments Manager
User Permissions
Active
User ID # - email@example.com
Allowed AI Model Providers & Key Usage
Configure which model backends this user is authorized to execute. If a model is unchecked, the backend blocks live calls for this user.
Assigned Interactive Lesson Modules
Choose which educational modules appear in this user's Tutorials menu.
Assigned Workflow Templates & Categories
Select which workflow template categories are visible and loadable by this user.
Live AI Model Access Required
🔐
Live AI Model Execution Guard
Non-logged in guests can build workflows and run interactive educational simulations freely. To execute real-time model calls with OpenAI, Gemini, Groq, or Tavily, please sign in or register for administrator approval.
AI FlowLab Video & Visual Demonstration
Video Tutorial Breakdown
00:00
Overview & Grade Modes
Learn how to switch between Grade 4-6, Grade 7-12, and Adult modes.
00:10
Loading Educational Templates
Explore preset workflows for Story Books, Customer Support, and Search Agents.
00:20
Connecting AI Blocks & Tools
Drag blocks onto the canvas and draw Bezier connection wires.
00:30
Live Execution & Python Export
Watch real-time workflow runs and export clean Python/LangChain scripts.
Block Output
port: response Completed
AI Architecture Masterclass: How Prompts, Models & Tools Work Together
Full Execution Pipeline Diagram
Watch how user prompts flow through tokenization, LLM decision making, external tool calling, and final synthesis in real time!
1
User Prompt
System + Query
➔
2
LLM Brain
Tool Decision
➔
3
Tool API
Flight / Weather
➔
4
Final Synthesis
Formatted Output
Phase 1: User Input & Context Assembly
Phase 1 of 4
Internal Reasoning & Thought
User asks: "What is the cheapest flight to Paris and what is the current weather there?"
Action Executed
Assemble System Prompt + User Query into standard API payload.
Live API Payload Inspector
// Payload content loading...🧠 Understanding Prompts & System Roles
In modern AI models (like GPT-4o, Claude, or Llama 3), a "Prompt" is not just a simple search query. It is a structured conversation composed of distinct Roles:
1. System Prompt
Defines the AI's identity, tone, rules, and available tools. It acts as the constitution that governs all subsequent behavior.
"You are an expert travel agent. Always verify prices using get_flight_price before answering."
2. User Prompt
The input or question provided by the human student or user requesting help or task execution.
"Find a flight from NYC to Tokyo under $900."
3. Assistant Response
The output generated by the AI model, which can be natural language text OR a structured Tool Call request.
"I am searching flights to Tokyo..."
⚡ Key Prompting Techniques for Students
- Zero-Shot Prompting: Asking the model directly without giving prior examples.
- Few-Shot Prompting: Providing 2-3 sample input/output pairs inside the prompt to guide formatting.
- Chain-of-Thought (CoT): Instructing the model to "think step by step" before giving the final answer, which drastically reduces mathematical and logic errors.
🛠️ How LLMs Talk to External Tools
AI models cannot directly browse the live web or access your local files on their own. Instead, they use Function Calling / Tool Integration.
Step A
Declare Tool Schemas
We provide the LLM with a list of JSON functions describing what each tool does and what arguments it accepts.
Step B
LLM Emits Tool Call
When asked for real-time data, the model stops generating plain text and outputs a structured tool request: get_weather(city="Paris").
Step C
App Executes Code & Returns Result
Our Python code executes the API request, gets raw JSON data (e.g. {"temp": "18°C"}), and feeds it back to the LLM.
Step D
Final Synthesis
The LLM reads the tool output and translates the raw data into a friendly, helpful answer for the user.
🔄 ReAct Agents: Synergizing Reasoning & Acting
ReAct (Reason + Act) is the standard architecture behind modern autonomous AI agents. Instead of answering in a single step, the agent loops continuously through 3 phases:
🧠 1. THOUGHT
Analyze state: "I need to check stock prices first before predicting trends."
➔
⚡ 2. ACTION
Invoke Tool: fetch_stock_price(ticker="AAPL")
➔
👁️ 3. OBSERVATION
Receive Tool Output: {"price": 224.50}
🔁 Loop repeats until the goal is accomplished!
#️⃣ Tokens & Tokenization: How AI Reads Text
AI models don't read words like humans. They break text into tokens — small pieces that can be whole words, parts of words, or even single characters. Understanding tokens is crucial for prompt engineering, cost estimation, and working within context limits.
🔤 What is a Token?
A token is the smallest unit of text that an AI model processes. Here's how the sentence "Hello, world!" gets tokenized:
Input Text:
"Tokenization helps AI understand language!"
⬇️ Tokenizer (BPE)
Token
ization
helps
AI
understand
language
!
ID: 3869
ID: 2065
ID: 8477
ID: 9552
ID: 3619
ID: 4221
ID: 0
Token
A piece of text (word, sub-word, or character) that the model processes as one unit. "running" might become ["run", "ning"].
Vocabulary
The complete dictionary of all tokens a model knows. GPT-4 has ~100,000 tokens. Each has a unique ID number.
BPE (Byte Pair Encoding)
The algorithm that builds the vocabulary by finding the most common character pairs and merging them repeatedly.
Token Limit
The maximum tokens a model can process at once. GPT-4o supports 128K tokens. This includes BOTH input AND output.
💡 When to Think About Tokens
- Cost Estimation: API pricing is per-token. GPT-4o costs ~$2.50 per 1M input tokens. More tokens = higher cost.
- Context Limits: Your system prompt + user message + AI response must all fit within the token limit.
- Prompt Optimization: Shorter, precise prompts save tokens and money while often producing better results.
- Rule of Thumb: 1 token ≈ ¾ of a word in English. 100 tokens ≈ 75 words.
🧮 Inference & Context Windows: How AI Thinks
Inference is the process of running an AI model to generate an output from an input. Understanding the inference pipeline helps you optimize speed, cost, and quality of AI responses.
🔄 The Full Inference Pipeline
Text Input
Raw text
→
Tokenizer
Text → IDs
→
Embedding
IDs → Vectors
→
Transformer
96+ Layers
→
Logits
Probabilities
→
Sampling
Pick token
→
Output
Text reply
📏 Context Window Visualizer
Every model has a fixed context window. Your entire conversation must fit inside it:
GPT-4o Context: 128,000 tokens
System Prompt (5%)
User Messages (15%)
AI Response (10%)
Available (70%)
Inference
Running a trained model to produce output. Each token generated requires a full forward pass through the neural network.
Context Window
The maximum number of tokens the model can "see" at once. GPT-4o: 128K, Gemini 2.5: 1M, Claude: 200K.
Temperature
Low (0.0-0.3): Deterministic, factual. Med (0.5-0.7): Balanced. High (0.8-1.5): Creative, surprising.
Top-P (Nucleus)
Controls diversity by only sampling from the top P% of probable tokens. Top-P 0.9 means only the top 90% probability mass is considered.
⚡ Practical Tips
- Use low temperature (0.1) for code generation, data extraction, and factual Q&A.
- Use high temperature (0.9) for creative writing, brainstorming, and storytelling.
- Cost formula: Total cost = (Input tokens × input price) + (Output tokens × output price).
- Groq vs. OpenAI: Groq achieves ~500 tokens/sec vs ~80 tokens/sec for GPT-4o, but model quality may differ.
🦜🔗 LangChain & LCEL: Composable AI Architecture
From raw API calls to robust production pipelines: Learn Declarative Runnables, Document Ingestion, Chroma Vector Stores, and ReAct Agents.
The LCEL Unix Philosophy: Composable Pipe Operator (|)
chain = ChatPromptTemplate | ChatModel | StrOutputParser()
⚡ Every component is a Runnable with standardized invoke(), stream(), and batch()
1. Ingest & Split
DirectoryLoader & RecursiveSplitter
DirectoryLoader & RecursiveSplitter
2. Embed & Index
HuggingFace / OpenAI ➔ Chroma DB
HuggingFace / OpenAI ➔ Chroma DB
3. Retrieve & Ground
VectorStore.as_retriever(k=3)
VectorStore.as_retriever(k=3)
4. Tools & Agents
ReAct Agent + Tavily Search Tools
ReAct Agent + Tavily Search Tools
LCEL Runnables
Standardized protocol replacing spaghetti code. Runnables automatically support sync, async, streaming tokens, and parallel execution.
Document Loaders
Standardized abstractions for extracting unstructured data from 70+ sources (Markdown files, PDFs, CSV, SQL, Web pages) into LangChain Documents.
Recursive Chunking
Splits long text along natural paragraph and sentence boundaries, preserving semantic context and adding overlap so sentences aren't truncated.
Chroma VectorStore
Stores mathematical embeddings of text chunks. Computes cosine similarity to find passages relevant to the user query in milliseconds.
Lesson 1: Pure LCEL Pipeline (Prompt | Model | OutputParser)
In LangChain Expression Language (LCEL), components are combined using the Python pipe operator (|). The output of the left component becomes the input of the right component.
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI
import os
# 1. Define prompt with input variable placeholders
prompt = ChatPromptTemplate.from_messages([
("system", "You are an expert AI Architect. Explain topics concisely with bullet points."),
("user", "{question}")
])
# 2. Instantiate LLM Model (OpenAI / Azure / Ollama)
model = ChatOpenAI(model="gpt-4o-mini", temperature=0.7)
# 3. Chain components declaratively with pipe operator (|)
chain = prompt | model | StrOutputParser()
# 4. Execute synchronously, asynchronously, or stream tokens
response = chain.invoke({"question": "Explain LangChain Expression Language in 3 key benefits"})
print(response)
# Streaming tokens in real-time
for chunk in chain.stream({"question": "What is LCEL?"}):
print(chunk, end="", flush=True)
Lesson 2: Scanning Knowledge Base Files & Token Counting
In real-world applications (such as our knowledge-base with 76 markdown files), DirectoryLoader batch-reads entire folder trees and tags documents with metadata (e.g. category, document type, file path). We use tiktoken to measure exact model token limits and costs.
import os
import glob
import tiktoken
from langchain_community.document_loaders import DirectoryLoader, TextLoader
# 1. Inspect knowledge base directory
knowledge_base_path = "./src/wrappers/basics/knowledge-base/**/*.md"
files = glob.glob(knowledge_base_path, recursive=True)
print(f"Found {len(files)} markdown knowledge-base files")
# 2. Count characters and compute exact model tokens using tiktoken
entire_kb = ""
for file_path in files:
with open(file_path, "r", encoding="utf-8") as f:
entire_kb += f.read() + "\n\n"
encoding = tiktoken.encoding_for_model("gpt-4o-mini")
tokens = encoding.encode(entire_kb)
print(f"Total KB Characters: {len(entire_kb):,}")
print(f"Total KB Tokens: {len(tokens):,} (Tokens/Char ratio: {len(tokens)/len(entire_kb):.2f})")
# 3. Load documents with directory metadata preservation
folders = glob.glob("./src/wrappers/basics/knowledge-base/*/*")
all_documents = []
for folder in folders:
doc_type = os.path.basename(folder)
loader = DirectoryLoader(
folder,
glob="**/*.md",
loader_cls=TextLoader,
loader_kwargs={"encoding": "utf-8"}
)
docs = loader.load()
for d in docs:
d.metadata["doc_type"] = doc_type
all_documents.append(d)
print(f"Loaded {len(all_documents)} categorized documents with metadata ready for splitting!")
Lesson 3: Text Splitting & Chunk Overlap Strategy
Large documents cannot be stored as a single block in a vector database because queries will retrieve too much irrelevant noise. RecursiveCharacterTextSplitter breaks documents by double newlines, single newlines, and spaces, adding an overlap buffer so context isn't sliced in half.
from langchain_text_splitters import RecursiveCharacterTextSplitter
# Configure recursive splitter with chunk size and overlap
text_splitter = RecursiveCharacterTextSplitter(
chunk_size=500, # Maximum characters per chunk
chunk_overlap=50, # Overlap characters between chunks to preserve context
separators=["\n\n", "\n", " ", ""],
length_function=len
)
# Split loaded documents into chunks while preserving metadata
chunks = text_splitter.split_documents(all_documents)
print(f"Split {len(all_documents)} source documents into {len(chunks)} searchable chunks")
print(f"Sample Chunk Preview:\n{chunks[0].page_content[:200]}...")
print(f"Sample Chunk Metadata: {chunks[0].metadata}")
Lesson 4: Indexing & Searching Chroma Vector Database
Text chunks are converted into dense vector arrays using an embedding model (e.g. HuggingFace all-MiniLM-L6-v2 or OpenAI text-embedding-3-small). The vectors are stored in a local Chroma database, which computes cosine similarity at query time.
from langchain_chroma import Chroma
from langchain_huggingface import HuggingFaceEmbeddings
# Alternatively: from langchain_openai import OpenAIEmbeddings
# 1. Initialize local HuggingFace embedding model (runs 100% free locally)
embedding_model = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
# 2. Create persistent Chroma Vector Database from document chunks
vector_db = Chroma.from_documents(
documents=chunks,
embedding=embedding_model,
persist_directory="./chroma_db",
collection_name="insurellm_kb"
)
print("Chroma Vector Store successfully built and persisted to disk!")
# 3. Perform semantic similarity search with score
query = "What coverage exists for electric vehicle batteries?"
results_with_scores = vector_db.similarity_search_with_score(query, k=3)
for idx, (doc, score) in enumerate(results_with_scores, 1):
print(f"\n--- Match #{idx} (Distance Score: {score:.4f}) ---")
print(f"Source: {doc.metadata.get('source', 'Unknown')} | Type: {doc.metadata.get('doc_type')}")
print(doc.page_content[:250] + "...")
Lesson 5: LangChain ReAct Agents with Tavily Search Tools
When users ask questions beyond static company documentation, a LangChain Agent uses the ReAct (Reason + Act) loop to invoke tools. Here we equip the agent with our newly configured Tavily Search API to ground answers in live internet facts.
import os
import requests
from langchain.agents import create_openai_tools_agent, AgentExecutor
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.tools import tool
from langchain_openai import ChatOpenAI
# 1. Define Tavily live search tool
@tool
def tavily_web_search(query: str) -> str:
"""Searches the live internet for recent news, facts, and breaking information."""
api_key = os.getenv("TAVILY_API_KEY")
res = requests.post("https://api.tavily.com/search", json={
"api_key": api_key,
"query": query,
"max_results": 2
}, timeout=10)
data = res.json()
results = [f"Title: {r['title']}\nSnippet: {r['content']}\nURL: {r['url']}" for r in data.get("results", [])]
return "\n\n".join(results)
tools = [tavily_web_search]
# 2. Create Agent Prompt with agent scratchpad
prompt = ChatPromptTemplate.from_messages([
("system", "You are an autonomous research agent. Use the tavily_web_search tool when asked about breaking news, live data, or current facts."),
("user", "{input}"),
MessagesPlaceholder(variable_name="agent_scratchpad")
])
# 3. Bind LLM with tools and AgentExecutor
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0.2)
agent = create_openai_tools_agent(llm, tools, prompt)
executor = AgentExecutor(agent=agent, tools=tools, verbose=True)
# 4. Run the Agent
result = executor.invoke({"input": "What is Google Antigravity and what did DeepMind announce?"})
print(result["output"])
Ready-to-Run LangChain Workflows in AI FlowLab
Click any template below to immediately instantiate the visual node workflow onto your canvas and run it with your active API keys:
LangChain Chroma RAG Flow
User Prompt ➔ Knowledge Base (Chroma) ➔ LLM Brain ➔ Grounded Chat Response
LangChain Tavily Agent
ReAct agent loop with Tavily live web tool calling and real-time citation synthesis
LCEL Sequential Pipeline
Composable Prompt ➔ Intent Router (RunnableBranch) ➔ Priority Triage Synthesis
📐 Vectors, Chunks & Vector Databases Masterclass
The complete guide to Semantic Embeddings, Recursive Text Splitting, Chunk Overlap, Cosine Distance, Chroma DB, and 3D t-SNE Clustering.
Interactive Document Chunking & Tokenization Lab
Production mechanics mirroringvector_1.py using langchain_text_splitters & tiktoken (cl100k_base).
Docs: 4,133 chars
Chunks: 0
Tokens: 0
Ratio: ~4.0 c/t
Chunk Size:
400 chars
Target maximum characters per chunk
Chunk Overlap:
80 chars
Overlap buffer carried into next chunk
Recursive Separators:
\n\n (Paragraphs)
\n (Lines)
" " (Words)
"" (Chars)
Hierarchical splitting priority order
Dense Vector Embeddings & 384-D Spatial Math Studio
Production embedding pipeline mirroringvector_1.py & vector_d3.py using HuggingFace all-MiniLM-L6-v2.
Model: all-MiniLM-L6-v2
Dimensions: 384 Dense Floats
Norm: ||v|| = 1.0000
Quick-Load Semantic Comparison Scenarios:
Computes continuous 384D representations via
sentence-transformers on the backend.
Cosine Similarity: cos(θ)
+0.4262
✓ Strong Semantic Alignment
Angle in 384D Space (θ)
64.8°
0° = Parallel, 90° = Orthogonal
Dot Product: A · B
0.4262
When normalized: cos(θ) = A · B
Dense Vector Coordinate Array (First 24 of 384 Dimensions):
Float32 array • 1,536 bytes in memory
Vector A coordinates (d1 - d24):
Vector B coordinates (d1 - d24):
Python Reproduction Code (SentenceTransformers / LangChain):
from sentence_transformers import SentenceTransformer
import numpy as np
# Load MiniLM 384-dimensional embedding model
model = SentenceTransformer('all-MiniLM-L6-v2')
# Encode sentences into dense float vectors
v1 = model.encode(["Carllm offers automated auto insurance risk assessment and instant quotes."])[0]
v2 = model.encode(["Vehicle coverage policy with telematics discounts and collision protection."])[0]
# Compute Cosine Similarity: cos(θ) = (v1 · v2) / (||v1|| ||v2||)
cosine_sim = np.dot(v1, v2) / (np.linalg.norm(v1) * np.linalg.norm(v2))
print(f"Dimensions: {len(v1)} | Cosine Similarity: {cosine_sim:.4f}")
Dense Vector Embeddings
Text translated into an array of floating-point numbers (e.g. 384 dimensions for all-MiniLM, 1536 for OpenAI). Synonyms have high mathematical similarity.
Chunk Overlap Safety Cushion
Prevents meaning from being split across boundaries. If a sentence begins in Chunk 1, the overlap ensures it is completely understood in Chunk 2.
Cosine Similarity Metric
Computes the angle between two vectors: cos(θ) = (A · B) / (||A|| ||B||). Ranges from -1 (opposite) to +1 (identical), independent of text length.
HNSW & Approximate Nearest Neighbor
Hierarchical Navigable Small World graphs. Allows sub-millisecond similarity search across millions of vectors without scanning every single vector.
Tutorial 1: Recursive Character Chunking & Metadata Binding
The standard Python pattern for chunking. Using separators=["\n\n", "\n", " ", ""], LangChain tries first to split on paragraphs, then sentences, then words, only splitting within words as a last resort.
from langchain_text_splitters import RecursiveCharacterTextSplitter
# 1. Instantiate the Recursive Splitter
# Recommended: chunk_size=1000, chunk_overlap=200 for long-form enterprise documentation
splitter = RecursiveCharacterTextSplitter(
chunk_size=1000,
chunk_overlap=200,
separators=["\n\n", "\n", ".", " ", ""],
length_function=len,
is_separator_regex=False
)
# 2. Split loaded documents into chunks while carrying forward metadata
# documents is a list of Document objects loaded via DirectoryLoader
chunks = splitter.split_documents(documents)
print(f"Total Raw Documents: {len(documents)}")
print(f"Total Generated Chunks: {len(chunks)}")
print(f"Average Chunks Per Document: {len(chunks)/len(documents):.1f}")
# 3. Inspect chunk structure
sample_chunk = chunks[0]
print("\n--- SAMPLE CHUNK ---")
print("Content:", sample_chunk.page_content[:200], "...")
print("Metadata:", sample_chunk.metadata)
# Outputs: {'source': 'knowledge-base/products/smartshield.md', 'doc_type': 'products'}
Tutorial 2: Dense Vector Embeddings (HuggingFace vs OpenAI)
Embeddings transform arbitrary strings into high-dimensional vector representations. You can run open-weights embedding models locally for 100% data privacy and zero API cost, or use cloud embedding APIs.
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_openai import OpenAIEmbeddings
import numpy as np
# OPTION A: Local Free HuggingFace Embeddings (384 dimensions, runs locally on CPU/Apple Silicon)
hf_embeddings = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
# OPTION B: Cloud OpenAI Embeddings (1536 dimensions, highly capable multilingual semantics)
# openai_embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
# Generate vector for a text sample
sample_text = "What is the policy deductible for water damage claims?"
vector = hf_embeddings.embed_query(sample_text)
print(f"Embedding Vector Dimensions: {len(vector)} (e.g. 384 numbers)")
print(f"Sample First 5 Coordinates: {vector[:5]}")
# Compute cosine similarity between two sentences
v1 = np.array(hf_embeddings.embed_query("Auto insurance deductible"))
v2 = np.array(hf_embeddings.embed_query("Car collision policy out-of-pocket costs"))
v3 = np.array(hf_embeddings.embed_query("How to make strawberry ice cream"))
def cosine_similarity(a, b):
return np.dot(a, b) / (np.linalg.norm(a) * np.linalg.norm(b))
print(f"Similarity (Auto Insurance vs Collision Cost): {cosine_similarity(v1, v2):.4f} (Very High)")
print(f"Similarity (Auto Insurance vs Ice Cream): {cosine_similarity(v1, v3):.4f} (Low/Unrelated)")
Tutorial 3: Chroma Vector Store: Ingestion, Persistence & Distance Scoring
Chroma stores the embeddings and runs HNSW index searches. When performing similarity_search_with_score, Chroma returns Euclidean distance ($L2$) — a lower score means closer similarity!
from langchain_chroma import Chroma
from langchain_huggingface import HuggingFaceEmbeddings
embedding_model = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
# 1. Create Chroma Vector Store from chunks and persist to local directory
vector_db = Chroma.from_documents(
documents=chunks,
embedding=embedding_model,
persist_directory="./chroma_vector_db",
collection_name="insurellm_knowledge_base"
)
print(f"Persisted {len(chunks)} chunks into Chroma Vector DB at ./chroma_vector_db")
# 2. Similarity Search with Distance Score
query = "What happens if a customer vehicle battery is damaged by flood?"
results_with_scores = vector_db.similarity_search_with_score(query, k=3)
for idx, (doc, distance) in enumerate(results_with_scores, 1):
print(f"\n[Rank #{idx}] Euclidean Distance: {distance:.4f} (Lower = Closer Match)")
print(f"Document Category: {doc.metadata.get('doc_type', 'General')}")
print(f"Source Path: {doc.metadata.get('source')}")
print("Content Preview:", doc.page_content[:220], "...")
# 3. Filtered Search (Restricting search by metadata)
products_only = vector_db.similarity_search(
query,
k=2,
filter={"doc_type": "products"}
)
print(f"\nRetrieved {len(products_only)} product-specific chunks via metadata filter!")
Tutorial 4: 3D Semantic Vector Space with t-SNE & Plotly
Directly mirroring your imports in vector_1.py! We use t-SNE (t-Distributed Stochastic Neighbor Embedding) to reduce 384-dimensional vectors down to 3 coordinates $(x, y, z)$, rendering an interactive 3D scatter plot of semantic clusters using Plotly.
from sklearn.manifold import TSNE
import plotly.graph_objects as go
import pandas as pd
import numpy as np
# 1. Extract raw vectors and metadata from Chroma
raw_data = vector_db.get(include=["embeddings", "metadatas", "documents"])
embeddings_matrix = np.array(raw_data["embeddings"])
metadatas = raw_data["metadatas"]
doc_types = [m.get("doc_type", "default") for m in metadatas]
print(f"Visualizing matrix of shape: {embeddings_matrix.shape} (N chunks x D dimensions)")
# 2. Reduce dimensions from 384/1536 down to 3D using t-SNE
tsne = TSNE(n_components=3, perplexity=min(30, len(embeddings_matrix)-1), random_state=42)
projections = tsne.fit_transform(embeddings_matrix)
# 3. Create DataFrame for Plotly 3D Scatter
df = pd.DataFrame({
'x': projections[:, 0],
'y': projections[:, 1],
'z': projections[:, 2],
'doc_type': doc_types,
'snippet': [d[:80] + "..." for d in raw_data["documents"]]
})
# 4. Generate Interactive 3D Scatter Plot
fig = go.Figure()
for category in df['doc_type'].unique():
subset = df[df['doc_type'] == category]
fig.add_trace(go.Scatter3d(
x=subset['x'], y=subset['y'], z=subset['z'],
mode='markers',
name=category,
text=subset['snippet'],
marker=dict(size=5, opacity=0.85)
))
fig.update_layout(
title="3D Semantic Vector Space (Insurellm Knowledge Base Chunks)",
scene=dict(xaxis_title="t-SNE Dim 1", yaxis_title="t-SNE Dim 2", zaxis_title="t-SNE Dim 3"),
template="plotly_dark",
margin=dict(l=0, r=0, b=0, t=40)
)
# fig.show() or fig.write_html("vector_space.html")
Tutorial 5: Complete Vector RAG Pipeline with LCEL
How everything connects into a unified production pipeline: Vector Retriever converts natural language question ➔ fetches top-k chunks ➔ formats into context string ➔ pipes into Chat LLM for verified answers.
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.runnables import RunnablePassthrough
from langchain_core.output_parsers import StrOutputParser
from langchain_openai import ChatOpenAI
# 1. Turn Chroma into a LangChain Retriever
retriever = vector_db.as_retriever(search_kwargs={"k": 3})
def format_docs(docs):
return "\n\n".join([f"--- Source: {d.metadata.get('source')} ---\n{d.page_content}" for d in docs])
# 2. Design the RAG Grounding Prompt Template
prompt = ChatPromptTemplate.from_template("""You are an expert AI customer assistant for Insurellm.
Answer the user's question truthfully using ONLY the provided verified context.
If the context does not contain the answer, say "I do not have sufficient information in the knowledge base."
Context:
{context}
User Question: {question}
Answer:""")
# 3. Instantiate Model
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0.0)
# 4. Compose the complete LCEL RAG Chain
rag_chain = (
{"context": retriever | format_docs, "question": RunnablePassthrough()}
| prompt
| llm
| StrOutputParser()
)
# 5. Execute Query
answer = rag_chain.invoke("What benefits does VoltGuard provide for electric vehicle battery degradation?")
print(answer)
Ready-to-Run Vector & Chunking Canvas Templates
Load these pre-wired vector workflows directly into your canvas:
Chroma Vector RAG Pipeline
User Query ➔ Vector Retriever (Chroma) ➔ LLM Brain ➔ Grounded Answer
Text Chunking & Splitter Inspector
Raw Policy Document ➔ Text Splitter (500 chars, 100 overlap) ➔ LLM Chunk Analyzer
Hybrid Vector & Web Search Agent
Dual-grounding: Chroma vector retrieval combined with Tavily live internet web search
🏛️ Multi-Domain Knowledge Bases Architecture & Vector Indexing
Designing production RAG knowledge bases across specialized enterprise verticals: Medical, Legal, DevOps, E-Commerce, and InsureTech.
Medical & Clinical
./knowledge-bases/medical
Healthcare Protocols & Pharmacology
Clinical guidance, AFib diagnostics, Metformin contraindications (eGFR < 30), and HIPAA compliance guardrails.
symptoms
treatments
medications
guidelines
Legal & Compliance
./knowledge-bases/legal
Enterprise Contracts & Auditing
Mutual NDAs, Master Services Agreements (MSA), GDPR Article 17 erasure, SOC 2 Type II controls, and Delaware dispute arbitration.
agreements
compliance
licensing
disputes
DevOps & Cloud SRE
./knowledge-bases/devops
Incident Triage & Infrastructure Runbooks
Kubernetes Pod CrashLoopBackOff triage (Exit Codes 137, 1), HPA v2 autoscaling, DB failover postmortems, and Vault secret rotation.
kubernetes
incidents
cicd
security
E-Commerce & Retail
./knowledge-bases/ecommerce
Customer Care, RMA & Product Catalogs
30-day customer return policy, RMA barcode workflows, international DDP customs duties, and UltraSound / ProBook specifications.
returns
shipping
products
faq
InsureTech Enterprise
./src/wrappers/basics/knowledge-base
Underwriting Policies & Commercial Client SLAs
SmartShield telematics, VoltGuard EV battery policies, employee roles, corporate financials, and enterprise client custom development hours.
products
employees
company
contracts
Knowledge Base Engineering Strategy Matrix
| Domain | Optimal Chunk Size | Overlap Buffer | Primary Metadata Key | Search Technique | Hallucination Guardrail |
|---|---|---|---|---|---|
| Medical & Clinical | 400 - 600 chars | 50 chars | medication_class, contraindication |
Strict Cosine (>0.82 threshold) | Disallow unverified drug substitution |
| Legal & Compliance | 800 - 1200 chars | 200 chars | clause_num, governing_law |
Hybrid Dense + BM25 Lexical | Exact legal clause quotation |
| DevOps & SRE | 500 - 800 chars | 100 chars | exit_code, k8s_resource |
Error Code Metadata Filtering | Direct CLI runbook command verification |
| E-Commerce & Retail | 300 - 500 chars | 60 chars | sku, policy_window_days |
Dense Cosine + SKU Match | Clear RMA condition enforcement |
| InsureTech Enterprise | 800 - 1000 chars | 150 chars | doc_type, endorsement_code |
Partitioned Chroma Collection | Coverage deductible limit validation |
Python Implementation: Multi-Folder Knowledge Base Ingestor
Production pattern for loading structured directories into Chroma DB with automated category detection from subfolder names:
import os
from pathlib import Path
from langchain_community.document_loaders import DirectoryLoader, TextLoader
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langchain_huggingface import HuggingFaceEmbeddings
from langchain_chroma import Chroma
def build_domain_vector_store(domain_path: str, collection_name: str, chunk_size=800, overlap=150):
print(f"Indexing Knowledge Base from {domain_path} into collection '{collection_name}'...")
# 1. Load all markdown files recursively
loader = DirectoryLoader(
domain_path,
glob="**/*.md",
loader_cls=TextLoader,
loader_kwargs={"encoding": "utf-8"}
)
raw_docs = loader.load()
# 2. Automatically bind parent folder as category metadata
for doc in raw_docs:
doc_path = Path(doc.metadata.get("source", ""))
category = doc_path.parent.name
doc.metadata["category"] = category
doc.metadata["doc_name"] = doc_path.stem
# 3. Split into domain-tuned chunks
splitter = RecursiveCharacterTextSplitter(
chunk_size=chunk_size,
chunk_overlap=overlap,
separators=["\n\n", "\n", ".", " ", ""]
)
chunks = splitter.split_documents(raw_docs)
print(f"Generated {len(chunks)} chunks across {len(raw_docs)} documents.")
# 4. Embed and persist into Chroma DB
embedder = HuggingFaceEmbeddings(model_name="all-MiniLM-L6-v2")
vector_db = Chroma.from_documents(
documents=chunks,
embedding=embedder,
persist_directory=f"./chroma_{collection_name}",
collection_name=collection_name
)
return vector_db
# Example: Build separate vector stores for Medical and DevOps
medical_db = build_domain_vector_store("./knowledge-bases/medical", "medical_kb", chunk_size=500, overlap=50)
devops_db = build_domain_vector_store("./knowledge-bases/devops", "devops_kb", chunk_size=650, overlap=100)
Dialog Title
Action Confirmation
Step 1 of 6
Welcome to AI FlowLab!
This guided tour will show you step-by-step how to build, run, and export visual AI workflows!