Deep Neural Network policy learning with mathematical foundations. Policy Gradient Methods Policy Parameterization Policy $\pi_\theta(a|s)$ parameterized by neural network with weights $\theta$. Objective Function Maximize expected return: $$ J(\theta) = \mathbb{E}{\tau \sim \pi\theta}\left[\sum_{t=0}^{T} \gamma^t …

Graph-based Retrieval-Augmented Generation for enhanced context and relationship understanding. Core Idea Graph RAG represents knowledge as a structured graph (entities and relationships) rather than flat document chunks, enabling multi-hop reasoning and relationship-aware retrieval. Mathematical Foundation Traditional …

Image embeddings convert visual content into dense vector representations that capture semantic and visual features, enabling similarity search, classification, and retrieval. Core Idea Image embeddings map images to fixed-size vectors in a high-dimensional space where semantically similar images are close together. …

High-level Keras API for building neural networks quickly. Installation 1# Keras is included in TensorFlow 2.x 2pip install tensorflow 3 4# Standalone Keras (multi-backend) 5pip install keras Sequential Model 1from tensorflow import keras 2from tensorflow.keras import layers 3 4model = keras.Sequential([ 5 …

ONNX (Open Neural Network Exchange) for converting models between frameworks. Installation 1pip install onnx onnxruntime 2pip install tf2onnx # TensorFlow to ONNX 3pip install onnx2pytorch # ONNX to PyTorch PyTorch to ONNX 1import torch 2import torch.onnx 3 4# Load PyTorch model 5model = MyModel() …

Essential PyTorch operations and patterns for deep learning. Installation 1# CPU version 2pip install torch torchvision torchaudio 3 4# GPU version (CUDA 11.8) 5pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 6 7# Check installation 8python -c "import torch; …

Q-Learning algorithm theory with mathematical foundations. Markov Decision Process (MDP) An MDP is defined by the tuple $(S, A, P, R, \gamma)$: $S$: Set of states $A$: Set of actions $P$: Transition probability $P(s'|s,a)$ $R$: Reward function $R(s,a,s')$ $\gamma \in [0,1]$: Discount factor Value Functions State Value …

Retrieval-Augmented Generation techniques for enhancing LLM responses with external knowledge. Core Idea RAG combines retrieval from external knowledge bases with LLM generation to produce accurate, up-to-date responses without retraining the model. Mathematical Foundation The core retrieval mechanism uses cosine …

Audio embeddings convert sound signals (speech, music, environmental sounds) into dense vector representations that capture acoustic and semantic features, enabling similarity search, classification, and retrieval. Core Idea Audio embeddings map audio waveforms or spectrograms to fixed-size vectors in a …

Tensor mathematics fundamentals and backpropagation theory with detailed mathematical derivations. Tensor Basics What is a Tensor? A tensor is a generalization of scalars, vectors, and matrices to higher dimensions: Scalar (0D tensor): $x \in \mathbb{R}$ Vector (1D tensor): $\mathbf{x} \in \mathbb{R}^n$ Matrix (2D …

Essential TensorFlow operations and patterns for deep learning. Installation 1# CPU version 2pip install tensorflow 3 4# GPU version 5pip install tensorflow[and-cuda] 6 7# Check installation 8python -c "import tensorflow as tf; print(tf.version)" Basic Tensor Operations 1import tensorflow as tf 2 3# Create …

TensorFlow Lite for deploying models on mobile and embedded devices. Convert Model to TFLite 1import tensorflow as tf 2 3# Load Keras model 4model = tf.keras.models.load_model('my_model.h5') 5 6# Convert to TFLite 7converter = tf.lite.TFLiteConverter.from_keras_model(model) 8tflite_model = converter.convert() 9 …

Text embeddings convert textual content into dense vector representations that capture semantic meaning, enabling similarity search, classification, and retrieval in natural language processing. Core Idea Text embeddings map text sequences (words, sentences, documents) to fixed-size vectors in a high-dimensional space …