Multi-Label vs Multi-Task vs Multi-Task Multi-Label
Most confusion comes from treating these as three points on one line. They are not. There are two independent questions:
- Ranking
- Recommendation
- Training
- Vision
- Serving
Approximate Nearest Neighbor (ANN) Search
The serving primitive of embedding systems. Once everything is an embedding, every retrieval problem reduces to: given query vector q, find the K vectors nearest to q among N =…
- Embeddings
- Training
- Retrieval
- Graphs
- Serving
Contrastive Learning — InfoNCE, Triplet, Siamese
Goal: learn an encoder $f(\cdot)$ that maps raw objects (images, sentences, user behavior histories, accounts) to vectors such that semantically same things land close together…
- Embeddings
- Training
- Retrieval
- Transformers
- Vision
Negative Sampling and Hard Negatives
In retrieval, metric learning, and link prediction you almost never have explicit negatives — you have positives (clicks, matched pairs, edges) and an ocean of "everything else.…
- Training
- Embeddings
- Retrieval
- Graphs
- Evaluation
Softmax, Sampled Softmax, and the logQ Correction
This note explains, from scratch, the three terms that appear constantly in retrieval papers: softmax over a huge catalog, sampled softmax / in-batch negatives, and the logQ cor…
- Embeddings
- Training
- Retrieval
- Recommendation
- Serving
Transformers from Scratch
The Transformer is the encoder you'll propose for any sequence: words, user click histories (Sequence Recommenders - DIN, SASRec, BERT4Rec), account event streams (AM-05 The Sia…
- Training
- Embeddings
- Transformers
- Retrieval
- Recommendation
Feature Interaction Models — Wide&Deep, DeepFM, DCN, DLRM
These are the heavy rankers: models that score a few hundred candidates with rich features. Their shared obsession: feature crosses.
- Ranking
- Recommendation
- Embeddings
- Transformers
- Serving
Multi-Task Learning — Shared Bottom, MMoE, PLE
Real rankers predict many outcomes at once: pClick, pLike, pComment, pShare, pDwell>30s, pHide, pReport. Why multi-task instead of seven separate models?
- Ranking
- Recommendation
- Training
- Serving
- Fraud
The Two-Stage (really Multi-Stage) RecSys Funnel
Every industrial recommender — LinkedIn feed, YouTube, Roblox discovery, ads — has the same skeleton, because of one arithmetic fact: you cannot run an expensive model on 10⁸ it…
- Recommendation
- Ranking
- Embeddings
- Retrieval
- LLMs
Two-Tower Retrieval Networks (Dual Encoders)
The workhorse of Stage-1 retrieval everywhere (and the template for account-matching candidate generation).
- Recommendation
- Ranking
- Embeddings
- Retrieval
- LLMs
BM25, Dense Retrieval, and Hybrid Search
BM25 scores a document for a query by summing, over query terms, a TF-IDF-like quantity:
- Retrieval
- LLMs
- Embeddings
- Transformers
- Training
RAG System Design (Databricks favorite)
Retrieval-Augmented Generation: ground an LLM's answers in retrieved enterprise documents. The interview is 20% LLM, 80% retrieval + evaluation + ops.
- Retrieval
- LLMs
- Embeddings
- Evaluation
- Ranking
Graphs and Message Passing from Scratch
Some data's most important information is relational: who connects to whom. LinkedIn's member graph, Roblox's friend/trade graph, the account–device–IP identity graph, transacti…
- Graphs
- Transformers
- Fraud
- Embeddings
- Training
Heterogeneous Graphs and R-GCN
Real industrial graphs have multiple node types and edge types. The account-matching identity graph:
- Graphs
- Transformers
- Embeddings
- Serving
- Training
Anatomy of a Fraud Detection System
The generic trust-&-safety architecture. Account matching is a specialization of this; payment fraud, spam, bot detection are siblings.
- Fraud
- Ranking
- Graphs
- Serving
- Evaluation
CLIP and Multimodal Models
Train on ~400M (image, caption) pairs from the web with a symmetric InfoNCE loss over the batch: each image must pick out its own caption among all captions in the batch (and vi…
- Vision
- Embeddings
- Retrieval
- LLMs
- Training
Planning, Imitation Learning, and the Waymo Stack
The long tail rules AV ML: millions of miles are boring; value is in rare events. The loop:
- Vision
- LLMs
- Serving
- Training
- Embeddings
BEV Representations and Sensor Fusion
A camera image is a perspective projection — depth is lost. To place camera evidence in the bird's-eye-view plane you must reason about depth. Two canonical mechanisms:
- Vision
- Transformers
- Evaluation
Lidar Networks — PointNet to CenterPoint
Lidar returns a point cloud: ~100–300k points per sweep, each (x, y, z, intensity). Two properties break standard nets: it's an unordered set (any permutation is the same scene…
- Vision
- Evaluation
- Transformers
Motion Prediction — VectorNet and Beyond (Waymo core)
Task: given each agent's past track + the HD map + traffic context, predict each agent's next ~8 seconds. Sits between perception and planning; Waymo wrote the canonical papers,…
- Vision
- Embeddings
- Transformers
- Graphs
- Training
CNNs and ResNet
A convolutional layer slides a small learned filter (e.g., 3×3×C) across the image, computing a dot product at each location → a feature map. Three built-in assumptions (inducti…
- Vision
- Training
- Transformers
- LLMs
- Graphs
Object Detection — R-CNN to DETR
Task: localize (boxes) + classify every object. Know the two-stage/one-stage/set-prediction trichotomy and the shared vocabulary: IoU (intersection-over-union of boxes), NMS (no…
- Vision
- Transformers
- Training
- Evaluation
- Serving
Vision Transformers (ViT)
Cut the image into 16×16 patches; linearly project each patch to a vector ("patch embedding"); add position embeddings; prepend a learnable [CLS] token; run a standard Transform…
- Vision
- Transformers
- Embeddings
- Training
- LLMs
Anomaly Detection Methods
When labels are scarce or the attack is novel, you detect deviation from normal instead of similarity to known-bad. Taxonomy by how much supervision you have:
- Fraud
- Embeddings
- Vision
- Training
- Retrieval
GCN, GraphSAGE, GAT
The three names you must distinguish crisply. All are instances of message passing; they differ in how they aggregate and whether they generalize to unseen nodes.
- Graphs
- Training
- Embeddings
- Transformers
- Fraud
Link Prediction
Task: given a graph, predict missing/future edges. Two flagship applications in your interviews: PYMK ("People You May Know" — predict future friendships, LinkedIn Prep) and sam…
- Graphs
- Embeddings
- Training
- Evaluation
- Retrieval
Scaling GNNs — PinSage and Sampling
Full-graph GNN training needs the whole adjacency + all activations in memory — impossible at 10⁹ nodes / 10¹⁰ edges. Know four scaling strategies plus the PinSage case study.
- Graphs
- Training
- Serving
- Embeddings
- Retrieval
Bi-Encoder vs Cross-Encoder
The single most reusable architectural dichotomy in this vault. It is the same tradeoff as retrieval-vs-ranking in recsys, and Layer-1-vs-Layer-2 in account matching.
- Retrieval
- LLMs
- Embeddings
- Ranking
- Recommendation
Fine-Tuning — LoRA and Friends
Updating all weights of a 70B model needs the weights + gradients + optimizer states ≈ 16 bytes/param with Adam (fp16 weight 2 + grad 2 + fp32 master 4 + two moments 8) ≈ over a…
- LLMs
- Retrieval
- Training
- Ranking
- Transformers
LLM Serving Internals (Databricks favorite)
Why LLM inference is its own systems discipline, in five concepts.
- LLMs
- Retrieval
- Serving
- Transformers
- Evaluation
Cold Start and Exploration
Every recsys interview ends with "what about new users/items?" Have a layered answer.
- Recommendation
- Ranking
- LLMs
- Embeddings
- Retrieval
Position Bias and Counterfactual Learning
Training data for rankers comes from logged impressions of the previous ranker. Click probability factors (to first order) as:
- Ranking
- Recommendation
- Vision
- Training
- LLMs
YouTube RecSys Case Study (the two papers everyone cites)
Knowing these two papers cold lets you anchor almost any recsys interview, especially at Google.
- Recommendation
- Ranking
- Embeddings
- Retrieval
- Serving
Sequence Recommenders — DIN, SASRec, BERT4Rec
A user is not a bag of features; a user is a sequence of actions. Order and recency carry intent: someone who viewed {crib, stroller, bottle} yesterday is in a different state t…
- Recommendation
- Ranking
- Transformers
- Retrieval
- Embeddings
Class Imbalance, Calibration, and PU Learning
Three intertwined topics that dominate fraud/abuse interviews.
- Training
- Embeddings
- Evaluation
- Fraud
- Ranking
Loss Functions Field Guide
Every architecture in this vault is "encoder + loss." Interviewers care that you pick the loss that matches the decision the system makes. This note covers each loss, its formul…
- Training
- Embeddings
- Fraud
- Evaluation
- Retrieval
Embeddings from First Principles
Neural networks consume vectors of real numbers. But most industrial data is categorical: user IDs, item IDs, words, device models, country codes. How do you feed "user84629173"…
- Embeddings
- Training
- Retrieval
- Recommendation
- Transformers