Jason Chen

Focus

Fused Embedding Kernel

Scoring a candidate means reading its embedding and reducing it against the user's. As two kernels, the gathered embeddings are written to memory and immediately read back. Fused, they never leave registers. The arithmetic is trivial — the whole problem is memory traffic, so removing passes removes time.

Gains compose multiplicatively, so the stack matters more than any one trick. Fusion and precision are exact byte ratios you can derive from the shapes; the microarchitectural factors are typical magnitudes for a bandwidth-bound gather kernel, not measurements. All of it is chasing the same ceiling — a dot product does two FLOPs per element read, so arithmetic intensity sits far below the ~10–20 FLOP/byte where an accelerator turns compute-bound. Stacked, this class of optimization lands around an order of magnitude at the kernel level; end-to-end gains land nearer 2× once the stages it never touches start to dominate.

Architecture

Retrieval Funnel

Millions of candidates in, a handful out. Both retrieval paths run in parallel, then merge. Drag a stage — everything the early stages drop is GPU time never spent.

Illustrative model. Per-stage cost is a fixed price per candidate examined.

Bounded Top-K Buffer

Ranked candidates arrive faster than the latency budget allows sorting. Keep the best K — but pay neither a full sort nor a heap update per item. Admit into a buffer of m·K; when it fills, partition once and keep the best K. The discarded half is overwritten, never reallocated.

Raising m buys fewer partitions with linearly more memory. Worst case the partition work amortizes to m/(m-1) per admitted item — 2.00 at m=2, 1.50 at m=3. In practice the boundary suppresses admissions so hard that admits grow like K·ln(N/K), and partitions are rarer still. Average O(N) overall, against O(N log K) for a heap.

Bloom Filter

Learned retrieval optimises for relevance, but eligibility is a hard constraint — who an advertiser will pay to reach, what budget is left, what policy allows. A model can happily surface a candidate that fails all three. Checking exactly is expensive on high-cardinality attributes, and an ineligible candidate that survives doesn't just waste compute: it occupies a slot in a fixed-width funnel that an eligible one could have had. A Bloom filter answers "definitely not present" in constant time and constant space, paying for it with false positives. Cheap rejects up front, exact check only on what survives — the filter is never the authority.

False positive rate is (1 − e^(−kn/m))^k, minimised at k = (m/n)·ln2. Measured value samples 4,000 absent keys. More hashes help until they fill the array, then hurt — the reason a filter has to be sized against the load it will actually see. The closed form assumes perfectly independent hashing, so once the array saturates it drifts from what any concrete hash function really does; showing both numbers is the point.

Request Slicing

To spend more capacity on one request you split it across workers — and the split has to be exactly disjoint. Slicing on index position is cheap, until replicas refresh on their own schedules and the same item sits at a different position on each. Slices then overlap and gaps open: candidates scored twice, others silently dropped. Slicing on a stable attribute of the item costs more and is worth it, because a retrieval path that quietly loses candidates is worse than a slower one.

Position-based slicing is correct only while every replica agrees on where an item lives. Staggered index updates break that agreement, and the failure is silent — no error, just a quieter funnel. Keying the split on an attribute that travels with the item makes the partition replica-independent, which is the property that actually matters.

Interleaved Weighted Round Robin

Weighted sources, one ordered output. In round r, every queue with weight ≥ r emits once — so weight sets share, not position, and nothing arrives in a burst. This is the shape of running several retrieval models in parallel and deciding how much of the result each one owns.

Deterministic, and order-preserving within each queue.

Unique-Value Blender

Supplementary sources keep surfacing candidates the main queue already has. Naive dedup credits whoever emits first. This one remembers main membership and reattributes the duplicate, so a supplementary slot is only spent on genuinely new supply. The catch: main is a bounded top-K filled on the fly, so membership is only knowable inside the window.

Reattribution costs the supplementary source its slot but not its credit. Items evicted from the bounded window before a supplementary match arrives cannot be reattributed — those are counted separately.

Selected Work

1. Neural retrieval, in production

   Built serving integration for an embedding-based matcher: request-level experiment configuration, dry-run evaluation, statistics transport, embedding and threshold sync. Added recall-aware dynamic thresholds.

2. Inference reliability on GPU

   Model and data staleness tracking, bounded concurrency on embedding downloads, model-loading fail-safes, and guards blocking experimental models from fallback tiers.

3. A cheaper first stage

   Introduced a lightweight matching stage ahead of the main retrieval path. Onboarded matchers, added cross-stage data passing, reduced CPU per request.

4. Shared multi-stage ranking layer

   Replaced per-route heuristics with one ordering and selection interface spanning four ranking stages. Onboarded partner teams across multiple surfaces.

5. Configuration tuning, automated

   Gradient-based system that proposes, ramps, and evaluates serving configurations — replacing hand-authored experiments.

6. Request-level tracing

   Per-source observability through the serving path: queue sizes, emitted volume, deduplication and attribution, plumbed into request traces.

Background

2021 — present

Meta · Staff Software Engineer, Retrieval Platform · New York

Georgia Tech

M.S. Computer Science

Florida

B.S. Computer Science

USAMO

Finalist