Skip to content
Main Site News Console

What Frontier Labs Are Buying: A Deep Dive into Data with Handshake AI — Garrett Lord & Team

· Latent Space Translated
播客深度访谈

Handshake scaled from launch to $1 billion in just one year, pushing the boundaries of expert data for frontier labs. This is our first deep dive into a data lab, featuring concrete case studies across finance, biology, and alignment.

swyx .ai

Nov 9 · 164 min

Watch on YouTube ↗

This is Latent Space’s first deep dive into an AI data company. In just about a year, Handshake’s AI training business grew from launch to $1 billion in gross annualized revenue. What are frontier labs procuring at such a massive scale? On which evals do models still fall far short of human-level performance? Garrett Lord brought along the team building these reinforcement learning environments to show us live. For the first time, we’re doing hands-on, live demos!

To cover the sheer breadth of Handshake’s business, we teamed up with podcast hosts across science, engineering, research, and finance for this very special crossover episode, as Handshake’s reach spans all of these domains and beyond.

Following the general introduction, look out for these three sections:

  • Finance. In the finance demo, an agent has to locate its task in a bank’s inbox, figure out which documents drive the deal, and deliver a workbook. Pay close attention to what happens with a line item from a client email regarding existing goodwill. The agent pulled the number into Python code and subtracted it. Collin points out that when asked directly, the model can explain its tax treatment. The error occurs when it must recognize that this specific line of code requires applying that rule.

  • Biology. Nick Magazine and Jonas Mueller showcase VIALS, Handshake’s benchmark for interpreting images used by working scientists. Nick displays nine brain tumor histology slides on screen and asks which two exhibit the most severe vessel necrosis. Take a moment to think it through before seeing his answer. Nick chose H and I. The model picked both as well, but threw in an extra G. Watching Nick point out what was missing from the tissue is far more illuminating than looking at the score alone.

  • Coding. In a simulated file access task, the team can inspect which files the agent read, whether it verified copies, and what it did with the shell history afterward. Looking only at the transcript of the agent’s final response misses most of what Steve Kelling and Nishant Ranka are actually evaluating.

  • 🟣 AI Data Business — Garrett Lord, Sahil Bhaiwala, and Nishant Ranka discuss the expert data market at the beginning and end of the show.

  • 🌸 Life Sciences — Nick Magazine and Jonas Mueller demo gel electrophoresis and brain tissue section tasks on screen.

  • 🟢 Finance — Collin Schweiker and Sahil Bhaiwala walk through Project Falcon and its Excel auto-grader.

  • 🟠 Alignment — Steve Kelling and Nishant Ranka trace agent tool calls in a safety simulation environment.

Show Notes

  • ATLAS-Finance — Explore this benchmark designed for agents working across a simulated bank’s inbox, documents, and financial models.

  • Inside a Bank — Handshake traces another public transaction, “Project Osprey,” from the client email all the way to grading decisions. On screen during the interview, we see Project Falcon.

  • VIALS Sample Tasks — Try these scientific image tasks, including the gel electrophoresis and brain tissue tasks featured in this episode.

  • How Handshake Trains Models to Interpret Scientific Figures — The team shares how they use intermediate image descriptions to train and evaluate vision-language models (VLMs).

Timestamps

0:00 Deep dive into Handshake AI
0:39 Garrett, Sahil, and Nishant on frontier training data
5:15 How to build a banking environment for agents
13:07 How frontier labs procure data
15:51

#播客#深度访谈#4Allapi.com

Published by the 4All API team

Original link:https://latent.space/p/handshake/

Need a mainstream LLM API? 4All API gives you one key to call OpenAI, Anthropic, Google Gemini, Qwen, DeepSeek, and dozens more — at official-pass-through pricing, with enterprise-grade reliability, integrated in 5 minutes.

Sign up for the 4All API console →