Andrew Ng

Arize's team teaches agent evaluation as structured error analysis: instrument the agent for step-level observability, then choose evaluators, test examples and metrics per component as well as end-to-end, mixing deterministic code checks with LLM-as-judge for open-ended output.

The case for having agents write the tests around code agents write, backed by real misbehaviour: infra bugs that took humans weeks to find, a password-reset security hole added for convenience, reward hacking on test code, and an agent running rm *.py in a working directory.

Ng argues the "10x" pattern was confined to engineering because most jobs have a physical ceiling on output. As AI absorbs information-processing work, he expects 2x-and-climbing gaps to spread across knowledge roles, with leverage coming from decisions rather than throughput.

Ng's counter to context-maximalism: when you can cheaply judge the result, throw a terse prompt — or just the stack trace — at the model and iterate. He names the conditions where it fails, including specs so partial no competent human could infer the intent either.

Hugging Face's smolagents course argues code agents beat sequential tool calls by emitting one executable action block, cutting round trips and failure surface. Covers when that wins, safe execution via a constrained Python interpreter and E2B sandboxing, and tracing to debug the loop.

A US district court ruled that training LLMs on copyrighted books is fair use, comparing the authors' complaint to objecting that schoolchildren learn to write. Read alongside the argument that data access, chips and regulatory capture are the real brakes on AI progress.

With agentic coding compressing the cost of building to a spec, the constraint moves upstream to deciding what to build. Ng calls it the Product Management Bottleneck and argues teams need product intuition fast enough to keep pace with how quickly agents ship.

GRPO, the algorithm behind DeepSeek's R1, replaces labeled training examples with Python reward functions — a fit for tasks with verifiable outcomes like math and debugging. It can work with under 100 examples and lifts reasoning in smaller, cheaper models.

Ng's case against treating evals as a large up-front project: start with five examples and unoptimized metrics, then improve the metrics and the system together, progressively shifting the judging burden off humans and onto automation.

A dottxt-built course separating the approaches to schema-conformant LLM output: vendor structured-output APIs, re-prompting libraries such as instructor, and constrained decoding that blocks tokens violating the schema at each step. Built around a social-media analysis agent.

A three-course PyTorch certificate taught by Laurence Moroney: fundamentals of tensors, datasets and training loops; then profiling, learning-rate schedulers, overfitting and Optuna-based tuning across TorchVision and Hugging Face; then Siamese nets, ResNet, DenseNet, transformers and deployment.

A LiveKit-built course walking the production voice-agent stack: speech-to-text, text-to-speech and LLM composition, where latency accumulates at each hop, modular pipelines versus speech-to-speech APIs, WebRTC transport, and a cloud deployment serving many concurrent callers.

Ng's argument from Davos: scattered bottom-up AI experiments have failed to pay off. Using a loan pipeline as the worked example, he shows why replacing one manual review with an agent is a minor efficiency gain, and why compressing the whole flow to ten minutes changes the product itself.

Ng on the control problem in voice systems: realtime voice-to-voice APIs make prototyping trivial and low-stakes conversation workable, but you lose the checkpoints text pipelines provide — verifying a refund against policy, confirming the API call actually fired — before the user hears it.

Google Cloud and IBM Research teach the Agent2Agent protocol: wrapping ADK and LangGraph agents as A2A servers so they discover and call each other, chaining them sequentially and hierarchically, pulling external data through MCP, and deploying on IBM's open-source Agent Stack.

Ng ranks agent speedup by function: frontend gains most, since models are fluent in React and TypeScript and can close the loop by driving a browser; backend lags on corner cases and subtle data corruption; migrations still demand care; research benefits least.

Ng on the weak joint in voice agents: voice activity detection inherited the turn-taking paradigm from text chatbots, and because it is trained to detect human speech it cannot tell the user from the next table over. Car noise is easy; a crowded room breaks turn boundaries and interruptions.

A VocalBridge course on bolting voice onto an agent you already have: roughly ten lines to give it speech without rewriting prompts, RAG or tools; outbound calls through a make_phone_call function with transcripts streaming back; and call-scoring evals to catch quality regressions.

Andrew Ng argues agent breaches stem from weak sandboxing and describes OpenWorker's approach on NVIDIA OpenShell: pass in only task-relevant files, block secrets and arbitrary web access by default, enforce limits in deterministic code, and log every action.

Andrew Ng draws on AI Fund's cadence of roughly one startup per month to outline speed-focused practices for building with current AI tools, and where he sees opportunity across the AI stack.