Vibeleaderboard
← All Intel
Intel / video

AI & Cybersecurity: Dan Boneh Interviews Nicolas Carlini

Source
youtube.com
Author
Stanford Online
Date

Context

Hiding a 's weights (its learned numerical settings) does not reliably protect it from adversarial examples, because inputs crafted against one model often work on another, Nicholas Carlini tells Stanford's Dan Boneh in this interview. An adversarial example is an input altered to make a model misbehave, such as a picture changed so a classifier labels a cat as guacamole. Common attack methods need the weights to compute the alteration, so secrecy sounds protective. Carlini says an attacker can build the example on a different model, trained on different data, and replay it on the target. It works more often when architectures and data are similar, and querying the target to adapt improves the odds.

Carlini is explicit about the limit: transfer attacks do not always succeed, so secrecy raises an attacker's cost, as spam defenses do, without delivering robustness. Carlini, whom the interview's introduction places at Anthropic working on security, adds that such attacks matter more now that language models make real decisions, such as screening uploads.

Terms in this piece · Glossary
  • Model — A computational system that uses learned patterns to produce an output from information supplied to it.
  • Training — The process that adjusts a model using examples and a measure of how well it performs.
  • Large language model — A model trained on large amounts of data to process and generate language.
Read the source www.youtube.com
More from Stanford Online
Recommended reads
Comments

Checking sign-in…

Loading comments…