Speaker 1: Bloomberg Audio Studios, Podcasts, Radio News. Speaker 2: Hello and welcome to another episode of the Odd Lots podcast. Speaker 2: I'm Jill Wisenthal, and I'm Tracy all the way, Tracy. Speaker 2: One thing about AI is that lots of lines that Speaker 2: go up. Speaker 3: Yes, famously, there is perhaps one line that has captured Speaker 3: the attention more than others when it comes to lines Speaker 3: going up. Speaker 2: Yes, but we're recording as April seven. Did you see Speaker 2: the anthropic revenue chart, by the way, Oh. Speaker 4: It's just extreme. Speaker 2: Okay, it's just the number of lines going up. I Speaker 2: mean there are some really. Speaker 3: Let me caveat that. Up until recently, there was one Speaker 3: chart of a line going up exponentially that became I Speaker 3: think it's fair to say the most viral chart in AI. Speaker 4: Right, Yes, I would absolutely agree with that. Speaker 2: So one of the many lines that go up, or Speaker 2: there are various lines that sort of capture This is Speaker 2: essentially just measures of AI progress of what they could do, Speaker 2: what the models are capable of, and so forth. And Speaker 2: you know, there's all different benchmarks out there and hobbyist Speaker 2: benchmark creators, et cetera, all kinds of benchmarks out there. Speaker 2: Organization called Meter based out in San Francisco, and they Speaker 2: measure how well AI models are doing at various sort Speaker 2: of engineering tasks, et cetera. And they have these charts Speaker 2: showing how long, you know, certain tasks, how long would Speaker 2: take a human to do them, and then whether AI Speaker 2: could do them. And yes, the lines just almost vertical. Speaker 2: I think there was someone one of the ones that Speaker 2: came out maybe very early this year or late last year, Speaker 2: showing the latest Claude model. Speaker 4: It, yes, like this is crazy. Speaker 3: When I look at these charts, they're called time horizon charts. Speaker 3: When I look at them, I like, intuitively I kind Speaker 3: of understand what they're saying, and you can kind of Speaker 3: see the leap in progress between some of the previous Speaker 3: models and Claude, right the latest claud model. And that's Speaker 3: what got everyone excited, was you had this big exponential Speaker 3: shift up in the capability of that particular AI model. Speaker 3: But then when I start like diving into what it Speaker 3: actually says on Meter's website about what these charts represent, Speaker 3: I start getting really confused. I know everyone wants to Speaker 3: get excited about AI and charts going up in general, Speaker 3: but I think there's a lot of nuance here and Speaker 3: we should probably talk about it, because the other thing Speaker 3: going on with meter right now is they've become sort Speaker 3: of the industry standard benchmark, and so a lot of Speaker 3: investment decisions are being based on these charts. And if Speaker 3: you oversimplify them as just like okay, lines going up Speaker 3: and then suddenly it goes up even more, obviously, people Speaker 3: are going to start to get like maybe a little Speaker 3: over excited. Speaker 2: Can I see one other thing too that I'm very Speaker 2: curious about, Like, I'm really glad that there are people Speaker 2: designing various benchmarks for measuring a progress. Seems like an Speaker 2: important thing to get a handle on. But like if Speaker 2: I were, like, say, like talented or smart enough to Speaker 2: be like doing these things, I would go work for Speaker 2: one of the labs and make ten million dollars a Speaker 2: year or something like that. And so I'm actually curious Speaker 2: to a lot of the nonprofits, et cetera. It's like, Speaker 2: do you really want to be like working at the Speaker 2: cutting edge of AI in a nonprofit? I mean, I Speaker 2: guess Open Eyes owned by a nonprofit weirdly enough, but Speaker 2: you know what I'm saying, Like I would want the money. Speaker 3: We should talk about it with our guests who are Speaker 3: currently sitting right here. Speaker 4: That's exactly right. Speaker 2: I'm very excited to say we have the two perfect Speaker 2: guests to talk about the best of viral and maybe Speaker 2: important chart in AI. Right now, we're going to be Speaker 2: speaking with Joel Becker. He is a member of the Speaker 2: technical staff at METER, and we're also going to be Speaker 2: speaking with Chris Painter, the president of METER. So, uh, Speaker 2: Joel and Chris, thank you so much for coming on oud, Speaker 2: lots for having us. Yeah, really excited to chat with Speaker 2: both of you. Chris, and you're the president. I'll start Speaker 2: with you, like, what is METER? How long has it Speaker 2: been around? What is this organization? What's its goal? Just Speaker 2: give us the sort of six sixty second synopsis of Meter. Speaker 5: Yeah, totally. Speaker 6: I can try and you know, sometimes I give a Speaker 6: long version. I can try and do a short version here. Speaker 6: So Meter is a research nonprofit based in the Bay Area, Speaker 6: like you said, dedicated to advancing the science of measuring Speaker 6: whether and when AI systems might pose catastrophic risks to Speaker 6: humanity as a whole, focused specifically on threats that come Speaker 6: from AI autonomy or AI systems themselves. So when you Speaker 6: talk about there's kind of this whole field and AI Speaker 6: of dangerous capability evaluations. People seeing ken this AI system Speaker 6: assist with a chemical or biological weapon attack, can it advance? Speaker 6: Kind of like bad actors' ability to execute cyber attacks Speaker 6: on a really large scale. METER is sort of specialized Speaker 6: in specifically assessing how autonomous are AI systems, what is Speaker 6: the scale and like length and difficulty of tasks that Speaker 6: they're able to do by themselves, partially because we think Speaker 6: it sets the stakes for conversations about AI misalignment. So Speaker 6: we sort of see ourselve being on the hook for Speaker 6: at any given point in time, giving humanity the bits Speaker 6: of evidence that are most informative for establishing the stakes Speaker 6: of are we reliant on AI systems as a society Speaker 6: in a way that could make it really bad if Speaker 6: they are misaligned. Speaker 3: I'm going to let Joe ask the question about why Speaker 3: you're both working in a nonprofit instead of one of Speaker 3: the labs later, But one question I do have is Speaker 3: when I think of METER, you guys always come up Speaker 3: in the context of these time horizon charts. And I Speaker 3: don't mean this as an insult or anything, but I Speaker 3: hardly ever hear anyone talk about the actual safety aspect Speaker 3: of your mission. Why do you think that is. Speaker 6: Yeah, So I think there's some distinction between our motive Speaker 6: for assessing time horizons and the kind of how it Speaker 6: gets used then by the rest of the world or Speaker 6: kind of like what the origin of the rest of Speaker 6: the world's interest in it for meter. I think the Speaker 6: reason that we work on things like the time horizon Speaker 6: charts is because if we're trying to establish the stakes Speaker 6: for talking about could AI systems go rogue or one Speaker 6: day could they like try to take over and subvert Speaker 6: human control? Three years ago, if you went back to Speaker 6: around when it meters started about fourish years ago, and Speaker 6: if you it was started by Beth Barnes Paul Cristiano Speaker 6: and this was kind of the initial motive. Is if Speaker 6: you went back then and you said, why don't I Speaker 6: think that AI systems are going to go rogue and Speaker 6: like take over or overthrow humanity today the kind of Speaker 6: most intuitive you know, you can come up with a Speaker 6: lot of abstract reasons debates about the goals AI systems Speaker 6: might or might not eventually have, but the kind of Speaker 6: most damning in the moment reason is the AI system Speaker 6: just can't do much right. It doesn't make sense to Speaker 6: talk about a question answer system that like can't even Speaker 6: reliably answer programming questions saying like is it going to Speaker 6: hack my systems or like backdoor me in some way. Speaker 6: It just doesn't make any sense to talk about. Speaker 3: It's going to write you a poem that you asked. Speaker 6: Right, or won't even at the time they couldn't do Speaker 6: anything for themselves. And so if you're like, kind of Speaker 6: being able to subvert human control depends on agency, And Speaker 6: so we wanted to come up with a measure that Speaker 6: kind of tracks agency over time to kind of say, Speaker 6: when would this argument no longer? When are AI systems Speaker 6: now able to kind of do long, complex enough actions Speaker 6: by themselves that the argument kind of the goalposts almost Speaker 6: move somewhere else to like, well, we would catch the Speaker 6: AIS or the AIS don't want to subvert human control. Speaker 6: And so I agree that there is a distinction between Speaker 6: like how I think partially the exercise of trying to Speaker 6: come up with these measures throws off things that are Speaker 6: very like grounded and intuitive measures of AI progress that Speaker 6: might be more intuitive than just benchmarks. Right, So if Speaker 6: you a lot of people are in the game of Speaker 6: making just benchmarks, where you say, like here's my harm Speaker 6: bench or something the AI gets seventy percent. That's much Speaker 6: less of a kind of grounded or long lasting metric. Speaker 6: Like it's hard to say what that means or how Speaker 6: that generalizes, but the idea with time horizon is like, Speaker 6: maybe it's more intuitive, and I think that helps both Speaker 6: for safety and for like business understanding. Speaker 4: So let's talk about what this charge. Speaker 2: I go the main chart here at meter dot org, Speaker 2: right on the front page, it's this time horizon chart, Speaker 2: and it shows Claude Opus four point six as a Speaker 2: February twenty twenty six able to complete a task length Speaker 2: in eleven hours and fifty nine minutes with a fifty Speaker 2: percent success rate. I have to admit, the first time Speaker 2: I saw this chart or version of this chart, what Speaker 2: I assume, and I suspect others assume, is that it Speaker 2: was able to go off and work on a task Speaker 2: for eleven hours and fifty nine minutes then come back Speaker 2: with an answers. But apparently it's not that. What do Speaker 2: you walk us through what's really being measured here? By Speaker 2: the way, the previous high was all was GPT five Speaker 2: point three codex. That was five hours of fifty minutes. Speaker 2: So I guess part of the reason this charge is Speaker 2: blew people in mind because literally that's basically a double Speaker 2: But why don't you talk to us about what's really Speaker 2: being measured here? Speaker 7: Yeah, so fundamentally, you know, in simpler terms, we are Speaker 7: plotting the difficulty of tasks the AIS are able to Speaker 7: complete over time. And you know, the particular way that Speaker 7: we measure the difficulty of tasks is in how long Speaker 7: it takes humans to complete those same tasks that we're Speaker 7: asking the AIS to do. So in this case, you know, Speaker 7: talking about for a PUS four point six, something like, Speaker 7: tasks that take humans twelve hours to do, we predict Speaker 7: that it will succeed at those tasks around fifty percent Speaker 7: of the time. And yeah, you know, it turns out Speaker 7: that when you plot using this particular difficulty measure how Speaker 7: performance AIS are relative to how long it takes humans Speaker 7: to complete these tasks, we see an exponential increase in Speaker 7: capabilities for AIS. And what that ends up meaning is Speaker 7: that you keep on having these doublings of capabilities every Speaker 7: let's say four months. Speaker 5: It seems on recent trends. Speaker 7: Where you know, the next model is not merely going Speaker 7: to have necessarily, you know, an hour longer time horizon, Speaker 7: but perhaps be having some multiple of the. Speaker 5: Time horizon of the previous model that's come out. Speaker 2: So then explain how the number of their twelve hours Speaker 2: is established. So there is some engineering task and you say, okay, Speaker 2: this is a test that would require twelve hours, but Speaker 2: humans of all different types of talent capabilities, how do Speaker 2: you establish that? Okay, this was a twelve hour TESK, Speaker 2: this was a six hour tel whatever. Speaker 7: Yeah, So the simple answer is, literally, we get humans Speaker 7: to sit down and complete the tasks that we give Speaker 7: to AIS and as close to identical conditions as possible. Speaker 7: So first we come up with the tasks, and that's Speaker 7: you know, that's whole god a killer fish. We can Speaker 7: talk about exactly how we do that. And then, using Speaker 7: essentially the same tools that we're about to give the AIS, Speaker 7: we take talented humans, you know, not people who have Speaker 7: seen this particular type of task before, but people who Speaker 7: have relevant expertise. So if it's a software engineering task, Speaker 7: you know, they have software engineering expertise. Machine learning task, Speaker 7: they have machine learning expertise, and then we time them, Speaker 7: we see how long it takes for them to complete Speaker 7: those tasks cescfully and then roughly we call the difficulty Speaker 7: of the task as measured in human time to complete, Speaker 7: as the average time it took these humans to complete Speaker 7: the task. Then we'll run the AIS on this same Speaker 7: set of tasks. Typically today for the very easiest tasks Speaker 7: that they're more or less always going to succeed, there's Speaker 7: some mid range of tasks where you know, perhaps they Speaker 7: succeed fifty percent of the time, or perhaps for some Speaker 7: tasks in that range they succeed zero percent of the Speaker 7: time and for others one hundred percent of the time. Speaker 7: And so they're getting fifty percent on average, let's say, Speaker 7: and then for the much harder tasks, perhaps they're getting Speaker 7: closer to zero percent. And then the point of which Speaker 7: we predict, you know, in the middle of all these Speaker 7: zero percents and one hundred percents by task, the point Speaker 7: at which we predict that they'd have a fifty percent Speaker 7: chance of succeeding. That is, either a fifty percent chance Speaker 7: of succeeding on some task or fifty percent of the Speaker 7: tasks or of that's difficulty that we think they would Speaker 7: succeed on. That's what we're going to call the time Speaker 7: horizon of these models. Speaker 6: I think one thing also that could be good to explain. Speaker 6: Here is the task distribution. I mean, this is not Speaker 6: this is not all activities that humans do. We are Speaker 6: specifically here interested in or the like. There's some question Speaker 6: in what tasks are you know, like Joel mentioned, we're Speaker 6: having people come into our office do the task to Speaker 6: get a sense of how long it takes. We're not Speaker 6: having them come in and like, you know, paint paintings Speaker 6: or write novels or you know. We're focused here specifically Speaker 6: on things that are in the distribution of work that Speaker 6: a engineer at a like. We like to think of Speaker 6: it as like a frontier AI lab, the tasks that Speaker 6: they might be doing. So this is things like software engineering. Speaker 6: It's fine tuning at AI models, it is like software Speaker 6: machine learning, that kind of task. Speaker 3: Wait, can I just ask why did you decide to Speaker 3: focus on engineering? Because you could have winded out to Speaker 3: you know, if we're talking about AI being capable of, Speaker 3: you know, taking over the world, there are all sorts Speaker 3: of substantive tasks that would fall under that category. So Speaker 3: why just do engineering? Speaker 6: Yeah, I think that for one thing, maybe other people Speaker 6: in the team or maybe Jolis thoughts about this, but Speaker 6: I think my particular motive and being interested in the Speaker 6: time horizon on software tasks is that first of all, Speaker 6: it's the thing that the industry is very like already Speaker 6: even before we started working on this, is very focused on. Speaker 6: So it's one of the capabilities that you should expect Speaker 6: to come along for the ride earliest. It's the thing Speaker 6: that like a lot of optimization pressure is being exerted on. Speaker 6: And then I think that it is kind of the Speaker 6: like thing that you would expect as an early warning Speaker 6: kind of sign of this AR and D automation. So Speaker 6: to some extent, METER thinks of itself as trying to Speaker 6: build you know, science that are advanced science that can Speaker 6: say when are we getting to the point that aisism Speaker 6: could improve themselves or speed up the pace of AI development. Speaker 6: When will AI research kind of feed on itself? And Speaker 6: the kind of core capability for that might be software Speaker 6: engineering and machine le learning research ability. There are other Speaker 6: skills that could be relevant to taking over the world. Speaker 6: I think other people have done time rising some like Speaker 6: cybersecurity sense. Speaker 3: But I suppose it is true like the basilisk isn't Speaker 3: going to paint its way into like power or something Speaker 3: like that. Okay, it might. Speaker 2: Deceive you it might be very convincing or cunning in Speaker 2: some way, and handover the cues. Speaker 5: I always say for your mental models. Speaker 7: You know, we don't have perfect evidence of this whatsoever, Speaker 7: but my rough sense, sort of colloquially or you know, Speaker 7: my prior before evidence comes in, is that if we Speaker 7: did study tasks on these very different distributions, you know, Speaker 7: not machine learning, not software engineering. I'm not sure about Speaker 7: painting exactly, but you know, perhaps or other kinds of Speaker 7: task distributions that we could enumerate that basically we would Speaker 7: see this similarly shaped exponential progress over time where every Speaker 7: I'm not sure exactly, but let's say, you know, for month, Speaker 7: six months, something like that, the level of capabilities as Speaker 7: measured in time horizon would be doubling at something like Speaker 7: that pace, maybe from a much lower level. So you know, Speaker 7: one example that we do have better evidence of is Speaker 7: that the ais today are much less performance at you know, Speaker 7: anything that requires vision capabilities, seeing what's on a screen, Speaker 7: clicking around at a computer, but they're getting you know, Speaker 7: tremendously better that sort of thing over time. Speaker 5: I just do mention quickly. Speaker 6: We did actually do a very kind of brief investigation Speaker 6: of this another task distributions that's on our website somewhere Speaker 6: like cross domain time horizons. I think we looked at Speaker 6: data from the Tesla's shared on self driving and forgetting Speaker 6: the other there's like os world. Maybe some of these Speaker 6: are like somewhat similar, still kind of in the distribution Speaker 6: of software tasks, but trying to get further afield into Speaker 6: things like vision. Speaker 3: How big is the sample size on the humans who Speaker 3: are actually doing work? And also is it getting harder Speaker 3: getting like human engineers into the room to compete with Speaker 3: like Claude Opus four point six versus say, if I Speaker 3: was a mediocre engineer, and I'm not, I'm a non Speaker 3: existent engineer, but if I was a mediocre one, I Speaker 3: would like maybe I would feel good about going up Speaker 3: against like GPT three or something, and maybe I would Speaker 3: feel a lot worse about myself going up against like Claude. Speaker 7: Yeah, you know, on these tasks, I'm in a pretty Speaker 7: similar position myself to you. So we have approximately three, Speaker 7: although it varies quite a lot across tasks. Human baselines Speaker 7: per tasks, so you know, typically we're ever going over Speaker 7: something like three. I think the final numbers, it's my Speaker 7: impression that they're not going to be so sensitive to Speaker 7: the particular baselines that we use. Speaker 6: Aren't the longer tests week more weekly baselined. Speaker 7: Yeah, So indeed, I think it will get a lot Speaker 7: harder to baseline these tasks as the length of task Speaker 7: AIS are able to successfully complete gets longer and longer. Speaker 7: You know, you might think at some points the length Speaker 7: of task that they can complete is longer than the Speaker 7: time in four months time, they're going to be able Speaker 7: to complete tasks of more than four months, and then Speaker 7: it's you know, kind of becomes paths close to impossible Speaker 7: to get these four months long baselines. Of course we're Speaker 7: not at that point yet, but you know, definitely has Speaker 7: become more difficult to get these baselines as time has Speaker 7: gone on. Speaker 5: At the moments, not impossible, but very challenging. Joe. Speaker 3: These are the future jobs for displaced engineers, right. It's Speaker 3: competing against the codes for benchmark first benchmark evaluation, we Speaker 3: found the jobs. Speaker 2: So we mentioned at the beginning the most viral CHARLINEI Speaker 2: is this chart that you have on the front of Speaker 2: your website. Your website defaults to this and it shows Speaker 2: you know, this doubling. So if we actually go back Speaker 2: to November let's say November twenty twenty five, Gemini three Speaker 2: pro three hours and forty four minutes, claud Op was Speaker 2: four point six twelve hours. Those are the fifty percent Speaker 2: success benchmark. If we go to the eighty percent benchmark, Speaker 2: which the website doesn't default to improve the price of Speaker 2: improvement looks a little less impressive to me. So okay, Speaker 2: now it's like it does not have the same gap Speaker 2: pretty clearly. Now eighty percent is still not one hundred percent. Speaker 2: And I know that this is your meter's goal is about, Speaker 2: like you know, human safety and all this stuff. But Speaker 2: when we think about people look at this and they Speaker 2: use it as a stand in for how performance are Speaker 2: these models? Even eighty percent, you know, certainly for like Speaker 2: any business application. I understand you're not like serving business Speaker 2: here per se, but probably businesses care about this. Even Speaker 2: eighty percent may not be very good enough. And it Speaker 2: does not look as crazy when you look at the Speaker 2: eighty percent chart as it does at the fifty percent chart. Speaker 2: Why the focus on the fifty percent chart? And given like, Speaker 2: why not look at the chart that just does not Speaker 2: look as impressive. Speaker 5: Yeah, maybe two central things to say. Speaker 7: One to my eyes, the eight percent shot it's basically Speaker 7: does look as impressive. Well, the doubling time is about Speaker 7: the safe cope. Speaker 5: On it's the same. Speaker 6: It's the same, it's say an afseet of it's the Speaker 6: same pace of progress. Speaker 7: You know, it's something like five times smaller than the Speaker 7: fifty percent than the fifty percent number. Speaker 5: But you know that only takes you too doublings. Speaker 7: And if each doubling takes around four months, that means Speaker 7: that in eight months time you're going to have the Speaker 7: same eighty percent success rate roughly as you do fifty Speaker 7: percent success ray today. Speaker 5: That's one thing to say. Speaker 7: Maybe a second thing to say is, you know, remember Speaker 7: at the beginning I said, essentially what we're doing is Speaker 7: plotting the difficulty of tasks that these ais can complete Speaker 7: over time, just with this particular measure that ends up Speaker 7: showing this clean exponential trend. And we've picked a particular Speaker 7: number as our difficulty number, and you know that is Speaker 7: this fifty percent reliability threshold. We could have picked a Speaker 7: different one. I think there are reasons for picking the Speaker 7: fifty percent one. In particular, it's the one that statistically Speaker 7: we're better able to measure. For some technical reasons, it's Speaker 7: the one that shows up in previously. It's show that Speaker 7: there are some couple of a couple of other reasons Speaker 7: why we can go for fifty per cents rather than Speaker 7: eighty percent. Maybe a final thing to say is that Speaker 7: this fifty percent number is sort of equivocating between these tasks. Speaker 7: It's able to complete fifty percent of the time and Speaker 7: fifty percent of the tasks it's able to complete one Speaker 7: hundred percent of the time, and fifty percent it's able Speaker 7: to complete zero percent of the time. And actually, I Speaker 7: think the situation is it's somewhere in between, but it's Speaker 7: a little bit closer to the latter, where there are Speaker 7: some tasks that it's completing with near perfect reliability and Speaker 7: some tasks in that range that it's completing with very Speaker 7: low reliability. And for downstream economic applications or for applications Speaker 7: inside of these maju AI companies or something, you know, Speaker 7: you might think that that's more favorable in some sense, Speaker 7: that there are some of these tasks where we're getting Speaker 7: one hundred percent reliability, even even for very challenging tasks. Speaker 6: I think two other things make maybe it could be Speaker 6: useful to just explain when you said that they are Speaker 6: technical reasons why it's easiest to measure it fifty percent. One, Like, Speaker 6: it is just the case that it is fifty percent Speaker 6: is the point at which it is like least sensitive Speaker 6: to Like the distribution is kind of thickest, right, I mean, Speaker 6: correct me if this is wrong, But my I mean Speaker 6: there are like to resolve something like ninety five percent, Speaker 6: you would need way more samples because then you need Speaker 6: to have some that are like you need way more Speaker 6: simples to be able to resolve that level of precision. Speaker 7: I think there are some caveats to that picture. But Speaker 7: let's say even more extreme. You know, let's say that Speaker 7: we cared about you know, ninety nine percents. In that case, Speaker 7: if we had one percent label noise quotes unquote, yeah Speaker 7: it is you know, if sometimes we were accidentally grading Speaker 7: some of the failing tasks, passing some of the passing Speaker 7: tasks as failing, then we just never be able to Speaker 7: estimate that reliably right, And at fifty percents, this comes Speaker 7: a little bit closer to washing out. Speaker 6: And I think one other intuitive thing here, or one intuition, Speaker 6: is that if you give me a task and you Speaker 6: give me the model, it is the point at which Speaker 6: I think that the model, all you tell me is Speaker 6: the time or the length of task that it takes Speaker 6: a human to do the task. The fifty percent time Speaker 6: horizon is the point at which I think it is Speaker 6: more likely that the model will be able to do Speaker 6: the task than that it can't. And I just find Speaker 6: that intuitive. Speaker 3: Yeah, how much interest do you get on these charts Speaker 3: from potential investors specifically? And the reason I ask is Speaker 3: because I was just messing around and like googling some Speaker 3: stuff and when the OPUS chart, the latest opis chart Speaker 3: came up, someone posted it on Breddit and I think Speaker 3: like the second comment on it was someone going, how Speaker 3: do I invest in open Ai? And like and like Speaker 3: people were they were trying to club together to like Speaker 3: invest in these companies. So clearly there are people out Speaker 3: there who are using these charts as investment tools. Speaker 6: I would say, you know, we don't get an enormous Speaker 6: amount of inbound from investment firms. I mean, sometimes, you know, Speaker 6: vcs or whatever we're based in the Bay area will Speaker 6: reach out to us. I think that there's some kind Speaker 6: of principle of our goal is to inform the public Speaker 6: and give them the best evidence that we can about Speaker 6: when we might get to this point of kind of Speaker 6: you know, AI being you know, fully autonomous or able Speaker 6: to improve itself. And there's some principle at play here Speaker 6: of like I kind of want to enable people to Speaker 6: do whatever they will do with that information, and I Speaker 6: think that we don't engage a ton in kind of Speaker 6: the like business side or investment implication of the work. Speaker 6: One kind of thought experiment I sometimes say to myself Speaker 6: is if I do believe that at some point we're Speaker 6: going to get this AI that's improving itself, and where Speaker 6: like AI research is automated, and you have all these Speaker 6: fears about a singularity, would I rather that like all Speaker 6: of Wall Street like falsely didn't think that was coming Speaker 6: when I believed it was coming, Or would I want Speaker 6: them all to know that it was coming, given that Speaker 6: I believe it's coming, and I think all of human Speaker 6: Maybe this is more a personal view, but I think Speaker 6: if this is possible that we will automate AI research, Speaker 6: I think all of humanity being aware of it, aware Speaker 6: of where we're heading, is sort of a precondition for Speaker 6: us all being able to figure out what to do Speaker 6: about it. And so I don't kind of want like Speaker 6: certain people or one side or one team to kind Speaker 6: of like selectively be in the dark because they might Speaker 6: invest on the basis of this or something like that. Speaker 6: But we don't, you know, it's not where we put Speaker 6: our time. We're focused on informing the public. The public Speaker 6: includes some investors. Speaker 3: So on that note, like what is the actual level Speaker 3: at which we're all presumably supposed to panic or at which, like, Speaker 3: if you're a policy maker, you would start to get Speaker 3: worried about AI being able to automate and improve on Speaker 3: itself in a way that eventually becomes detrimental to humanity. Speaker 7: I don't know exactly what the level is on this time, Speaker 7: horise and measure. I think, you know, one thing to Speaker 7: say is we have made real progress on the science Speaker 7: of measuring these AI systems and how capable they are. Speaker 7: But I think there's a long way to go, and Speaker 7: in an important sense, I think we're behind on this task. Speaker 7: We're measuring some underlying technical trend and at some point Speaker 7: I do think that implies greater risks. Speaker 5: So astonishing things happening. Speaker 7: Although Chris can speak more to other arguments that we Speaker 7: might back out to for why even if AIS are Speaker 7: very capable, we still might not see castrophic dangers emerge Speaker 7: in the short term. Speaker 5: Yeah, I'm sure, you know. Speaker 2: I think part of the reason why the AGI cheddar Speaker 2: has really picked up, particularly in the wake of like Speaker 2: everyone using Chlord code, is it's very easy to emerge Speaker 2: in it. So like you're sitting there, it's like, yeah, Speaker 2: do this, do this. It's like I don't even need Speaker 2: to be here, right, I think you sort of get Speaker 2: a very intuitive feel for like how the human could Speaker 2: come out of the loop. What helps today, because I'm Speaker 2: sure there's been tried. Like if you go to like Speaker 2: ch juput and you say, here's a you hear you Speaker 2: have cloud code access, go build something, and the AI Speaker 2: is what actually happens today when AI is working with A. Speaker 7: Yeah, my sense is that at some point, you know, Speaker 7: further away points than would have been true some time ago, Speaker 7: the AIS will more or less full on their faces Speaker 7: that you know, there are some things they're not so Speaker 7: capable of today, like collaborative hallucinations. Speaker 4: Will they're just like, you know, just like devolved terribles. Speaker 5: Yeah, I think all sorts of ways can go. Speaker 7: You know, at some point they're going to need to Speaker 7: rely on external resources, and today that they're not as Speaker 7: capable at managing these external resources effectively. I think they're Speaker 7: less capable at sort of ideation and sort of self Speaker 7: awareness about where they are in the problem today than Speaker 7: they are at these kind of raw software engineering skills. Speaker 7: You know, you know, as you mentioned, the ways in Speaker 7: which AIS are autonomous today or close to autonomous today, Speaker 7: is the human has the idea and then you know, Speaker 7: submits that idea to cloud code or a code or Speaker 7: one of these other agenticie tools, and then they handle Speaker 7: the software engineering components and possibly there's still still some Speaker 7: intervention after that. I do imagine that the sort of Speaker 7: circle of autonomy or something gets larger over time. I Speaker 7: do think there's no fundamental barrier. It seems to me Speaker 7: today as having those ideas and so we moved to Speaker 7: a great level of abstraction. But if we were purely Speaker 7: relying today on these fully autonomous capabilities, you know, could Speaker 7: you manage research departments, any any apartments of your choice Speaker 7: inside of a major AI company. Speaker 5: No, my guess is probably not. Speaker 3: Actually on this note, this reminds me something I wanted Speaker 3: to ask. So when you look at the domain specific Speaker 3: time horizon charts, so the ones that show like I Speaker 3: think you call them task suites or something like that. Speaker 3: Like I guess productivity by a specific job, and you Speaker 3: see these different lines, So sometimes you see like almost Speaker 3: horizontal lines and sometimes you see squiggly or steeper lines. Speaker 3: What is actually happening there? Like, how are we supposed Speaker 3: to interpret that? Like is this a measurement problem? Or Speaker 3: is it say something very fundamental about like what AI Speaker 3: can and can't do under current conditions. Speaker 6: The thing that I think would be good for Jill Speaker 6: to explain is that I think that there is a Speaker 6: distinction here between will AI like the time horizon charge Speaker 6: doesn't by itself, I think, tell you will productivity in Speaker 6: one specific kind of job increase because of access to AI? Speaker 7: Yeah, maybe one thing to say on that chart showing Speaker 7: the time horizon on these different task distributions relative to Speaker 7: my guesses ahead of time, You know, I think those Speaker 7: time horizons are remarkably similar. I think the doubling times Speaker 7: the pace of progress in AI seems more similar than Speaker 7: I put of guessed to the original trend that we Speaker 7: that we published, although you know, imperfectly so on this Speaker 7: difficulty translating what we might call raw AI capabilities, in Speaker 7: some sense you know, capabilities on benchmarks or something to Speaker 7: real world productivity. I think there are a number of Speaker 7: differences in a number of ways, in particular in which Speaker 7: the benchmark results are overestimating what we might see in Speaker 7: the wild, you know, not not hugely overestimating. I think Speaker 7: we do see that people are getting real utility out Speaker 7: of these modern agentic a IOLs, but overestimating to some extent. Speaker 5: One is that the. Speaker 7: Scoring implicitly is different in real problems, I'm scoring based Speaker 7: on something a bit more holistic than these algorithmic scoring procedures, Speaker 7: these automatic scoring procedures that we're using at Meter and Speaker 7: many other people that are using in the in the Speaker 7: benchmark world. There's some notion of code quality if you're Speaker 7: if you're working in software engineering, but for other tasks Speaker 7: there's there's. Speaker 3: Beautiful code, elegant code. Speaker 5: People talk that yeah, yeah, yeah, for other tasks that's Speaker 5: going to be coding. Speaker 1: This is. Speaker 7: One more thing is that the tasks that come up Speaker 7: in the wild are more likely to be messy in Speaker 7: some sense. They involve working with other people, They involve Speaker 7: working in much larger code bases or sort of more Speaker 7: open ended problems, maybe with something even adversarial going on Speaker 7: in the in the software engineering context, that might be Speaker 7: that someone's trying to make a change to the part Speaker 7: of the code base that you're currently working on and Speaker 7: you need to and you need to work around that, Speaker 7: and we do tend to see that the AIS are Speaker 7: less capable working on these more messy problems. Speaker 5: I don't want to overstate that. Speaker 7: You know, it's not an enormous effect, but you know, Speaker 7: that's one thing that gets in the way of these Speaker 7: productivity increases, you know. And then I do think there's Speaker 7: something to the reliability question right where you know, if Speaker 7: it was true that for a certain type of task Speaker 7: you only had you know, eighty percent reliability, then every Speaker 7: time you're going to need to go back and verify Speaker 7: the work of these AIS, and not only verify the Speaker 7: work of dcais, but without the context of how they Speaker 7: implemented the solution relative to if you went about the Speaker 7: task yourself, you'd already have that in your head, and Speaker 7: so this verification step quote unquote would take less time. Speaker 7: You know, I don't expect these frictions to be sort Speaker 7: of so fundamental in some sense, or I imagine they Speaker 7: go up levels of abstraction I think not only as Speaker 7: the underlying technical progress real, but I think that the Speaker 7: productivity improvements that are also going to show up increasingly. Speaker 5: But yeah, there are these frictions. Speaker 2: Tracy alluded to this question when she asked about VCS Speaker 2: and investor interest. So people see these charts and regardless Speaker 2: of what Meter's point. Speaker 4: Is, like, this is incredible. I got to invest in this. Speaker 2: But this brings me to this broader thing that I Speaker 2: find very strange about AI, which is this kind of odd, Speaker 2: sort of Baptist and bootlegger relationship between the AI labs Speaker 2: people who are building this stuff and the sort of Speaker 2: alignment safety people, and they sort of go back and forth, Speaker 2: and like you have the heads of the lab saying yes, Speaker 2: this might destroy the world and take all your jobs, Speaker 2: and the safety people in the alignment people. Speaker 4: Says, yes, this might destroy the world. Speaker 2: And like, I'm very strange industry right, Like the only Speaker 2: thing that I can think of a cigarettes, where like Speaker 2: they warn you that smokey is bad, except they had Speaker 2: to do that because they lost a lawsuit. Speaker 4: I don't think they were particularly inclined to do that. Speaker 2: I can't think of any other industry where the most Speaker 2: enthusiastic people about it are also warning and dooming about Speaker 2: how bad the thing they're building could be. So I'm Speaker 2: sort of curious, like you know, first of all, like Speaker 2: and I talked about this in the intro, like who Speaker 2: is the type of person that's like working it meter? Speaker 2: It is like skilled enough to do like advanced evaluations, Speaker 2: and like where's the funding coming from? But like talk Speaker 2: to us about like who's behind meter and why they're there. Speaker 6: Yeah, totally, So. I think one thing to say on Speaker 6: the history of kind of people caring about AI safety Speaker 6: in the day area is that this concern goes back Speaker 6: like quite a ways, I could say for over a decade. Speaker 6: There are many people who got into the field because Speaker 6: they saw this trend of deep learning, Like what if Speaker 6: deep learning works and it kind of goes all the Speaker 6: way to artificial general intelligence and then superintelligence, and if Speaker 6: that works, then it could affect everything. I think possibly Speaker 6: when people worry about this, there's a future that they Speaker 6: have in mind with super intelligence that's even more capable Speaker 6: than what people who think of themselves as like AGI Speaker 6: pill today think of. They're imagining AI systems that can Speaker 6: run you know, the entire economy and I think people Speaker 6: who kind of a while ago or many years ago Speaker 6: saw that vision and were sort of alarmed about the Speaker 6: stakes of it. Many people had this intuition that the Speaker 6: thing to do is go and work in the industry Speaker 6: because if you're like helping build it, you know what's Speaker 6: the best way to shape the future, It's to build it. Speaker 6: And I think that there's obviously you could have questions Speaker 6: about how sincere that is for many of the people Speaker 6: who are in the industry, or if there's kind of Speaker 6: a mix of different motivations and like you know, different Speaker 6: wolves inside of them where maybe they partially are motivated Speaker 6: by that, but also they're like there's kind of this Speaker 6: like Oppenheimer, like, it feels good to feel like you're Speaker 6: in the position of making something that's dangerous made. Speaker 2: Someone wants described Open aiyet of me, this is years ago. Speaker 2: Friend said it was like Open AI I was sort Speaker 2: of like the Manhattan Project, except the goal was to Speaker 2: not build the bomb at the very end, if that Speaker 2: makes any sense. So to your Oppenheimer point, it's like Speaker 2: very strange. Speaker 6: And I think one thing to emphasize is, you know, well, Speaker 6: it could be that there's a mix of motivations now Speaker 6: there are definitely many people, I think in the Bay Speaker 6: Area who sincerely believe that the technology is headed to Speaker 6: someplace that will be very difficult for a huge where Speaker 6: it will be very difficult for humanity to stay kind Speaker 6: of in the driver's seat or like stay in control Speaker 6: and kind of a meaningful sense. Speaker 2: It does seem is though, like people talk about all Speaker 2: the big AI labs have like a pr problem or Speaker 2: something like that. They keep bringing this up, and it's Speaker 2: like maybe they just believe it. Speaker 6: So I think that this concern is quite old, and Speaker 6: I think many people have this intuition that they're like, Speaker 6: I can influence the thing by building it. But now Speaker 6: there's this problem that that logic kind of always recommends Speaker 6: that you continue building more advanced technology or like more Speaker 6: advanced AI systems. And now you have this problem where Speaker 6: there's all of these companies and they all say that Speaker 6: they need to build it because if they don't build it, Speaker 6: another company will. And then even if all the and Speaker 6: they could all have doubts about each other's commitment to Speaker 6: safety or to these principles. Famously, the leaders of the Speaker 6: labs really do not get along. They're not friends. It's Speaker 6: not easy for them to kind of sort out the Speaker 6: safety thing among themselves. And then even if all the Speaker 6: USAI labs kind of agreed to do that, they then Speaker 6: have this kind of external bogeyman of China, Right, well, Speaker 6: what will the Chinese companies do? And so there's this Speaker 6: sense in which just like even the concern is real. Speaker 6: I think a lot of people then who are in Speaker 6: the industry have the instinct that they kind of there's Speaker 6: no guiding principle for what they should do on safety Speaker 6: other than to like build leverage for themselves for later. Speaker 6: And I think that is a concerning state of affairs Speaker 6: for AI development to be in globally. You know, obviously Speaker 6: we're trying to do something different by like informing the Speaker 6: public or kind of giving like you know, you could Speaker 6: imagine that this situation would be better if or like Speaker 6: one gap that exists right now in that picture is Speaker 6: that it's the people building the technology who most believe Speaker 6: that it's going to be destabilizing and sort of all encompassing. Speaker 6: Maybe if the public and governments all were on the Speaker 6: same page and believed the same thing, if it were Speaker 6: true that it was headed there, then there would be Speaker 6: kind of like more time for society to figure out Speaker 6: a response from people who are not trying to build Speaker 6: leverage over the technology themselves directly, or you know, control Speaker 6: the technology via some kind of like public action or government. Speaker 3: Can I just ask very quickly since you brought up Speaker 3: China and I don't want to forget to ask this question. Speaker 3: But Quinn doesn't show up on your like main charts. Speaker 3: I think you did a preliminary assessment of it a Speaker 3: while ago, but like, what's the difference between assessing one Speaker 3: of the closed models in America versus one of the Speaker 3: open source models over in China. Speaker 7: I think one thing to say is that the capabilities Speaker 7: are lacking behind We think that they're they're lacking behind it. Speaker 7: I'm not sure that of it. Speaker 3: They just like don't make it onto the chart. Speaker 7: So we do try to prioritize just because MITA has Speaker 7: has limited resources staff time in particular, that the models Speaker 7: that we anticipate being on the frontier and in general, Speaker 7: the Chinese models have been something like, you know, nine Speaker 7: to twelve months let's say, behind the US models. And Speaker 7: I think the gap by time horizon is probably even Speaker 7: larger than the gap by benchmark scores, where there's some Speaker 7: I'm not sure how scientific I can make this, but Speaker 7: there's some cloaqu real sense or something that the Chinese Speaker 7: models are stronger according to benchmark scores than they would Speaker 7: be on you know, truly held out problems in some. Speaker 3: Sense like gaming the benchmark. Is that what that means. Speaker 7: Or I'm not sure you know exactly how that shakes out, Speaker 7: but something spiritually spiritually close to that. I'm not sure Speaker 7: that's true for all Chinese models. I'm sure it's true Speaker 7: for lots of models outside of China, but I think Speaker 7: that's the least more possibility. Speaker 3: I'm very curious when you talk to external actors in Speaker 3: all of this, and I'm going to group them into Speaker 3: I guess policymakers, investors, and the labs themselves, like who Speaker 3: are you interacting the most with at the moment? Speaker 6: I think that in practice we end up interacting a Speaker 6: lot with AI labs because there's some amount of sorting out, Speaker 6: getting access to models, working with them to set new Speaker 6: precedents and things related to third party red teaming and Speaker 6: third party risk assessment. We think of our audience as Speaker 6: being sort of like high context members of the public, Speaker 6: so the kind of like people, you know, who are Speaker 6: maybe like you do, right, people who are kind of. Speaker 3: Like people listening to this podcast. Speaker 6: So people listening to this podcast people with kind of Speaker 6: who have to make important decisions that will be informed Speaker 6: by the pace of AI progress or like the kind Speaker 6: of profile of AI capabilities. Overall, Because we're based in Speaker 6: the Bay Area, I think we like disproportionately end up Speaker 6: interacting with people who are building the technology and like Speaker 6: closer to it. Partially, I think back to Joe's point before, Speaker 6: I think this is kind of because it is the Speaker 6: case that to kind of care about a lot of Speaker 6: these frontier problems, you're kind of selecting for people who Speaker 6: are building the technology themselves. There's some sense in which, Speaker 6: like the companies in the industry spends more time thinking Speaker 6: today about frontier capabilities assessment than the government does. I Speaker 6: think like one day you could imagine us getting to Speaker 6: the point where the government is like very focused on Speaker 6: this and dedicating a lot of resources to it, and Speaker 6: at that point I would expect Meeter to be spending Speaker 6: more time talking to governments. Speaker 3: That's kind of what I was getting at because our Speaker 3: senses and a lot of the conversations, like we talk Speaker 3: to people and they'll say something about like, oh, it's Speaker 3: important to have a social safety net for an AI Speaker 3: enabled future, but no one seems to be really thinking Speaker 3: about it in a lot of detail. Speaker 2: And when you say, you know, it's easy to imagine Speaker 2: or maybe the government will care more about this, not Speaker 2: so easy for me to imagine. It seems like they Speaker 2: mostly care about you know, data centers and like where Speaker 2: they located and stuff like that. It would be nice Speaker 2: if we had policymakers really looking at like frontier capabilities Speaker 2: and stuff. Still seems kind of a way off, but Speaker 2: it is interesting. You know, you're like talking about like Speaker 2: the sort of like capitalist dynamic, right, there's competition, and Speaker 2: it's like you have a lot of people that are Speaker 2: really worried about, oh, what if the other guys get Speaker 2: to ASI or AGI first, or what if the Chinese, Speaker 2: et cetera. How much does the fact of like free Speaker 2: market capitalism and the demand you know, the big investors Speaker 2: at the VC funds, like they want to return, they Speaker 2: want an ipo if we might get some big AI Speaker 2: IPOs this year in fact, how much do you find Speaker 2: that to be perhaps intention with the safety element? Speaker 6: Yeah, I maybe, Yeah, people on our team wou have Speaker 6: different views on this. I personally don't feel there's, yeah, Speaker 6: there's some thing you're like investors are key decision makers. Speaker 5: And you know they're people too. Speaker 6: That sounds strange to say investors or people do I Speaker 6: sound like Mitt Romney or something. But I think that, like, Speaker 6: I think that the element of this that feels like Speaker 6: it could be intention is if you build a bunch Speaker 6: of financial obligations to keep kind of the pedal to Speaker 6: the metal no matter what the risks are going into Speaker 6: the future. So, like, one thing I think a lot Speaker 6: about is if you're like building up a huge amount Speaker 6: of debt to build data centers and then say that Speaker 6: you do find evidence that you're now worried about about Speaker 6: the you know, loss of control from AI systems, you Speaker 6: do find instances of AI systems going rogue. Do you Speaker 6: now have like a financial commitment to build up those Speaker 6: data centers and like continue kind of the pace of progress. Speaker 6: I think that is one place where I feel the Speaker 6: tension pretty acutely, Like you're building these expectations into the Speaker 6: market that could kind of force you to continue development Speaker 6: when you otherwise would rather invest more in safety or Yeah, Speaker 6: like it at least gives you a kind of financial Speaker 6: obligation to continue scaling at least compute. I think that Speaker 6: like the people themselves being informed about the progress does Speaker 6: not seem bad to me. I think it's like good Speaker 6: in some ways for everyone to be on the same Speaker 6: page about capabilities that could be related to subverting human Speaker 6: control later on. But I think in the world beyond Speaker 6: like the information that Meter shares, I do think there Speaker 6: is a tension, like the fact that private companies are Speaker 6: building this I think could cause really acute tensions in Speaker 6: the future where people make these commitments that they wouldn't Speaker 6: if they were trying to like slow or you know, Speaker 6: maximize social resilience of the technology. Speaker 5: Yeah. Speaker 7: I'm not sure how these things shake out, but I Speaker 7: think there are some forces on the other side, right, Yeah, Speaker 7: you know, some safety promoting technologies quote unquotes or techniques Speaker 7: do make the models more useful, you know, if they're Speaker 7: better complying better complying with your whale in some sense, Speaker 7: and so have capitalist incentives standard capitalist incentives to invest Speaker 7: in that kind of research. Maybe that doesn't cover you know, Speaker 7: the broad suite of safety research that seems important. It Speaker 7: certainly doesn't rule out capabilities progress as being an important Speaker 7: taxis on which you do want to scale. But you know, Speaker 7: I think there are some some forces in each direction. Speaker 3: Since you mentioned compute just then, can you talk a Speaker 3: little bit more about I guess the relationship between like Speaker 3: the time horizon improvements and the cost of compute at Speaker 3: the moment, and like what you've actually seen and how Speaker 3: that impacts it. Speaker 7: Yeah, so, so one extraordinary fact from my perspective. I'm Speaker 7: not sure how to how to fit these facts together, Speaker 7: but something like the R and D spend on compute Speaker 7: of these companies has risen exponentially, of course, and in Speaker 7: fact it's risen exponentially at essentially the same rate as Speaker 7: time horizon progress. You know, I think there's nothing necessary Speaker 7: about that. You know, it doesn't mean by itself that Speaker 7: if computer progress lows then capabilities progress will also slow, Speaker 7: but you know, it's clearly an important input into into Speaker 7: AI progress. I expect that to continue to be through Speaker 7: in future. Sometimes people ask us if we think it's plausible, Speaker 7: or how plausible we think it is that that capabilities progress, Speaker 7: this exponential capabilities progress might slow down at some point Speaker 7: at some point in the future. And you know, one Speaker 7: reason it seems it's hard for me to consider it Speaker 7: plausible that it will slow down in the next at Speaker 7: least small number of years. Is that a lot of Speaker 7: those computes are and the investments basically already bigged in, right, Speaker 7: Like the data centers have already been built, you know, Speaker 7: plans for data centers even beyond twenty twenty seven twenty Speaker 7: twenty eight are presumably you know, coming coming to fruition Speaker 7: coming about, and so some of these input investments are Speaker 7: already baked in in some sense. So it would be Speaker 7: surprising to see capabilities slow to the extent that computes Speaker 7: has been has been an important input. After that, maybe Speaker 7: maybe you need to think about, you know, other arguments Speaker 7: for how capabilities might slow. Speaker 5: But that's roughly how I think about it. Speaker 2: There's a very good or interesting critical subject post called Speaker 2: against the Muter grav by someone named Nathan Woodgen who Speaker 2: brings up one an interesting point that I wouldn't have Speaker 2: thought of heading out Reddit, which is you're paying the Speaker 2: software engineers to come in and perform these tasks, right Speaker 2: it seems, you know, maybe this will be the last Speaker 2: job of humans, is just doing benchmark If I were Speaker 2: like a good software engineer and you say, Joe, come Speaker 2: in and do this task. Speaker 4: How do you prevent me? Oh man, this is taking Speaker 4: me a long time. Speaker 2: Mean, why I keep getting one hundred dollars an hour Speaker 2: for like looking at my computer and time? Who this Speaker 2: is tough. I'm gonna have to come back tomorrow and Speaker 2: keep working on this. How do you avoid the sort Speaker 2: of conflict of interest where the person who's paid to Speaker 2: work on this problem may be encouraged to take as Speaker 2: long as possible to solve it, and with only three Speaker 2: people working on it at times, I don't know, like Speaker 2: this does not It seems like a conflict of interest Speaker 2: to me. Speaker 7: Yeah, So the shulds onset is, you know, in general, Speaker 7: we are incentivizing these people to complete the task because Speaker 7: you know, it's possible, in particular, to complete the task Speaker 7: faster than that is who are attempting the same task Speaker 7: the time that it would take for them. Speaker 3: They task a bonus if they do it faster than Yeah. Speaker 7: Yeah, approximately, there's a bonus if they complete it faster Speaker 7: faster than anyone else. Speaker 5: You know. Another thing to say. Speaker 7: Is I think it just is true that our baselining methodology, Speaker 7: or the ways in which we compare to humans in Speaker 7: some ways leaves a lot to be desired that you know, Speaker 7: ideally we would have invested, you know, one hundred times Speaker 7: as many resources in having one hundred baselines human basedlines Speaker 7: per task, and those would have come from, you know, Speaker 7: perhaps the very best software engineers or machine learning engineers Speaker 7: in the world. Maybe that would be the Maybe that Speaker 7: would be the comparison that we're making. And indeed, we'd Speaker 7: be doing all of this procedure over many more tasks, Speaker 7: not just many more tasks, many more tasks, over wider Speaker 7: task distributions than just software engineering or machine learning engineering. Speaker 7: I mean, I do think time horizon still represents progress Speaker 7: over over what's come before in the science of measuring Speaker 7: AI capabilities. But you know, in some ways I'm sympathetic Speaker 7: to a lot of criticisms of time horizon. I do Speaker 7: think that some of the details, at least for the Speaker 7: work we've done so far, you know, aren't going to Speaker 7: matter as much as you might naively think. So trueing Speaker 7: the shortest baseline time that we end up observing or Speaker 7: the longest time you know, it's actually not going to Speaker 7: make that much difference to the final measurements. Speaker 5: You know. Speaker 7: Of course, we do think these people are talented software Speaker 7: engineers or cybersecurity people or someone depending on the task. Speaker 7: But you know, perhaps we could have found even more Speaker 7: talented people. They would have completed it in half the time. Speaker 7: And so you know, naively, it would seem like the Speaker 7: time horizon that we estimate of these models would be Speaker 7: half as long as we actually end up observing. But Speaker 7: of course that that wouldn't change the doubling time. It Speaker 7: would mean you'd get to the same level after another Speaker 7: four months. In some sense, the big picture that I Speaker 7: want time horizon to point to is less this like Speaker 7: Opus four point six is twelve hours in particular, and Speaker 7: more that we're seeing this remarkable pace of progress that Speaker 7: shows no signs of slowing in the recent past, and Speaker 7: I think in the near future as well. You know, Speaker 7: in fact, it shows some signs of speeding up. Speaker 3: Well, I was going to ask about this because I Speaker 3: think recently the statistic that you would always hear was Speaker 3: like a doubling every seven months something like that. How Speaker 3: fast do you see it going in the near future? Speaker 7: Yeah, so I was a doubling over every seven months. Speaker 7: Person that there was there was controversy in our team Speaker 7: about about what to believe here because when we originally Speaker 7: published this work approximately a year ago, you'd see, you know, Speaker 7: if you plotted a single straight line, a single exponential Speaker 7: you'd get something like, you know, six or seven months, Speaker 7: let's say. Speaker 5: But if you. Speaker 7: Restricted to just the time since I think JPT four Speaker 7: to oh, since the twenty twenty four models onwards, you'd Speaker 7: see something closer to this sort sort. Speaker 5: Of like four or five month trend. Speaker 7: And some people believed in that, and you know, some Speaker 7: people like me had the intuition that, well, we have Speaker 7: so few data points, we should we should really be Speaker 7: estimating over this larger number of data points than a Speaker 7: large number of. Speaker 5: Data points says every six or seven months. Speaker 7: There are a couple of things that have changed my Speaker 7: mind and made me realize my colleagues were right. Since Speaker 7: since then, One is that for the models that have Speaker 7: that have come out, since you know, what trends has Speaker 7: has better predicted how performance those models would be. And Speaker 7: it's very clear that the answer to that is the Speaker 7: four month doubling time and not this seven month doubling time. Speaker 7: You know that there's some some possibility that could speed Speaker 7: up again. We've seen it. We've seen it speed up Speaker 7: once I think there are some reasons in principle why Speaker 7: you might expect it to speed up again. I think Speaker 7: there are some caveats about this, you know, these are Speaker 7: these are maybe some some takes that my colleagues would Speaker 7: agree with, and so you know, maybe maybe you should Speaker 7: discard that, or you know, you should think that they're Speaker 7: going to commits me in the way that they did Speaker 7: with the with the four month versus seven month doubling times. Speaker 7: I have some suspicion that the tasks that meter is Speaker 7: measuring performance on are you know, in some sense more Speaker 7: and more narrow slice of possible tasks, and in particular, Speaker 7: and more and more narrow slice that is perhaps similar Speaker 7: to the kinds of tasks that you'd expect these major Speaker 7: AI companies to be training on in the first instance. Speaker 7: And so in some sense, we're increasingly more so than Speaker 7: was the case before, measuring progress on the exact types Speaker 7: of tasks that they're trying to get better at. You know, Speaker 7: you might think, for instance, the kinds of tasks that Speaker 7: would make for good reinforcement learning environments, the kinds of Speaker 7: tasks that you can score quickly and cheaply and automatically. Speaker 7: I think that progress is real. I think that progress Speaker 7: generalizes to some extent to other types of tasks I see. Speaker 7: I think we're saying, you know, remarkable progress and these Speaker 7: more messy tsks. For example. Speaker 2: I have one last question, which is like how big Speaker 2: is your team funding? And like also how many people Speaker 2: Meter are basically like really rich from AI and they're like, Speaker 2: you know what, I'm good. I don't need to pursue Speaker 2: like stick around for the IPO or whatever. I'm set Speaker 2: and now I want to work on something that like humanity. Speaker 3: No. Speaker 2: I've seen like there are other independent air researchers and Speaker 2: they talk about this. It's like, I want to be Speaker 2: able to talk about what I saw. Miles Brundage, someone Speaker 2: who has like a little think tank, He's talked about this. Speaker 2: What's like, how many people are like rich already and Speaker 2: they're like, Okay, now I want to work for something Speaker 2: that's public facing. Speaker 6: Yeah, so Meter right now is about thirty people that Speaker 6: we're growing and hoping to grow fast. We are hiring Speaker 6: I should say meter dot org slash careers and yeah, Speaker 6: you were touching before and kind of the thing about Speaker 6: is it difficult to be a nonprofit? Speaker 5: You know, we can't pay people in equity. Speaker 4: We got to get an io, right. Speaker 6: Yeah, there's no no ibo or for Meter, but we Speaker 6: do try to pay competitively on cash compensation, right, So Speaker 6: that's an area where we feel we can like somewhat Speaker 6: compete with labs. And it's true that I think a Speaker 6: lot of our team is just motivated by trying to Speaker 6: kind of do something different like not you know, all Speaker 6: the companies to some extent or in this business of Speaker 6: kind of like building somewhat redundant products kind of competing Speaker 6: for the same role in the world. And Meter is Speaker 6: in a really unique position at the moment where I Speaker 6: think that we have like access and the ability to Speaker 6: communicate these ideas and explain the state of AI research Speaker 6: to a number, like a lot of audiences that might Speaker 6: be hard for like individual researchers inside of a company, Speaker 6: Like we get to talk to a lot of governments directly. Speaker 6: We get to come here and talk with you all, Speaker 6: And that's kind of different. I think if you look Speaker 6: at all the actors that are working on the frontier Speaker 6: of AI research or AI safety, you kind of if Speaker 6: you compare us to AI lab staff, I think that Speaker 6: our work gets to be we get to kind of Speaker 6: every day work on whatever research we think will be Speaker 6: most informative to the like public decision. Speaker 2: Do you have ex AI, not XAI, but ex as Speaker 2: a former AI lab staff who maybe there was a Speaker 2: tender at some point and now they work at mater. Speaker 6: Yeah, we do, okay of those. Yeah, so we do Speaker 6: have some people who previously worked at AI labs. I Speaker 6: do think that as time goes on, I think one Speaker 6: hope that I have is that more, you know, there Speaker 6: will be more and more researchers who have kind of Speaker 6: like made the money that they need from working in Speaker 6: the industry and now are excited and kind of like Speaker 6: lifting all boats by working on kind of like inside Speaker 6: of an organization where the north star can be what Speaker 6: is most informative to the rest of the world outside Speaker 6: of these like relatively small set of companies. Speaker 7: Chris is very polite. I think that's I think that's wonderful. Speaker 7: I'm tempted to be a little bit, a little bit Speaker 7: more aggressive in this conversation. I think we have spoken Speaker 7: through mister's work on some of the most important problems Speaker 7: in the world, problems that are going to define the Speaker 7: future I think for not just the next years, but Speaker 7: you know, coming coming decades, maybe maybe even coming centuries. Speaker 7: And we've also spoken about some of the ways in Speaker 7: which me to work is not might not what you Speaker 7: might want it to be. That there's a long way Speaker 7: to go in the science of evaluating these ais. Why Speaker 7: have we not made more progress? You know, maybe maybe Speaker 7: a couple of reasons. I think clearly the central reason Speaker 7: is that we are bottlenecked on technical talent, on incredibly Speaker 7: capable people to come work on these questions. I was Speaker 7: on a meter work retreat recently where we were brainstorming, Speaker 7: you know, twenty thirty of these what seemed like world Speaker 7: important problems, problems that we think no one else is Speaker 7: going to get to if we do not get to them, Speaker 7: and we are able to conduct research on how many Speaker 7: of those problems, I think it's one. Speaker 5: Two. Speaker 7: You know, maybe if we do an extraordinary job this quarter, Speaker 7: it might be three. As Chris alludes to, I think Speaker 7: if you're interested in, you know, less working on redundant Speaker 7: products at these major area companies and more advancing our Speaker 7: understanding on some of the most important questions in the Speaker 7: world that are going to shake the world for years Speaker 7: to come. Meters is a great place to go. Speaker 6: Well, yeah, One more thing to say about that is Speaker 6: like the vibe inside of Meter is a state of triage, right, Speaker 6: And I think people often tell themselves externally. People might guess, oh, Speaker 6: you know, meters A, it's outside of any of the Speaker 6: AI labs. So the thing it might most struggle with Speaker 6: is things like access to AI models. You know, you Speaker 6: can't do the research you want because you don't have Speaker 6: you're not building the thing yourself in practice, or that's Speaker 6: the story that people always tell us. You have to Speaker 6: build you know, the future to shape it in practice. Speaker 6: I think our experience at METER is that, like when Speaker 6: we want to try new types of research that would Speaker 6: require new kinds of structured access, our experience at this Speaker 6: point has been that AI labs are like pretty game Speaker 6: to play ball on that. And the thing that is Speaker 6: more happening is that we're having to turn down opportunities Speaker 6: to do stuff like that because we don't have the Speaker 6: staff that we need to make those things happen. Speaker 2: Interesting Joel and Chris, thank you so much for coming Speaker 2: on odd Laws. Absolutely fascinating conversation and I appreciate your Speaker 2: taking your time. Speaker 5: Great to have you in the studio. Speaker 6: Yeah, thank you so much, so much, having us. Speaker 2: That was a really interesting conversation to that we're starting Speaker 2: from the end sort of the idea of like, Okay, Speaker 2: here are some really important questions, like let's just set Speaker 2: everything aside. Speaker 3: And there's thirty people working on there, there's. Speaker 2: You know, and like how many people want to do it, Speaker 2: and it's like, okay, we try to match cash comp Speaker 2: et cetera. Yeah, that seems like kind of a tricky Speaker 2: issue if like, if you accept the premise that these Speaker 2: are some big questions we have to get right and Speaker 2: you got to land this plane hopefully, Like that's a Speaker 2: bit of an issue. Speaker 3: Yeah. The other thing I thought was really interesting was Speaker 3: the Chinese models not really making it on the charts Speaker 3: even though, like we know, in the market itself, like Speaker 3: when deep Seak, when that new version came out, that Speaker 3: was like this huge thing where everyone started to panic Speaker 3: and to not see it even like land on the Speaker 3: time horizon chart. It's kind of interesting. Speaker 4: I guess it's interesting. Speaker 2: I mean, I guess I buy the reasoning from their Speaker 2: perspective that the only interesting question from meters perspective is Speaker 2: like the most cutting edge slightly adjacent to the most Speaker 2: interesting chart for like business, right, So it's like, Okay, Speaker 2: we know the deep sea and Quinn and Kimmy and Speaker 2: all those are like very impressive. Do they push like Speaker 2: the very frontier? Perhaps not, but just in general, I Speaker 2: find this space so weird because it's like, here you Speaker 2: have these people who are like clearly quite alarmed at Speaker 2: the potential here, and most people, I think, look at Speaker 2: these charts and they say like, wow, this is like Speaker 2: I want to invest in this, or this is. Speaker 4: Like no, I know, I know. Speaker 3: Like that's why my first question was like, you're here Speaker 3: for AI safety purposes, but everyone seems to get excited Speaker 3: about the line go up charts right, Like there's a Speaker 3: disconnect all connected. Like I say, when an industry basically Speaker 3: says it's worried by itself, you should pay attention. Speaker 2: It's really strange. This gets back to, you know, very Speaker 2: It's very strange where you have the CEOs of these Speaker 2: companies who are in many cases the most alarmist, and Speaker 2: there's this sort of cynical thing. And I don't totally Speaker 2: discount the cynical interpretations like oh, they're saying this because Speaker 2: they want to get investors and so forth, and they Speaker 2: need all this money. But look, it was also true Speaker 2: that open AI and Anthropic but open AY a little Speaker 2: more were like founded with these very exotic corporate structures Speaker 2: of like a private company owned by nonprofit et cetera, Speaker 2: which they presumably did because they took pretty seriously the Speaker 2: fact that this technology is science. It was like very Speaker 2: strange and not just like it's not just enterprise office right, Like. Speaker 3: They were self limiting in a way. Speaker 2: One other interesting thing too, that this idea is like, okay, like, Speaker 2: first of all, what's the difference between seven months and Speaker 2: four month time doubling? Speaker 5: Not much? Speaker 1: You know. Speaker 3: It's like these people's like, oh, I can't but it's exponential, Speaker 3: isn't it. Speaker 4: I guess it's exponential, But it's still funny to me. Speaker 2: It's like, oh, I think like AI is going to Speaker 2: destroy all white collar work in two years, and someone Speaker 2: else is like, no, no, I think it's gonna be three years. Speaker 2: Is if that makes any different whatsoever? But one thing Speaker 2: to consider all sort of alluded to this. You know, Speaker 2: you had like open ay shut down. It's like video efforts, Speaker 2: et cetera. So perhaps part of the story is just Speaker 2: this intense focus now on the software engineering side, as Speaker 2: what these labs are working in Yeah, and sort of Speaker 2: like all these other side quests are not as important, Speaker 2: So maybe we will see even more rapid progress on Speaker 2: some of these technical benchmarks, because clearly, from the labs perspective, Speaker 2: that's where the action is more than some of these Speaker 2: consumer things like making making images or videos. Speaker 3: Yep, all right, shall we leave it there, Let's leave Speaker 3: it there. Okay, this has been another episode of the Speaker 3: Auth Thoughts podcast. I'm Tracy Alloway. You can follow me Speaker 3: at Tracy Alloway. Speaker 2: And I'm Joe Wisenthal. You can follow me at the Stalwart. Speaker 2: Follow our guest Chris Painter He's at Chris Painter yup. Speaker 2: And Joel Becker He's at Joel Underscore b k R. Speaker 2: Follow our producers Carmen Rodriguez at Carmen armand dash Ol Speaker 2: Bennett at Dashbot, kil Brooks at Kilbrooks and Kevin Lozano Speaker 2: at Kevin Lloyd Lozano. And for more odd Laws content, Speaker 2: go to Bloomberg dot com slash odd Lots where the Speaker 2: daily newsletter and all of our episodes and you can Speaker 2: chat about all these topics twenty four to seven in Speaker 2: our discord Discord dot gg slash lots. Speaker 3: And if you enjoy Odd Lots. If you like these Speaker 3: AI episodes, then please leave us a positive review on Speaker 3: your favorite podcast platform. And remember, if you are a Speaker 3: Bloomberg subscriber, you can listen to all of our episodes Speaker 3: absolutely ad free. All you need to do is find Speaker 3: the Bloomberg channel on Apple Podcasts and follow the instructions there. Speaker 3: Thanks for listening.