Clip / Education

Live confidence-gated planning, and why the fuzzy score still works

From Full Walkthrough: Writing & Using Skills — Nick Nisi and Zack Proser · ≈1:01:02

“there's a value in thinking. It's like, you know, the same way that a good engineer in a whiteboarding session would kind of draw the same stuff out of you.”
AI Engineer

“And so right there, it did this confidence score. It's based on the problem clarity, it has a 20. Goal definition 18. Success criteria, it doesn't really know what I'm asking for. So that's the lowest one. Uh scope boundaries and then consistency. So those all add up to 100 and I got a score of a 90 out of 100.”
AI Engineer

“I read and review the contract and then it's going to build from there these phases that I can execute or these specs that I can execute in phases uh and then go from there uh so that I can clear the context for each one and have like a fresh context going.”
AI Engineer

“the way I would say that is like is the math here tight? No. Uh does it matter? No, because the value is in the iterative loop of like clarifying and and clarifying your own thinking by by responding.”
AI Engineer

What’s in it

A demo plus an honest defense of a technique people dismiss as fake precision: the score's job is to force clarifying questions, and it ends in a reviewable contract and phased specs.

Watch this moment

Clip transcript

know. Or requesting code. >> So, this is an example of the ideation scale. I gave it that that simple command and now it's saying like, "Oh, what do you like I don't fully understand what you mean. Uh what kind of fun are you looking for?" I'll say, uh a visual gag. Uh ASCII art gallery, sure. Hidden Easter egg. Uh yeah, we'll go listed but subtle. So like I I give it one sentence. And it's like, "Well, what do you mean by that?" And it's like pulling all of that out out of me. >> there's a value in thinking. It's like, you know, the same way that a good engineer in a whiteboarding session would kind of draw the same stuff out of you. >> Yep. And so right there, it did this confidence score. It's based on the problem clarity, it has a 20. Goal definition 18. Success criteria, it doesn't really know what I'm asking for. So that's the lowest one. Uh scope boundaries and then consistency. So those all add up to 100 and I got a score of a 90 out of 100. So it doesn't It's not going to just be like, "Okay, I know what you want." It's going to ask me uh some more things. Like, "Oh, we'll do that and we'll just have minimal up." I'll say zero config. I just want it to go fast. And so now I'm at 96 out of 100. So it understands what I want and now it's going to write a um a contract for me to read. I read and review the contract and then it's going to build from there these phases that I can execute or these specs that I can execute in phases uh and then go from there uh so that I can clear the context for each one and have like a fresh context going. >> Yeah, the way I would say that is like is the math here tight? No. Uh does it matter? No, because the value is in the iterative loop of like clarifying and and clarifying your own thinking by by responding. >> Yeah. Oops. And so there's the contract that it's it's loading. Uh and it tells like what success criteria means, the scope boundaries, what's in scope, what's specifically out of scope, any future considerations, how we plan to execute it. This is an easy one, just a single phase. Uh and so it's going to just create that spec for me, which it did here. And then I could run this uh and go. And so it was all gated on that that confidence score. >> Cool.