Cody, Gonzalo, and Jesus discuss the realities of implementing AI in enterprise, cutting through the hype while rock climbing.
In the inaugural episode of AI for Luddites, hosts Cody, Gonzalo, and Jesus hit the rock climbing gym to discuss the practical realities of artificial intelligence in the enterprise. They cut through the industry hype to explore how companies are actually using AI, the truth behind tech layoffs, and the process of building semantic data layers. The conversation also covers open-source models, agentic programming, and strategies for token optimization to save costs.
In this episode
- AI hype versus practical enterprise implementation
- Tech layoffs and using AI as a corporate scapegoat
- Apple Intelligence and disciplined product releases
- Building a semantic data layer using Claude
- Model Context Protocol (MCP) and connecting data sources
- Modernizing legacy technology in the trucking industry
- Open-source models versus frontier models
- Agentic programming and adversarial code review
- Cost savings through token optimization and request routing
Chapters
| Time | Chapter |
|---|---|
| 00:00 | Introduction and AI Hype |
| 03:00 | Tech Layoffs and AI Scapegoating |
| 04:30 | Apple Intelligence Strategy |
| 06:15 | Building a Semantic Data Layer |
| 08:30 | Model Context Protocol (MCP) |
| 10:00 | Rock Climbing and Career Backgrounds |
| 17:40 | Transitioning to AI and Legacy Tech |
| 23:40 | Open Source vs. Frontier Models |
| 28:00 | Agentic Programming and Adversarial Review |
| 33:00 | Cost Savings and Token Optimization |
Worth quoting
The loudest people about AI coming to take your job… are the people selling the technology.
If you don’t know what you’re doing, you can’t really guide an LLM.
I have actually over the last two and a half years saved about half a million dollars annually in a recurring expense.
Full transcript
Episode 1, recorded at Vertical View climbing gym in Idaho. Speakers: Cody, Gonzalo, and Jesus. This transcript was machine-generated and lightly cleaned for readability; it may contain small errors. Timestamps match the audio.
Introduction and AI Hype [00:00]
[00:00] Gonzalo: And welcome to our first podcast of AI for Luddites.
[00:04] Jesus: There you go.
[00:05] Gonzalo: With Cody, Gonzalo, Jesus. Because we're not going to be just blowing up smoke, we're also going to be talking about what's hyped, what isn't working, and what needs fixing. And we're not at a studio, we're at Vertical View. Vertical View, we're going to do some rock climbing.
[00:24] Cody: That's right.
[00:24] Gonzalo: So we're going to be moving. So if you hear some heavy panting, it's not because of fear. It's because we're being active. Or because we're too excited. We're going to be talking about Kimi 3, about open source large language models, and why you shouldn't be token maxing on Fable 5 just because.
[00:44] Cody: That's right.
[00:46] Gonzalo: And before we start, everybody's talking about AI, but we'd like to think we're actually using it and implementing it in real work, not for views on YouTube. Yeah. And nothing, nothing that I have against YouTube, but there's a lot going on today. And with that…
[01:05] Cody: There's a lot of noise.
[01:06] Gonzalo: Tell us about the noise, aside from the music that we're hearing, which hopefully we'll be able to master later.
[01:14] Cody: It's difficult, you know, I think for a long time we've lived in a sales-focused world. And because of that, there's always noise. And then you get something where, you know, 40% of our economy and the CapEx expenditure that's being put in is being put into this AI sphere. And so you have all these people trying to sell smoke. And I think that's where a lot of the noise comes from, you know.
[01:41] Gonzalo: And selling just becomes excessive. Sometimes even like, you know, I'm at the gym and somebody's like, hey, do you want peptides? I'm like, no man, I'm at the sauna. Can I have a time to breathe?
[01:53] Cody: Right.
[01:54] Gonzalo: Where does the hype end and where does reality begin?
[02:00] Cody: We, I think we're all involved in projects where if we spend too much, we might not have a job, and that's important.
[02:08] Jesus: Yeah.
[02:09] Cody: Some of us have kids. I don't want to tell them, hey, Fable 5 will feed you. Because it won't. You know, I also think, I mean, maybe someday in the future some of those things will replace a lot of people. I don't know. But for now, for all the projects I'm working on, you know, if it's not an intelligence layer, it's human in the loop somewhere. I think my goal from like a corporate standpoint, right, is as we grow as a company, I want these technologies to help absorb some of that impact. And so our current teams can handle that volume, not to replace the people that are currently working there, but to kind of augment them so that we can get more done.
[02:47] Gonzalo: So this fear mongering of everybody's going to run out of a job, it's actually been the opposite. I'm kind of seeing that you need more experts.
Tech Layoffs and AI Scapegoating [03:00]
[03:01] Cody: Yes.
[03:02] Jesus: Well, and also the loudest people about AI is coming to take your job, you need to get in on this technology, are the people selling the technology.
[03:09] Gonzalo: Selling it.
[03:10] Jesus: The other question is, where are we in an economy that everybody's blaming AI, but could it be that things are slowing down and everybody's blaming it?
[03:19] Cody: Yeah.
[03:20] Jesus: Well, yeah, I think a good example is that you see a lot of these tech layoffs that happened the last couple years, and I think a lot of people are using, like the executives at those companies are using AI as a scapegoat. But a lot of it is just overhiring from the cheap to free money that was being given out during the pandemic. It's like, I think Meta specifically, you know, they just laid off 8,000, which is only about 10% of their company. But they went from, I believe it was like 20 to 60,000 in over three years during the pandemic. It's like, they're not doing anything that sustains that amount of headcount. But they're just saying AI as kind of a scapegoat of, well, we planned badly.
[03:58] Cody: All that 1 to 2% interest money they took in, basically free. Yeah.
[04:03] Gonzalo: But there were some companies that weren't laying off, right? Like Apple didn't go through the heavy layoffs.
[04:08] Jesus: Nope. Apple's also been one of the more stable companies over the years though, and in like long term planning.
[04:13] Cody: They're very disciplined.
[04:14] Jesus: They're very disciplined. Yeah.
[04:15] Gonzalo: And they were kind of making fun of it at the beginning of like, why aren't you spending? You're probably not preparing for the future.
[04:20] Jesus: Everyone wrote off Apple as like, oh, you guys are so far behind, you guys are basically irrelevant in terms of the AI space. And Apple basically came out and said like, we don't, we don't do AI like that. We will incorporate it, but we don't need to build our own like LLMs, our own systems. It's like, we will just partner with them.
Apple Intelligence Strategy [04:30]
[04:36] Gonzalo: They have been doing a lot of research, however, would you say they're late to market, but they do it very taking into consideration what people actually will use?
[04:45] Cody: What people will use, and it has to fit their widget, right? Like that's their whole thing. They control the whole widget, they have the whole widget from the hardware all the way up to the software, and they want to integrate that all the way through. And it takes a lot of time to do that, I think.
[04:58] Jesus: Yeah.
[04:59] Cody: Yeah, I think, I think Apple's been one of the more disciplined ones of not just jumping on AI hype as far as both CapEx and as also just general trend following as well. And I think it's done them probably pretty well.
[05:11] Jesus: Yeah. Because like, say you have an iPhone, where is an LLM going to help you in your day-to-day iPhone usage? Just, just to make Siri a little bit smarter. That's about it.
[05:20] Cody: That might actually help though.
[05:21] Jesus: Yeah, no, I mean, I'm like, that's, realistically, that's, that's like their biggest implementation, which I know they've been trying to do, but they've been consistently, they, they said they were going to have a couple years ago, and then it's been pushed back, and they still haven't quite shipped it, but as far as Apple specifically, I think that's the biggest gain that they'll be able to get from an LLM system.
[05:39] Gonzalo: Yeah. Have you tested the beta?
[05:41] Jesus: I have not, no.
[05:42] Gonzalo: I have not either. I've, I've gotten on betas and the few times that I did, it messed up development. So for anybody who's either launching or publishing, I don't recommend it.
[05:57] Jesus: Yeah. Have a separate account and system for it.
[06:00] Gonzalo: Right, right. But I do miss it. Do miss testing some features early. I remember testing Siri when it was new. I liked their UI. I just like when things look nice and work well.
[06:12] Cody: Yes. Yeah.
[06:13] Gonzalo: But I think you, you, you've been mentioning you've been using Codex a lot.
Building a Semantic Data Layer [06:15]
[06:17] Cody: Uh, Codex, uh, a little newer too, actually. I've been using Claude Code a lot.
[06:21] Gonzalo: Okay.
[06:22] Cody: Um, and that's been going really well. And so we recently developed kind of an intelligence layer. Um, it's a whole like semantic layer with intelligence built in for the entirety of our data domains, our different knowledge domains within the company. So we're LTL, you know, so we have shipping data, we have operational data, we have finance data, finance data, we have payroll data, we have time punch data, truck data. We have all these different data domains, and they all mean something to each other in the middle. There's a way to interconnect them all. Um, and we've done that. And a big hurdle of that was just the semantics. You know, on the back end of a database, when we talk about something like a pro number, that, that relates to a shipment. In a database, a pro number could mean 17 different things. But in the context of a sentence, when I say, hey, can you track that pro, it means one thing. And so we have to create that semantic layer to give something like Claude an understanding of what that means. So when we ask questions about our data, it understands it and knows how to get us the right answer.
[07:25] Gonzalo: So it's taken some time to train too.
[07:27] Cody: Yes. We got, we got exceptionally lucky in a way. We spent the last two years building a data warehouse and BI dashboards and all that. And so we had basically already created an understanding of what we were looking for through the formulas. And so I was actually able to use Claude to go reverse engineer all of our formulas to automate creating the semantic layer for us, which sped this whole process up to, it took me a week to build this thing. And right now it connects eight or nine different data domains really well.
[07:56] Gonzalo: Yeah. And it's accessible via…
[07:58] Jesus: So you kind of took a side… these are massive amounts of information.
[08:02] Jesus: Yeah, well, I mean, it's, it's a bunch of information and, and it is a bunch of scattered information where some is over here, some is over here, and we knew, and like Cody was saying, we had been already putting the legwork in to build up the connections of what needs what, what needs to tie to what. But to connect all of those via a semantic layer and then make it into an MCP where we just connect it to Claude in the span of a week, like, for those who might not know what an MCP is…
[08:29] Cody: Model Context Protocol, which…
Model Context Protocol (MCP) [08:30]
[08:31] Jesus: Right.
[08:32] Cody: Model Context Protocol is essentially a way to give an LLM a set of tools and context to whatever you're connecting it to. So we can take our own tool, app, widget, data source that we've built, and we can use an MCP to connect that to Claude or ChatGPT or Kimi if we wanted.
[08:48] Gonzalo: Which is wild, right? We're speaking about these things. I, I still get, does it happen to you that you're often in awe?
[08:57] Jesus: Yeah.
[08:58] Cody: Yeah, all the time.
[08:59] Gonzalo: However, it doesn't take away the fact that you know your business. You have to know your business. You can't just be punching buttons and see what's going to happen.
[09:06] Cody: Right.
[09:07] Gonzalo: There's also flaws.
[09:08] Cody: Well, and you have to understand…
[09:10] Gonzalo: And errors, have you noticed them?
[09:13] Cody: In general or with this specific project?
[09:15] Gonzalo: Well, I think two weeks ago we were trying to do some security scanning and Fable 5 said, hey, we're done, we can't get any further.
[09:28] Cody: Oh yeah.
[09:29] Gonzalo: It literally was two brackets. And it's something that I was seeing and I go, come on, Fable 5, you should catch this. So if you don't know what you're doing, you can't really guide an LLM.
[09:40] Cody: You can't really guide it, yeah.
[09:41] Gonzalo: So you need expertise. Back to the… good people are irreplaceable. They're actually the ones who are training and creating these solutions.
[09:50] Cody: Yes. And I mean, so, you know, we were able to get this solution done in a week, week and a half, right? We were able to build this thing.
[09:55] Gonzalo: You guys want to get started?
[09:57] Cody: Yeah, let's get started.
[09:58] Gonzalo: One of us might… I'll watch. By the way, we're here with two expert climbers, I take it?
Rock Climbing and Career Backgrounds [10:00]
[10:08] Cody: Expert's a little generous, but…
[10:11] Gonzalo: I'm a newbie, and it's good to be a newbie. It's all right. Catch shiny things. And John, we will miss you, John. You'll be here next time. Find some climbs. By the way, we're looking at a massive wall. This is on 65 foot, I believe?
[10:29] Cody: 65 feet, 75 feet.
[10:30] Gonzalo: With an overhang? The overhang's probably…
[10:33] Cody: I take it this is the hard one.
[10:35] Gonzalo: Would you say that's 15 feet of overhang?
[10:36] Cody: This is the advanced stuff.
[10:38] Gonzalo: Oh my gosh.
[10:41] Cody: That's a lead-only wall, so we're not going to do that today because we don't have the equipment for it. You see how there's no ropes hanging? You have to clip yourself as you go, and we didn't bring any of that equipment, so.
[10:51] Gonzalo: Okay. These ones… is this the right… do I have…
[10:56] Cody: Yeah. Yep. All good.
[10:59] Cody: If we could find like a 5.6…
[11:01] Gonzalo: Yeah, there's purple here. There's a couple little lower ones. Should we all start at once, give it a pause, or one at a time while two talk? How do you guys want to do it?
[11:10] Cody: It has to be one at a time while two talk because we have to belay, so.
[11:13] Gonzalo: Oh, perfect. Oh, perfect. Yeah.
[11:15] Cody: If you want, I'll start here. Just kind of get going on this, I'll warm up on this one, and then if you want to follow after that, then you can give it a shot.
[11:23] Gonzalo: Absolutely. Yes, yes, yes. Oh my gosh. This is cool. Pulling cords. Equipment. Where is the AI here?
[11:33] Cody: No, this is as analog as it gets.
[11:36] Gonzalo: AI-assisted muscle movement. Gonzalo. Quite a nice balance. There's no technology here. So Jesus, you're going to be… this role is called a belayer, did you say?
[11:49] Jesus: Belay. To belay. Yep. To belay. Yep.
[11:53] Jesus: So I control the rope and make sure that if he falls or if he loses, he's safe. There's tension, it doesn't go down.
[12:01] Cody: And also a couple things, I'm going to intentionally fall, I'm going to intentionally sit on the rope so that you see what it's like, so that it's… if you don't see it, it's a little hard to just trust it.
[12:09] Gonzalo: Okay. I've done this before, just a long time ago.
[12:15] Cody: Yeah. I got you. Climb on.
[12:18] Gonzalo: He's going. He's just going to climb.
[12:21] Jesus: So on top rope, it's pretty easy. I just pull the tension out, make sure that it's locked within my assisted braking device. So even if for some reason my hands come off or whatever, the device will help stop it as a backup. Oh, nice. And I just manage the tension as he climbs.
[12:39] Gonzalo: Cool. Who lost his microphone? Microphone down. Which is actually a good thing, because we have a slack.
[12:49] Cody: Perfect. Nice.
[12:52] Cody: Okay, once it's your turn, we'll go through and show you the setup.
[12:59] Gonzalo: And Jesus, how did you get started in all of this? Because before AI, there's machine learning, and before machine learning…
[13:10] Jesus: You know, I come from a very non-traditional path. I haven't gone to school for anything. I've always liked computers growing up. I started in like the gaming space and video game development. Actually where Cody and I originally had met. We were at a local company here that did video game QA testing, all kinds of different testing. And we partnered with the Microsofts, the Sonys, the, you know, the Epic Games of the world, and worked with them on certain titles and this and that.
[13:44] Gonzalo: So gaming is your background?
[13:45] Jesus: Gaming, I started in gaming, where a lot of my specific experiences have started.
[13:49] Gonzalo: Just to fill in people, gaming is a larger market than music and movies combined.
[13:57] Jesus: Yes. Music, movies, and TV combined.
[14:00] Gonzalo: For anybody who still might go, well, how big is gaming? Pretty darn big.
[14:05] Jesus: Yeah, it's the largest in entertainment, for sure.
[14:10] Jesus: But yeah, so that's kind of where we got started. I definitely learned and grew a lot in that company in that capacity. Started out with like hands-on as a testing analyst, hands-on testing, and then kind of worked my way up to management. And then when I left that company, I decided I wanted to be more hands-on in a technical role. And Cody just happened to be hiring for a position at the company he was at at that time, and intercepted me. And really, I like to just stay up to date with what's happening, because I find it coming down, interesting. And I'm always about how to introduce tools, systems, workflows to increase productivity and make things streamlined. I'm a big efficiency guy.
[14:51] Gonzalo: So you've seen everything from user to developer to analytics and the communication in between. And the finances that drive the decision in the background as well.
[15:04] Jesus: Which is important, right. Finances, I think right now there's this idea of everything's going to be fine, just keep on spending. Yes. But will it? I saw both of you rolling your eyes. I also, I've never experienced an unlimited budget or money being thrown. Because it doesn't exist.
[15:26] Gonzalo: John B's students listen to this, I know. By the way, John B is our dear friend who's not here. Lead AI at BSU. And yeah, and by the way, he just tells me how often he tells students that behind technology is to never forget to be great human beings. And I think that's beautiful. If I notice something now more than ever is how important it is to connect, to inspire, to have people trust you. Like I'm not impressed by somebody's knowledge, especially when they're, I don't know, and they're not kind or, geez, whatever happened to being good to one another?
[16:17] Jesus: Being good to one another, yeah. It's important. Building good teams, good leadership, helping people, just helping people really, you know, treat others how you want to be treated. And it can be hard sometimes in business, I'm going to grant that, right? At the end of the day, business is about a company making a profit. Work is hard. And so it's like, out in the day-to-day world, you can just be kind to someone. In business, sometimes it takes a little extra thought on how you can do that in a way that's good for the business and can be kind to others. So it takes effort, but it's worth the effort. You have to put that in. Yeah. It can't be solely about the business at the sacrifice of the people.
[16:56] Gonzalo: So Jesus was telling me about his background. Cody, how did you get into AI? Because before AI…
[17:07] Cody: How did you get from AOL to AI? AOL to AI, you know? It's funny, I'm not really sure. I guess, taking it back, there's generations of AI, right? And so I think you go back and you start looking at the last generation, probably the last big generation would be like machine learning. I think that's where like Siri and those things initially came from. And I had some experience with those, but not a ton. I really, really started getting into AI as it started happening right through COVID. ChatGPT launched, and I saw the potential in what it could do and the fun things we could do with it and dove right in and haven't looked back. It's like anything else though, you know, it's a tool. You have to know how to use it, learn how to use it, utilize it in certain ways. But yeah, I think I just have a passion for tech and I like to learn new things and that's kind of where I got into it.
Transitioning to AI and Legacy Tech [17:40]
[18:07] Gonzalo: So you've been doing this for a bit, also in gaming?
[18:11] Cody: Yeah, yeah. I also worked at Lionbridge with Jesus. I was an IT manager over there.
[18:20] Cody: …for the game testing studios. By the time I left, I think we had, I had, I don't, I don't even know. All the game testing studios across North and South America under my purview and we were, it was a lot. And some of them were a little older and a lot of them were just like, you know, cutting edge technology and we're pushing petabytes of data across these networks a day and it was, it was a challenge.
[18:43] Gonzalo: And some, I just got back from Japan and I realized even though you might have cutting edge technologies, there's things that you have to be considerate of and still integrate. You were mentioning some technologies that worked, that were almost, not ancient, but archaic.
[19:00] Cody: Archaic, yeah.
[19:01] Gonzalo: You have to work with it and knowing about it is important. You can't just say, let's forget it, let's replace it.
[19:07] Cody: Right. So it's interesting. So, you know, I worked at Lionbridge with Asus on, you know, cutting edge game testing studios. I mean, this is like the coolest technology and equipment you can, you can run and manage. And it was very challenging. We were doing really challenging things and we need, that's why we needed cutting edge equipment. I thought coming into the trucking industry it was going to be easier, I'll be honest. It has been significantly more difficult working with outdated, older technologies and learning how they work, learning how to make them play well with modern technology and kind of pull them into the modern age. It's been significantly more challenging.
[19:38] Jesus: Yeah, it's the hardest part. I mean, they have so much outdated and archaic technology, which makes outdated and archaic processes, which, you know, try to bring those into the modern age or modernize it. It's like, some you can do, some you can't, and you have to get real creative with how that works.
[19:53] Cody: And as a carrier, we have to partner with other carriers. And so that means we have to exchange business data. And so even if we have, even if we had the most cutting edge modern system in the world, we have to interface with whatever they're using. And we have to figure out how to make that work.
[20:08] Jesus: We can only go as modern as the big guys will allow us, so to speak.
[20:13] Gonzalo: So it does take a lot of patience on your behalf.
[20:16] Cody: Oh, yeah.
[20:17] Gonzalo: Which is safe to say that that's something a few people have nowadays.
[20:21] Cody: They're like, you just need to throw everything out and re-do it.
[20:25] Jesus: Yeah, it's a mix of patience but also being open as well. There's a lot of, especially in the trucking industry that we work in now, it's like there's the big guys that drive, you know, majority of the revenue in the industry. They've done things the same way for the past 30 years. They're not open to changing it until they have to.
[20:42] Gonzalo: Who's coming up next? I didn't mean to interrupt.
[20:45] Cody: I think it's you.
[20:46] Gonzalo: It's me? Okay.
[20:50] Gonzalo: So if it's me, that means you guys are going to be talking about… Kimi 3?
[20:57] Cody: Kimi 3? I haven't used Kimi 3 yet.
[21:00] Gonzalo: You haven't used it?
[21:01] Cody: No, I've been on vacation.
[21:02] Gonzalo: Okay.
[21:03] Jesus: We have to figure out how we can. We're slow on the Chinese open models for sure.
[21:07] Gonzalo: Should we talk about that? Because I have my own concerns as well and some might be, what do you call it?
[21:14] Cody: I'm going to help you with this. So you're going to go through the bottom hard point here.
[21:18] Gonzalo: Okay.
[21:19] Cody: Up through the top one.
[21:20] Gonzalo: This bottom? The bottom here?
[21:21] Cody: Yep. And then through this top one right here.
[21:24] Gonzalo: Totally out the clear?
[21:26] Cody: Yep. Right on. Tie you in here. Nice. Double figure eight knot.
[21:33] Gonzalo: Oh, yeah.
[21:35] Jesus: We're just going to do 10 easy climbs. We're just doing the easy ones and just doing two or three times.
[21:39] Cody: There you go. Yeah, we've been super inconsistent, so we're kind of just getting back into the swing of things.
[21:44] Jesus: Just getting it in.
[21:45] Gonzalo: And I'm going to need to trust them.
[21:47] Jesus: Yeah, we just wanted to climb.
[21:49] Gonzalo: Trust.
[21:53] Cody: This rope will never break with you on it.
[21:56] Gonzalo: This is the feeling of, are you okay with Claude taking over your terminal? Sure.
[22:06] Gonzalo: Okay, I'm going to take this end. When did you know it was the right time to let an LLM into your terminal?
[22:16] Cody: Oh, that's a good question. I don't think there's ever a right time. If you set up the permissions and environment right, you could do it sooner than later. Definitely. And you know, I think there's, there's also, you know, like with any process, there's ways to separate what it's doing from production where you don't have to worry about it. I mean, you can jump in and test it out and build something and do whatever you want. And then…
[22:41] Gonzalo: But how did it feel at the beginning? It's scary to trust.
[22:45] Cody: It's a little nerve-wracking for sure.
[22:48] Jesus: It definitely is.
[22:54] Cody: Sometimes it's like climbing a rock wall. You just got to grab a hold and one at a time, you just keep going.
[22:59] Jesus: That's right. Next thing you know, you're 15 feet off the ground and…
[23:03] Cody: Give it a little time, you're 50 feet off the ground and…
[23:06] Jesus: That's right. You start with, okay, you know, you can use the terminal to read the architecture, read only. And then you can have it update architecture, but no provisioning. And then you go from there.
[23:20] Gonzalo: Well, I'm trusting you guys.
[23:23] Jesus: Yeah. Because for you, you're like, this is figure eight.
[23:28] Cody: Yeah, I recommend when you get to a point, sit on the rope and kind of let go so you feel it.
[23:33] Gonzalo: Yeah, yeah, just hang out for a second. Like let go and fall off the wall so to speak.
[23:37] Jesus: He's going. He's crushing it.
[23:39] Gonzalo: Next topic is open source and open weight models. It's going to be an interesting one because it's a hot topic right now.
Open Source vs. Frontier Models [23:40]
[23:46] Cody: It is a hot topic. I mean, I could see us using it in some of our workflows for like, you know, some code generation, things like that. But I think a lot of the projects I've been working on recently do require more like frontier level performance. But I'm also really excited to see where some of these open source models are in about a year. And if we can get some of that, like the contextual awareness of what the frontier models have into an open source model.
[24:16] Jesus: Yeah, definitely. And more like taking a model that has like general knowledge but being able to fine-tune it for a specific thing can squeeze a lot out of a less powerful model in theory.
[24:30] Cody: Yeah, I mean, the gap that, you know, the open models have closed to the frontier one in the past year has been insane.
[24:37] Cody: Okay, now just sit back on the rope.
[24:41] Gonzalo: There you go.
[24:45] Jesus: I know you've been on vacation. I don't know if you've seen everything going on right now with like, you know, was it Kimi? Was it K3 is the most recent one that came out?
[24:53] Cody: Yeah, and I think the frontier guys are accusing it of distilling.
[24:58] Jesus: Yes. So Anthropic accused it of distillation. And so that prompted conversations…
[25:04] Cody: Good job, man.
[25:10] Gonzalo: How do you feel? 45-foot wall.
[25:12] Cody: Oh, this is exhilarating. It's so much fun, right?
[25:15] Gonzalo: What's the biggest problem? The mind?
[25:18] Cody: The mind, yeah. Absolutely the mind.
[25:21] Cody: You hit physical limits on this, but like oftentimes, like even just like if I go to lead, I have to come down like a whole grade or two to do it because it's scarier and my mind gets in the way and I grip tighter and I burn out faster.
[25:34] Gonzalo: Is it calming yourself down? Is it breathing?
[25:39] Cody: Breathing. Breathing is so important on the wall. It's the most important thing. Remember to breathe.
[25:45] Jesus: Breathing is the most important thing in anything.
[25:48] Cody: Yeah, true.
[25:50] Gonzalo: Have you ever done scuba diving?
[25:53] Jesus: I have not, no.
[25:55] Gonzalo: So when you scuba dive, depending on your breathing rate is your consumption of oxygen.
[26:01] Jesus: Yep.
[26:02] Gonzalo: I remember mine was consumed like at half and the majority of people had just such little. And then I realized this is the type of anxiety I live with, right? So you were talking about breathing.
[26:23] Gonzalo: How often do you use Claude remote?
[26:27] Jesus: I've actually started using it more to really play with it. I have it, yeah. I like to use it for, you know, keep things going. So a lot of times like I'll have something running through lunch and so I'll check up on it on lunch and see what's going on. I have all my Claude sessions to be remote by default. And so I can check out whatever's running at one time. It definitely works well, but I still do my majority on my actual machine.
[26:54] Gonzalo: Question. Do you think it's a lie that you can just walk away? I've noticed that even when I'm on remote, my mind is still thinking about it. There's no such thing as you're gonna… you're still… to really disconnect, you have to disconnect.
[27:10] Jesus: Yeah, you talking about disconnect from in general or Claude specifically?
[27:13] Cody: I think because here's what you're saying, right? Like if you could really walk away and disconnect and it could be just a fully remote session, you wouldn't need remote on your phone. You wouldn't have to ping back in. You could just walk away and let it do its thing. But because you have to keep that remote connection, even though you're a mile away, you're tethered to what you're doing via your phone.
[27:30] Gonzalo: Have you completely gone like rails off like don't ask me for anything?
[27:34] Jesus: No.
[27:35] Gonzalo: Okay, good. Good.
[27:36] Jesus: I definitely not at work. There are some at home where I've played with it a couple times on just, you know, projects specifically set up to do that. It didn't work well. And I feel like the tradeoff you get is you can give it enough context up front, you can try to give it enough context up front, and say, you know, execute, do it, don't ask me for nothing. Two things you're going to either end up with is it's going to be not enough context that it needs to actually implement, or its end result is going to be so far off from what you initially imagined, it's not helpful. Like AI is not there. I think, you know, potentially some of these guys on Twitter that are like at Anthropic and OpenAI like semi-achieve that with this new loop, looping prompting or whatever, but…
Agentic Programming and Adversarial Review [28:00]
[28:18] Gonzalo: Yeah.
[28:19] Jesus: …they also have full infrastructures and setups and stuff to support that, which I still only think half of what they actually say is true.
[28:27] Gonzalo: I agree. I'm concerned about this focus on still selling.
[28:34] Jesus: Oh, yes. It's going to be like that for I don't know how long.
[28:38] Cody: And I agree with what you're saying. And I think there's also a caveat there. So, I'm also, I stay very involved in the whole thing, architecture all the way through. Sometimes I have it spin up a couple of like side features or something on a main, you know, piece, but it's all following the architecture and the plan. But what I've found is you can record, start a recording for a conversation, sit down with one of your people and discuss the project from start to finish, what it should look like, what it needs, have, you know, an adversarial conversation about, oh, we can't do this, it needs to have that. Sit down and talk for an hour, two hours, three hours.
[29:11] Gonzalo: For those who don't know what an adversarial perspective is…
[29:15] Cody: You're on one side of the fence and the other person is intentionally on the other side and they're trying to poke holes in what you're saying.
[29:21] Gonzalo: So a devil's advocate approach to whatever objective or idea is, which is very healthy as a human being.
[29:30] Cody: Yeah. And it's even more helpful sometimes when you're doing development or trying to create a project. And you can have a one, two, three hour long conversation, you know, structured specifically about this project, record it, have it transcribed, and that's your prompt.
[29:44] Jesus: Yeah.
[29:45] Cody: The whole thing. Put the entirety of a three hour conversation as your prompt.
[29:49] Jesus: It now has more context than you were ever going to write out.
[29:53] Gonzalo: So what you're saying is that it's really important what you also, your input is very important.
[30:00] Cody: So important. Yeah.
[30:02] Gonzalo: And the clarity of your input has a big impact on the output.
[30:06] Cody: Yeah. And there's a ton of nuance in conversation that just doesn't come out when you're writing, right? Like if you're prompting for something, there's a lot of nuance that Jesus and I might talk about a bug or a feature or something about whatever we're working on. And in that conversation, there is just a lot of nuance in what we were saying and the way we were saying it. And that gets lost in just trying to prompt out something. So recording that conversation, transcribing it has a huge, huge, huge bonus benefit to what you're doing.
[30:30] Gonzalo: Yeah. So the actual creating of an input also, it's almost like a self-reflection and you find out even more elements of what you're trying to find out.
[30:42] Cody: Yeah. Or especially, you know, you bring in that second person and you just have that conversation, right? Like you poke holes and…
[30:50] Jesus: Vice versa. You go in with an assumption and then we talk through it and realize, oh, this is not the best solution or this won't work for whatever reason. So, and this is kind of one of the things you have to keep in mind when doing agentic programming, especially if you're more of a solo developer, is that between yourself and the models, they have the knowledge that they have, the assumptions that they make on both sides, is that especially in this degree, the LLMs are going to be pigeonholed into a certain way of thinking or a certain type of knowledge that from an LLM perspective, you literally can't unless you give it additional context, plug it into different systems, or you bring in completely separate models. Just the way they may approach it might be completely different than the model you're using will approach it. And so for like solo development, you know, it's definitely, I would say, recommend getting other people in, but also getting other models in. And so that's why like over the past week, you know, we do a lot of our actual development with Claude and, you know, Fable [?] and Opus, but bringing Codex and GPT's models in, it approaches it from a completely different way of thinking that it'll find potential exploits, bugs, or inefficiencies that Claude couldn't think of coming up with because it's pigeonholed into a certain way of thinking.
[31:57] Gonzalo: Interesting. So you have found using…
[32:00] Gonzalo: multiple models do adversarial reviews of the code.
[32:03] Cody: Yeah.
[32:04] Jesus: Oh, interesting.
[32:05] Gonzalo: Yes. Yep. One model will review a different model's code.
[32:07] Jesus: Do you think people are also starting to become very uh allegiant to a specific…
[32:13] Cody: Yes. LLM? Absolutely.
[32:15] Jesus: Kind of like Mac and Android or cell… what do you call it?
[32:20] Cody: Mac OS and PC and…
[32:23] Jesus: Nikon and Sony cameras… everyone has their brands.
[32:27] Cody: A project I'd love to start sometime in the future…
[32:31] Gonzalo: I wish John were here because he'd be telling us a lot about uh open source.
[32:35] Cody: Yes.
[32:36] Gonzalo: And we will be talking about this. This is really important and I love learning. I've learned already a lot just listening to both of you. It's refreshing to being with other people who have I feel more of a level-headed approach as opposed to like, well, they told us to spend as much as we can and now what's happening is, you know, even finance is saying no more.
[32:56] Cody: I'd like to put in for context from a business perspective, from the time I've started this job until today, I have spent as little as possible and saved as much as possible across every project, including AI. I have not increased spend anywhere. I've actually over the last two and a half years saved about half a million dollars annually in recurring expense.
Cost Savings and Token Optimization [33:00]
[33:14] Gonzalo: Recurring expense. Yes. So your approach has been use as minimum as possible and when there's cost…
[33:23] Cody: And there's ways to spread that out, right? So like if I'm going to build a part of a data warehouse project, how much do you think that would cost in developers or in-house development and time and expertise and stuff to build that? It's a tremendous amount of money when you're working in corporate environments. Or you can buy two uh, you know, high usage max plans, spend $400 in a month and pretty much get to the same spot that somebody else would have gotten in six months across $50,000 in spend. So there's ways to balance that, right? And so I'm not saying don't spend, I'm just saying, you know, don't hit up an API and start burning through tokens at the fastest rate possible.
[34:00] Gonzalo: We never jumped on token maxing.
[34:02] Cody: Never. Token optimization. Uh, you know, it's funny, you can use a lot of the models to work as a router to tell you which model should be the best approach for a certain part of your project.
[34:14] Gonzalo: Oh, I love that.
[34:16] Cody: Yeah. So because they have a really good understanding of their own limitations. So Sonnet 3.5 might tell you, actually, you know what, yeah, this is really great for me, but this piece here should be done by Opus or something, or Opus can tell you, you know, this piece should be actually probably a Haiku thing, but we can bump this one down to Sonnet and save tokens here. And so you can kind of route your requests to different pieces of your project and sub-agents to do that.
[34:34] Gonzalo: Request routing. I love that.
[34:37] Cody: Yeah, and that's what I want to work on in the future is our own router so we can start like have a layer that whenever we send it to it, it auto-routes to the different models and what level or layers.
[34:45] Gonzalo: This is brilliant. That would be a great second…
[34:49] Cody: Because I mean most people, and I'm definitely guilty of this at times, just go and like, okay, whatever I'm working on, they don't properly assess the actual skill needed and so they just default to like Opus or Haiku and they're just using way more either of their limits or API costs than the actual thing, or even just the most basic things.
[35:09] Gonzalo: Could you imagine GPT-5 [?] on scheduling?
[35:11] Cody: Yeah. Yeah, and I mean eventually I think the…
[35:13] Gonzalo: I'm gonna crawl under a table because I've been uh guilty of that.
[35:18] Cody: Yeah, no, I'm definitely guilty of it as well, but you know, definitely I think custom options that work for the business and stuff and their environment you're working in, I honestly think in the probably near future given how this accelerates, um, the main consumer systems like the Claude codes and the, you know, I think they're going to get to that spot eventually where they're going to have built-in routers for their own specific models. Um, or maybe not, maybe they'll just try and…
[35:44] Gonzalo: I think so now with open source models that are starting to become…
[35:48] Cody: I think so. And I'm definitely excited to talk about the topic of open source models. I think it's an important one.
[35:53] Gonzalo: Let's leave that for the next session and um, when John's here.
[35:57] Cody: With that said, thank you for making this happen.
[36:00] Gonzalo: Yeah, we'll uh, we're probably going to be bringing in even some anti-AI individuals from PSU which I think…
[36:07] Cody: Oh, AI doomers? Oh, I'd love to talk to them.
[36:10] Gonzalo: Yeah, I think it's important. I think it's really important. And uh, thank you again for making this happen and let's continue climbing.
[36:16] Cody: Let's do it.
About the show
AI For Luddites is a plain-English podcast for the skeptical, the resistant, and the merely exhausted — no hype, no jargon, no promises that everything is about to change forever. Hosted by Cody, Gonzalo, Jesus and John Wee, from 210 Digital Marketing in Eagle, Idaho.
More from 210 Digital: our HIPAA-safe healthcare AI implementation roadmap · AI capabilities.