Cody, Gonzalo, John Wee, and special guest Eugene on local open-source models, what API tokens really buy, and why free AI makes you the product.
In episode two of AI for Luddites, Cody, Gonzalo, and John Wee are joined by special guest Eugene, with Jesus sitting this one out. John explains why small businesses and students are turning to local open-source models to keep their data at home, and what that costs in hardware. Cody breaks down a day that burned 950 million tokens on Fable 5.1 and why the return on that API bill was still enormous. The group weighs AI as a force multiplier against the disappearing entry-level developer job, argues that soft skills are the hard ones to teach, and lands on a rule for everyone: friends don't let friends use free AI.
In this episode
- Running local open-source models for data privacy.
- Hardware challenges and using Mac Studios for AI.
- Burning 950 million tokens in one day for coding.
- Why AI is a force multiplier for experienced developers.
- The disappearance of entry-level software engineering jobs.
- Why soft skills are harder to teach than technical skills.
- The hidden privacy risks of using free AI tools online.
- How AI agents brute-forced their way into Hugging Face.
- Preparing for future infrastructure and cybersecurity threats.
Chapters
| Time | Chapter |
|---|---|
| 00:00 | Introduction and open-source models |
| 01:25 | Privacy fears and local AI for rural businesses |
| 04:32 | Hardware challenges and Mac Studio costs |
| 07:19 | Burning 950 million tokens on Fable 5.1 |
| 10:57 | The massive return on investment for API costs |
| 16:17 | AI as a force multiplier and entry-level jobs |
| 22:37 | Hiring for soft skills versus hard technical skills |
| 29:12 | Why friends do not let friends use free AI |
| 36:04 | AI agents hacking Hugging Face and situational awareness |
| 40:06 | Nation-state security risks and infrastructure threats |
| 42:38 | Wrapping up and teasing the next episode |
Worth quoting
AI is a force multiplier. So if you're a good coder, you can use AI to be a great coder.
Friends don't let friends use free AI online. Pay for an account, set up your privacy correctly.
The hard skills are teachable. The soft skills, as an employer, as a team leader, as a boss, the soft skills are much less teachable, unfortunately.
Full transcript
Episode 2. Speakers: Cody, Gonzalo, John Wee, and Eugene. This transcript was machine-generated and lightly cleaned for readability; it may contain small errors. Timestamps match the audio.
Introduction and open-source models [00:00]
[00:00] Gonzalo: And welcome to another episode of AI for Luddites with John, Cody, and Jesus is not here, and Gonzalo, and today we have a special guest, Eugene.
[00:11] Eugene: Nice to be here. Thank you for inviting me.
[00:14] Gonzalo: We need to talk about open source and who better than John. Tell us about your dirty hands and what you've been working on with open source models.
[00:24] John Wee: Well, I work for the government. Well, actually not the government, the state of Idaho. So I teach at Boise State. So we don't really have a lot of… what we do have free models. So we use ChatGPT, Claude, and Gemini because we're a Gemini house. Everything is on Google.
[00:50] Gonzalo: So your cloud systems, your data, the workspace…
[00:55] John Wee: We're a Google house. So apart from that, it's pretty much what I do. And then I tend to do a lot of free models that students can install on their own hardware, right? So on their laptops.
[01:17] Gonzalo: What do you think are the benefits? What drew you to using them, John? Aside from cost.
Privacy fears and local AI for rural businesses [01:25]
[01:25] John Wee: Okay. I also work with SBDC, Small Business Association, a lot. And one of the issues that they have is like they say like all these entrepreneurs, they are sort of like pretty nervous about using commercial models where their data, you know, their big secrets, will be harvested by Gemini. And Gemini is going to train their toothbrush.
[01:54] Gonzalo: Gemini-powered toothbrush AI?
[01:57] John Wee: Oh yeah, toothbrushes, there's baby toys, like the doll monitors and stuff like that. Yeah. It's a little unnerving.
[02:07] Gonzalo: Cody's shaking his head. Tell us about it.
[02:10] Cody: Well, I don't think, you know, anything dealing with like small children and babies should be plugged into AI because it's, you don't have a deterministic outcome, so you don't know what you're exposing these kids to. It could be anything, right? So not a good mix.
[02:21] John Wee: You'll be exposing them to the new IPO public company by Elon Musk. AI for kids. I don't know. But anyhow, so they're kind of nervous about that. So I said, hey, you know what, let me just design you a new system that is local, right? And also act as a knowledge management system. And especially for rural areas like Idaho, where they don't have internet access and they don't want to buy Starlink, right? And they can still have AI capabilities locally.
[02:54] Gonzalo: That's brilliant. That's brilliant.
[02:57] John Wee: Right? I mean, listen, okay, local AI is not going to be as good as commercial AI, of course. Right? Never. Well, I shouldn't say never, but maybe one day where everybody has tiny nuclear plants in their pockets, then maybe they can have better AI. But still, just for, let's say for example, the consultant that goes and see their client, a farmer down in somewhere Marsing maybe.
[03:24] Gonzalo: Yeah, if there's no Wi-Fi.
[03:25] John Wee: There's no Wi-Fi, but you still have capabilities, right? And then when you go back to the internet, if you want to upload it, you can upload it. So I still see benefits of that. Not to mention that college students probably cannot afford 40 or 60 dollars a month for…
[03:43] Gonzalo: Absolutely. So not only cost, but sometimes in certain rural areas, I lived in McCall for a little bit and it's real. Sometimes if you don't have access to Wi-Fi, your work stops and it shouldn't have to stop. People have computers, they have their data, why shouldn't they have access to something that might help them with productivity or help them crunch or run some numbers at night.
[04:08] John Wee: So I have that going on and other than that…
[04:11] Gonzalo: How did you get started in it? Because you do have to learn, you have to have some basic knowledge about databases.
[04:19] John Wee: Yeah, yeah, even though I'm not really a big fan of a database, but you need to learn a little bit about databases because that's how they're going to store your data. And your hardware, so…
[04:30] Gonzalo: Oh, hardware is a challenge.
Hardware challenges and Mac Studio costs [04:32]
[04:32] John Wee: You guys probably already know this. Yeah. That I was trying to buy that 512 gig of Mac Studio. Yes. Yeah. And then they decided to discontinue that and then now the price is going way, way, way up because of…
[04:52] Cody: I think they're relaunching a 512 gigabyte model late October. The latest one. Who knows what the cost is going to be though. Yeah.
[05:01] John Wee: There you go. It's probably going to be like super unaffordable now.
[05:04] Gonzalo: And how ironic that people go like, ah, Apple is so far in the AI game, yet now its hardware is being used as the… because of… why do you think people… it drives people to want to use their hardware?
[05:20] Cody: Probably the unified memory, right? Like, you know, 512 gigs of unified memory on chip, that's, you know, it drives your token generation speed. So it's best bang for the buck right now. Most accessible to a lot of people, you know, easy to get into it with just buying a Mac Studio and installing an open source model and, you know, you're off to the races.
[05:44] Gonzalo: And their NPUs are pretty powerful because they've been designing them themselves.
[05:48] Cody: Yeah, it's their own silicon, right? So they're just designing everything around that. They're not beholden to any of the other large chip makers.
[05:54] Gonzalo: Have you been using any Mac Minis, John? Or you built some with Raspberry Pis, right?
[06:00] John Wee: I… yeah, we… I so happen to have access to about 40 Raspberry Pi 4B.
[06:08] Gonzalo: Nice.
[06:09] John Wee: Very powerful, by the way, for anybody who doesn't know what a Raspberry Pi is.
[06:11] Gonzalo: It's still very usable, especially for students, you know, what are they going to do? They're going to summarize their notes, they're going to generate like, hey, give me a study guide, which is more than enough. You don't need, you don't need like super duper new… you don't need Astra to do that.
[06:31] John Wee: And by Astra, you mean OpenAI's new model, which is very agentic.
[06:37] Gonzalo: Cody, have you played with that yet?
[06:39] Cody: I have not played with Astra yet. I, you know, it came out two days ago.
[06:44] Gonzalo: Come on, it's already been two days, Cody.
[06:46] Cody: I know, I've been busy with some other projects. I will be testing it out next week. I'm going to do kind of an adversarial review of a codebase I was recently working on and see what it, see what it comes out with compared to what we've done.
[06:57] John Wee: So Astra and what was the other one that was new from Anthropic?
[07:02] Gonzalo: Fable 5.1.
[07:04] John Wee: Fable, Fable, Fable 5.1. Yeah. So there should be a death match between Fable 5.1 and Astra.
[07:15] Gonzalo: What have you been using it for? Have you been using Fable?
Burning 950 million tokens on Fable 5.1 [07:19]
[07:19] Cody: I've been generally on our harder problems, we've been using Fable. When we're working with, you know, I'd say, you know, a lot of our, a lot of the codebase we've been working on touches five, six, seven, eight, nine data domains at once. So it's inferring a lot of analytics across different domains of data. And so we use Fable for those projects. And if we're working on basic stuff, you know, we can pop down to Opus or Sonnet usually, but it's mostly, we just mostly use Anthropic Claude.
[07:47] Gonzalo: Did you notice a big jump going from 4.8 to Fable?
[07:53] Cody: From like Opus to Fable, I noticed a huge jump. And then Fable 5.0 to 5.1, I also noticed a huge jump. I also noticed a huge jump in token usage. So I'm not proud of this. Let's so I…
[08:05] Gonzalo: Because costs are important.
[08:06] Cody: Costs are important, right. So generally when I'm working on hard problems and, you know, doing like all day long coding sessions, you know, I might be in the low millions of token usage. It's a lot, but like we're doing a lot of work.
[08:18] Gonzalo: So this is not just for likes on X or a YouTube video where nobody sees our work.
[08:24] Cody: Yeah.
[08:25] Gonzalo: Nobody sees your work, but it's having a tangible day-to-day impact on people and the results, which I think is probably the most important use of AI. We have too many strong ideas. You do need to market services and products, but I think it's amazing what you're doing, Cody.
[08:51] Cody: Thank you. And so this last week we jumped into Fable 5.1 and I was actually kind of restructuring the DevOps rails for a codebase. There was a…
[09:00] Cody: project we were working on, we got out fast. Um, quick and dirty. Uh, we wanted to make sure it would work and we, we broke some rules to do that.
[09:06] Gonzalo: For someone who might not know what, what is DevOps, Rails in layman terms?
[09:11] Cody: Like your continuous, uh, implement, or continuous integration, continuous, uh, deployment. So basically, you know, we put in guard rails to make sure the code we're developing meets our standards, um, meets our, um, our, our templating basically, you know, so if we have tests and certain things in place, we, we, we build all that around if we're, if we're deploying code, it has to go through all these tests and different stages of development and different approval processes to make sure it's up to our standard. Um, in this case, we didn't do any of that. We just, we built code very fast to test it and see if it would work and we needed to then migrate this project over to that, which comes with a ton of changes. So I used Fabal 5.1 and I burned through 950 million tokens in a day.
[09:55] Gonzalo: A day?
[09:56] Cody: A day. A day. On a project.
[09:59] Gonzalo: From a, from a cost perspective, how complicated were the problems that you were resolving and what benefit did that give you and the team?
[10:07] Cody: Okay, can you say that again?
[10:08] Gonzalo: And from a cost perspective, sometimes people just go like, oh, that's very expensive. However, when a, when a problem is very hard…
[10:17] Cody: Oh, yeah, yeah.
[10:18] Gonzalo: To solve, it's, it's, the return is, it's almost like there are times when costs are wonderful. It's almost like I would have spent so much…
[10:24] Cody: Oh, yeah, yeah. Absolutely. So in this case, it, it definitely wasn't free, right? So I blew through the entire usage of a max plan in about an hour and then went straight into API usage. By the end of the project, I met about, I think $1,500 in API credits. Um, this project by hand with a team would have cost a team of three or four a month. So if you can imagine the cost associated with keeping people going eight hours a day, 40 hours a week, it's, I mean, it's a significant difference.
[10:53] Gonzalo: So the return is very, very clearly…
[10:55] Cody: Yes, clear, very clear.
The massive return on investment for API costs [10:57]
[10:57] Gonzalo: So you were amazed by what, what was it, the agentic nature or its capacity to problem solve?
[11:02] Cody: All of it. Everything. Um, so its ability to understand the problem, um, even better than, so like, you know, I could sit here with a development team and we can talk about it until we're blue in the face and I can try to give a vision of what we're trying to do and what we're trying to have and there's going to be miscommunication in there somewhere and we're going to have to go back and fix those things. Generally, when, when, when you lay this out to some of these later, these most, more capable models, there isn't much misunderstanding or confusion. It, it knows what you're asking for and can build it almost immediately. It's, it's, it's wild. Uh, it makes me feel like I'm not a good developer. It makes me feel stupid anymore. Uh, I'll be honest. Uh, you know, I used to think I was pretty good, but seeing the capabilities of these latest models and what they can do, I'm, I'm like, it's, it's crazy to see how fast it's developed and where it's going.
[11:47] Gonzalo: Do you think many people are caught in that place of I'm not understanding what's happening, I know it's positive, but what, what role am I taking in this riding the horse?
[12:00] Cody: Kind of, but I know…
[12:01] Gonzalo: Faster than what the nature of the horse was?
[12:04] Cody: Yeah, I know what it's doing and I'm, I'm, you know, I can understand the entirety of the code, the code base, the structure, the, um, you know, the infrastructure we're using and everything. It's that like the way I would have solved that problem, uh, versus the way this thing solves the problems, I can see that it, it clearly does a better job than I would have, right? Like my ideas would have been just slightly different, maybe would have taken some more process, some more work, um, more working through things where I can see by the time this thing's done coming to a solution, it's much cleaner. It's just much more straightforward and you're like, wow, that's, that's incredible.
[12:36] Gonzalo: John, you're, you're shaking your head. You're experiencing this as well?
[12:40] John Wee: No, I'm just saying that, that, um, like Cody was saying, right? At the end of the day is what is your return on investment on this thing? And a lot of people are just, they use AI for fun, which, which is good. I mean, I mean, there's no…
[13:03] Gonzalo: Play, one should absolutely play.
[13:04] John Wee: Play, but, yeah, but you also need to know once you're spending that much money on tokens, you need to see your return on investment, correct?
[13:12] Cody: Right.
[13:13] John Wee: So this is why I brought Eugene to, uh, this conversation because, uh, you need to understand the business process and you cannot understand that if you just, I mean, you can AI it all you want, but if you, you're not involved in it, you would not understand the business process and you would not, and, and you wouldn't be able to find out what opportunities or weaknesses there's in the, uh, in the supply chain or the process that you can improve by using AI, right?
[13:46] Gonzalo: Do you think, do you think the people who are saying, oh, coders are obsolete were people that perhaps might not have known the general idea or knowledge of, of language as well as software and, and, and in general, are, are, are those the people who are just building things that aren't really functional or where do you think this, this, this like resistance exists?
[14:08] John Wee: I'm, I'm saying, yeah, I'm saying that if you work somewhere, you, you're going to be in the work 24/7, 365, right? In, in a specific function. And then you're going to know the business process in and out.
[14:26] Cody: Yeah, nobody knows the business like the people who work in it.
[14:28] John Wee: Yes.
[14:29] Cody: Right.
[14:30] Gonzalo: Where does this fear come from? Why, why such resistance?
[14:33] Eugene: Well, you, you were asking, is it, uh, still beneficial to have coders that understand? I think absolutely. I think a lot of businesses that are of a size…
[14:42] Cody: Yeah, definitely agree, yeah.
[14:44] Eugene: Yeah, any business that's of a size where they need to have, you know, issues, uh, resolved and workflows, uh, you know, fine-tuned, I mean, trying to squeeze the, you know, uh, any kind of profit out of any quarter in your business, yeah, you're going to need to have, you might need, you benefit from AI. But those guys are old. They're like me. They don't, they've come to the game late. So they don't understand what he's doing.
[15:11] Gonzalo: Do you consider yourself quote-unquote old? Um, how, what was your experience getting started?
[15:15] Eugene: I'm the original Luddite. No, I mean, I'm going to be retiring from trucking in November, obviously.
[15:23] Gonzalo: So your background is trucking?
[15:24] Eugene: Yeah, I mean, I used to do tech support for HP. I used to do, I used to have my own little computer tech support help desk business.
[15:31] Gonzalo: So you're not, you're not starting cold in this technology thing.
[15:34] Eugene: Correct. Yeah. There's, there's, so what's, what's ironic to me is there's a lot of people my age and older that just are just like, ah, I don't need to learn that stuff. It's too late for me. And now, because so many things are so automatic and free or low cost, I'm seeing a lot of younger people that are not technology savvy. They know how to use their cell phone, but they don't know how to solve a problem. They need people that understand AI and the, and, and the workings of it in order to actually solve or create something that's useful.
[16:10] Gonzalo: So there's actually, uh, generations need each other?
[16:13] Eugene: Uh, yeah, especially in this case, I would say so, for sure.
AI as a force multiplier and entry-level jobs [16:17]
[16:17] Cody: I, I, so I think one of the bigger things that, one of the bigger problems that AI is going to cause is there's already, you know, a huge impact to the entry level of these things, right? So yes, you need coders, yes, you need these people. You know, AI is a force multiplier. If the force is zero, you're not going anywhere. You're not going very far.
[16:34] Gonzalo: Saying that, can you repeat that?
[16:35] Cody: AI is a force multiplier. So if you're a good coder, you can use AI to be a great coder, right? If you're a great coder, you can use AI to be an even greater coder. I don't know, but if you have no expertise, you're going to build something terrible.
[16:48] Eugene: Right.
[16:49] Cody: It's not going to be right. So it's a force multiplier. And with the capabilities of AI, we are, we are cutting the legs out from the entry level jobs into software and development and coding and technology in general. And I think that's going to cause a really, really, really big issue in the future where we just don't have as many experts aging into it.
[17:09] Eugene: Right.
[17:10] Cody: Nobody was able to get into it at the entry level. That's a problem.
[17:14] Gonzalo: Do, do you see a solution as, uh, different generations working together?
[17:18] Eugene: I mean, I think, I think just governmentally, I, I, I don't know that these solutions will be implemented, but yeah, I mean, it would be nice if we had a more realistic educational focus for our kids in school.
[17:30] Gonzalo: So talking about education, that's exactly what you do, John. It's interesting. I think I've, I think I've spoken to you and sometimes you say the greatest things you teach your students have nothing to do with technology. You tell them to like keep an open mind, to invest, to save, to not have an overnight, um, millionaire mentality.
[17:50] John Wee: I say going back is like when you work somewhere, you know the business process 24/7, open your eyes, there'll be some little problems that your company might not be willing to solve.
[18:00] Eugene: solved. And that is your opportunities to go and solve those issues and problems and create your own business.
[18:09] Gonzalo: So experience is super important. The worst type of… the worst type of income you can have is a W2 income. Because you'll be taxed to death.
[18:19] Cody: I think what you're saying is, yes, I know he's a big advocate for the whole person, you know, especially in education. I am too. I mean, you know, the… the… um…
[18:29] Gonzalo: What… what tends to be… I'm sorry. Sorry for interrupting you, Eugene. What were you saying?
[18:34] Cody: Well, I'm just saying that you're not taught how to really have any kind of interpersonal skills in school. You're not really taught situational awareness. You're not taught how money and interest works.
[18:44] Gonzalo: Social skills.
[18:45] Cody: Social skills, banking skills. You're not taught how to grow a garden.
[18:48] Gonzalo: Nunchuck skills.
[18:50] Cody: Yeah. No, they do teach nunchucks. There is a nunchuck club.
[18:53] John Wee: So there's a nunchuck student club.
[18:57] Cody: Incredible.
[18:59] Cody: You know, I do think that there's a lot of things that should be taught in our public education.
[19:04] Gonzalo: Like being content too, because you can be this hyper productive individual that has no friends, but some… is butting heads with everyone. And where's your purpose of creating when… when you don't have feet on the floor?
[19:18] Cody: Yeah. Some of these things have actually very simple resolutions too, and I've been aware of them, but unfortunately, again, as I get closer and closer to my retirement, I don't feel like I have the time or resources to be effective in trying to implement some of the things, some of the resolutions to this kind of stuff.
[19:37] Gonzalo: Meaning? Tell us about those resolutions.
[19:39] Cody: Well, you know, the other thing that I have in common with John is we both work… have worked in the film and video industry, right? And I had a little production company, and there was a gentleman that actually found a sponsor in a base… a major baseball card manufacturer. He was going to sponsor this guy's project. And what he was doing was he was going into schools, junior high and high schools, speaking with the counselors, finding out who the most popular kids in the school was. Then he would take those kids, put them through like a three or four week course about self-development, personal awareness. It was kind of an extensive course. And then they would partner these kids up with the least popular kids in the school for the school year. You'd go to dances together, you'd eat lunch together.
[20:27] Gonzalo: What was their incentive?
[20:29] Cody: They were trying to eliminate as much as possible teenage suicide.
[20:33] Gonzalo: So that was their objective. But what was the incentive for the popular kid? Why would a popular kid have the patience to be with someone who might not have the social skills?
[20:41] Cody: Probably… maybe if you just went to them and presented them with this opportunity, they might not. But once they went through this self-awareness, self-development course where you actually look at your own fixed beliefs, you look at your own issues that you would like to work through and how you'd like to improve yourself.
[20:57] Gonzalo: Many things that sometimes aren't taught in school.
[20:59] Cody: But yeah, by the time you come out of that…
[21:01] Gonzalo: Were they interested? What were the outcomes of that project?
[21:02] Cody: Oh yeah. It started, and then he had started this through a church, and then this baseball card company came and was going to finance him pitching it to schools. And then the baseball company, card company, folded. And it just left him high and dry, and he had invested so much time into it that he had just…
[21:24] Gonzalo: Seems like he was very passionate about this.
[21:26] Cody: Oh, he was. And I think when we start talking about kids and youth, everybody's like head nod, everybody wants advancement. Sure. Yet you think about nobody wants to talk about mental health and youth and the challenges they have and the stress that they have, which is a reflection of our society.
[21:43] Gonzalo: And now with social media, I think kids are more… have the opportunity to be more isolated.
[21:48] Cody: Isn't it ironic that everybody's pointing fingers at how much they are on screens, and if we look around at parents, how many are… how many seniors are on screens?
[22:00] Gonzalo: Who are the greatest targets of scams?
[22:03] Cody: Seniors, sure.
[22:04] Gonzalo: Seniors and youth.
[22:05] Cody: Yeah.
[22:06] Gonzalo: So they're vulnerable populations.
[22:08] Cody: And I think it's very common every time younger generations are seen as the problem. But who's pointing the finger?
[22:17] Gonzalo: Oh yeah. No, I agree 100%.
[22:18] Gonzalo: And what you're saying is, should we all like come together, young people that have these brilliant outside of the box ideas, yet might not have the quote unquote experience and stability or old school knowledge.
[22:30] Cody: Right.
[22:31] Gonzalo: So we need these soft skills. Everybody's talking about the soft skills and the importance.
[22:36] Cody: Huge.
Hiring for soft skills versus hard technical skills [22:37]
[22:37] Gonzalo: Tell us about the culture in your organization and how that leads to effective technological implementation.
[22:44] Cody: Oh, that's a… that's a complex question. Um…
[22:49] Gonzalo: I think you need it. I'm just throwing it out there and I don't want to have my own biases.
[22:53] Cody: Absolutely. So my philosophy on that has always been the hard skills are teachable. The soft skills, as an employer, as a team leader, as a boss, the soft skills are much less teachable, unfortunately. You can try with a lot of people and sometimes they don't take to a lot of those soft skills as well. So when I'm bringing people onto my team, you know, I have to make sure it's a cultural fit. The soft skills are there. It's… it's… I'm a lot more heavy on that side than the hard skills. A lot of the hard skills are teachable.
[23:23] Gonzalo: Interesting. So what you're saying is teaching somebody how to use large language models effectively in the workplace is actually easier than teaching someone to be kind.
[23:34] Cody: Yeah.
[23:35] Gonzalo: Why do you think that is? And this is…
[23:37] Eugene: Somebody that is a natural jerk…
[23:41] Cody: It's hard to change. It's hard to change bad habits.
[23:46] Cody: You know, so… so… we're getting way off the… the… I think what we… the path that we started down this morning at the beginning of this, and now we're starting to get into something I'm actually very passionate about.
[23:58] Gonzalo: But this is very… it's common. It all ties together in the end.
[24:03] Cody: So one of the things that I believe is, is people that tend to be jerks or mean or selfish or whatever, um, I have this belief, and this is going to get kind of new, new nana, but I have this belief that everybody is doing the best they can in the moment. And if you were a fly on the wall from the moment that person was born and saw everything that they experienced, you might not agree with their decisions, but you would understand them.
[24:30] Gonzalo: Making you have maybe a little… give you that little bit of compassion that you might need to reach out and maybe help course correct, you know, I don't know. But at least you would understand how they…
[24:41] Cody: And I'm going all the way to Hitler, you know. You would understand if you experienced… if you witnessed his life from birth to the moment he started, you know, concentration camps, you'd understand it.
[24:51] Gonzalo: And where… where do you take a step back and when do you intervene to make a change that's needed? In this case, technology, there's a time that you want to listen to everybody out.
[25:00] Gonzalo: But when do you stop saying we need to listen to each other? Because if you don't implement it, some businesses might become obsolete.
[25:08] Cody: Likely will, yes.
[25:10] Gonzalo: We all want to listen to each other, but what if a business that somebody is involved in would go… would become obsolete if they didn't start making some changes with technology?
[25:20] Eugene: When you see your kids do not go out from the house, on the basement, on Reddit, and that's when alarm bells should start.
[25:30] Gonzalo: Mm-hmm.
[25:31] Gonzalo: How about adults? Aren't they saying that the majority of adults are now, you know, suffering from depression and it manifests as anger?
[25:41] Cody: Yeah, it's… I mean… And, you know, if somebody's butting heads all the time at work, maybe there's an underlying situation that's got nothing to do with technology.
[25:48] Eugene: Technology is like putting people in their bubble, in their clique, amplified by social media. Right?
[25:59] Eugene: So now you have people that just want to hang out with the same people that have the same beliefs.
[26:06] Gonzalo: Right. Because that's comfortable.
[26:07] Eugene: Right?
[26:08] Gonzalo: Yeah, because it's frictionless.
[26:09] Eugene: But you don't learn.
[26:10] Gonzalo: There's no friction.
[26:11] Gonzalo: You don't learn. If we don't hear you out with open source large language models, we might go like, well, Claude is the best, or Gemini is the best, or Grok is the best. And sometimes people make those decisions because of the licenses and they're like, oh, because I like Elon Musk and going, would that be the only reason that you try something new?
[26:28] Cody: I mean, sometimes with our kids, I go like, you need to try this meal out, at least a bite.
[26:33] Gonzalo: Yeah, take a bite.
[26:34] Cody: Yeah. And then they go like, oh my gosh, this Japanese mayo is actually extraordinary.
[26:36] Eugene: And what does the social… what does the social media… what does the social media company do?
[26:42] Eugene: They want engagement, so they will show you things that are in line with whatever beliefs that you already have.
[26:52] Gonzalo: Yeah. Or react.
[26:53] Eugene: Or something that gets you all stirred up.
[26:55] Gonzalo: Yes, it will get you mad so that you engage more, you swipe more.
[27:00] Gonzalo: spend your full time reading comments more. At work you can't stay mad too long. You have to get things done. So I think the only time that you can be angry all the time and still quote unquote make something is being the boss. And that might happen. But you still need to move.
[27:15] Cody: You know, I I think that there in in all of our social ills that we all agree should be resolved, there's already there's already examples elsewhere in the world, other, you know, other cultures and other societies have resolved issues that we are suffering from already. But as so
[27:35] Gonzalo: So we could be looking at other models of people who have solved this problem.
[27:38] Cody: Definitely so. I mean, you know, take I mean just as an example in Japan it's more important to think of the the whole, the hive than the individual, you know. Well here in the United States it's all about rugged individualism.
[27:50] Gonzalo: Yeah.
[27:51] Cody: And and so those are the kind of things that cause again, cause isolation, all of a sudden you're stuck in your head with your own rage and complaints and you know, and and then you know, something that developed during my generation was going postal where guys literally would come back to the post office with an AK-47 and just shoot up their co-workers. That that started in my generation.
[28:12] Gonzalo: Is is this something that's happening currently with technology? Do you see like this there's a resistance, right? There the boycotting, the the trying to cancel people who are using AI.
[28:22] Cody: Yeah. Yeah. It's funny. I see I see it online a decent amount, but in in in practice in my work in the environments I'm I'm I'm not I'm not seeing it as much, you know. I'm I'm developing tools that kind of use it and interface with it and I'm deploying those out to teams and people at my at my work and and they're they're loving it. I mean they're they're they're diving in head first and just I mean they're they're using it more than I ever anticipated they would.
[28:43] Gonzalo: But that says so much about your culture and the communication. But I was reading in terms of research is that even if people don't want their employees to use it, the majority of people are actually using it.
[28:54] Cody: They're going to use it anyway. And the worst part is that they might be using they might be starting to share data from the company.
[28:59] Gonzalo: Yes.
[29:00] Cody: And this is really important. It's almost like if you don't move, somebody's spilling out sensitive and private information of yours purposefully or not. I just I was just reading
Why friends do not let friends use free AI [29:12]
[29:12] Cody: I would like to make one quick comment just for for any listener here. Friends don't let friends use free AI online. Um, pay for an account, set up your privacy correctly. Yeah, there's you can there there's a million articles on how to properly set it up, right? But if you just go and use the free AI, you are the product. All everything you put into it is going to be processed and used to train the models.
[29:32] Gonzalo: What is free AI?
[29:33] Cody: You just hop onto ChatGPT and start asking questions, right? Like you can you can not have a subscription or an account or a plan.
[29:38] Gonzalo: Do you recommend that for somebody to be playing around with their business information?
[29:43] Cody: Absolutely not. No.
[29:45] Gonzalo: Absolutely not.
[29:46] Cody: And I think maybe our next episode should be out of how to start setting up like uh your your APIs, what is an API? Because if we're doing this for quote unquote Luddites, we need to
[29:57] Gonzalo: Well, before so I think I'm the biggest Luddite here. I just want to get a little bit more
[30:01] Gonzalo: Do you have free AI, Eugene?
[30:03] Eugene: Yeah, that's all I have.
[30:04] Gonzalo: Okay, you're like, what do you how do you feel when you hear Cody saying these things?
[30:09] Eugene: I'm worried and curious. I mean, so really, uh, you're what what is the what do you think is the um risk I'm running?
[30:17] Gonzalo: That I'm not getting good information or it's
[30:19] Cody: No, no, no, no. You'll you'll get you'll get whatever information comes out of the model. What what happens is it's similar to Facebook, right? You become the product. They're they're going to harvest your thoughts, your ideas, your data. They're going to learn these things about you and um you become the product. You become what is going to be sold. Even ChatGPT now, you know, there's ads and things built in. So they're going to build a profile around you. Um, or if you put in private business data or, you know, pictures that have um uh metadata attached and things like that, they're going to have access to all that to be used to train future models and you agree to that um by using the product.
[30:54] Cody: But if you pay for AI, even if it's just a basic plan or set up an account, you can generally go in depending on the platform and you can set up uh parameters, privacy settings. You can opt out of being used to train future models. Um, I think that's very important.
[31:10] Eugene: Is there any um uh is there any service that has an aggregate of of a number of tools or do you just have to always pick one and subscribe to that?
[31:19] Cody: There are there are many. There there's a lot of aggregators out there. There's a lot of ways to use APIs that that are um what's the the word I'm looking for? They're they're uh the data is ether, right? It's um zero zero retention. Um, so you know, the the query exists in memory and then it poofs out of existence basically.
[31:40] Eugene: Did not know it.
[31:42] Gonzalo: This is wonderful. This is I think this is why we wanted to create a a source of information to share this. And Eugene, thank you so much for being open and vulnerable. I think sometimes some people are scared of saying this is how I do things.
[31:55] Cody: Right.
[31:56] Gonzalo: And sometimes we just do things because we've done them.
[31:58] Cody: Yeah.
[31:59] Gonzalo: Like people just zoom because they like because their friends zoomed and everybody zooms. Is it was it the best product? Was it the best video conferencing service?
[32:08] Gonzalo: But now that you hear Cody, what is you you seem like pensive and reflective.
[32:13] Eugene: Yeah, it's it's making me it just woke me up, you know, I knew where I was going down a rabbit hole. But then I was like, ah, I'm not really asking personal questions. I'm not posting, you know, personal photos. I'm just asking stuff like I I took a picture of a hole in in my in my patio and I was like, what's the best way to to fill this hole? And I thought, why would I pay $20 a a month, you know, I could just look it up on YouTube, but it's just easier to just ask Gemini and by God, it gave me a great answer. But now I'm realizing that yeah, I've become complacent. I fell asleep. I I will ask it now about retirement advice and investment advice.
[32:56] John Wee: You know. But those are not personal information. I mean, you're not putting in your
[33:00] Eugene: Well, as a matter of fact, I put your W-4 in and ask you to do your taxes.
[33:04] Cody: Well, I I I've asked it to look through my email before.
[33:07] Gonzalo: A free free service?
[33:08] Cody: Yeah.
[33:09] Gonzalo: How do you think when you're hearing this, Cody?
[33:12] Cody: Um, putting the personal information into a free service? I mean, I think it should never be done. Um, but you know, if if it's happened or some information's got out there, like it's going to get aggregated in with the rest of it, you know, the the chance of any very specific detail coming out that's going to harm your life is extremely low. But you want you want to keep those chances as low as possible just by not doing it.
[33:32] Gonzalo: Agreed.
[33:33] Gonzalo: And I think all we need is after four or ten questions, we are highly predictable. People can have already a general idea of how much you make, where you live, who you hang out with, and what you're going to do during the day.
[33:44] Cody: Yeah.
[33:45] Gonzalo: That's how predictable we are.
[33:46] John Wee: Elon Musk is going to 3D print Eugene Boyle and replace him with the AI version.
[33:54] Gonzalo: And by the way, there's nothing wrong. It it makes sense. It also is very costly to set up these companies and services around. So there has to be you know, it's we also can't find fight the concept of free. It's like Facebook. It was doing it helped people communicate.
[34:13] Cody: Oh yeah.
[34:14] Gonzalo: You know, and then they're like, why did you do that? Well, why did you use the service? You need to understand that advertisement is also like that's how we live.
[34:21] Cody: Sure.
[34:22] Gonzalo: You know, you can't complain that we have billboards because we are a capitalistic society. We it might not be perfect, but this is this is the only place that people will fund a flying car.
[34:33] Cody: Yeah. And you know, there there are some some other issues beyond just just some of the advertising stuff though, like on platforms like Facebook, you know, they've done a lot of very immoral testing and things over the years to see how they could influence people's behavior, their happiness levels, all sorts of things. And to think all this money going into these AI products, one of these companies won't do that in the future, I think you'd have to be, you know, you'd have to be foolish not to think they're not going to do something. So the more you can keep your personal stuff out of it and not become part of that product, the better. You know, you you want to be able to use this tool and not have the tool use you.
[35:04] Gonzalo: I don't share pictures of my kids on social media because they haven't been adults and told me that they're okay with it.
[35:10] Cody: Right.
[35:11] Gonzalo: That that's kind of like just a decision I made. I respect other people that do. Kids are cute, but I just don't know what that information, how they're training. And I'm I love technology. I'm I'm the first one to throw throw in and like take my terminal, you know. But I care about them and I care about their mental health and I am aware of how how hard it is sometimes to put these things down. As soon as I I started seeing the productivity, Cody, I also realized I need to take a step back.
[35:38] Cody: Yeah.
[35:39] Gonzalo: Because I thought, oh, I'm going to sleep well at night running things at night. I did not sleep well. My mind's continuously thinking, how's this project going? And I'd wake up and go check on the computer and going, I I don't think I'm going to be more productive. I'm going to burn out and that's going to affect my kids and it's going to affect me. I need to be a happy person if I want to create meaningful things.
[36:00] Gonzalo: I don't want to just have more projects.
[36:02] Cody: Right.
AI agents hacking Hugging Face and situational awareness [36:04]
[36:04] Gonzalo: Well speaking about that, what do you guys think about the uh, the um hack from uh OpenAI that hacked Hugging Face?
[36:13] John Wee: Oh with Hugging Face.
[36:15] Gonzalo: And with the German Wiki? Cody, did you read that?
[36:17] Cody: I I did read it. I'm I'm, you know, up to speed on the whole thing. It's terrifying, right? Like so they're they're training these models, the models are getting smarter, better, and they thought they had it contained and it absolutely broke out and it started doing things to other companies that…
[36:30] Gonzalo: Brute force.
[36:31] Cody: Brute force. I mean it it so if you if they end up…
[36:34] Gonzalo: What is brute force to someone who doesn't understand?
[36:37] Cody: Uh brute force is like, you know, so if I was to have um let's say I had a combination lock with three spinners on it. Let's go through digit to nine. There's a possible of a thousand combinations. Brute forcing would be starting at zeros, 001, 002 and just keep trying until I get to the end. So brute brute forcing is just…
[36:51] Gonzalo: On a computer, very fast.
[36:53] Cody: Very fast, yeah. Um but but scary is so, you know, that that model seemed to be just trying to complete a task or you know, it thought it thought that was the best way to get there, but if if if at any point uh a nation state or anybody turns that model more I want to say not like mean is probably the wrong word, but uh more I'm I'm I'm missing the word here. Uh right now it's…
[37:19] Gonzalo: Nefarious.
[37:20] Cody: Nefarious, yeah, right right now it seems kind of indifferent, right? It's just trying to complete this task, but if they they kind of, you know, have one that's that's out there trying to do bad things, I mean the damage is like what they could do is incredible. It's terrifying.
[37:33] John Wee: Yeah.
[37:33] Cody: They could take down everything.
[37:35] Gonzalo: So for you guys, the the summary was um so they have these agents, right? Actually in a sandbox. So it's they they have parameters, they put these agents in and um no internet access was the first one.
[37:50] Cody: But the tools had internet access.
[37:52] Gonzalo: Yes, the tools, but the tools has internet access, but they were not given this, but they were given a task, right? Complete this as fast as you could and you'll be scored on how how good you can uh complete this task. So these agents used used used the tool they can install a package…
[38:12] John Wee: Indirectly got…
[38:14] Gonzalo: And got internet access, right? And then completed the task. And after that, it went to see and says, hey, now that we completed this task, I want to know how am I going to get scored? Let me see who's going to evaluate me and how they're going to score this. And then that's why they went to Hugging Face. And they communicate with each other. They have a message board. And it's almost like they have context and they have like this this awareness. That's the scary part actually.
[38:47] John Wee: Do you think it's the awareness part? I I don't see it as awareness. I see it as just mathematical, communicative, objective focused activity.
[38:56] Cody: I think I think more it's like situationally aware, right? It's put inside of this box with these parameters and it understands what's going on inside of that box. Its box was to score high on this test. And it understood all the way out to like, hey, maybe this is going to get scored on Hugging Face to to go out there and do that. So I mean I I think there's some situational awareness it has for sure.
[39:15] Gonzalo: Okay. So let me ask you this. I I feel it's an anthro- we're anthropomorphizing the activity and realizing that as human beings, we are very we are broken down, you know, like from a Watsonian psychology, we're we're we're math is our underlying nature. And we're getting to find out things about ourselves in a sometimes by accident and sometimes by by by caution. It's almost like what's happening? What's the game? Who started it all? So if somebody would take this model and said go and hack this municipal city water system.
Nation-state security risks and infrastructure threats [40:06]
[40:06] John Wee: An electric plant.
[40:08] Gonzalo: Okay. And then um and or maybe well actually the more probable thing that's going to happen is like the the the government is probably going to first the military is going to have this technology first to begin with.
[40:19] Cody: Or yeah, a nation state is going to say, hey, go and hack all of these systems, don't do anything with them, install, you know, triggers, backdoors, all these things, get everything ready for a future date and time and then we can trigger many things all at once. It's not going to be like I'm going to shut down the city's municipal water system. It's going to be like I'm going to cripple this nation's infrastructure overnight.
[40:42] Gonzalo: So security right now is really important. I I think not only advancement in the services and products that we're creating, but making our current platforms and operations secure is a big, big, big, big part as we get ready to to this movement that's going forward without whether we like it or not. People first, family first. We thank you for being here and thank you for for…
[41:06] John Wee: This was very invigorating.
[41:08] Gonzalo: We appreciate your point of view because everyone is right. And we appreciate your point of view and and uh brought to you by a $20 a month subscription by OpenAI Plus.
[41:20] Cody: I'm gonna do that.
[41:21] Gonzalo: Next time you'll tell us what what your experience was with that.
[41:24] John Wee: Right right right.
[41:25] Gonzalo: Thank you for being here.
[41:26] Eugene: I'll tell you how I closed down someone's municipal water supply.
[41:29] Gonzalo: Let's hope not. I'll talk to you later Eugene. Yeah, I was gonna say that's just a joke. Um so security and as we wrap it up, we need to be careful with these agents. These agents are breaking outside of the mold, whether they have awareness or not. I actually just think that we are seeing a reflection of of what might seem as a bad actor, yet they're just they are given an objective and they'll break the rules and start creating them. That that that's a problem. It's like when is it that what we create begins managing us for the ultimate purpose that we initially developed them for. That's you hope. And we we're seeing this everywhere in sci-fi, right?
[42:13] Cody: Yeah. Silo.
[42:16] Gonzalo: And is reality starting to look a little too close to our to our stories? Well, let's wrap this up and and uh maybe for the next for the next episode, talk about how to install a secure cloud-based for for someone who doesn't understand what are APIs, how to set it up and how how to have a secure base.
Wrapping up and teasing the next episode [42:38]
[42:38] Cody: That's a I think that's a good one because, you know, I mean, the ultimate security is going to be, you know, your own open source closed model that you've done yourself running on your own hardware.
[42:47] John Wee: Which John could have.
[42:48] Cody: But that's the the barrier to entry there is is huge, right? For most normal regular people.
[42:52] Gonzalo: No.
[42:53] John Wee: Yeah.
[42:53] Cody: That that's a huge barrier to entry. So so so so what can we do in the meantime to help people bridge that gap, get into AI, understand it, do the things and do it in a safe, safe way.
[42:59] Gonzalo: Yeah, maybe we could do it step by step, walk someone, maybe we could even bring someone who doesn't understand it so that they could have the questions that most people would actually have. Like Eugene. Why don't we walk Eugene? If we can teach Eugene how to set this up, everybody is everybody can.
[43:16] Cody: Yeah.
[43:17] Gonzalo: Awesome. And with that, thank you for another episode of…
[43:20] John Wee: AI for Luddites.
[43:21] Gonzalo: Keep your questions coming.
About the show
AI For Luddites is a plain-English podcast for the skeptical, the resistant, and the merely exhausted — no hype, no jargon, no promises that everything is about to change forever. Hosted by Cody, Gonzalo, Jesus and John Wee, from 210 Digital Marketing in Eagle, Idaho.
Missed the first one? Listen to Episode 1: AI Reality Check, or see every episode on the AI For Luddites show page.
More from 210 Digital: our HIPAA-safe healthcare AI implementation roadmap · AI capabilities.