Talking Drupal #515 - AI with amazee.ai

Nic: This is Talking Drupal, a weekly chat about web design and development from a group of people with one thing in common. We love Drupal. This is episode of 515 AI with amazing. On today's show, we are talking about ai, how it can be privacy focused, and what am amazing AI is doing. To help with our guest, Michael Schmidt.

We'll also cover light LLM AI provider as our module of the week. Welcome to Talking Drupal. Our guest today is Michael. He is the founder and general manager of am Amazee IO and am amazing AI with a passion for open source technologies and a focus on data privacy. Michael is led the development of amazing AI's Drupal AI provider to empower content creators and site administrators without compromising on security and control.

Welcome back to the show and thank you for joining us.

Michael: Thanks for having me. Good to be back.

Nic: I'm Nic Laflin, founder nLightened Development, and today my co-host are joining us for the third week Rich Lawson, associate Director of Technology at Evolving Web. Welcome back. Thanks, Nic. It's great to be back.

Great to see everybody. And joining us as usual, John Picozzi, solution architect at EPAM. It's up everyone. And now to talk about our module.

We, let's turn over to Matt Glaman, principal software engineer, Acquia, and a maintainer of a number of Drupal modules of his own. Matt, what do you have for us this week? So,

Matt: I thought to be on point would be the AI provider module for light LLM. So this module is, was created in February of this year. There's versions 1.1 and 1.0 available that match the AI module itself.

Versioning. And the goal of the module is it integrates with light LLM, which is an AI gateway. I don't wanna spoil too much 'cause I'm sure there'll be a big part of the discussion today. But I feel like it, it is good light. LM is what a lot of organizations will end up going to when they wanna use AI and give it access to their developers.

So this module lets you have Drupal talk to the AI gateway at light LLM instead of directly to open AI or Bedrock or your other providers. It's actively maintained. The maintainers are the three developers from freely Give Marcus, Andrew, and Mark. It works by extending the open AI module itself and just tweaks it a little bit so that way you can use all the models available inside your light LLM instance.

And another cool thing about light LLM is it is open source. So it's an open source AI gateway.

John: Very cool. I'm always, I'm always very impressed with, I don't know how Marcus has all this time. 'cause it seems like he is literally, you know, in, in the ai module itself, in all of these other sub modules and you know, commenting and doing all, all of the things.

But good to see. Good to see. He is, he is maintaining this and yeah, I mean, I, I think there are, you know, a ton of these modules out there. The ecosystem is very rich and I'm sure we're gonna talk about a lot of them today. But yeah, I mean, I think this is one that Matt, have you ever, have you ever used Light Light LLM?

And if, if so, what have you used it for?

Matt: Yes, so that's one where I, like I said, I don't wanna see a bit of the show, 'cause I'm sure that's exactly what Michael's gonna dive into. But it has been for its governance features and costs and just, you know, if, if anybody's worked with AI providers, all the APIs are just a little bit different.

And one nice thing about light LLM is it's the open AI API. So if you're talking to AWS Bedrock, you just use the open AI API framework or API spec, basically with that. So it just helps normalize your access.

Nic: Interesting. So, so Drupal's already doing some of that by allowing you to swap providers, but it sounds like this is more inter, it's better for inter project compatibility too.

Like if you have a large, if it's a large organization, you have multiple teams integrating with ai, it's great to use this because then everybody's integrating to the same, the same point. Yeah. It's, I could say like,

Matt: yeah, for internally, for us, it's to make sure that developers can have access to LLMs at scale and monitor costs between departments.

John: Okay.

Nic: Hmm. Very cool. I I also love their pricing model, which is, if you're open source, it's free, and if it, everybody else is considered enterprise and you have to get in touch. So no way to know how much is this is gonna cost you.

John: Interesting. So, because, because I feel like I'm a, I'm a dummy. When, when like, like, I'm like, oh, ai, I got the ai, I know what AI is.

Oh, LLMI know what, I know what the acronym stands for, but I'm wondering like, where does this sit kind of in the chain of like your Drupal site, right? So like, let's imagine like I'm a user and I'm, I'm interacting with like Drupal chat chat bot, right? Like so is Drupal and going through light. LM to your, you know, your model backend and getting the data.

And then, so it's basically just providing that kind of like, a normalization layer basically. Right. Am I, am I describing that right?

Michael: Yeah. Some

John: people call it like an AI

Michael: gateway where instead of talking to the L LMS directly, you add, basically it's like in a world you look like an a, a reverse proxy.

It like another topic. Instead of an API gateway,

John: you have a, like a AI gateway. Yeah. Right. Okay, that makes sense. And instead of

Michael: like, as a company, instead of, that's, that's what Matt referred to is like instead of giving everybody your, their own. Let's say open AI API keys directly, and you need to manage all that all on open ai.

And then you need to do the whole also on cloud. And then you have another one that you wanna use, I dunno, deep and whatever. Instead you just give them one API key on light, lm. And then in light you can configure all these different endpoint and the other, so that means you can also monitor the token usage and all that stuff.

Mm-hmm. Plus, yes, the API normalization, your developers basically need to learn one API, which is the most common use. It's pretty much the standard right now is the open AI API, and in the background, it doesn't matter where the LLM is. Well, the API stay speak.

John: Now, if you're using light LOM and can you specify what, what backend you want it to use?

Like, Hey, I want you to use Claude. Go get this information from them, correct? Yeah. So you can actually,

Michael: so you can give the developer the choice or you can do the, you can configure it in light lm, so I can, I can. Provide you the models. I can, I could literally tell you you're using cloud, but in the background you're actually using something else.

Yeah, yeah. And there's also failover and local lancing capabilities so that if one of them does not respond in a specific amount of second, send the request to another one and all that stuff. Yeah. It's like an, an ai, an L LM gateway is probably the right thing.

Matt: Yeah. And that's the main feature of the module.

Yeah. Is the open AI module itself has some very specific logic that checks the model name. So it's very open AI model specific. This just does a little bit of magic to make sure like, oh, you're using anthropic Claude, great. It works just fine, even though it doesn't say GPT-3 0.5 or whatever in the model.

Right.

John: So it just, it, it kind of normalizes it for you, so you don't have to think about like, oh, I gotta do the, you know, how, phrase it in a certain way or do a certain thing for this, for this model. Yeah. Huh. Very cool.

Nic: I mean, I, I mean, I've been seeing a lot of these types of services where like. The idea is that AI is moving so quickly, which model is good at which particular thing changes day to day?

And being able to pivot quickly seems like, seems like a good a good service to provide. Right. So it seems like there's a lot of tools like that

John: just makes me feel very basic in my AI use. 'cause I just use open AI for everything, so.

Michael: Well that's right. That, that's okay. Like for one individual people, that's perfectly fine.

But yeah, if you like, like we have customers that think about how do I roll out now LLM access to a thousand people. Yeah.

Nic: Yeah.

Michael: Like telling them and like they're saying, well, some of my people want to use cloud, other ones want to use that. And then I have, and now my infrastructure team just told me they bought some cheap use.

Now we wanna do on-premise stuff. Like how do you manage all of this? And so having one centralized point where everything goes through just makes it a lot easier. And that's what light does it,

Nic: it. It also depends on the integration side too. 'cause I have a friend that you know, part of his job is writing AI for the company that he works for, right?

The, the actual LM side and, and logic. And so he uses a lot of AI internally and he's told me that it's not just the model or the company, it's also it, it's also the integration that you're working with. So for example, we all know Gemini General is pretty, pretty bad, but they just had a new release 2.5 I think, and he said that 2.5, if he uses it through like an IDE integration, it's the worst thing he is ever used.

But if he uses it directly on the Gini portal, it's currently the best code writer out there, which I don't know if Lite LLM can help with that case.

Michael: I mean, you can, I mean, there's a lot more that light alarm can do. Like one of the big things is guardrail. So you actually have another LLM that analyzes your pump first before it's sent there.

And you could basically check the, the, the prompts, different things. There's also like a telemetry observability part. Yeah. So there is a lot of things that you can modify the request if you really wanna. Yeah.

Nic: All right. I think you found a good on-topic module of the week. Matt, if our folks wanted to suggest a module leak, we get in touch with you.

What's the best way for them to do that?

Matt: Thanks Nic. Listeners, if you wanna suggest a module of the week or just connect, you can reach out to Martin who normally runs this segment on Drupal Slack, or his handles are Mandclu on socials.

All

Nic: right. Thank you so much for joining us this week, Matt. Appreciate it. See you.

Okay, so we can move on to our primary topic. So Michael, it's been, it's been a while. We were discussing before the show it where you were last on episode 2 57. We're currently on episode 515. It's been a few years what have you and am amazing. Been up to since the last time we talked.

Michael: Yeah, it just track five.

It's almost exactly five years ago. We were all locked down. It was in a very different place. Yeah, lots have happened. Obviously we're still doing what we have been doing and what we talked about five years ago, which is Kubernetes, making Kubernetes easy for customers to use without ever needing them to access Kubernetes.

And in the recent years, it was really, I would say, when the AI craziness happened couple of years ago now as well. We originally, the first, the first goal that we actually tried to pursue is to run models directly on our infrastructure. So, because nobody really knew about these like foundation models at that point.

So we actually worked with some customers that had their own models that they trained and running them, so like figuring that out. And then pretty fast it, I mean, over some time it became clear that running the models themselves might not actually be the most important thing or that we have these foundation models and specifically with RAG and now with tool calling that you actually can teach these generalistic Foundation models a lot just on the go.

And so that's, I wouldn't say pivot, but we just test some things and we realize that running the models directly, it's actually also extremely expensive. Because of all these cheap use and loading and unloading all these modules, even though we can do it, it's not what customers really wanna. And so what we have been now focusing on around the last year is really how do we make the access to these models, the existing foundation models, easier, private, and also how do we connect it into Drupal?

John: So that, that's interesting to me. I think you just said something there that I, I didn't, didn't necessarily think about or don't necessarily think about when, when interacting with, with ai, right. You, you said that you know, a ma AI is a privacy focused company and you guys are trying to figure out how to implement AI in a privacy focused way.

Why, why do I need to worry about that? Where's the inherent privacy risk? What, what am I not seeing, Michael?

Nic: Yep.

Michael: So we have always been privacy focused as am Mai io, just a regular hosting, and we do this with allowing customers specifically choose the region or the country in most cases, that their data stored and processed.

And in some cases we can do this even in. Cloud account of the customer. So if a customer has an existing AWS Google Azure account that they wanna run their stuff in we can do that because that gives them safety risk reduction, because technically they could shot us as the, as the management provider or as the maintenance provider any minute.

And because everything is open source, they could literally just run the whole infrastructure themselves. So that's a big point. And so we've always spent a lot of, or most of our customers have these higher requirements. So that's from governments to enterprises and some cases even on premise, like we have cases where we like run hospital literally in their own data center, in their basement because they say This is all patient data, this cannot leave.

Mm-hmm. The, the premise is literally, and so as am amazing, we've always worked in that environment. And when the whole AI thing came up, again, like at the beginning, we looked at running models themselves, but then what we saw is just like. Everybody started to use Open Air and Tropic as these these service providers, either through chat gt, so like a UI that you talk to or just the APIs.

And again, we are very privacy focused. So what we've done is I read these privacy policies and I literally like could not lift my jaw from the table anymore because it was so bad. And, and what is interesting though, if you look at their marketing claims, like a lot of things they're saying, oh, we're not using your data for training, which might be true and that's what interestingly a lot of people are concerned about, but they still have access to all your data.

They store it in perpetuity, meaning they never delete it. They actually have a lot of subprocesses that they use for all their content moderation and all that stuff. So any question that you ask in chat GPT or through their APIs, basically they have connected to you, right? And it's connected to your name.

If you have a credit card, it's actually connected to your identity. And that's really the concern that we have or that I have, is that this data will somehow be used against you in one way or another, or maybe it's not against you, but this creates massive honeypots like we see recently with the, with the app that that got hacked, right?

So This's just like the company may be behind, didn't wanna do anything bad, but they had a fire raise account open. And so the data is gone. And I mean obviously that's negligence on the company side, but also some case. So it just creates these honey pots of a lot of data that can be used massively against you.

And I think. People today. And I get it, like there's still this honeymoon phase. AI is extremely exciting and it's cool. And look what I can do and look at this tool and that tool. But what I'm really trying to do is wake these wake people up and saying, this is all cool, but be aware that all your ai, I, if I go to chat GBT, I'm handling basically any data that I put in there.

I need to be aware that this maybe is public and connected to my name at one point. So you need to be very aware that what you put in there is basically public data. Well, it, I think

John: that's, I, you know, I think that's, that's a really interesting PSA for folks. And maybe not so much people listening to our, our podcast 'cause you know, maybe, hopefully they know better.

But over the last couple of weeks I've seen two, two different scenarios where one was a, post or an article somewhere, some global paper. I don't remember which one was talking about how, you know, people are, people are developing, you know, AI girlfriends relationships with ai, like Right. So like, you know, maybe they're telling them things that, you know, they would typically tell somebody that they're very close to.

Right. Not realizing that that data is now stored, as you said, in perpetuity and, and could, could come back to bite them. Right. So that's one use case where you're, you're saying these things and I'm kind of like, oh, well that kind of stinks for that person. Right. The other is that somebody was telling me just the other day that they know somebody that named their AI and was you know, referring to it.

Outwardly at work as like, almost like a coworker. Oh, well I'm gonna tell, you know, Scotty, Scotty, ai, that, you know, I need to do this, this, and this, and it's gonna do it. So now you're potentially taking company data, right? And putting it into this thing. And now again, linked to you, linked to now your company, now they know where you work.

Right. Those things can be very, very dangerous, right. As you, as you kind of just highlighted for us. I'm just wondering, how is am amazing ai like how are you guys being more privacy focused? Is it, is it you know, people have control over their data, people have control to tell you, like, delete everything you know about me or, or, you know, is there something else there that's that's make, you know, enhancing the, the solution?

Michael: So, I would say we have three pillars of how we make it better. First everything we do is completely open source. Mm-hmm. So you can actually verify and check if what we're using, if we're claiming that this is the right thing or not. So that's the first thing. The second is we don't store any of your prompts.

So what you ask the LLM. I don't care because my business is not based on selling that data to somebody else. My business is not training models because all these big companies, they're, while they maybe say they don't train your data specifically on your data, they can still in fear a lot of information of what people are asking about to then train.

So like they're interested in knowing what you're doing. So that's the, that's the second pillar. All we log is we have some meta information of like, who, how many tokens to use, because that's, some of the pricing models are based around that, but that's it. Mm-hmm. And so, so I know which API key has used, how many tokens?

That's it. I don't know what you asked. I don't know. What you've implemented it with ever. And the third one is we allow people to select a specific region. So like with the hosting, you can tell me you want it, for example, in Switzerland, and then your data. So your AI requests, they go to a server in Switzerland, they're processed on that server and they're also and if you have vector dbs, so like you wanna store some data, this is also happens all on this server infrastructure.

I will tell you which infrastructure provider, it's so, we have data protection agreements where we outline very clearly which infrastructure provider we use. And that's another thing that like in, in an open AI case or in tropics case, you just send it to some server, you have no idea in which country is stored.

And specifically in these higher data protection needs, the country is actually extremely important. Yeah. Because the data protection of the contest themselves apply to the data. So like for an European company. It's extremely important that the data does not leave Europe. Mm-hmm. For Switzerland company, and we see this popping up.

Like I'm said, this is the business that we're in. We see in every country they're launching their own version of GDPR, and one of the main thing is this data is not allowed to leave the country. So I can guarantee you that if you say, I want this in Australia, I want this in Japan, I want this in Europe, in the UK and Ireland, whatever we have like.

25 different regions right now, it will stay there and will not leave that country. So that's the three pillars. And then for the customers that are even wanna go even more, we can go even, we can add a a fourth and a fifth. The fourth would be we can run it in your a Ws account, Google leisure account again, or if you wanna go even more, we can run it on premise.

It's gonna be interesting where you get all the GPUs from because they are very high commodity right now. And I've heard like all crazy stories of how they try to convince Nvidia to send cheap use to you first and not Google or, or OpenAI. But if you have GPUs, we can also do it on premise. And that's obviously then the, I would say the top standard or the gold standard because then literally you can, you can see the physical data in front of you.

Nic: So,

I wanna kinda circle back a little bit before we move on to something that you mentioned that I think has an impact on privacy and it, it's something that I don't know a whole lot about you, you mentioned kind of. A distinction of AMA is that you work on the foundation. I'm not sure if I'm getting that right, if you get the foundation foundations rather than just running the LMS directly.

Can you so talk a little bit more about what, what that, what that means and why that is, is helpful in this case.

Michael: So there is a word of foundation models. Obviously I have to say all of these, this is like all brand new stuff and it's probably changing maybe in a couple of months. But basically people refer to foundation models as these models that are not trained on one specific topic only, but they're generally trained on a lot of things.

So all your chat g, your GPTs, your clots, your lamas, your deep seeks, these are called foundation models, which give you a foundation if you wanted to, to build other stuff. And that is either with like in context and rag. So you basically teach. Let's say Claude, in our case, that we use our most right now with Drupal, you teach it about Drupal.

In that moment, when you ask a question, you say, Hey, Claude. These are all the, the content types that this site has. Select the one that is the best for this specific question that the user has. For example, like that's an in context learning. Or you can actually, some of the, if they're open source, you could take these models and so-called fine tune them or fine train them where you then train them on one specific task.

So we see this, like for example, there are now models coming out that are specifically trained for coding where they take foundation models and they slap a coder. That's most of the time, this is dash coder versions of these models. You could still ask them about. Whatever history of, of, of a country, they might still know a little bit about it, but they're actually really focused on coding and we see similar things in terms of like, some of them are specifically trained on law to ask law questions and things like that, but that's the foundational model.

You can take them to make them specifically for specific topics.

Rich: Thank you. So Michael I think you did a great job of outlining some of the different privacy concerns and also some of the pri protections that you've baked into the infrastructure. Are there other differentiators besides privacy or other key benefits to using your infrastructure?

Michael: Yeah, there is a couple. One of them is latency. So we see specifically, like if you, let's say, are in Australia and you need access to some of these models running them in Australia by itself, you just don't have the hundred, 200 milliseconds of latency to another server. So that's, in some cases it's faster.

The other thing is we're actually looking a lot on to pricing models. A lot of times these token or these, these providers, they just charge you by the token. And a token is a very weird concept. That is like in the machine learning world, AI world, everybody knows what a token is, but outside of this, like nobody has an idea.

So people also asking like, so is a token a word? Or like, or like, how can I influence the amount of tokens that is general? And you can't, if I literally go to the AI and say like, write me a really long answer, this request is gonna generate more tokens and you, you can't stop it. So we're also looking into different pricing models to basically say, instead of charging everybody individually for tokens, which is then like very unclear.

Nic: We're

Michael: looking at more pricing models that we say, Hey, we give you a fixed price per month. And as long as you stay, like in a regular usage of normal people, that's what you wanna pay. Like, you don't have to worry about that. Suddenly one month you pay $1, and then the next month you pay a hundred. Now, if you're a developer, that's obviously cool because you pay for that, but specifically what we are working with enterprises, they want predictability.

And that's actually another thing where we separate ourselves that our pricing model is actually a fixed price per month rather than token based. And like this token anxiety that I've heard before is yeah, is just gone and we, we basically cover or the fact that we have multiple customers, we can then basically smooth out the cost for all of us.

Mm-hmm.

Nic: Yeah. I mean, I prefer services that I use. I prefer that too. Right? Like I prefer a service like where it's like, Hey, I am paying you and you give me a server that has one CPU and two gigs of ram. I can serve as much traffic as I can, get that server to support. And then if the traffic goes up and I don't increase the resources, the site goes down.

But if I need, if I get sustained traffic, I just increase the resources and pay more versus the the other provider model, which is like, Hey, we charge you based on the number of page views you get. Your site's not gonna go down, but surprise, here's a 40,000 bill because AI started scanning your blog for no reason.

Yep. And so, yeah, I, I prefer that as well. Absolutely.

So can you talk about. Some of the more concrete use cases of, of this service. So you mentioned like you can do some incon training of like, here are the con content types for the site. How exactly are those integrating with Drupal here? Are they Yep. Doing changes to the website, are they guiding changes?

Are they documenting things like that? How, how are your clients utilizing it in the context of Drupal?

Michael: Yeah, I think that's, honestly right now the most interesting question is that we have built, or Drupal as a AI framework is extremely cool. And I think we have a running list of like feature ideas that we could come up and I think it's like a two, 300 different use cases of how you could use AI in Drupal.

But that's also the problem, right? It's so wide and so vast of what you could do. I think. For the next, we're gonna buy new ones. But what I can tell you right now is a lot of customers are asking about search. And that's mostly because the existing search systems with like solar elastic search, while they're also adding AI capabilities obviously, but if you just look at what has been used so far, they're actually quite dumb.

They're really good in finding keywords, but they don't actually understand the question of the user. So, we have a lot of our demos, like if you look in recipes like a sold already chokes between the difference from, for Italy and Italian, it's already different foreign solar. So if I ask, do you have any recipes from Italy?

And the word Italy never appears in any of the text, but it's just Italian. It's like you cannot do anything like it will respond with no, no answers. Now you can go to the solar and configure keywords and matching, but like then you're gonna reimplement the whole English language. And then obviously an AI model understands if somebody asks like, Hey, do you have any recipes from Italy?

That it also, it tell, and recipes cannot respond. But not only that, because the model, again, these are foundation models, so they have been trained on language and they understand that even though the word Italy or Italian doesn't actually need to appear in the content, the model still understands that.

Like a pasta or pizza, rest recipe is most probably has to do with Italy. So they can actually respond now to stuff that never, ever would be able to do because they can analyze the content in a completely different way. And that's for a lot of customers. What. Right now the use case that most people can understand because AI can also show it very easily, have a demo site.

It's like, here's the current search, this is the new one. And that's, so for a lot of search heavy sites, we were immediately able to show that we just like, so we just analyze all the searches that people do and they're clicks. We can literally show the same in an AI world and how the responses are immediately better.

So, and then obviously the other thing is that like, like in, in the regular Drupal search, every search is a completely new start. Like you start from the beginning. Like if you ask for something and you have 500 results, you can say, okay, now within these five results, show me. Or I'm actually looking, I dunno, for an, for a recipe of Northern Italy and, and AI can do that.

So you actually have an. Natural language conversation with it. And so that's right now, definitely the use case that most people understand and could see how this can help them. But again, this is one of 200, so I'm happy to talk about others, but yeah, that's definitely one of them. Let's list all 199 other Yeah.

John: And in alphabetical order. No, I, I mean I think that's a, that's a, a, a great use case, right? That people are very familiar with even now, because you can, you, you go to Google and you say, Hey, I'm looking for this right? Or what's the answer to this question? And it says, oh, well, yes, you can, you know, you can do X, Y, or Z or this is the information that you're looking for and here's the page that we got it from.

A little link. Right? And, and like that experience is just more, feels more natural. Right. You know, back, back in the day, and I'm, I'm using air quotes for those people that are listening. Back in the day, right? You used to type something into Google and then you had to go through the pages of results to be like, oh, what, what, what's the right answer here?

Now? It just seems like it's being offered up to you in a, in a more digestible way. Correct. I, I've also seen people you know, dropping images in like, Hey, add, add my friend John to this image. Or hey you know, something I use it for, I actually just used CHATT this morning to write a LinkedIn post because writing, writing things for, for social media is painful to me.

So, like, you know, I think all of those are pretty interesting use cases. But depending on what industry you are in or what company you're, you, you work for, right? You may wanna, you may wanna use a private private LLM for that, or a private model for that.

Rich: One quick question that I had as a follow up to your search example Michael for Drupal sites, are you often seeing your clients wanting to basically set up like the natural language processing and, you know, through AI and then continue to use like an Apache Solar or an elastic search, or you seeing them moving to vector databases or like a mix?

You know, I'm, I'm curious about that.

Michael: I would say that's really like the discovery place that we are in. So technically you can implement it in both ways. You can completely forego the regular search part and say, Hey don't or just use vector dbs. And and also gave me the response as like, it looks just.

You only talk to your content, then you obviously can, like we see this a lot like with governments for example. And so like, right, if I can say, Hey, what are the requirements for me as a Swiss person to integrate into Australia, instead of reading a document, it can literally give me an answer right back.

So that's where like, and, and natural language processing is really good, but there's other cases where you still expect like a list of answers. You just want them ranked by with natural language processing. And that's possible. So you can actually. Drupal is called Boost. So you can actually use the AI to inject an additional boost into the solar responses that Solar says, Hey, these are 50 results, but I don't really know which of them in which order.

So then it asks an ai, in which order should I do these 50 responses? And then the AI will be say, Hey, this article is most probably based on the question, more important, then we'll put that one on top. So there's a combination and right now we see multiple ways. It's, that's, I really wanna say this, like there is obviously not best practices that you can say today that like, this is how you have to implement.

It's still a lot of new things. It's brand new. Mm-hmm. And the modules are evolving. We're adding new features to Drupal AI pretty much every day now. So there is also a lot of updates and it needs a little bit of also experimentation and see what really works and whatnot.

Nic: So, so. I, sorry, John, I just wanna interject there.

There's one thing that's always incredibly painful with search that I think doesn't go away with AI no matter what, because I think it, it, it, but it's a feature request that people, that general, people at companies that generally care about search, always want, they wanna manually boost specific pieces of content for specific requests.

Is there, is there a tool that helps make that less painful or even explain. That that feature is not something that you should implement because that's really what I want. I wanna be able to explain to people you don't search doesn't work that way. People don't interact with search that way. You don't wanna manually ring things.

Michael: I would've, two answers yes.

Nic: The

Michael: first one is I would just give it a try. And what I've seen so far, people were really excited of what answers that an AI just was able to give back outta the blue. Like, or without actual specific training or like specific injections and stuff like that. However, we also have cases where yes, customers, like there's a content on the website that they really don't want to be ever used and there are a lot of po.

So that would be the second answer. We still have capabilities to change the behavior of it, and there's a lot of different ways. So a, you can just not index it. If it's not in the Vector db, the LM is not gonna find it. So that's another alternative. So you can really have a flag or a. Taxonomy term or a field on a note that does not index in a director db, that it'll never arrive and your LM will never answer to it.

So that's one way. The other way is that, so within the Vector DB space, like it's still a search, like, so it still looks instead of like a two dimensional just keywords, it looks, looks into a multidimensional part. And you can still change the behavior of specific responses. So you could still have your, your blog post that you think is the most important thing, and you would have on the PO at the top.

There's still ways in the Vector DB to make sure that this blog post is more likely to respond. And the last thing, and that's actually a new thing, right? All of these requests, they always have a system prompt. So that's basically where you tell the LLM, that hey, LLM, you are talking to your user, you have access to this data.

And this was the question of the user. So that's like the system prompt, which basically packages all the data, the in context the request of the user, the history, what have you talked about, like all these things goes into the system prompt. And in there, of course. You can, again, skew the direction into a way.

Like, for example, we had a case or I had, I did a demo where somebody said like, what is Python? The question is, are we talking about the programming language or the animal? Yeah. So like you can obviously steer that, that saying, Hey, this is in the context of animals Never respond about Python, the code.

Because let's say there's a children's website where they ask about different animals. So there is ways for you to steer these things into a specific direction. And then obviously the very last thing is it's just an API, right? So you could have another layer in between that basically analyzes what comes back, and you go in saying like, Nope, this, like, this answer I do not like, or whatever, like, can try again, or things like that.

So there's a lot of multitude if you really wanna, but yes. Then my first answer still says, let's try it out first. At least what I've heard so far is that people are really excited about the responses that they gave.

John: And I mean, I think. On that, on that point, right. One thing that, that Drupal AI is doing very well is giving a lot of control to the, to the site owner, right?

It's allowing you to say, Hey, when somebody types this in, like it, make sure it's in this context. Or make sure if they're, if they're doing a, a text image that you're using this service with this context, right? So it is, it is providing a lot of those tools kind of out of the box. But one thing I'm, I'm curious about, and this is probably gonna be like an ai dumb, dumb question, so brace yourself, but the getting back to kind of amazing AI service and being privacy focused, right?

You're working with, people's own kind of LLMs and, and models. Right? What if I wanna use, you know, chat GBT or Claude in a more secure way? Is there, is there a method for me to still use the a ma Z AI service? Or am I just gonna have to go kind of use, use those, those other services directly?

Michael: No. So that's actually a cool thing. So we can actually offer catchy PT or clot in a private way. So how this works is that, so today we have proprietary models and we have open source models.

Nic: Mm-hmm.

Michael: An open source one is like TSE or Llama. I can download them and I can run them on my servers. No problem at all.

And I can guarantee you, like nobody else has access with this proprietary models. There is now where actually these model creators, so Entropic or OpenAI, actually give the models to specific infrastructure provider. So Google, Azure, AWS, they mm-hmm. They actually get the models from, let's say, Tropic so they can run clo, for example, on servers in Switzerland.

And the data is never sent to Entropic or anyway. So these models just run within these AWS servers that AWS guarantees me that they're in Switzerland only, so I can use them in a privacy private manner without the fear of these models talking back to Tropic. And that's what we're doing. So we basically, that's how we give access to these proprietary models in a private way.

Plus we also have the open source models that obviously I can run anyway, I, I

John: have a. I'm gonna double down on my dumb dumb question here. But like, when we talk about models, right? So like, is the model really just the, the mechanism in which it's taking your tokens and and separating them and then searching the LLM for the data?

Michael: No, the model is basically is a neural network. Okay. That has like input and output neurons mm-hmm. And trillions of ones in between. And that's basically, and in the, in the LLM world or in the machine learning world, they're, they have weights and biases, how these neurons are connected with each other.

Mm-hmm. And so basically the model is basically the definition of these neurons and, and the weights and biases between them. So that's the, if you download. Let's say Lama four, that's what you would see. And if you could look inside of it, you would see a neural network that has input. And the input is basically your text.

Got it. So when

John: we, when, when we say model, we really mean LL, like it's just a shortening of LLM, is that fair to say?

Michael: Correct. But that's just the other models beside of LLM, right? So you have video models, you have you have got it, got it, got it. Image vault, you have audio models. So model is like the top term, and l LM is one of them, but there's other sisters or brothers left to LLMs.

And,

Nic: and it's a, it's a combination of several things. Like what was the training data, how long did the training data, it, how long did they process the training data? How many connections were they allowed to make? And that's one of the ways that, for example, open Ag gets away with, they, they'll say that it's open source, but they're not open sourcing, like all of the actual pieces that you really need to be able to build it yourself.

Right. And a a, a common one is, I don't remember the exact, I don't know the exact numbers, but if you look at LAMA for example, they have an 8 billion weight model and then they have a, what, 80,080 billion weight model. Right. And what that means is like how it takes a lot of processing to be able to say, I'm gonna ingest this information and make connections between it.

8 billion means like, Hey, we just let you run for, I don't know, exact numbers or any of that stuff. It's like, Hey, we let you process it for an hour. Once you were done with an hour, we, and this is really simplifying it, but once an hour is up, we pull the plug. And what, that's as good as you get the $80 billion, 80,000,000,001, they let it run for three weeks, a month, two months, whatever.

And so it has much deeper connections and much better fidelity.

John: And when you say run, Nick, you, you mean just like kind of like. Training. Basically training, right. Processing the data and making those connections and, and, and understanding whatever it's being trained on, right? Yeah.

Michael: And you can actually, like, there's all kind of great stuff.

So they train only the big model and from that you can actually make a smaller model from it and

John: like until baby models.

Michael: Yeah. And, and that's like if you have access to, to the, and biases and the neurons themselves, you can actually continue training on top of them. Right. So you can, that's one of the things.

Basically have the model just to ingest more and more and more about one specific topic and then you have your deep seek. I don't know how to talk like a baby model that can only talk like a baby type of thing.

John: We're bordering on horrifying here because now we're talking about neurons and like pathways and I'm like, oh my God, it's gonna, it's a brain, it's gonna, it's gonna take over the world.

I mean there,

Nic: there's the reason why It's called a neural network, right?

Michael: Yeah. Yeah. That, that's always like, where is, where does hi stop and end? Like that's obviously like all these models

John: as like a, a Star Trek fan. It makes me very excited because I always think of like, when I hear neural net, I always think of like Lieutenant Commander data and the fact that he's like, you know, a a, a Android and I'm like, oh man, if we could get to that, that would be kind of cool.

I think we will just a matter of time. That would be awesome. I look forward to that future.

Rich: Earlier we talked a little bit about how there are different use cases or, you know, different things that people would wanna build. Like we talked a little bit about search. If we think about what you're providing at MA with this AI infrastructure in terms of, you know, individual dral developers, how, how does it apply to them?

Like, what are, what sorts of things are they thinking about?

Michael: Yeah, I think the most important thing is that we understand the architecture. So an LLM is basically like an external service that you consume from a Drupal point of view. So that could, you can compare it to a solar, a database, a Redis, whatever.

So it's an, it's basic a capability that Drupal itself does not have. You cannot build an LLM in PV or at least I haven't seen one yet. And so you have basically have an endpoint, an API endpoint. So Drupal needs basically needs to know how to talk to these providers. And it needs to prepare because the an LLM expects.

One request, which is natural language. So it expects language to tell it what you should do. And Drupal, basically, the AI framework knows all of that. So it knows about system prompts and things like that. And then you add providers to these AI frameworks. And then the provider basically teaches the AI how to talk to one specific provider.

So they talk to us. For example, our provider tells Drupal, which, where does the API key need to go and how does the, the structure of the request needs to be? And that's why there's all these providers out there because every provider has slight differences. We try to use as much as open standards. So we actually use the open ai, not as the company, but the standard of their API is what we use because it's like defacto standard, but like Cloud, for example, decide that they're gonna use something else.

Or ENTROPIC decided they use something else. So that's why you have all these different providers. But they basically teach the AI framework what to do. And the AI framework itself teaches Drupal, Hey, this is what. AI is, this is how you talk to No. The other piece that we also have is Vector dbs. So that's a new way of storing data.

But in a Drupal term, we already have a framework for databases or searches, which is search API. So basically our model or a module from Drup the me AI model basically talks to search api. So it teaches search API, Hey, instead of talking to Solar or Elasticsearch, now you can talk to a Vector db, which in our case is a Postgres.

So you just need to be able to talk to Postgres, but it's very similar there. You also have your Postgres provider basically that you enabling search API and then search API that knows how to index stuff and how to send it and what have I sent already and on Crown, I need to update. Like all that stuff that the framework does.

We'll all be using, like we're not implementing any of this. So these are really the two external things. Now, the difference is a little bit that some of these LLMs. You can actually run locally. So that's possible. So yes, if you have a powerful enough computer with a tool like ola, which is basically an app that you install in your computer, now the LLM runs locally and there's also a provider for for Drupal to talk to.

But that only works locally. So if you would wanna do this in the cloud or in any production, there's no LAMA anymore. So then you basically need to talk to our provider. And the really cool thing is what we also see in the Drupal world is that's why it's actually called am Mei ai and not io you can use AM mei AI on any Drupal hosting.

And it's not just limited to ours because we realize there's all these hosting providers out there that it takes them a little bit longer about how to implement this. But a lot of people wanna use ai. So for us, yeah, you just install the provider, you deploy it, and then Drupal through the internet, obviously all encrypted, they can talk to our endpoints and it teaches Drupal about LMS and back to dbs.

So. That's basically how the architecture works and that's what Drupal developers need to use. So they need to, they need to install Drupal ai, they need to choose a provider. There's multiple, you can install multiple providers if you wanna, and then they build whatever cool features that they build with Drupal ai.

So that's the search. And we see all these other capabilities of like, you can literally ask Drupal now to create content types. I've seen a lot of people using that. Like instead of going in and creating content types and like you just exported Chas and give it to Drupal and saying Here, and then it goes back and just runs, run runs and five minutes and says, Hey, I created all the content types for you because it's literally faster than clicking through the UI of creating all the fields and stuff like that.

Yeah. And so that's like another use case that we see people is, that they use in our ForSight building. So.

Nic: So let's say, I'm, I'm hearing this, I'm getting pretty excited. I want use this, like, what are the, what are the practical steps? Like if I want to use the piece you just said, I want to, I have a fairly standard set up for a client for some content types.

We've been working on setting up a recipe, but, hey, maybe it's easier just to document this. Do how, how do I get started here? What's the, yeah.

Michael: So the first thing you need is the Drupal AI module, or I call it the framework because it's, it actually doesn't provide any capability outta itself. So, but you download the AI module and through org you install it.

And there's already a lot of sub modules based on what you want to do. So what you want to install is specifically the agent system when, and then there's a chat bot system, so that gives you this chat bot inside Drupal. So that's when you're locked in with a specific role. You can see this little chat bot that you can talk to and then you need to choose for a provider.

Right now there's over 20 different providers. I'm happy to say that I think right now we're the only ones that offer a free trial. So you can download our provider. It's just another Drupal module. You go to the settings of it. We need one email address, that's all to identify you. And then we give you a 30 day trial so you can completely use this completely for free.

There's no limits. And you can go right ahead and then what you need to do, if you only install one provider, that provider becomes the default. So your little chat window should work out of the box already. If you have multi provider, there's a configuration setting that you choose, like, which provider I wanna use.

But if you only have one, let's say hours that you installed, this is the default anyway. Okay. We actually expose models already with the correct Drupal correlation. That's obviously one of the things that we can do. If you, let's say, install the, the OpenAI module, it ask you of these 50 models, which one do you want to use?

And you're like, I have no idea. So you try all of them. What we do is for, we, we expose the actual models that we use, like Cloth three, five but we also have just a Drupal version that we know tested. So you don't have to worry about any of this. And then you can go right ahead. You can ask your Drupal about content types and things like that.

John: What, what happens after

Michael: the 30 day free trial? So right now it just runs out and it asks you to subscribe. We have different tiers. There's like a $30, which is like the lowest level. And then we have a hundred dollars ones for the people that really use it like a lot. So like, yeah, if you use it like for the search where your actual visitors of the website using it, they're gonna use much more versus but actually the, the differences is really about, also about the support that you want from us.

So with the higher model with the higher pricing, you really get access to our team. We help you implement stuff. And the $30 really for the people that just wanna that feel free, like they don't want a lot of the help, like the enterprise. White glove type of thingy that we do. Mm-hmm. So that's it.

You pay $30 a month, as I said, right now, there's no limit in amount of tokens or anything. Obviously if you start to like reindex the whole Wikipedia in your Drupal site, maybe we're gonna have to have a talk just to understand what you're trying to do. Yeah. But so far, like I said, we really believe that we want to give people access to it and wanna make it easy.

And so tokens would prevent all of that. So that's why we do the 30 day trial. And and also after that, it's not token based.

John: Interesting. So I'm wondering, you know, there's a, there's a broader technical landscape out there different platforms different tooling that people have kind of integrated with Drupal.

I'm wondering if you're seeing, may z ai and, and just AI in general kind of used with other, other, these other systems like CRMs or, or personalization, right? I mean, personalization is a huge one.

Michael: Yeah. I mean, I think we could have a whole lot of talking Drupal about this, but basically that's where we bring in MCP.

So m to shortly explain MCP is basically a way for an LLM to talk to another data source or even another LLM. So you could literally, you could say, Hey Drupal talk to my CRM and and send a newsletter to all my CM contacts.

How this works is basically the Drupal itself would first generate the email, like the content based on all the different, it would generate this, and then it would talk to the MCP server of the CRM and says, Hey, I have an email newsletter you should send out to all the CRM. And then basically the MCP finds the LLM how, like what API endpoints that it has, what can I talk to, what can you do?

So it literally, like, it can then talk to each other. And the really cool stuff is Drupal because we have ECA which most people still, I, I hear people call it just rules. So, but this basically this whole approach of doing this whole action based, you can actually combine all of them. So, you could literally define an ECA, workflow where you're saying, Hey, you talk to this CP first, then you get that data, you enrich it with Drupal data, and then you go there and then you do that. So there's a massive world I think we're just discovering now where in Drupal, and that's obviously because of all the, of all the way that Drupal is built, but also like of all the entity and virtual entities and all that stuff.

You can build massive tools that automate most of it. And I see we've seen this already with some customers where they basically interest emails into Drupal to analyze them and then do business decision based with these emails automatically and then assign them to people. And the people don't actually, like, they used to have like a shared emailing box that they tried to go through and now they log into Drupal and they see the emails that are assigned to them.

And Drupal now starts to suggest to you already like this. Here's a response I already created for you. You just have to approve it and then it sends it back. So there's massive opportunities you can do here. This,

John: this is like, I don't know, this sounds amazing to me, and I'm obviously like paying attention to what's going on in the AI space, and I'm, and you know, I'm, I'm familiar with ECA MCP not so much, but you're, you're, what's that,

Nic: what does that

Michael: stand

Nic: for?

Michael: Or model, context, protocol. It's basically a way for an LLM to talk to an to an API. Okay.

John: Which, which like ratchets up the. The, the, the, the, the power, the interoperability, I don't know, whatever term you wanna put in there. Like, it, it just feels like it, it doubles down on AI's ability to be more, more useful.

And I think your, your example there of, hey, I, I wanna send this blog post out to a group of people that may be interested in it based on data they've put into my CRM is like a prime example of, of a, a really great use case as to where you know, I feel like AI and MCP like together could very, very easily solve a lot of the personalization problems that many, many sites have where it's like, Hey, I just bought this product and now you're sending me a discount to buy the product.

Which is like, the boat ship sailed, right?

Michael: Yeah. And I think that's really like right. In the past, if you would have thought about, let's say you wanna automatically send. A LinkedIn message after you post a blog post. Like what I see so far, people are doing, they're writing the blog post maybe with AI outside.

Let's say they with chatt, then they copied into Drupal, and then they go back into CHATT and create the a, the short term for LinkedIn, and then they copy and they go to LinkedIn. All of that now would literally be possible. You can ask Drupal to write a blog post. It knows about all the other content on your Drupal site.

So we can also cross reference other blog posts that you have written before. It'll write it in your language because it has access to all the other blog posts. And then at the end, after the blog post is published, it can talk to the, to the LinkedIn MCP server to actually post the, the message immediately to there so there's no copy pasting.

And then because it has access to the, to the LinkedIn, it could literally, when you go back to your Drupal node and because you're locked in, it can show you stats how good that LinkedIn message did. And it can actually learn from that so that the next time that it publishes it, it knows, oh, of these three LinkedIn posts, this one worked really well because I send it on a specific time or I use specific words.

So it'll, the next time it'll do it, it'll learn, it'll analyze the last post, which worked really well. So like all of that can be just so much easier. And because Drupal has this interopability yeah, that's obviously allows a lot of things that today if I would come to you and say, I want you to build this in Drupal, you would need to like.

A LinkedIn module, and then you can only post, but it cannot log. It's just like, how do you do all of this? And with, with this ai, with EC, a and m ccp, it opens up a world of things that I don't think, like we could only have dreamed of. So easy

John: to get. We need to get on the recipe building though, because ECA, you know, workflows still confuse the crap outta me.

But anyway, so the really cool

Michael: thing is actually you can ask AI to create your workflow. So, so you could literally go to EC saying, Hey, write me a blog, write me a workflow that does X and it builds it for you.

John: Oh, I'm gonna have to try that one. Sorry, Nick, go ahead.

Nic: No, I, I was just gonna ask on a more practical note, like one of the things that, so we, we use Google Docs for show notes, right?

And then it is not a huge amount of time, but a decent amount of the time that I spend each week working on the show is. Converting the show notes into kind of a, a prepared show notes document in, in Hack MD and then copying and pasting that to like five different places Right now, we don't wanna really move away from Google Docs at the moment because we, we use it for a lot of things.

Like there's pre-show notes, this post-show notes, there's resource, there's a bunch of stuff that it does. But how real, if we're being realistic, if I wanted to build a system where I can just write kind of the finished notes in Drupal and then have it post out those results to the four or five places we publish to, like how, how much work are we looking at?

Like, is that, you know, 50 hours of developer time to build? Is that a hundred hour? Because some, some of the endpoints maybe don't have, like, I dunno if Paton has an API for pushing content up or

Michael: Yeah.

Nic: Or anything. Like how, how complex is it to build some of these workflows you're talking about? So building the workflow itself should actually be pretty simple.

Michael: Now with ECA and access to lms, you can literally, like, you could give it the Google Doc that is like, full of stuff that should not go public. And you could, you could like basically ingest that and, and say like, create me a show note that can go public and like do not mention anything or pre and post and like all this internal information, like ignore this.

I only wanna like talk about what you can see under the primary topic or whatever. Like something like that. And so that, that would be possible. You could if, and there are MCP servers for Google Docs, you could literally, like, all you have to do is paste the link in and then Drupal with through an M CCP server can scrape the Google Doc automatically.

And you could do that in, in ECA to then create the note Drupal notes that is unpublished and you can then look at it and go in and change whatever you wanna do. And then when you click publish, another ECA is triggered. That then talks to again endpoints, MCP servers of all these providers.

And so, yes, the question is, do all these endpoint or these places that you wanna publish, do they have an MP server? What is happening right now is

party. It's a conversion of an existing API that already exists to basically a machine like an LLM readable, API. Technically there are CPS that all they do is just they convert open API schemas into MCP and that doesn't really work that well. Because the way that open API or like how we have built APIs in the past, if you just do like a one-off thing, it might be okay, like posting a LinkedIn message is maybe okay because it's one API endpoint anyway.

But as soon as you have multiple of them, the other line is a bit confused that it needs to like, first this and this and this and this. What it expects is I just wanna do all of it in once. And then the MCP server basically splits it up into individual API calls. So that would be possible. But yeah, it, I would claim if you, if assuming there's MCP servers for all of them with like.

Half a day, a day of work, you should automate, be able to automate a lot of that just outta the

Nic: box because that would save me a lot of time then, because the other, the other side of it that, that's complex and, and again, I, I don't mean to use this show to debug this thing, but I think it's good to have a practical thing to, to talk about sometimes, especially when it comes to ai.

But the other thing that we struggle with it, or the other piece that I struggle with for setting this up is it still requires putting the finished assets somewhere. Right? The, the, the finished product for the video is anywhere from two and a half to five gigabytes. Right? So I, I can't,

Michael: yep.

Nic: You know, I can't upload that to our hosting provider and then have it, have it move.

I mean, that would use up all of our space, right. And. You know, so we, we rely on YouTube for hosting the actual assets. The, the audio version's obviously much smaller, but even that's a hundred megabytes. We would run outta space pretty quickly. So figuring out how to resolve these. But our new edit editing system does provide URLs for most of the videos.

It doesn't for the audio yet, but it, it's possibly, it might be interesting like being able to automate some of that stuff. It might be a good test case. And I

Michael: mean, one of the other things is like, we haven't talked about this at all, but like you can also run, like you can install cloud, let's say, or there's open web ui, which is like a local hosted jet interface and you can actually run some of the MCP servers locally.

So there is one for YouTube. So you could actually tell your local, like. It's a bit hard what it is, but it's technically an MCP client or your chat interface locally that it should first upload this to, to YouTube, and then when it gets the link, Drupal cannot only talk cp. It can also actually be an MCP client.

So your Drupal can receive requests through MCP as well. So you can actually run the Drupal MCP server locally and connect it to your Drupal site. So you literally paste the, the YouTube or the link the video, link the file locally, tell it to upload it to YouTube. When it's done, it goes to Drupal. Then, so like this, this gives me an idea so I could

Nic: have a local copy of the talking drupal.com website in d dev with the MCP server configured there.

That then communicates with the actual production talking Drupal site. So I can basically like upload the file to a local server, which will take no time at all and then kick off the process and it, or just even just have a rule for where the files are in the system. It'll kick off a process and publish the actual, or not publish, but create the actual that.

Yep. That would be an interesting

Michael: use case. You can run Drupal locally, or there's all these MCP clients that you can run locally as well. So it doesn't even have to be Drupal. It can just be any, like, if you install cloud locally, that is then basically an MCP client by itself, and you could connect to Drupal or you can also just run Drupal locally.

That's also totally possible.

Nic: Very interesting. I, I am intrigued and, and, and I, sorry, rich, 'cause I know you have the next question, but there's one other question that comes to mind because I am,

I would love to use AI to solve that type of problem. Right. I, I love the, like, I don't have to copy and paste anymore. I don't have like, we're, but I'm very resistant to. Using AI to generate content or generate notes, right. We, we've been, we've experimented a I'm not, I know you're not,

John: as the person that creates the show notes, I'm thinking, man, could I use, could I use Gemini to create, create our show notes going forward?

Hmm.

Nic: So, so, so I'm curious if there's a way, if you found a way to lock down that piece so that if it's doing this, it's not like, I'd be very, I'd be very, for example, I would not want to use this if I found out that during the MCP process where it's moving content from talking dral to Patreon, to Libson, to whatever, then it would also modify the content.

Or if it's taking stuff from, you know, the, the one thing that we've used, we've been experimenting with is. Kind of merging our primary topic show notes that we've talked about with the actual trans transcript of the show. Right. And pulling out key moments, because we go off topic all the time. Our listeners will know, probably may not know this, but we have a list of primary topics and bullet points that we talk about.

And then John and I and our guest hosts ask questions about other pieces. And, and historically I've just kind of had to make a note of those as I'm editing the show or listening to it, or just remembering. But since we now have transcripts, there's a tool that can just kind of summarize that and I can then pull out some of those pieces.

And, and even that, I mean, that's, that's about the limit of what I am, I'm willing to do because it's pulling from a transcript, which is like a repeatable thing that we can see if it's worth, like, it's not generating like, Hey, this is what talking Drupal is talking about. It's like, this is analyzing something we've already talked about, but is there a way either with Maze AI or CPS.

Of helping kind of limit the generation aspect of it and make sure that it's really just kind of handling the make work aspect of it.

Michael: Yeah, and that's really like, that really depends on what you're telling the LLM that it actually should do. So like the LLM itself does not have an opinion about stuff, or it doesn't have like, bad thoughts that are like, okay, I'm not gonna inject these things.

Like all these hacks that we've seen there is actually literally somebody like injecting, like a system prompt or something that, that tells the LM do X, y, and Z. So if you, let's say if you create, if you decide you wanna still wanna create the, the short term description, let's say that you post a LinkedIn, whatever that you wanna do yourself, and you go to the, to the MCP or the LLM and says, Hey, post this on LinkedIn.

There's no need for the LLM to go in and modify it unless you tell it, Hey, can you rewrite it? So at least that's in my experience that this never happened or this wouldn't happen. But I think this is also brings up a very important point. It's like people always ask me, oh, look, amazing is gonna replace me with ai.

And I'm like, no. What, what is gonna happen is that somebody using AI is gonna replace you if you don't learn how to use it. Like that's what I see in my team, like my marketing team, they're still the same people. But they just learned how to use AI to make their work easier. But I still want the human to actually read the LinkedIn message before it goes out because we had cases where it completely, it summarized something completely off wrong.

So you need somebody that checks this. So in your case, I would, yeah, if you blindly hope that it perfectly transcribes what we've talked about and creates topics, I wouldn't trust this. So you still need somebody to read through it and maybe updates it a little bit, but, and least in my experience, that work is easier and faster than having a wide screen.

Then you're like, oh, shit, now I need to create, like, like the, what, what did we talk about today again? Yeah. So it, it's really just like, that is much easier.

Nic: And, and that's the one thing that I, and I'm, you know, like I said, we've been experimenting with it and I, I've found that it catches some of those rabbit holes better than, better than I have.

And I, like I said, I think that's because it's pulling it from a transcript of the show rather than just kind of just, you know, making it up wholesale. But I do find that I have to, I do edit it, right? I can't, I, I can't just copy and paste it. It doesn't. It doesn't work like that. Yep. But but yeah, no, it's, it's interesting to hear.

'cause like I said, the, one of the more tedious aspects of publishing is after everything is wrapped up and, and been rendered and all that, then I have to then post it to five different locations. And then I, I know that Steven posted to a couple different locations as well. So if there's a way to kind of automate some of that, Hey, we split it up, we put this piece here and that piece there, it would, it would save us both some time.

Yep. Yeah.

Rich: One thing I wanted to circle back to is, Michael, before you had mentioned that there's the ability to download open source models like Deepak and Llama to your platform, to your service. So I was curious, you know, open source models, are you seeing that you know, happening quite a bit on amazing AI and like what sorts of use cases are there?

Michael: Yeah, I think there's a very interesting thing overall happening in terms of AI and open source. And that's just the speed that things that come out first in proprietary models are ime like days, weeks, maximum months later, immediately available in open source. And I think, honestly, I've not heard a lot of people talk about this.

Like if you look back, like in the Linux space, like until Linux was really able to keep like the same quality and experience that you had, let's say had in Windows, it took years, right? Mm-hmm. Like until they came up. Obviously we spend a lot of time in the CMS world, and I still think today, like if you really compare like open source CMSs compared to some of the proprietary, unfortunately the proprietary are still much better.

But in the AI space, I think that's actually common. Not true at all. Or sometimes the other way around where like these open source models actually push the proprietary models ahead and, and I think that's something extremely cool. And obviously yesterday we still use a lot of proprietary models just because they have been first.

But yeah, with specifically what comes out interestingly from the Chinese so the, and, and the deep seeks they're extremely good. So we, I would say we see a slow uptick of more and more open source used, and some customers actually specifically ask for it because they just wanna, they want something that, that is open source, that is in our ethos, right, of Drupal itself.

Mm-hmm. The customers saying, Hey. If I can choose between a proprietary and like giving, like, because every time, so I pay the infrastructure depending on what I use, but some of them, I, some of the money goes back to entropic. Like if I run their cloud model while they can't see the tokens that go in and out and what, what we ask them, there's still this company that gets a little bit of a cut of it.

And if we can use open source and then therefore make it cheaper or better to use. I mean, yeah, that's definitely something it's, I have to say though, it's still, it's still a lot of discovery of which models really work and we've trained, we've tested Clot three five a lot together with Drupal just because that was the one that was easy accessible in all of these regions.

So right now, if you use a Maci ai, you use Cloud three five, most of the case because that's just a solid choice. But we can give customers access to more models if they're interested to. And we have a couple of customers that trust, like some of these more open source ones.

John: Thank. So where, I mean, where does am Maie AI go from here?

Right? What, what, what's the long-term plan for the service? It sounds like right now you're, you're, you're working towards, or, or you're achieving, I won't say you're working towards you, the privacy angle, right. But are there other services areas, things that you wanna kind of morph into? What, what's on on the roadmap?

Michael: Yeah. There's definitely multiple ideas that we're looking at. First of all, in Drupal, we're gonna stay up to date what happens in Drupal. That's why we became a strategic Drupal AI initiative sponsor to ensure that. They're really good stuff, and I have to really say this, like all the other open source CMSs, they're miles or years behind what Drupal can do.

And we wanna, and that's obviously as, as somebody that has a part of its heart beating Drupal since I think 15 years now. I'm very excited about this. So that's why we're actively sponsoring this and pushing forward and like, organize everything. Like I have at least two or three people that work now full-time just with the triple community to make sure that we have demos and marketing materials and like all that stuff existing and things like that.

So. We're gonna continue pushing into Drupal. The other things is that we see other frameworks so Symphony just announced their AI stuff. There's a WordPress team that works similar and funny enough, they actually all copy what happens in Drupal. So we are looking into these spaces. There have to say though, it's even harder to show really cool use cases because again, they don't have this framework system that Rupa has yet.

But in the end, right, it's an api so we can, it's, and because we use Open AI's API standard and Postgres standard, we can actually very simply connect other CMSs. And I, I believe that like in a couple of years, if the CMS does not have any AI capabilities. Like, that's gonna be the, the, the system that weeds out of all these CMSs we have out there.

So that's obviously the, the place that we're gonna next at the, at a completely different one, and that has nothing to do with Drupal, is that we see a lot of companies actually asking for like a track GPT like interface with private AI behind. So that's one thing that we're working on. We're gonna probably, there's a be coming soon and we're gonna launch a ga behind that.

But basically that's then when you could just talk to your. Model that has, again, because it has MCP support, so it has access, for example, to all your data in Google Drive. It has access to your Slack, it has access to your Drupal. But you could actually ask this tool to to do your tasks or like to write a blog post of the last three Drupal stuff because that's what we really see.

We also see Drupal morphing into, it used to be the only tool that a lot of people used, and now it's just, it's, it's, it's part of a group of tools that are used. And doing everything in Drupal all the time also doesn't make always sense. So this, this capability to have a separate tool where you have a jet interface and you can ask it questions and do tasks for you, but then you can talk to all the other tools.

And that's really something that we see right now. You can't do this, like you can go tot or clo, but then again, you center all the data and. The alternative is that you like download open web UI locally and you run it locally, but that's for the four of us here. Possible. But tell this to other people.

Yeah, just download Docker and API keys and like all that stuff. So, so basically how did you create chat like interface that is completely private, has, has guarantees of private, but also has access to your data to ask questions about? So that's another world that we're going into. It's obviously very new for us.

This is more like a SaaS type of solution, but yeah, that's where we're gonna

Nic: very definitely keep me updated on that because I, I think I may have a client or two that's interested in that. Cool. So Michael, thank you for joining us. It's been far too long. It's been 250 what, eight episodes since you've been on, so we'll have to have you on sooner than that.

Thank you for joining us.

John: Thank you

Nic: for having me.

John: Do you have questions or feedback? Reach out to talking Drupal on the socials with the handle Talking Drupal or by email with [email protected]. You can connect with our hosts and other listeners on Drupal Slack in the Talking Drupal channel,

Nic: and you can promote your Drupal community event on Talking Drupal.

Learn [email protected] slash td promo.

John: Get the Talking Drupal newsletter to learn more about our guest hosts, show news, upcoming shows, and much more. Sign up for the [email protected] slash newsletter.

Nic: Thank you patrons for supporting talking Drupal. Your support is greatly appreciated.

You can learn more about becoming a [email protected] and choose the Become a Patron button. All right, if our listeners had any questions or about your service or you, Michael, how can they best get in touch with you?

Michael: Yeah, you'll find me all in different places under the name Schnitzel. So I'm very excited to go back to Vienna, by the way, Vienna.

But yeah, find me on schnitzel, either on Slack on or just via, there's a contact there.

Rich: And Rich, how about you? You could definitely go to reach out to me at on Slack, RK Lawson. I'm Rich lawson.co and you know, for my website. And then definitely check out evolving web com for evolving web.

Perfect.

Nic: And John, how about you?

John: Well, when, when this comes out, I do believe rich Michael and I, as well as Stephen Cross will be at Drupal GovCon in College Park, Maryland. So if you're listening to this the week of the 11th and you haven't got plans you know, Thursday and Friday, you, you, you're, you wanna travel over to College Park, you can, you can find us there.

Personally, you can find me at picozzi.com. You can find me on the social media and drupal.org at John Picozzi, and you can find out about EPAM at epam.com

Nic: ComCom, and you can find me pretty much everywhere at nicxvan N-I-C-X-V-A-N.

John: And as usual, if you've enjoyed listening, we've enjoyed talk. How a good one, everyone.

Nic: Thanks everybody. Thank you.