A Chat with the Game Modder Making Waves with Generative AI

Generative AI is all the rage across countless industries, thanks to the mindblowing advancements spearheaded by OpenAI and its ChatGPT product. Needless to say, this is soon going to affect the gaming industry, which has always been very technology-driven due to its very nature.

So far, we’ve heard of publishers like Ubisoft using AI to squash bugs or, more recently, to help them write backgrounds of some non-player characters. Companies have created AI-based tools to massively accelerate the creation of art assets like character models. What gamers have been really dreaming of, though, is seeing generative AI integrated directly into games so that they might have realistic chats with NPCs instead of being forced to select from a handful of predefined options.

Given the ever-expanding development timeframe of games, especially the triple-A ones, we’re probably a few years out from seeing that. However, a veteran Mount and Blade modder calling themselves Bloc has provided a groundbreaking first look at the incredible potential of generative AI integration into sandbox games. Earlier this year, Bloc released a few videos demonstrating a full ChatGPT integration into Mount and Blade II: Bannerlord, where the modder now had the option to discuss nearly anything he wanted with any NPCs roaming the world of Calradia.

Last month, Bloc actually shared a playable mod using a different but similar system from Inworld, a development platform (with plugins for Unreal Engine and Unity) that creates real-time generative AI characters in games and other entertainment. Amazed by the modder’s work, I reached out to Bloc to discuss the pros and cons of each implementation, whether this could be applied to a game like The Elder Scrolls V: Skyrim (spoiler: it’s coming), and how the future of gaming looks like with this new, powerful tool at the industry’s disposal.

How did you first become interested in implementing generative AI systems into games?

I came up with this idea ~4-5 years ago when I was experimenting with my RNN models. However, the technology was not advanced enough to be integrated into games. I continued to explore new technologies as much as possible. For example, even though it’s not related to games, I published this post in the forum 2 years ago just for fun. It used GPT-2 to generate nonsensical blog posts about Bannerlord and make fun with TaleWorlds’ community manager ???? I also wanted to explore a similar idea with Galactica, but that one was taken down relatively quickly.

Your first experiments were with ChatGPT 3, which you successfully integrated into Mount and Blade II: Bannerlord. How long did it take, and what were its pros and cons?

Some people might find this shocking, but it only took me ~3-4 days to implement the initial video. However, I had already worked on some pet projects with ChatGPT, and I had a good background in Machine Learning and Large Language Models. The pros of ChatGPT were its ability to adapt to the story and the role and to generate coherent answers that suited RPG games. The cons were its slowness and its unpredictability due to the “responsible” restrictions OpenAI imposed. Because of these things, between my first and third videos, I spent a lot of time trying to make ChatGPT sound more natural and to integrate it with the quest/trading system.

As far as I’m aware, you haven’t released the ChatGPT 3 mod publicly. How come? Is it planned at some point?

ChatGPT version was using non-publicly available parts of ChatGPT. It was kind of a dirty workaround and wasn’t ready for the public. Currently, they have APIs where users can call and get the data much faster; however, these APIs are not free, so having a ChatGPT version of the mod wouldn’t be feasible for general use. So I’m not planning to spend time on that, to be honest. It’s costly even for me to test/implement around.

Are you going to experiment with the new ChatGPT 4 AI model?

When I get some time, I want to check it out. I’m not sure if I can use it for gaming, but I have some ideas. I have tried Bard and Bing Chat, which also use a version of GPT4, but not ChatGPT with GPT4, not yet.

More recently, you have showcased and released a mod that takes advantage of the Inworld AI model. Did you find Inworld on your own, or were you approached by Inworld? Can you talk about the differences in setting this up versus ChatGPT?

I heard about Inworld from an article I read a while ago, but I didn’t know they were ready for public-facing products. They reached out to me after my initial video, offering me a developer account so that I could play around and look at it without burning my wallet (unlike OpenAI ????). A couple of other similar companies also reached out to me, but Inworld was the most flexible. Setting up ChatGPT and Inworld had very fundamental differences. First and most importantly, Inworld was ready for “production” compared to ChatGPT. Even though they are in the early stages of the product (it might be in beta at the moment), it’s a fully functioning product with several integration choices and some SDKs.

Although their functionality didn’t exactly match what I needed, it was still possible for me to expand their functionality with some extra layers/workarounds to make that mod more interesting for the users – and they were cool with it. Not many companies enjoy random guys like me playing around with their systems and “using” parts that are not designed for that purpose (for example, Inworld doesn’t support runtime character generation or update; these functionalities were added by me just so that Bannerlord’s sandbox feeling could be fulfilled).

In the mod’s description, you wrote that Inworld is not integrated with the game system of Mount and Blade II: Bannerlord. Is this a limitation of the game or of the Inworld SDK? Can it be overcome? It would be much better if generative AI was integrated with the games’ systems.

This is a limitation of the Inworld System, unfortunately. It’s related to how their system works. I’m pretty sure it can be easily overcome with several changes in their system. Obviously, unlike a modder like me, they have actual customers, and they have to think about how they can implement something that can fit seamlessly for everyone, so it’s probably not easy for them to just go and create such a system. Nevertheless, I explained to them how this could be used, and I have been informed that they took notes and perhaps even started working on it; who knows. I do understand that the lack of real effects, like triggering a quest or haggling, might make the game feel a little dull, but I heard great stories from users who interacted with the Inworld Mod. I think even adding the ability to talk with NPCs, learn their “inner thoughts” or get some information or just to immerse yourself for roleplaying is very important, especially for a game like Bannerlord, where everything is dynamic and dialogues are extremely uninteresting/basic.

Do you think Inworld and/or ChatGPT could be a good fit for a game like The Elder Scrolls V: Skyrim? It would be nice if generative AI could be integrated into the game’s Radiant quest system to provide more realistic and believable quests.

Yes, I think Inworld or ChatGPT would be a good fit for Skyrim. I’m working on something already, which I will be showcasing when I have enough time to finish it (due to my work, I’m only able to work on it during weekends in my free time.)

Regarding generating quests, though, I will cautiously say no. I’m saying no because people think ChatGPT or other LLMs are magic wands that can be easily integrated into game systems. Unfortunately, this is not the case. Perhaps this will go into too much detail, but no GPT solution would be plug-and-play for game quest/interaction systems, no matter how sophisticated they are. What I mean by that is even if you generate quests in Skyrim, you still need to implement some checks and systems to validate within the game that checks the conditions of your quest and reward or punish the player according to those conditions. And you need to do these checks in a more “template” manner so that you can have a working system. This means you need to map possible actors to ChatGPT responses and possible outcomes to check. If ChatGPT tells you to get a soul gem from Dragonsreach, you need to have a system that can understand “soul gem” and “Dragonsreach” and create a quest accordingly. And this system should also handle cases where the LLM hallucinates, like if it tells you to get an apple from Germany, it shouldn’t cause any crash or corruption. And once you have all the mapping done, there is not much need for a powerful GPT to create these quests for you.

You can use a randomizer to generate the same type of quests more reliably and with more control over them. Like in my last video, in that quest showcase, I didn’t use ChatGPT to generate the quest. It was only adjusting the prize based on my answer (sheep amount) and allowing me to talk/haggle with the villager. ChatGPT was just a natural quest-taking system for existing quests. I believe this has better value than random generation. Because with this, you can have some quests that you solely need to trust your instincts and ask questions before accepting the quest. Who knows, perhaps that villager was luring you into a trap – if you asked more questions before accepting it, you could perhaps smell the fishy story and wouldn’t accept it.

Do you believe we’re going to see a substantial integration of these generative AI systems into games? If so, how long do you think this will take?

I’m sure of that. I think that given the fact that AI is now very accessible and growing exponentially, there is no chance that it won’t disrupt the gaming industry. We are already seeing some AI integrations in games, research papers about them, or games in development that use AI Art or AI Tech. I don’t think it will take long before some AAA companies utilize AI (which I believe Ubisoft has already started working with AI to ease the writing burden in their games). However, I also must say that AI/LLMs are not suitable for all games or genres. I don’t think every game will have GPT in the future, and it shouldn’t. GPTs are good if your game requires some role-playing and chatting. If you allow people to chat with NPCs and interact with the world by using language to increase immersiveness, then it makes sense. If you are making a detective game like Inworld did here, it also makes sense so that you can use your inquisitive nature to get answers and investigate. But if your game doesn’t have such elements, and if it’s Doom-esque or trying to tell a specific tailored story, having GPT in it would only add overhead.

Fair enough, and thank you for your time. Keep up the good work!