RPG-related Interactions with ChatGPT

I think that is super interesting in how people are using it. As an experiment I tried to have it write my Turn in combat for my IRL game character, Red Dragonborn Conquest paladin with shield master feat and it gave a pretty good description:

Prompt
I am playing dnd in a play by post format. I am going to give you a description of my action then you will provide a more narratively rich description of that action.
"I use my action to attack twice with my Longsword, using a 1st level smite on any hits. I also use my bonus action to do a shield shove attack"

Response
"With a resolute determination, you draw your Longsword, its blade gleaming in the dim light of the dungeon. You step forward, the weight of your armor no hindrance to your agility as you make two swift, calculated slashes towards your foe. As your blade connects with the enemy's flesh, you channel the divine energy within you, infusing each strike with the power of your 1st level smite, causing radiant energy to burst forth upon impact.

But your assault doesn't end there. With a deft movement, you pivot on your heel, bringing your shield crashing into your opponent with the force of a battering ram. The impact sends them staggering backward, momentarily stunned by the unexpected attack. It's a flurry of strikes, each move executed with precision and purpose, as you fight to turn the tide of battle in your favor."

Obviously in PbP the dice rolls could determine if those actually hit or not, and if the Shield Shove works but makes for something interesting

I've used it to generate poems and songs and rhyming prophecy, I've used it to brainstorm "head canon" and speculative fiction in established IPs (like Forgotten Realms, Middle-earth, "the Wizarding World", and others), I've used it to generate lists and random roll tables, and to help brainstorm plot for RPGs.

It's a great tool that in some cases does things I could "never" do (come up with actually good rhyming songs/poems/prophecy) and in some cases it just lightens the burden of GMing (spitting out lists and generating additional ideas for me to tinker with).

I have found that for extended conversations about narrow topics (which I think using it as a GM would fall under), you absolutely have to, as someone else put it, provide it periodic summaries of the conclusions you've come to (when GMing, this would be: what's happened so far in the plot) to keep it on track and prevent it from "losing the thread".

But given how popular solo RPGs have become, I think there's a HUGE audience/market for an AI GM.

I've experimented using LLMs as an early play tester of draft rule sets. Because they can reference the rules of nearly any other system, you can ask them to compare and contrast, ask them for mechanical advice, ask them for technical edits/copy edits/proofreading, and even ask them to run through 100 iterations of X scenario and then provide statistical analysis of the results (if you're looking to balance out some given mechanic). You definitely must be aware of hallucinations. Even the newer models are prone to making stuff up out of thin 1s and 0s. (ChatGPT's o1 reasoning model seems much more effective when you ask it to stick to "rules as written" analysis.) I've also played with using LLMs to teach more complex rulesets by having them develop escalating training paths (quizzes/scenario-based questions, etc.) and then take me through those training paths. For some reason, this method seems to make the models less prone to hallucinate. I haven't yet found one that effectively works for long-term GMing due to the number of tokens they can work on as input and lack of long-term memory - though these are mostly computational restraints that we should expect to overcome as time moves on. ChatGPT-4o, for example, is able to update its own memory during a conversation and that does appear effective in keeping things somewhat on-track.

For an interesting experiment, try the following prompt in one of the big three (ChatGPT, Claude, Gemini): "Please refamiliarize yourself with the Dungeons & Dragons module, The Keep on the Borderlands. Once complete, please work to run a game for me that plays through the module. Please use the Dungeons & Dragons Basic Set rules. I'd like for you to play as the DM and 3 player characters while I also play as a player character. Please also walk me through character creation. I'd like you to play the game as close to rules as written as possible though you may take liberties with getting us through the module. Please ask me any questions you might have before starting. Thank you." You'll find that you'll hit content filters quite a bit. :) You'll also find that the LLM often puts you in the driver's seat for decisions, often giving NPCs conflicting opinions that leave you as the deciding vote rather than trying to establish its players' agency (however dubious agency may be in an LLM). And you'll also find that the LLMs will throw things into the narrative that have no grounding in the source texts. The upshot though is that this prompt can show how far these things have come (and how far they have yet to go). I'm not necessarily an advocate for using LLMs, given their environment impact and the questionable ethics of their training, but I am super curious about how they work and what they can and can't do.