Building AI Agents in the MUD Metaverse
The purpose of this workshop is to teach participants how to create intelligent, interactive AI agents in the MUD Metaverse using XR Creator Studio.
Building AI Agents in the MUD Metaverse
The purpose of this workshop is to teach participants how to create intelligent, interactive AI agents in the MUD Metaverse using XR Creator Studio. Through hands-on guidance, attendees will learn to integrate OpenAI’s API, design agent behavior with system prompts, enable memory for conversational context, and visually represent agents using 3D models and animations. By the end, participants will be fully equipped to build and deploy their own AI-powered agents in immersive XR environments.
Instructor
Yoan Escobar brings over a decade of experience in software engineering, with a diverse background that includes interactive application development in VR/AR, machine learning, computer vision, web technologies, and system integration. Throughout his career, he has led and mentored cross-functional teams while continuously learning alongside them. He believes that strong teams grow through collaboration, active communication, knowledge sharing, and continuous self-improvement.
Building AI Agents in the MUD Metaverse
Step into the frontier of immersive intelligence. In this hands-on workshop, participants will learn how to build intelligent, interactive AI agents inside the MUD Metaverse using XR Creator Studio. Leveraging the power of OpenAI’s API, this session will guide you through creating fully contextual, visually embodied agents capable of dynamic conversation and world interaction.
By the end of the session, you'll have the skills to build your own AI-powered agents from scratch with persistent memory, custom instructions, and animated 3D presence.
1. OpenAI API Integration
-
Set up an OpenAI Developer account.
-
Generate and securely manage your API Key.
-
Understand basic prompt engineering and usage limits.
2. Creating the AI Agent Logic in XR Creator Studio
-
Add and configure the AI Agent Logic component in your XR scene.
-
Explore agent triggers, event handling, and communication models.
3. System Prompts & Agent Instructions
-
Define system-level prompts to shape your agent’s personality, tone, knowledge scope, and behavioral rules.
-
Use XR Creator Studio's text interface to dynamically edit and test system prompts in real-time.
4. Memory Persistence with Persistables
-
Understand the importance of context retention for multi-turn conversations.
-
Implement Persistables in XR Creator Studio to store and recall information during interactions.
-
Configure memory scope: local, session-based, or persistent across scenes.
5. 3D Embodiment: Models & Animations
-
Attach 3D character models to your AI agents.
-
Animate the agents with gestures, facial expressions, and idle behaviors to enhance immersion.
-
Use interaction triggers to coordinate animation states with conversation events.
-
Duration: 1 hour
-
Skill Level: Beginner to Intermediate
-
Tools Used: XR Creator Studio, OpenAI Dashboard, MUD Metaverse and 3D Asset Library (models + animations provided)
By the end of the workshop, you will be able to:
-
Set up an AI agent in the MUD Metaverse using XR Creator Studio.
-
Write custom instructions that define how your agent thinks and responds.
-
Implement persistent memory to make your agents smarter and more consistent.
-
Give your agent a body with a 3D model and lifelike animations.
-
Deploy and test your AI agent in a live XR environment.
-
XR Developers & Designers.
-
AI Enthusiasts and Prompt Engineers.
-
Game Developers.
-
Educators & Interactive Storytellers.
-
Metaverse Builders.
-
Laptop with XR Creator Studio installed (installation instructions will be provided ahead of time).
-
OpenAI account (we’ll help you set one up if needed).
-
Curiosity and creativity!
Verse
mud-ai-agents
Step into the frontier of immersive intelligence. In this hands-on workshop, participants will learn how to build intelligent, interactive AI agents inside the MUD Metaverse using XR Creator Studio. Leveraging the power of OpenAI’s API, this session will guide you through creating fully contextual, visually embodied agents capable of dynamic conversation and world interaction.
XR Controls Overview
Getting started? Select your device to see the basic controls.
Use arrows or W, A, S, D
Use W, A, S, D + Hold Shift
Use Q and E or left mouse button
Right mouse button
Left mouse button
Use the joystick
Touch + Drag
Touch
Left controller thumbstick
B or Y buttons to move faster
Right controller thumbstick
Right controller trigger
Left or Right controller trigger
SUPPORT MUD FOUNDATION EVENTS AND PROGRAMS
Tech-Centered Art, Art Creation, and XR Art Access: Together, let's build the future of art on digital media. By supporting our art + tech programs, our vibrant art-tech community, and the XR tech infrastructure of our non-profit community partners, we're ensuring that our programs remain accessible to everyone, becoming a collaborative ecosystem for all!
Your contributions go to the MUD Foundation INC, a 501(c)(3) organization based in Miami, Florida, and are used at our discretion for charitable purposes. They are tax-deductible in the U.S. to the fullest extent permitted by law.
Sponsors
