Vibe Coding a Multi-player 3D FPS Game using AI
~5000 lines of code - Using Grok 3, Three.js, Cursor, Sonnet3.7, CSM.ai
"There's a new kind of coding I call "vibe coding", where you fully give in to the vibes, embrace exponentials, and forget that the code even exists." - Karpathy
Introduction
I was interested in knowing how far we can go with LLMs and generative models to make a simple but functional multi-player game, without explicitly coding it myself. I wanted something simple but it ended up being a fully functional 1v1 FPS game with close to 5000 lines of code :o - there is no way I could have created this within a weekend without AI. There are known bugs but it is surprising that it worked at all.
This approach matters because it represents a significant shift in how games can be created - potentially democratizing game development and allowing creators to focus on vision rather than implementation details. What if the barrier to entry for creating 3D multiplayer experiences could be dramatically lowered?
Before explaining how the game was created, here’s the game and here is the AI generated source code.
You can use CSM to generate your own avatar and profile image. It is saved in your local cache and should reload every time you come back. You can use text prompts to generate a new asset or load a pre-existing AI generated asset from your CSM profile (see the video above).
Credits: Anton Baumann for brainstorming, play testing and helping write this blog post.
The Methodology
The algorithm is simple and has three key steps:
Use Grok 3 to generate the first version of the game. Of all choices, this is the best LLM that can do full three.js games, perhaps due to its context window and reasoning abilities. I have tried other LLMs in the past but this one wins by a large margin in terms of (a few iterative) prompt to games. I can’t wait for it to be plugged into cursor and have access to its API.
Take this into cursor (composer) + sonnet 3.7
Play test the game and feed this back into step [2]. Repeat until convergence. Multi-player play testing and feeding back problems into the LLM is the most time consuming part of the process.
The Iterative Process
I am sharing a sequential list of my prompts in Grok 3, where you can see the rough process unfold. After this stage, the composer within the cursor was used to edit the entire code base.
Initial Setup
I want to create a 3d multiplayer COD style game
I want it browser based
I am just going to use grok 3 :) and CSM 3d assets
For now just use boxes and then we can replace later
what is the right next thing to add to iteratively build the game
Let's do option 2.
Gameplay Testing & Refinement
From here on out, I play test with every revision:
<I pasted code> + "this works so far"
Make an iconic counter strike style map that is highly addictive for browser play
player height is too high, it is hard to see walls and I can't pass through some doors
now I am unable to move!
Movement Mechanics
add jump and walking up and down movement. also add crouch. make the walls higher?
ok great but when I am on a block, it shakes
still the same problem. only on the blocks!
ok make the map bigger and have some shading so its easier to walls etc! lol
ok now while walking, should there be slight up / down movement or no?
ok I changed to const bobFrequency = 3; // Speed of bobbing and its good.
Visual Styling
can we give it a white line drawing look?
wait I want black edges and white background. I also want lighting on the white surfaces etc
ok visual is ok but now wasd is not moving the character ...
ok great this works. can the black shadows be felt more naturalistic on white things
yes its great. now add logo.png on the bottom right
logo.png:1 GET http://localhost:3000/logo.png 404 (Not Found) Image (anonymous) @ (index):231
ok works! what's next
Multiplayer & Combat System
how do I know which team is which, score etc? Let's do UI
when I have two players in two browsers, I can't see each other and shooting docent take scores down ..
now I can see it but when I shoot the other team member, it says: Shot fired (index):392 No valid target or same team: undefined (index):371 Shot fired
I am shooting red from green but its saying: ot fired {playerTeam: 'green', otherPlayers: Array(1)} (index):388 Raycast hit: {hitId: undefined, hitTeam: undefined, playerTeam: 'green', allTeams: Array(1)}
I am shooting red but its still: Raycast hit: {hitId: undefined, hitTeam: undefined, playerTeam: 'green', allTeams: Array(1), otherPlayersKeys: Array(1)}allTeams: ['red']hitId: undefinedhitTeam: undefinedotherPlayersKeys: ['8MD9Q1ENFzd_XmitAAAT']playerTeam: "green"[[Prototype]]: Object
Shot fired {playerTeam: 'red', otherPlayers: Array(1)} otherPlayers :
lets make the map bigger so its a really fun 1 vs 1 FPS
… etc
Multi-player Testing
Right before posting it I realized there were several errors when multiple people joined the game. This is perhaps the trickiest part of the process. I asked Anton to play test it with me together and report back problems. Here were some of the problems and prompts associated (Cursor composer with Sonnet3.7):
There is a bug. The other player sees me and also another copy of me (same meshes) but I can't see him. Please fix any bug in the logic of multi agents
For big systemic bugs, I had to first copy the entire codebase in Grok 3 and then feed the output as a prompt in cursor (composer + agent):
<index.html> + <server.js> + I am unable to see the other players. debug the logic of multiple rooms and make it rock solid
There is a bug. The other player sees me and also another copy of me (same meshes) but I can't see him. Please fix any bug in the logic of multi agents
sometimes I can see the opponent but I can't shoot it. sometimes it works.
As soon as a new player joins, or old browser window reconnects, have a stylistic button saying, 'Enter Game World' - otherwise too many stale windows reconnect when game is on
After the code base crossed 3k+ lines of code, Grok 3 UI and generation speed became unmanageable and I had to resort to cursor composer.
Reflections & Insights
What worked surprisingly well was how the LLMs could understand and implement gameplay mechanics based on my experiential feedback rather than technical specifications. Instead of debugging code, I was describing gameplay feelings - "it shakes on the blocks" or "the player height is too high."
The most challenging aspects were the multiplayer synchronization and hit detection, which required more specific technical prompts and several iterations to get right. Even sophisticated LLMs sometimes struggled with the complex spatial reasoning required for 3D collision detection.
Conclusion
This approach to game development - where AI handles the implementation details while the creator focuses on experience and feel - points to an interesting future for creative tools. The boundary between technical and creative work becomes increasingly blurred as we can simply describe what we want and how it should feel rather than specifying how to build it.
I believe the most important missing piece for this direction is to automate play testing. We need a general-purpose foundation model for game play, and SIMA from DeepMind might be the answer.