Interactive-LLM-Powered-NPCs

Why we included this project

Open-world games rarely let you talk to the random guard or shopkeeper, but this project shows how to change that. It takes your microphone input, runs it through speech-to-text, feeds the transcript along with character context to a language model, and then renders a lip-synced facial animation that gets overlaid onto the NPC's face in the running game. The system identifies which character you're addressing, gives each one a generated personality, and uses vector stores plus pre-conversation files so characters keep consistent memory and distinct speaking styles. It also reads your webcam expressions to adjust responses, and it does all this without modifying the game's source code by replacing the displayed face pixels. It's a research-style prototype rather than a polished product, but for developers experimenting with LLM-driven in-game dialogue and real-time speech-to-speech pipelines, it's a useful reference.