Academic
mAIte: AI Companion Prototype
A Unity prototype for talking with an AI companion through an Oculus microphone, with GPT-4o handling the contextual voice response.
I built the conversation loop for a friendly, non-clinical mouse companion. The prototype records controller-triggered speech, sends it to GPT-4o and plays the reply, with clear feedback when something goes wrong.
The short version
- Recorded Oculus microphone input and converted it to WAV before each request
- Sent Base64 audio and conversation history to GPT-4o through UnityWebRequest
- Split recording, API calls, playback and error handling into separate Unity scripts
What I worked on
- Mapped the Oculus A and B buttons to microphone checks, recording and turn submission in MicrophoneCapture.
- Used WavUtility to convert AudioClips. GPT4AudioCaller sent each request, decoded the response and played the returned audio.
- ConversationManager kept the interaction state and history in one place without mixing it into the recording or API code.
What came out of it
The prototype let users talk to Ted, the first mouse companion, and hear replies that remembered the conversation. Beeps, logs and fallback messages covered missing microphones, empty recordings and invalid responses. The character’s final visual style was left for a later iteration.
What I learned
Testing Ted showed me how much a visible character changes the rhythm of a voice conversation. Recording, waiting and playback had to feel clearly different. Keeping the scripts small and handling errors early made the prototype much easier to adjust.
Tools and methods
- Unity
- C#
- Oculus Microphone
- UnityWebRequest
- WAV Audio Processing
- GPT-4o Audio
- MVP