我为过敏朋友打造 Magnus AI：基于 Gemini Live API 的 Windows 语音助手

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

My friend Vashkar Ghosh has to plan his meals around his allergies. That means reading labels, double-checking ingredients and searching for safe options, again and again.

So I built him Magnus AI, a voice-first assistant for Windows. Instead of typing and searching, he can just say what he needs. He tells Magnus what he can't eat, and Magnus suggests meals that avoid those ingredients and keeps his preferences saved for next time.

Magnus does more than meals, because a helper that lives on his PC should handle the small daily things too. He can:

• Open and switch between apps by voice
• Set volume and brightness
• Run one-phrase routines like "Work Mode", "Study Mode" and "Night Mode"
• Search the web and play YouTube
• Set reminders and read his clipboard
• Show Magnus his screen or webcam and ask for help

Demo

https://drive.google.com/file/d/137xF8eviOjMR5DFXnfyZKzUN_IJyS8Br/view?usp=sharing

Code

https://github.com/RIDDHIDEV-OPS/Magnus-AI

How I Built It

Real-time voice loop. Microphone audio is resampled to 16 kHz mono and streamed over a WebSocket to the Gemini Live API. Replies come back as 24 kHz audio. There's about 250 ms of turn detection, and barge-in lets Vashkar interrupt mid-sentence the way he would with a person.

Self-describing tools. Every file in actions/ exposes one typed function. At startup, Magnus turns the docstrings into function declarations, so adding a new skill means dropping in a single file.

Windows control. Win32 APIs (SetForegroundWindow, AttachThreadInput), psutil and PyAutoGUI let Magnus find, launch and bring windows to the front.

Interface. A PyQt6 HUD with an animated orb, live audio waveforms, and CPU/RAM/GPU telemetry served from a small local FastAPI dashboard.

Local memory. Preferences and learned context are stored in a local JSON file with thread-safe atomic writes, and it's git-ignored so personal data never reaches the repo.

Resilience. If an API quota error hits, Magnus steps down a fallback ladder of Gemini models instead of going silent.

Why Does Open Innovation Matter?

I want to be upfront: the model at the heart of Magnus is Google's Gemini Live API, which is not open-weight. What is open is the code (MIT-licensed), the Python ecosystem it's built on (PyQt6, psutil, PyAutoGUI, FastAPI), and the Open-Meteo data behind its weather tool.

Building this for a friend taught me what that trade-off costs. Meal and allergy information is personal, and with a hosted model, his voice and screen go to a server he doesn't control. Open-weight models would fix that by running on his own laptop, working offline, costing nothing per use, and letting me tune the assistant to him. My next step is an offline mode built on an open-weight model so his data never leaves his machine.