用 Whisper 模型识别巴西葡萄牙语拍卖俚语与游戏出价

Posted on Oct 3 AI-assisted

This is a submission for the Hacktoberfest Weekend Challenge: Build for a Friend

What I Built

One friend of mine had an idea to create a multiplayer game around auctions where players bid using their voices in real time. With that idea the main problem emerges, how to capture and understand what action player wants in the middle of an auction, even if it is a idiom, slang, or other funny word that can be interpreted by humans to a monetary usage?

To solve this, I built Auction-ASR-PTBR: a real-time speech processing and phonological parsing pipeline designed to run locally alongside the game on players' PCs, translating chaotic speech into exact number bid values with tactical game flags such as player wants to add number or multiply the bet.

This system needs to be accurate with a low latency reponse time at same time it runs by side of game.

Demo

This video in pt-br made by my friend demonstrate a POC made already integrated into unity environment, recognizing real-time voice bids and colloquial Brazilian Portuguese idioms.

Direct link: Watch the demo on YouTube

Code

main branch: Standalone Python server with an embedded Web Audio push-to-talk interface running on CUDA / local CPU via Flask.
unity branch: Pluggable integration code ready for whisper.unity runtime environments.

How I Built It

This first Hacktoberfest project was made using antigravity-cli with Google`s Gemini Flash 3.8 High model, but is only made possible using open-source openai/whisper models.

The pipeline combines open-source ASR models with a deterministic linguistic post-processor.

Why Does Open Innovation Matter?

This project fundamentally requires open-source AI models:

Domain-Specific Fine-Tuning & Prompting: Open models allows the community (such as brunoqgalvao/whisper-small-pt-br on Hugging Face) to fine-tune Whisper on Brazilian Portuguese speech, giving us full control over decoder prompts and logits.
Zero Cloud Latency: In a competitive multiplayer auction, roundtrip network latency (500ms–1500ms) to a closed cloud API breaks the game. Local inference decodes bids instantly.
Zero Runtime Cost for Players: Because open-weight models can be embedded locally alongside Unity (whisper.unity), players don't need API keys, paid credits, or an active internet connection to play.

Prize Categories

Overall Hacktoberfest Challenge Entry: Hacktoberfest Weekend Challenge: Build for a Friend