跳到正文
原文
Tomer Tunguz 博客(VC 分析)·· 2 小时前AI 评分35

在 DuckDB 连接串里加两个字母,播客处理迁移到云端

Small Data Becomes Big Data

AI 导读

在 DuckDB 连接串中加上 `md:` 两个字母,播客处理器就从本地迁移到云端 MotherDuck,10 个智能体可在夜间自动转录并总结播客。作者称 MotherDuck 的性价比是 Snowflake 3XL 的 2-4 倍速度、价格仅为其十分之一到百分之一。数据继续增长后可迁移到 DuckLake。

正文

In short : Two letters changed my infrastructure : adding 'md:' to my DuckDB connection migrated my podcast processor from local to cloud, enabling 10 agents to work while I sleep with 2-4x better price-performance than Snowflake.

I sleep better knowing my agents work through the night. Less work for me in the morning.

My podcast processor transcribes & analyzes conversations. I started on my laptop, needed a little database to collect podcast data & metadata, & booted up a DuckDB instance.

But then the data started to grow, & I wanted the podcast processor to run by itself. I changed two little letters, & the database moved to the cloud :

# Before : local only
conn = duckdb.connect('podcasts.db')

# After : cloud-native
conn = duckdb.connect('md:podcasts.db')

Now, in the small hours, 10 robots listen & summarize podcasts for me while I sleep.

As I collect more & more podcast information, my data has grown. I’m using a larger instance of MotherDuck.

Relative query time across cloud data warehouses

Source : ClickBench

Aside from ease of use, there are real price-performance advantages. MotherDuck systems are two to four times faster than a Snowflake 3XL & from a tenth to a hundredth of the price.

Cloud data warehouse cost comparison by instance

Source : ClickBench

As the amount of data expands & I process more technology podcasts every day, I’m sure I’ll need a data lake. At that point, I can migrate to DuckLake.

Small data becomes big data faster than you know it.

From Small Data to Big Data - The progression from local development to cloud-scale analytics

Two letters changed everything. In this era, when those letters aren’t AI it’s worth paying attention.

Get the next one in your inbox

The 1-minute read that turns tech data into strategic advantage.
Read by 150k+ founders & operators.

GP at Theory Ventures. Former Google PM. Sharing data-driven insights on AI, web3, & venture capital.

Bloomberg • WSJ • Economist

来源:Tomer Tunguz 博客(VC 分析) · tomtunguz.com