哈佛研究：AI 提升咨询顾问效率但在战略任务上降低准确率
Centaurs & Cyborgs : The Jagged Frontier of AI
哈佛商学院分析了 758 名咨询顾问使用 GPT-4 的表现。第一项实验中，使用 AI 的顾问平均多完成 12.2% 的任务、速度快 25.1%、质量提升 40%，低绩效者受益最大（提升 43%，高绩效者为 17%）。
原文归纳了哈佛对 758 名咨询顾问用 AI 的实验数据，读者可以据此理解 AI 在哪些任务上提升绩效、在哪些任务上反而降低准确率。
In short : Harvard study shows AI boosts productivity but can reduce accuracy in strategic tasks, highlighting the need for effective human-AI collaboration.
Last week, Harvard Business School released its analysis of LLMs on 758 consultants’ performance when using AI. Three key insights emerged.
In the first experiment (inside AI’s capability frontier), consultants randomly assigned access to GPT-4 AI completed 12.2% more tasks on average & 25.1% faster. Quality improved by 40%.
Lower performing consultants benefit the most from AI augmentation, increasing performance by 43% compared to 17% for higher performers.
This rising-tide effect seems common across AI applications. The benefit to the lower quartiles is dramatic across sales, customer support, & consulting. This has broader implications, something I aim to write about later this week.
In the second experiment, consultants analyze business data to offer strategic recommendations. Here, AI reduces performance - those using AI are 19 percentage points less likely to produce correct solutions.
The “jagged frontier” conceptualizes how AI profoundly increases productivity on some tasks but provides no value or even diminishes performance on seemingly similar tasks.
As with any tool, we need to learn the best applications & techniques for hewing knowledge work effectively.
Last, the paper discusses two types of human/AI collaboration : centaurs & cyborgs. Centaurs ask for high-level help with AI. Cyborgs train the AI to act as a character. The difference remains a bit nebulous to me, but the last section of the paper in Appendix E highlights different techniques, some of which I hadn’t explored.
| Persona | Practices & Descriptions |
|---|---|
| Centaur | Ask AI to map a problem domain |
| Refine human input & improve its presentation | |
| Cyborg | Ask the AI to simulate a type of personality or character (e.g., a software buyer) |
| Ask the AI to edit its own work | |
| Ask the AI to explain its logic & expose contradictions | |
| Disagree with the AI |
I haven’t yet asked an LLM to impersonate someone but given the impact to education, I could imagine LLMs becoming sales & customer support trainers for new employees in those roles.
Especially since the impact on lower quartiles (presumably many of which are ramping) is so marked.
Get the next one in your inbox
The 1-minute read that turns tech data into strategic advantage.
Read by 150k+ founders & operators.
GP at Theory Ventures. Former Google PM. Sharing data-driven insights on AI, web3, & venture capital.
来源：Tomer Tunguz 博客（VC 分析） · tomtunguz.com