arXiv：cs.CL· Yue Qiu, Zekang Du, Yiqun Diao, Bingsheng He, Qinbin Li·· 3 小时前AI 评分39
从提示词到树结构：面向小样本表格分类的高效 LLM 引导决策树生成
From Prompts to Trees: Effective LLM-Guided Tree Generation for Few-Shot Tabular Classification
AI 导读
研究提出一种三阶段框架，通过让 LLM 生成规则再组织成树，将 LLM 知识蒸馏为可解释决策树，用于小样本表格分类。该方法在多个真实表格数据集上取得了优于现有基线的准确率与可解释性，且提示词开销显著降低。该工作已被 EMNLP 2026 主会接收为 oral 报告，代码已开源。
正文
Abstract:While Large Language Models (LLMs) possess rich world knowledge and impressive generalization capabilities, their direct application to tabular data classification is hindered by high inference costs and limited interpretability. In contrast, decision trees are fast and transparent but often underperform in low-data regimes. In this work, we propose a novel framework that bridges these paradigms by distilling LLM knowledge into interpretable decision trees under a few-shot learning setting. Instead of directly prompting the LLM to generate full trees, which is often unstable and inefficient, we develop a three-stage paradigm that prompts the LLM to generate rules and organize the rules into a tree. Experiments on multiple real-world tabular datasets demonstrate that our method achieves superior accuracy and interpretability with significantly lower prompting overhead compared to existing baselines.
| Comments: | Accepted to EMNLP 2026 Main as an oral presentation. Code available: this https URL |
| Subjects: | Machine Learning (cs.LG); Artificial Intelligence (cs.AI); Computation and Language (cs.CL) |
| Cite as: | arXiv:2610.10227 [cs.LG] |
| (or arXiv:2610.10227v1 [cs.LG] for this version) | |
| https://doi.org/10.48550/arXiv.2610.10227 arXiv-issued DOI via DataCite (pending registration) |
Submission history
From: Yue Qiu [view email]
[v1]
Wed, 7 Oct 2026 15:18:11 UTC (4,470 KB)
来源：arXiv：cs.CL · arxiv.org