{
  "id": "llm-decision-trees",
  "title": "大语言模型决策树：从 Tree Prompting 到智能体分类",
  "eyebrow": "学习程序结构与提示词",
  "description": "Tree Prompting 如何学习由语言模型提示调用组成的决策树，以及它与 Aug-Tree、GPTree 和 Agentic Classification Tree 的联系。",
  "lead": "Tree Prompting（EMNLP 2023）学习一棵决策树，由接收提示词的语言模型完成每个节点的分支判断。学到的提示词与控制流程共同构成可重复用于分类任务的程序。这一机制直接连接了今天的大语言模型决策树、自然语言问题树和智能体分类。",
  "original_papers": [
    "tree-prompting"
  ],
  "terms": [
    "大语言模型决策树",
    "提示决策树",
    "自然语言决策树",
    "智能体分类",
    "可学习的提示程序",
    "动态提示"
  ],
  "demo": "tree",
  "related_guides": [
    "probabilistic-attention"
  ],
  "reading_order": [
    "tree-prompting",
    "aug-tree",
    "gptree",
    "act"
  ],
  "sections": [
    {
      "id": "original-method",
      "title": "Tree Prompting 提出了什么",
      "paragraphs": [
        "John X. Morris、Chandan Singh、Alexander M. Rush、Jianfeng Gao 和 Yuntian Deng 在 EMNLP 2023 发表了 Tree Prompting；Morris 与 Singh 的贡献相同。预印本于 2023 年 10 月 21 日公开。每个内部节点让冻结参数的语言模型执行一条提示，再根据输出选择分支，最终由叶节点给出预测。",
        "这棵树从带标签的样本中学习。候选提示包括采样得到的示例、指令以及动态生成的提示。在动态版本中，iPrompt 针对到达当前节点的样本生成候选提示，再根据划分质量选出提示。节点提出的问题与树的分支结构都从数据中学习得到。"
      ],
      "citations": [
        "tree-prompting"
      ]
    },
    {
      "id": "program-view",
      "title": "把提示树理解为程序",
      "paragraphs": [
        "在一个客服消息分类示例中，第一个节点询问消息是否要求退款；如果回答“是”，第二个节点再询问消息是否涉及重复扣款。这条路径返回账单类标签；回答“否”则进入树的其他部分。这个固定示例展示控制流程；论文中的树则从数据中学习。",
        "这个程序由条件、分支和返回值组成。传统条件检查数值阈值或精确字符串；基于提示的条件让语言模型按指令理解输入。指令规定要问什么，语言模型给出针对当前输入的判断。",
        "路径日志能区分三种错误：节点问了无关的问题；问题相关，但模型回答错误；前面的判断都正确，叶节点却分配了错误类别。记录实际经过的路径，就能区分这些错误。修改叶节点无法修复上游错误的语义判断。"
      ]
    },
    {
      "id": "learning-and-cost",
      "title": "学习哪些内容，预测需要多少计算",
      "paragraphs": [
        "任务适配体现在选出的提示词和树结构中，无需更新语言模型权重。当每个节点的特征需要一次模型调用时，深度为 D 的路径需要 D 次调用，无需遍历整棵树。原始代码提供了兼容 scikit-learn 风格的分类器接口。",
        "训练决策树的开销与预测开销应分别衡量。训练时搜索候选问题，并在带标签的输入上评估它们；部署时执行保留下来的决策路径。因此，搜索阶段投入大量计算，并不意味着每次预测都要付出同样的开销。",
        "实现时需要明确标签集合、输出解析方式、停止规则，以及模型输出无效答案时的处理方式。除准确率外，还应测量平均路径长度和 token 用量。两棵深度相同的树，如果提示中包含的示例数量不同，计算开销也不同。保存训练好的提示词和树时，应同时记录模型版本：更换语言模型会改变这些可执行条件的含义。"
      ],
      "citations": [
        "tree-prompting"
      ]
    },
    {
      "id": "related-methods",
      "title": "比较语言模型在系统中承担的角色",
      "paragraphs": [
        "Aug-Tree 在训练时使用大语言模型扩展文本特征，部署后通过可解释的特征检查完成预测，无需调用语言模型。GPTree 构建由大语言模型支持的决策树，并允许专家通过反馈修改决策路径。ACT 则通过基于不纯度的选择与 TextGrad 优化，学习自然语言二元问题。",
        "这些区别对应不同的设计选择：生成特征、回答语义问题、搜索更好的问题，以及修改得到的程序。"
      ],
      "table": {
        "columns": [
          "方法",
          "节点或划分方式",
          "语言模型的作用"
        ],
        "rows": [
          [
            "Tree Prompting · 2023",
            "在输入上执行学到的提示词",
            "执行分支判断；动态版本还生成提示词"
          ],
          [
            "Aug-Tree · 2023；预印本 2022",
            "扩展文本特征的逻辑或组合",
            "在训练期间构造特征"
          ],
          [
            "GPTree · 2024",
            "语言模型辅助的决策规则与划分类型",
            "构建决策，并结合专家反馈修改"
          ],
          [
            "ACT · 2025",
            "学到的自然语言二元问题",
            "提出、评估并优化问题"
          ]
        ]
      },
      "citations": [
        "tree-prompting",
        "aug-tree",
        "gptree",
        "act"
      ]
    },
    {
      "id": "when-to-use",
      "title": "什么任务适合学习问题树",
      "paragraphs": [
        "当任务具有可重复使用的决策结构，而且带标签的样本能揭示哪些区分最重要时，这种设计就有用。意图路由、文档分类和归类任务都适合逐步提出语义问题。树把任务专属的结构保存在模型权重之外。",
        "问题容易读懂，并不等于语言模型回答问题的过程也已被解释。先把路径记录为实际执行的决策流程，再用改写、歧义输入和改变后的事实测试节点。如果改变了决定性事实，节点仍选择同一分支，它就没有完成预期的判断。",
        "选择不同的示例消息，查看它们经过固定树的路径。交互示例演示控制流程，不测量分类准确率。训练分类器时，使用原论文和代码，在训练样本上拟合，在验证集上选择树的大小，再用留出的样本评估保存下来的树。"
      ]
    },
    {
      "id": "original-notebook",
      "title": "运行原始 Tree Prompting 笔记本",
      "paragraphs": [
        "作者的<a href=\"https://github.com/csinva/tree-prompt/blob/master/demo.ipynb\">原始笔记本</a>使用 GPT-2 和 Rotten Tomatoes 样本拟合 <code>TreePromptClassifier</code>，预测留出样本的标签，比较单条提示词，并通过 <code>plot_tree</code> 绘制学到的树。请按<a href=\"https://github.com/csinva/tree-prompt\">代码仓库的安装说明</a>准备环境；该笔记本会下载模型和数据集。"
      ],
      "citations": [
        "tree-prompting"
      ]
    }
  ],
  "updated_at": "2026-11-11",
  "url": "https://yuntiandeng.com/research/zh/llm-decision-trees/",
  "papers": [
    {
      "id": "tree-prompting",
      "title": "Tree Prompting: Efficient Task Adaptation without Fine-Tuning",
      "authors": [
        {
          "given": "John X.",
          "family": "Morris"
        },
        {
          "given": "Chandan",
          "family": "Singh"
        },
        {
          "given": "Alexander M.",
          "family": "Rush"
        },
        {
          "given": "Jianfeng",
          "family": "Gao"
        },
        {
          "given": "Yuntian",
          "family": "Deng"
        }
      ],
      "year": 2023,
      "venue": "EMNLP 2023",
      "paper_url": "https://arxiv.org/html/2310.14034v1",
      "summary": "学习可重复使用的决策树，其内部判断由接收提示词的语言模型调用完成；候选提示包括少样本示例、指令和动态生成的提示。",
      "connection": "基于提示的决策树原始工作：通过修改树结构和提示词适配任务，同时保持语言模型参数不变。",
      "bibtex_type": "inproceedings",
      "citation_key": "treeprompting",
      "source_urls": [
        "https://arxiv.org/html/2310.14034v1"
      ],
      "pdf_url": "https://aclanthology.org/2023.emnlp-main.384.pdf",
      "code_url": "https://github.com/csinva/tree-prompt",
      "doi": "10.18653/v1/2023.emnlp-main.384",
      "equal_contribution": [
        "John X. Morris",
        "Chandan Singh"
      ],
      "arxiv_id": "2310.14034",
      "arxiv_first_posted": "2023-10-21",
      "abstract": "Prompting language models (LMs) is the main interface for applying them to new tasks. However, for smaller LMs, prompting provides low accuracy compared to gradient-based fine-tuning. Tree Prompting is an approach to prompting which builds a decision tree of prompts, linking multiple LM calls together to solve a task. At inference time, each call to the LM is determined by efficiently routing the outcome of the previous call using the tree. Experiments on classification datasets show that Tree Prompting improves accuracy over competing methods and is competitive with fine-tuning. We also show that variants of Tree Prompting allow inspection of a model’s decision-making process.",
      "abstract_source": "https://arxiv.org/html/2310.14034v1",
      "updated_at": "2026-11-11",
      "description_language": "zh-Hans",
      "abstract_language": "en"
    },
    {
      "id": "aug-tree",
      "title": "Augmenting interpretable models with large language models during training",
      "authors": [
        {
          "given": "Chandan",
          "family": "Singh"
        },
        {
          "given": "Armin",
          "family": "Askari"
        },
        {
          "given": "Rich",
          "family": "Caruana"
        },
        {
          "given": "Jianfeng",
          "family": "Gao"
        }
      ],
      "year": 2023,
      "venue": "Nature Communications 14, 7913",
      "paper_url": "https://www.nature.com/articles/s41467-023-43713-1",
      "summary": "Aug-Tree 使用语言模型生成的扩展内容，为树的划分构造可解释文本特征。",
      "connection": "与 Tree Prompting 遍历树时调用语言模型的方式不同，它在训练阶段完成特征扩展。",
      "bibtex_type": "article",
      "citation_key": "augtree",
      "source_urls": [
        "https://www.nature.com/articles/s41467-023-43713-1"
      ],
      "first_public": "2022-09-23",
      "doi": "10.1038/s41467-023-43713-1",
      "container_title": "Nature Communications",
      "volume": "14",
      "issue": "1",
      "article_number": "7913",
      "updated_at": "2026-11-11",
      "description_language": "zh-Hans",
      "abstract_language": "en"
    },
    {
      "id": "gptree",
      "title": "GPTree: Towards Explainable Decision-Making via LLM-powered Decision Trees",
      "authors": [
        {
          "given": "Sichao",
          "family": "Xiong"
        },
        {
          "given": "Yigit",
          "family": "Ihlamur"
        },
        {
          "given": "Fuat",
          "family": "Alican"
        },
        {
          "given": "Aaron Ontoyin",
          "family": "Yin"
        }
      ],
      "year": 2024,
      "venue": "arXiv:2411.08257",
      "paper_url": "https://arxiv.org/html/2411.08257v1",
      "summary": "使用语言模型辅助的划分构建决策树，并结合专家反馈修改决策路径。",
      "connection": "后续的大语言模型决策树系统，支持专家交互及多种划分类型。",
      "bibtex_type": "article",
      "citation_key": "gptree",
      "source_urls": [
        "https://arxiv.org/html/2411.08257v1"
      ],
      "arxiv_id": "2411.08257",
      "arxiv_first_posted": "2024-11-13",
      "publication_type": "preprint",
      "container_title": "arXiv",
      "updated_at": "2026-11-11",
      "description_language": "zh-Hans",
      "abstract_language": "en"
    },
    {
      "id": "act",
      "title": "ACT: Agentic Classification Tree",
      "authors": [
        {
          "given": "Vincent",
          "family": "Grari"
        },
        {
          "given": "Tim",
          "family": "Arni"
        },
        {
          "given": "Thibault",
          "family": "Laugel"
        },
        {
          "given": "Sylvain",
          "family": "Lamprier"
        },
        {
          "given": "James",
          "family": "Zou"
        },
        {
          "given": "Marcin",
          "family": "Detyniecki"
        }
      ],
      "year": 2025,
      "venue": "arXiv:2509.26433",
      "paper_url": "https://arxiv.org/html/2509.26433v4",
      "summary": "通过基于不纯度的划分选择与 TextGrad 优化，学习由自然语言二元问题组成的分类树。",
      "connection": "后续以智能体方式学习问题决策树的工作，将问题优化与树结构学习结合。",
      "bibtex_type": "article",
      "citation_key": "act",
      "source_urls": [
        "https://arxiv.org/html/2509.26433v4"
      ],
      "arxiv_id": "2509.26433",
      "arxiv_first_posted": "2025-09-30",
      "publication_type": "preprint",
      "container_title": "arXiv",
      "updated_at": "2026-11-11",
      "description_language": "zh-Hans",
      "abstract_language": "en"
    }
  ]
}