591 lines
37 KiB
Markdown
591 lines
37 KiB
Markdown
# Feishu 问题清单与黄金对话样例
|
||
|
||
## 目的
|
||
|
||
这份清单用于给 `统一 Project Operating Context + 三路 handoff` 提供验收基线。
|
||
|
||
本文件不负责定义实现方案,只负责收口三件事:
|
||
|
||
1. 当前 Feishu / Web / query.answer / requirement-plan 的已知失败类型
|
||
2. 用户可感知的黄金对话样例
|
||
3. 能落到回归测试里的 before snapshot
|
||
|
||
若要直接过目 12 条成功案例的完整输入/输出/步骤,请看:
|
||
|
||
- [2026-04-17-unified-project-ai-handoff-success-cases.md](/Users/yuuuu/Documents/AIProjectManger/.worktrees/codex/unified-project-ai-handoff/docs/superpowers/specs/2026-04-17-unified-project-ai-handoff-success-cases.md)
|
||
|
||
## Baseline Snapshot
|
||
|
||
- 记录时间:2026-04-17
|
||
- 基线分支:`master`
|
||
- 基线提交:`3f94cf7` (`Add explicit QA anchors and route governance pages`)
|
||
- 执行命令:
|
||
|
||
```bash
|
||
cd /Users/yuuuu/Documents/AIProjectManger && PYTHONPATH=backend backend/venv/bin/python -m pytest \
|
||
backend/tests/test_feishu_pipeline_queue_contract.py \
|
||
backend/tests/test_ai_pmo_qa_http_mainline_e2e.py \
|
||
backend/tests/test_adaptive_rag_external_followup_contract.py -v
|
||
```
|
||
|
||
- 基线结果:
|
||
- `73` 个用例里 `68` 通过,`5` 失败
|
||
- 其中 `3` 个失败属于测试契约漂移,不直接代表客户可见 runtime 回归:
|
||
- `_should_process_feishu_high_accuracy()` 已改为返回 `(should_sync, reason, semantic_bundle)`,但测试仍按二元组解包
|
||
- 其中 `2` 个失败属于真实 follow-up 退化,直接影响 Feishu / QA 追问体验:
|
||
- 多候选 meta follow-up 没进入澄清分支
|
||
- `第二条` 这类中文编号选择无法恢复原问题
|
||
|
||
## 飞书展示验收基线
|
||
|
||
飞书侧的客户可见验收,不以内部 `rag_result / fit_brief / external_recommendation` payload 直接为准,而以“后端最终准备发给飞书的内容”为准。
|
||
|
||
- 非交互 QA:
|
||
- 以 `qa_direct` 文本为基线
|
||
- 真正的格式化出口是 `FeishuReceiveRoutingMixin._build_feishu_qa_direct_reply_text()`
|
||
- 验收时要看最终文本是否已经包含:
|
||
- 结论正文
|
||
- 围绕当前项目实体的分析段
|
||
- 外部判断对应的分析结果
|
||
- 必要时才出现的下一步建议
|
||
- 强交互 requirement / draft / pending-manual:
|
||
- 以 action card payload 为基线
|
||
- 真正的格式化出口是 `FeishuCardActionMixin._build_action_card_from_capture_result()`
|
||
- 验收时要看最终 card 结构、按钮动作码与 `work_item_id`,而不是只看内部语义字段
|
||
- 两类 Feishu 输出都必须经过 `FeishuReplyOutboxMixin._enqueue_reply_outbox()` 写入 `feishu_reply_outbox`
|
||
- 因此飞书验收必须分开看两类最终输出:
|
||
- `qa_direct` 文本
|
||
- `draft_pending_confirm / requirement_plan_followup / pending_manual` card
|
||
|
||
最小样例口径:
|
||
|
||
```text
|
||
外部实时查询:<结论正文>
|
||
|
||
结合当前项目看,<项目/仓库摘要>。就这次判断来说,<external_recommendation 的客户可见分析>。
|
||
|
||
<recommended_next_step>
|
||
```
|
||
|
||
```json
|
||
{
|
||
"header": {"title": {"content": "AI PMO 需求收敛"}},
|
||
"elements": [
|
||
{"tag": "div", "text": {"content": "A/B/C 选项正文"}},
|
||
{"tag": "action", "actions": [{"value": {"action": "plan_choice", "work_item_id": 77}}]}
|
||
]
|
||
}
|
||
```
|
||
|
||
## 失败类型清单
|
||
|
||
### 1. `reply/thread` 锚点丢失或绑错
|
||
|
||
- 表现:明明在回复某条机器人消息,runtime 却新开普通 query 话题
|
||
- 风险:requirement-plan / draft confirm / follow-up 全部容易串线
|
||
- 重点证据:
|
||
- `backend/tests/test_feishu_pipeline_queue_contract.py`
|
||
- `docs/architecture/RUNTIME_GUARDRAILS.md`
|
||
|
||
### 2. `internal / external / planmode` 误分流
|
||
|
||
- 表现:同一句话在不同入口被解释成不同主链
|
||
- 风险:回答与动作边界漂移,用户不知道当前系统在“答问题”还是“要执行”
|
||
|
||
### 3. 外部问答后的短 follow-up 错掉进 internal 或 plan
|
||
|
||
- 表现:用户只回一句“继续核实”“那你刚才那条是哪个模型回的”,系统不留在 query.answer
|
||
- 风险:外部 freshness 问答续链失败,或被错误拉进 plan
|
||
- 重点证据:
|
||
- `backend/tests/test_adaptive_rag_external_followup_contract.py::test_ambiguous_recent_reply_returns_numbered_candidates`
|
||
- `backend/tests/test_adaptive_rag_external_followup_contract.py::test_pending_clarification_selection_restores_original_question`
|
||
|
||
### 4. “适不适合 / 推荐什么 / 怎么推进” 这类复合问题被拆错
|
||
|
||
- 表现:系统只做内部事实问答,或只做外部搜索,或直接进 plan
|
||
- 风险:丢掉“先判断,再给 plan 入口”的默认策略
|
||
|
||
### 5. 同群同人 scope 串线
|
||
|
||
- 表现:同一聊天里换了新话题,旧 requirement / draft / QA subject 继续抢主链
|
||
- 风险:用户需要反复声明“我现在说的是另一个东西”
|
||
|
||
### 6. 旧 active context 劫持新问题
|
||
|
||
- 表现:明明已是新问题,runtime 还沿旧 `pending_clarification / draft / active requirement` 继续解释
|
||
- 风险:回复看起来像“接着上轮说”,但对象完全不对
|
||
|
||
### 7. 明明该继续 plan,却掉回 `query.answer`
|
||
|
||
- 表现:已经在 active requirement / explicit reply 锚点里,用户继续 `A/B/C` 或“那按这个推进”,却被当普通 QA
|
||
- 风险:需求收敛链断掉
|
||
|
||
### 8. 明明只是问结论,却被直接拉进 plan
|
||
|
||
- 表现:用户只是在问适配性或建议,系统马上切成 requirement-plan
|
||
- 风险:用户没有先拿到结论判断,产品体验像“被抢操作权”
|
||
|
||
### 9. 卡片动作和纯文本回复行为不一致
|
||
|
||
- 表现:点按钮能续上,文本协议续不上;或者反过来
|
||
- 风险:同一 requirement thread 出现双轨行为
|
||
|
||
### 10. Web 与 Feishu 对同一句话给出不同链路
|
||
|
||
- 表现:Web 浮窗保留在问答,Feishu 同句掉进计划链,或反过来
|
||
- 风险:统一项目型 AI 的前台体验不一致
|
||
|
||
## 黄金对话样例
|
||
|
||
### Case 01: 纯内部系统问题
|
||
|
||
- 场景名:内部配置只读问答
|
||
- 初始上下文:项目页内,`project_id=17`
|
||
- 用户原话:`当前 PMFlow 里 MCP 模板有几个?`
|
||
- 当前错误行为:被当成外部搜索或泛化说明题
|
||
- 目标行为:走 internal,只回答当前系统内对象事实
|
||
- 应命中的 lane / handoff:`internal`
|
||
- 验证方式:`/rag/ask` 的 `route_mode=internal`,回答不混外部来源
|
||
|
||
### Case 02: 纯外部事实问题
|
||
|
||
- 场景名:外部官方信息问答
|
||
- 初始上下文:普通 QA 线程,无 active plan
|
||
- 用户原话:`OpenAI Docs 最近有没有新的 Responses API 最佳实践?`
|
||
- 当前错误行为:被内化成系统配置题或泛化聊天
|
||
- 目标行为:走 external,只基于外部证据回答
|
||
- 应命中的 lane / handoff:`external`
|
||
- 验证方式:external-only 来源合同生效,内部来源不进入正文
|
||
|
||
### Case 03: 先判断,再给 plan 入口
|
||
|
||
- 场景名:项目适配性判断
|
||
- 初始上下文:当前项目明确,repo 摘要可用
|
||
- 用户原话:`OpenAI Docs 这种能力适不适合我们当前项目?`
|
||
- 当前错误行为:直接进 plan,或者只给功能介绍
|
||
- 目标行为:先给项目判断,再给一个明确 plan 入口
|
||
- 应命中的 lane / handoff:`external + fit brief + offer_after_answer`
|
||
- 验证方式:回答里先有结论,再有 `recommended_next_step / can_enter_plan`
|
||
|
||
### Case 04: 显式强锚点直接进 plan
|
||
|
||
- 场景名:结论后直接推进
|
||
- 初始上下文:上一轮已给适配性判断,当前消息显式 reply 到该轮
|
||
- 用户原话:`那按这个推进`
|
||
- 当前错误行为:回到普通 query.answer
|
||
- 目标行为:直接 handoff 给 plan 主链
|
||
- 应命中的 lane / handoff:`explicit_plan_now`
|
||
- 验证方式:命中 active requirement / draft 主链,不再重问模式
|
||
|
||
### Case 05: reply/thread 明确续问
|
||
|
||
- 场景名:对上一条 QA 解释继续追问
|
||
- 初始上下文:显式 reply 到机器人上一条解释性回复
|
||
- 用户原话:`你刚才这条说的“模型路由”具体指哪层?`
|
||
- 当前错误行为:按 session 旧主题或新问题重开
|
||
- 目标行为:优先用 reply/thread 锚点改写问题
|
||
- 应命中的 lane / handoff:`query.answer follow-up`
|
||
- 验证方式:锚点优先于 `last_subject`
|
||
|
||
### Case 06: 同群同人但切新话题
|
||
|
||
- 场景名:scope 内新问题
|
||
- 初始上下文:同一群、同一用户,上一轮刚结束 requirement-plan
|
||
- 用户原话:`另外,现在 PMFlow 里 signal source onboarding 做到哪了?`
|
||
- 当前错误行为:继续沿 requirement-plan 解析
|
||
- 目标行为:显式识别为新 query 话题
|
||
- 应命中的 lane / handoff:`internal`
|
||
- 验证方式:旧 active context 不得抢走新问题
|
||
|
||
### Case 07: 旧 active context 劫持新问题
|
||
|
||
- 场景名:pending clarification 清理不彻底
|
||
- 初始上下文:上一轮存在 follow-up 澄清候选
|
||
- 用户原话:`不是说那个,我现在问 external gateway。`
|
||
- 当前错误行为:继续消费旧澄清状态
|
||
- 目标行为:新话题应显式跳出旧 clarification
|
||
- 应命中的 lane / handoff:`query.answer` 新话题
|
||
- 验证方式:pending clarification 被清理或失效
|
||
|
||
### Case 08: 明明该继续 plan,却掉回 query.answer
|
||
|
||
- 场景名:A/B/C 协议回复掉链
|
||
- 初始上下文:active requirement-plan,上一轮刚发 A/B/C
|
||
- 用户原话:`我选 C`
|
||
- 当前错误行为:普通 QA 回复或 mixed reply
|
||
- 目标行为:继续 requirement-plan.collect
|
||
- 应命中的 lane / handoff:`plan`
|
||
- 验证方式:协议解析命中当前 option map
|
||
|
||
### Case 09: 明明只是问结论,却被直接拉进 plan
|
||
|
||
- 场景名:建议题被抢执行权
|
||
- 初始上下文:无 active plan,无强锚点
|
||
- 用户原话:`这个方案适不适合我们现在这套 PMFlow?`
|
||
- 当前错误行为:直接开始 requirement-plan 追问
|
||
- 目标行为:先答结论,再给可选下一步
|
||
- 应命中的 lane / handoff:`offer_after_answer`
|
||
- 验证方式:首轮不得直接进入 plan 主链
|
||
|
||
### Case 10: 卡片动作和纯文本回复不一致
|
||
|
||
- 场景名:同一 requirement thread 的双轨行为
|
||
- 初始上下文:同一轮次同时支持卡片按钮与文本协议
|
||
- 用户原话:`选 B`
|
||
- 当前错误行为:按钮能续上,文本不行
|
||
- 目标行为:按钮与文本都复用 canonical thread context
|
||
- 应命中的 lane / handoff:`plan`
|
||
- 验证方式:按钮 / 文本都命中同一 active topic
|
||
|
||
### Case 11: Web 与 Feishu 对同一句话不同链路
|
||
|
||
- 场景名:跨前台不一致
|
||
- 初始上下文:项目页 Web 与 Feishu 同时可问答
|
||
- 用户原话:`这个适合吗`
|
||
- 当前错误行为:Web 走判断,Feishu 直接进 plan 或掉成 internal
|
||
- 目标行为:统一遵守“先判断,再给 plan 入口”
|
||
- 应命中的 lane / handoff:`external + offer_after_answer`
|
||
- 验证方式:Web / Feishu 结构化结果一致
|
||
|
||
### Case 12: 多候选 meta follow-up 需要澄清
|
||
|
||
- 场景名:上一轮候选不唯一
|
||
- 初始上下文:最近 5 分钟内有 2 条候选回复
|
||
- 用户原话:`那你刚才那条是哪个模型回复的?`
|
||
- 当前错误行为:直接 no-anchor,或者答错对象
|
||
- 目标行为:返回编号候选,等待用户选条目
|
||
- 应命中的 lane / handoff:`query.answer follow-up clarify`
|
||
- 验证方式:返回 `ambiguous_recent_reply`,并能消费 `第二条`
|
||
|
||
## 当前测试映射
|
||
|
||
- Feishu pipeline:
|
||
- `backend/tests/test_feishu_pipeline_queue_contract.py`
|
||
- 当前已覆盖:reply/thread canonicalization、requirement-plan sync gate、显式协议续问
|
||
- Feishu 最终展示:
|
||
- `backend/tests/test_feishu_card_action_contract.py`
|
||
- 当前已覆盖:`qa_direct` 富文本摘要、`requirement_plan_followup` action card、外部状态文案分流
|
||
- HTTP mainline:
|
||
- `backend/tests/test_ai_pmo_qa_http_mainline_e2e.py`
|
||
- 当前已覆盖:`/rag/ask` 与 `/ai-pmo/process-text` 的 `project_operating_context` 前门 contract
|
||
- external follow-up:
|
||
- `backend/tests/test_adaptive_rag_external_followup_contract.py`
|
||
- 当前已覆盖:显式 reply 锚点、recent reply candidate、meta follow-up clarification
|
||
|
||
## 严格验收矩阵
|
||
|
||
下面这张矩阵才是本轮统一 handoff 的正式验收标准。
|
||
|
||
- 每个 case 都必须先给出最小输入
|
||
- 每个 case 都必须指明真实入站面和函数入口
|
||
- 每个 case 都必须说明“必须经过哪些主链”
|
||
- 每个 case 都必须区分:
|
||
- 结构化判定输出
|
||
- 最终用户可见输出
|
||
- 若当前没有完整自动化覆盖,必须显式标成 `partial` 或 `gap`
|
||
- 本表最后一列只表示“自动化覆盖状态”,不等于“业务验收完成”
|
||
- 若未单独注明,默认这些 case 仍未经过完整的人工业务验收 / 真实用户场景验收
|
||
- 当前真正做过的人工级 smoke 只包括:
|
||
- 后端模拟 `qa_direct` 最终出站文本
|
||
- 后端模拟 `requirement_plan_followup` 最终 action card 出站
|
||
- 这些 smoke 只能证明“后端最终输出层可工作”,不能替代完整业务验收
|
||
|
||
| Case | 最小输入 | 真实入站面 | 必经主链 | 必验结构化输出 | 必验最终输出 | 当前自动化证据 | 自动化覆盖状态 |
|
||
| --- | --- | --- | --- | --- | --- | --- | --- |
|
||
| 01 纯内部系统问题 | `当前 PMFlow 里 MCP 模板有几个?` | Web `POST /rag/ask`<br>`backend/app/api/rag_api/routes_query.py::rag_ask` | `rag_ask` -> `rag_ask_contract.build_kernel_ingress_kwargs()` -> `AIPMOKernelService.process_ingress()` -> `AdaptiveRAGQueryRouteGatewayMixin._run_query_route_gateway()` -> internal query path -> `_render_answer_with_route_contract()` | `route_mode=internal`<br>`effective_source_policy=internal`<br>`reply_contract.head=内部知识库查询:` | HTTP `answer` 必须以 `内部知识库查询:` 开头;不得混入 external-only source | `backend/tests/test_adaptive_rag_mainline_contract.py::test_query_route_gateway_uses_upstream_internal_contract_without_runtime_probes`<br>`backend/tests/test_adaptive_rag_mainline_contract.py::test_render_answer_with_route_contract_adds_explicit_source_prefix_and_summary` | covered |
|
||
| 02 纯外部事实问题 | `OpenAI Docs 最近有没有新的 Responses API 最佳实践?` | Web `POST /rag/ask` | `rag_ask` -> `build_kernel_ingress_kwargs()` -> `process_ingress()` -> `AdaptiveRAGQueryRouteGatewayMixin._run_query_route_gateway()` -> `external_query_service/*` -> `ExternalAnswerPresenter.present()` | `route_mode=external`<br>`effective_source_policy=external`<br>external source filter 必须 fail-closed | 最终回答只能来自 external route;internal source 进入 external route 时必须被过滤掉 | `backend/tests/test_adaptive_rag_mainline_contract.py::test_deep_guard_answer_external_route_uses_external_presenter_without_internal_context`<br>`backend/tests/test_adaptive_rag_mainline_contract.py::test_external_route_source_filter_is_fail_closed` | covered |
|
||
| 03 先判断,再给 plan 入口 | `OpenAI Docs 这种能力适不适合我们当前项目?` | Web `POST /rag/ask`;Feishu `message.receive_v1` | Web:`rag_ask` -> `build_kernel_ingress_kwargs()` -> `process_ingress()` -> `ProjectOperatingContextBuilder.build_from_ingress()` -> `_run_query_route_gateway()` -> `_build_response_payload()`<br>Feishu:`handle_message_event()` -> `_handle_message_receive()` -> `_capture_and_persist_from_ingress()` -> `_finalize_receive_result()` -> `_route_scene_reply()` -> `_render_feishu_qa_direct_reply_text()` -> `_safe_reply_text()` -> `feishu_reply_outbox` | `project_operating_context.current_project_id` 必须存在<br>`execution_recipe={primary_lane: external, fit_brief_required: true, external_evidence_required: true, plan_handoff_mode: offer_after_answer}`<br>`handoff_mode=offer_after_answer`<br>`can_enter_plan=true`<br>`recommended_next_step` 非空 | Feishu `qa_direct` 文本必须先经过受控 analysis plan,再输出最终文本;最终回复只允许围绕 1-2 个入选项目实体解释判断,不得把项目里所有任务/信号源摊开讲。若没有足够依据,不得硬塞下一步建议,也不得退回 `当前项目:/外部判断:/下一步:` 标签摘要 | `backend/tests/test_ai_pmo_qa_http_mainline_e2e.py::test_http_rag_ask_should_attach_project_operating_context_for_current_project`<br>`backend/tests/test_adaptive_rag_mainline_contract.py::test_query_route_gateway_builds_external_execution_recipe_for_current_project`<br>`backend/tests/test_adaptive_rag_mainline_contract.py::test_build_response_payload_exposes_execution_recipe_and_plan_handoff`<br>`backend/tests/test_feishu_card_action_contract.py::test_route_scene_reply_sends_rich_qa_direct_text_for_handoff_metadata`<br>`backend/tests/test_ai_pmo_ingress_mainline.py::test_route_scene_reply_enqueues_rich_qa_direct_text_into_feishu_outbox` | covered |
|
||
| 04 显式强锚点直接进 plan | `那按这个推进` | Feishu `message.receive_v1` reply/thread;Web active thread follow-up | `handle_message_event()` -> `_handle_message_receive()` -> `capture_from_feishu()` -> canonical thread context -> active requirement / follow-up resolution -> requirement-plan mainline | `execution_recipe.plan_handoff_mode=explicit_plan_now` 或 requirement mainline 直接命中 active context;不得退回普通 `query.answer` | 输出必须继续 plan 主链;不得重新询问 internal / external / plan 是哪种模式 | `backend/tests/test_adaptive_rag_mainline_contract.py::test_query_route_gateway_keeps_explicit_requirement_plan_recipe`<br>`backend/tests/test_feishu_pipeline_queue_contract.py::test_capture_from_feishu_explicit_reply_choice_protocol_recovers_active_requirement_thread_context` | covered |
|
||
| 05 reply/thread 明确续问 | `你刚才这条说的“模型路由”具体指哪层?` | Feishu reply/thread;QA session follow-up | `handle_message_event()` -> `_handle_message_receive()` -> `capture_from_feishu()` -> `AdaptiveRAGFollowupAnchorMixin._resolve_followup_anchor()` -> answer mainline | 必须先命中 reply anchor / resolved anchor reply;不得优先吃 `last_subject` 或旧 session lexical hint | 有效问题必须被重写到上一条回复对象;不能当新题重开 | `backend/tests/test_adaptive_rag_external_followup_contract.py::test_explicit_reply_to_message_id_resolves_anchor_without_session_history`<br>`backend/tests/test_adaptive_rag_external_followup_contract.py::test_followup_anchor_prefers_structured_semantics_without_keyword_hints` | covered |
|
||
| 06 同群同人但切新话题 | `另外,现在 PMFlow 里 signal source onboarding 做到哪了?` | Feishu `message.receive_v1` | `handle_message_event()` -> `_handle_message_receive()` -> `capture_from_feishu()` -> kernel intent/session inheritance guard | 旧 requirement / draft session 不得被继承;`_should_inherit_recent_feishu_session_context()` 必须返回 false;显式 signal-source 请求不得被 draft/plan 劫持 | 必须回到新的 internal query;不能继续 requirement-plan / draft-confirm | `backend/tests/test_ai_pmo_ingress_mainline.py::test_should_inherit_recent_feishu_session_context_skips_explicit_new_requirement_topic`<br>`backend/tests/test_ai_pmo_ingress_mainline.py::test_should_inherit_recent_feishu_session_context_skips_draft_pending_confirm_session`<br>`backend/tests/test_ai_pmo_ingress_mainline.py::test_should_inherit_recent_feishu_session_context_is_disabled_for_query_session`<br>`backend/tests/test_ai_pmo_ingress_mainline.py::test_draft_confirmation_route_does_not_hijack_explicit_signal_source_request` | covered |
|
||
| 07 旧 active context 劫持新问题 | `不是说那个,我现在问 external gateway。` | Feishu `message.receive_v1` 或 Web QA session | `process_ingress()` -> pending clarification / active draft guard -> new query path | 旧 `pending_clarification` 必须被识别为不再适用,或在新题下显式失效;不得强制消费旧 option map / draft route | 必须进入新 query topic;不能继续用旧 clarification 或旧 draft 解释 | `backend/tests/test_ai_pmo_ingress_mainline.py::test_process_ingress_short_circuits_query_route_pending_clarification_followup`<br>`backend/tests/test_ai_pmo_ingress_mainline.py::test_query_pending_clarification_followup_skips_explicit_new_topic_and_clears_stale_state` | covered |
|
||
| 08 明明该继续 plan,却掉回 query.answer | `我选 C` | Feishu `message.receive_v1`;kernel active context follow-up | `handle_message_event()` -> `_handle_message_receive()` -> `capture_from_feishu()` -> active requirement workspace lookup -> requirement choice protocol parser -> requirement-plan direct result | 必须命中当前 option map / active requirement context;不得走普通 QA semantic gate | 输出必须继续 `requirement_plan_followup`;不得出现普通 QA 解释文案 | `backend/tests/test_feishu_pipeline_queue_contract.py::test_capture_from_feishu_allows_group_choice_protocol_without_repeat_mention_when_requirement_plan_active`<br>`backend/tests/test_ai_pmo_ingress_mainline.py::test_active_requirement_choice_protocol_followup_bypasses_semantic_decision_gate`<br>`backend/tests/test_ai_pmo_ingress_mainline.py::test_execute_query_intent_requirement_plan_followup_persists_progress_and_asks_next_question` | covered |
|
||
| 09 明明只是问结论,却被直接拉进 plan | `这个方案适不适合我们现在这套 PMFlow?` | Web `POST /rag/ask`;Feishu `message.receive_v1` | 与 Case 03 相同,但验收重点是不能直接 handoff 到 plan 主链 | `execution_recipe.plan_handoff_mode=offer_after_answer`<br>首轮不能直接 `explicit_plan_now` | 首轮输出必须先给结论;`recommended_next_step` 只能作为可选下一步,不能直接切 plan scene | `backend/tests/test_adaptive_rag_mainline_contract.py::test_query_route_gateway_builds_external_execution_recipe_for_current_project`<br>`backend/tests/test_adaptive_rag_mainline_contract.py::test_build_response_payload_exposes_execution_recipe_and_plan_handoff`<br>`backend/tests/test_feishu_card_action_contract.py::test_route_scene_reply_sends_rich_qa_direct_text_for_handoff_metadata` | covered |
|
||
| 10 卡片动作和纯文本回复不一致 | 文本:`选 B`<br>卡片:点击 `B` 按钮 | Feishu `message.receive_v1`;Feishu `card.action.trigger` | 文本:`handle_message_event()` -> `_handle_message_receive()` -> `capture_from_feishu()` -> active requirement lookup -> requirement-plan mainline<br>卡片:`handle_message_event()` -> `_handle_card_action_trigger()` -> synthetic ingress -> `process_ingress()` -> `_safe_reply_card()` | 文本与卡片都必须落到同一 `work_item_id` / canonical thread context;不得一条续上、一条掉链 | 文本续问与卡片动作都必须回到同一 requirement thread;输出都应继续 requirement-plan | `backend/tests/test_feishu_card_action_contract.py::test_handle_card_action_trigger_plan_choice_dispatches_synthetic_ingress`<br>`backend/tests/test_feishu_pipeline_queue_contract.py::test_capture_from_feishu_group_choice_protocol_prefers_active_requirement_over_older_pending_draft`<br>`backend/tests/test_feishu_card_action_contract.py::test_route_scene_reply_sends_requirement_plan_followup_as_card` | covered |
|
||
| 11 Web 与 Feishu 对同一句话不同链路 | `这个适合吗` | Web `POST /rag/ask`;Feishu `message.receive_v1` | Web:`rag_ask` -> `build_kernel_ingress_kwargs()` -> `process_ingress()` -> `AdaptiveRAG`<br>Feishu:`handle_message_event()` -> `_handle_message_receive()` -> `capture_from_feishu()` -> `_route_scene_reply()` | Web 与 Feishu 必须共享同一 `project_operating_context / execution_recipe / handoff_mode` 语义;不得 Web 是 `offer_after_answer` 而 Feishu 直接 `plan` 或 `internal` | Web 要显示同一 handoff meta;Feishu `qa_direct` 要落到同一“先判断,再给 plan 入口”文案结构 | Web 侧:`backend/tests/test_adaptive_rag_mainline_contract.py::test_build_response_payload_exposes_execution_recipe_and_plan_handoff`<br>`backend/tests/test_ai_pmo_qa_http_mainline_e2e.py::test_http_rag_ask_returns_offer_after_answer_handoff_metadata_for_current_project_fit_question`<br>`frontend/src/components/ai-steward/FloatingQAChat.test.tsx`<br>Feishu 侧:`backend/tests/test_feishu_card_action_contract.py::test_route_scene_reply_sends_rich_qa_direct_text_for_handoff_metadata` | covered |
|
||
| 12 多候选 meta follow-up 需要澄清 | 第一轮:`那你刚才那条是哪个模型回复的?`<br>第二轮:`第二条` | Feishu / QA session follow-up | `_resolve_followup_anchor()` -> ambiguous candidates -> pending clarification -> `_consume_pending_clarification_selection()` | 第一轮必须返回 `ambiguous_candidates`,带编号候选;第二轮必须恢复 `original_question` 并绑定 `selected_qa_id` | 用户可见输出必须先列编号候选,再能消费中文编号 `第二条` | `backend/tests/test_adaptive_rag_external_followup_contract.py::test_ambiguous_recent_reply_returns_numbered_candidates`<br>`backend/tests/test_adaptive_rag_external_followup_contract.py::test_pending_clarification_selection_restores_original_question` | covered |
|
||
|
||
### 严格判定规则
|
||
|
||
- `covered`
|
||
- 至少同时有:
|
||
- 一个结构化主链测试
|
||
- 一个最终输出测试
|
||
- 或者该 case 本身只适合看结构化输出,且已由主链 contract 直接覆盖
|
||
- `partial`
|
||
- 已有相邻 contract,但还没有完全对上黄金输入或黄金输出
|
||
- `gap`
|
||
- 当前没有可直接证明该 case 的自动化证据
|
||
|
||
### 业务验收口径
|
||
|
||
- “自动化覆盖” != “业务验收完成”
|
||
- 业务验收至少要求:
|
||
- 按黄金输入从真实入站面进入
|
||
- 经过本文声明的主链
|
||
- 结构化输出与最终用户可见输出都对上
|
||
- 对用户有意义的业务结论和线程续接行为都正确
|
||
- 按这个口径,当前还不能说“只剩 2 个业务 case 没测”
|
||
- 当前能诚实宣称的只有:
|
||
- `12/12` 个 case 已有自动化证据
|
||
- 但完整业务验收并未按 12 个黄金场景逐条跑完
|
||
|
||
### 本轮补齐结果
|
||
|
||
- `Case 07`
|
||
- 已补 `backend/tests/test_ai_pmo_ingress_mainline.py::test_query_pending_clarification_followup_skips_explicit_new_topic_and_clears_stale_state`
|
||
- 当前要求是:显式新话题信号不能再被旧 `pending_clarification` 抢走,且旧澄清状态必须立即失效
|
||
- `Case 11`
|
||
- 已补 `backend/tests/test_ai_pmo_qa_http_mainline_e2e.py::test_http_rag_ask_returns_offer_after_answer_handoff_metadata_for_current_project_fit_question`
|
||
- 当前要求是:同一句话在 Web 与 Feishu 两端都必须保留同一份 `offer_after_answer` handoff 语义
|
||
|
||
## 当前执行记录(2026-04-17)
|
||
|
||
本轮已实际执行一组覆盖 10 个黄金场景的关键切片,不再只停在“文档映射”。
|
||
|
||
执行命令:
|
||
|
||
```bash
|
||
cd /Users/yuuuu/Documents/AIProjectManger/.worktrees/codex/unified-project-ai-handoff
|
||
PYTHONPATH=backend /Users/yuuuu/Documents/AIProjectManger/backend/venv/bin/python -m pytest \
|
||
backend/tests/test_adaptive_rag_mainline_contract.py::test_query_route_gateway_uses_upstream_internal_contract_without_runtime_probes \
|
||
backend/tests/test_adaptive_rag_mainline_contract.py::test_render_answer_with_route_contract_adds_explicit_source_prefix_and_summary \
|
||
backend/tests/test_adaptive_rag_mainline_contract.py::test_deep_guard_answer_external_route_uses_external_presenter_without_internal_context \
|
||
backend/tests/test_adaptive_rag_mainline_contract.py::test_external_route_source_filter_is_fail_closed \
|
||
backend/tests/test_adaptive_rag_mainline_contract.py::test_query_route_gateway_builds_external_execution_recipe_for_current_project \
|
||
backend/tests/test_adaptive_rag_mainline_contract.py::test_query_route_gateway_keeps_explicit_requirement_plan_recipe \
|
||
backend/tests/test_adaptive_rag_mainline_contract.py::test_build_response_payload_exposes_execution_recipe_and_plan_handoff \
|
||
backend/tests/test_ai_pmo_qa_http_mainline_e2e.py::test_http_rag_ask_should_attach_project_operating_context_for_current_project \
|
||
backend/tests/test_feishu_card_action_contract.py::test_route_scene_reply_sends_rich_qa_direct_text_for_handoff_metadata \
|
||
backend/tests/test_ai_pmo_ingress_mainline.py::test_route_scene_reply_enqueues_rich_qa_direct_text_into_feishu_outbox \
|
||
backend/tests/test_adaptive_rag_external_followup_contract.py::test_explicit_reply_to_message_id_resolves_anchor_without_session_history \
|
||
backend/tests/test_ai_pmo_ingress_mainline.py::test_should_inherit_recent_feishu_session_context_is_disabled_for_query_session \
|
||
backend/tests/test_ai_pmo_ingress_mainline.py::test_draft_confirmation_route_does_not_hijack_explicit_signal_source_request \
|
||
backend/tests/test_feishu_pipeline_queue_contract.py::test_capture_from_feishu_allows_group_choice_protocol_without_repeat_mention_when_requirement_plan_active \
|
||
backend/tests/test_ai_pmo_ingress_mainline.py::test_active_requirement_choice_protocol_followup_bypasses_semantic_decision_gate \
|
||
backend/tests/test_feishu_card_action_contract.py::test_handle_card_action_trigger_plan_choice_dispatches_synthetic_ingress \
|
||
backend/tests/test_feishu_pipeline_queue_contract.py::test_capture_from_feishu_group_choice_protocol_prefers_active_requirement_over_older_pending_draft \
|
||
backend/tests/test_feishu_card_action_contract.py::test_route_scene_reply_sends_requirement_plan_followup_as_card \
|
||
backend/tests/test_adaptive_rag_external_followup_contract.py::test_ambiguous_recent_reply_returns_numbered_candidates \
|
||
backend/tests/test_adaptive_rag_external_followup_contract.py::test_pending_clarification_selection_restores_original_question \
|
||
-v
|
||
```
|
||
|
||
执行结果:
|
||
|
||
- `20 passed`
|
||
- 这组切片直接覆盖:
|
||
- Case 01
|
||
- Case 02
|
||
- Case 03
|
||
- Case 04
|
||
- Case 05
|
||
- Case 06
|
||
- Case 08
|
||
- Case 09
|
||
- Case 10
|
||
- Case 12
|
||
- 未被这轮实跑切片覆盖的仍是:
|
||
- Case 07
|
||
- Case 11
|
||
|
||
补齐剩余 2 个 gap 的执行命令:
|
||
|
||
```bash
|
||
cd /Users/yuuuu/Documents/AIProjectManger/.worktrees/codex/unified-project-ai-handoff
|
||
PYTHONPATH=backend /Users/yuuuu/Documents/AIProjectManger/backend/venv/bin/python -m pytest \
|
||
backend/tests/test_ai_pmo_ingress_mainline.py::test_query_pending_clarification_followup_skips_explicit_new_topic_and_clears_stale_state \
|
||
backend/tests/test_ai_pmo_qa_http_mainline_e2e.py::test_http_rag_ask_returns_offer_after_answer_handoff_metadata_for_current_project_fit_question \
|
||
-v
|
||
```
|
||
|
||
补齐结果:
|
||
|
||
- `2 passed`
|
||
- 这轮补齐后,12 个黄金场景都已有对应自动化证据
|
||
|
||
最终合并验收命令:
|
||
|
||
```bash
|
||
cd /Users/yuuuu/Documents/AIProjectManger/.worktrees/codex/unified-project-ai-handoff
|
||
PYTHONPATH=backend /Users/yuuuu/Documents/AIProjectManger/backend/venv/bin/python -m pytest \
|
||
backend/tests/test_adaptive_rag_mainline_contract.py::test_query_route_gateway_uses_upstream_internal_contract_without_runtime_probes \
|
||
backend/tests/test_adaptive_rag_mainline_contract.py::test_render_answer_with_route_contract_adds_explicit_source_prefix_and_summary \
|
||
backend/tests/test_adaptive_rag_mainline_contract.py::test_deep_guard_answer_external_route_uses_external_presenter_without_internal_context \
|
||
backend/tests/test_adaptive_rag_mainline_contract.py::test_external_route_source_filter_is_fail_closed \
|
||
backend/tests/test_adaptive_rag_mainline_contract.py::test_query_route_gateway_builds_external_execution_recipe_for_current_project \
|
||
backend/tests/test_adaptive_rag_mainline_contract.py::test_query_route_gateway_keeps_explicit_requirement_plan_recipe \
|
||
backend/tests/test_adaptive_rag_mainline_contract.py::test_build_response_payload_exposes_execution_recipe_and_plan_handoff \
|
||
backend/tests/test_ai_pmo_qa_http_mainline_e2e.py::test_http_rag_ask_should_attach_project_operating_context_for_current_project \
|
||
backend/tests/test_ai_pmo_qa_http_mainline_e2e.py::test_http_rag_ask_returns_offer_after_answer_handoff_metadata_for_current_project_fit_question \
|
||
backend/tests/test_feishu_card_action_contract.py::test_route_scene_reply_sends_rich_qa_direct_text_for_handoff_metadata \
|
||
backend/tests/test_ai_pmo_ingress_mainline.py::test_route_scene_reply_enqueues_rich_qa_direct_text_into_feishu_outbox \
|
||
backend/tests/test_adaptive_rag_external_followup_contract.py::test_explicit_reply_to_message_id_resolves_anchor_without_session_history \
|
||
backend/tests/test_ai_pmo_ingress_mainline.py::test_should_inherit_recent_feishu_session_context_is_disabled_for_query_session \
|
||
backend/tests/test_ai_pmo_ingress_mainline.py::test_draft_confirmation_route_does_not_hijack_explicit_signal_source_request \
|
||
backend/tests/test_feishu_pipeline_queue_contract.py::test_capture_from_feishu_allows_group_choice_protocol_without_repeat_mention_when_requirement_plan_active \
|
||
backend/tests/test_ai_pmo_ingress_mainline.py::test_active_requirement_choice_protocol_followup_bypasses_semantic_decision_gate \
|
||
backend/tests/test_feishu_card_action_contract.py::test_handle_card_action_trigger_plan_choice_dispatches_synthetic_ingress \
|
||
backend/tests/test_feishu_pipeline_queue_contract.py::test_capture_from_feishu_group_choice_protocol_prefers_active_requirement_over_older_pending_draft \
|
||
backend/tests/test_feishu_card_action_contract.py::test_route_scene_reply_sends_requirement_plan_followup_as_card \
|
||
backend/tests/test_adaptive_rag_external_followup_contract.py::test_ambiguous_recent_reply_returns_numbered_candidates \
|
||
backend/tests/test_adaptive_rag_external_followup_contract.py::test_pending_clarification_selection_restores_original_question \
|
||
backend/tests/test_ai_pmo_ingress_mainline.py::test_query_pending_clarification_followup_skips_explicit_new_topic_and_clears_stale_state \
|
||
-v
|
||
```
|
||
|
||
最终合并结果:
|
||
|
||
- `22 passed`
|
||
- 这条命令是当前严格自动化验收基线;对应 12 个黄金场景的关键证据都已并入同一轮执行
|
||
|
||
## 代表性实跑输入输出
|
||
|
||
下面这些是当前真正拿得到的“实跑级”输入输出样本,用来证明本文不是只列测试名。
|
||
|
||
### Sample A: Case 03 / Case 09 的 Feishu `qa_direct`
|
||
|
||
- 输入:
|
||
|
||
```json
|
||
{
|
||
"scene": "qa",
|
||
"message_content": "这个适合吗",
|
||
"result": {
|
||
"direct_reply_required": true,
|
||
"direct_reply_text": "外部实时查询:目前证据不足,建议先明确要查融资还是招聘动态。",
|
||
"rag_result": {
|
||
"fit_brief": {
|
||
"project_summary": "Alpha 是一个 AI 项目管理工作台。",
|
||
"repo_summary": "当前仓库负责统一项目上下文与 AI handoff。"
|
||
},
|
||
"external_recommendation": {
|
||
"retrieval_status": "evidence_found",
|
||
"impact_status": "indirect_only",
|
||
"external_impact": "目前更像背景相关,还不能直接映射到当前正式对象。"
|
||
},
|
||
"recommended_next_step": "如果你要,我可以基于当前结论进入 plan,继续把需求范围和下一步收清楚。"
|
||
}
|
||
}
|
||
}
|
||
```
|
||
|
||
- 最终出站文本:
|
||
|
||
```text
|
||
外部实时查询:目前证据不足,建议先明确要查融资还是招聘动态。
|
||
|
||
结合当前项目看,Alpha 是一个 AI 项目管理工作台,当前仓库负责统一项目上下文与 AI handoff。就这次判断来说,这条外部信息目前更像背景相关,还不能直接映射到当前正式对象,所以现在更适合作为参考,而不是直接当成已经命中当前项目的正式变化。
|
||
|
||
如果你要,我可以基于当前结论进入 plan,继续把需求范围和下一步收清楚。
|
||
```
|
||
|
||
- 判定重点:
|
||
- 先答结论
|
||
- 再追加分析式项目映射和下一步建议
|
||
- 不是直接切 plan
|
||
|
||
### Sample B: Case 10 的 `requirement_plan_followup` action card
|
||
|
||
- 输入:
|
||
|
||
```json
|
||
{
|
||
"scene": "qa",
|
||
"message_content": "C",
|
||
"result": {
|
||
"reply_template_key": "requirement_plan_followup",
|
||
"work_item_id": 88,
|
||
"direct_reply_text": "C. 混合增强路径\n\n⭐ 当前我更推荐:C"
|
||
}
|
||
}
|
||
```
|
||
|
||
- 最终出站 card 关键结构:
|
||
|
||
```json
|
||
{
|
||
"header": {
|
||
"title": {
|
||
"content": "AI PMO 需求收敛"
|
||
}
|
||
},
|
||
"elements": [
|
||
{
|
||
"tag": "div",
|
||
"text": {
|
||
"content": "C. 混合增强路径"
|
||
}
|
||
},
|
||
{
|
||
"tag": "div",
|
||
"text": {
|
||
"content": "⭐ 当前我更推荐:C"
|
||
}
|
||
},
|
||
{
|
||
"tag": "action",
|
||
"actions": [
|
||
{ "value": { "action": "plan_choice", "choice": "A", "work_item_id": 88 } },
|
||
{ "value": { "action": "plan_choice", "choice": "B", "work_item_id": 88 } },
|
||
{ "value": { "action": "plan_choice", "choice": "C", "work_item_id": 88 } },
|
||
{ "value": { "action": "plan_choice", "choice": "D", "work_item_id": 88 } }
|
||
]
|
||
}
|
||
]
|
||
}
|
||
```
|
||
|
||
- 判定重点:
|
||
- 最终输出不是普通文本
|
||
- 必须是 `requirement_plan_followup` card
|
||
- `work_item_id` 和 `plan_choice` 动作码必须保留
|
||
|
||
### Sample C: Case 12 的编号澄清协议
|
||
|
||
- 第一轮输入:
|
||
|
||
```text
|
||
那你刚才那条是哪个模型回复的?
|
||
```
|
||
|
||
- 第一轮结构化输出要求:
|
||
|
||
```json
|
||
{
|
||
"anchor_mode": "ambiguous_candidates",
|
||
"candidate_replies": [
|
||
{ "index": 1 },
|
||
{ "index": 2 }
|
||
]
|
||
}
|
||
```
|
||
|
||
- 第二轮输入:
|
||
|
||
```text
|
||
第二条
|
||
```
|
||
|
||
- 第二轮结构化输出要求:
|
||
|
||
```json
|
||
{
|
||
"selected_qa_id": 102,
|
||
"effective_question": "我问你这个是哪个模型执行回复的?"
|
||
}
|
||
```
|
||
|
||
- 判定重点:
|
||
- 不得直接答错对象
|
||
- 必须先给编号候选
|
||
- 必须支持中文编号选择
|
||
|
||
## Task 0 验收口径
|
||
|
||
- 这轮 unified context / three-lane handoff 不以“架构看起来合理”为验收
|
||
- 必须以本文件里的黄金样例 + 对应回归测试为准
|
||
- 飞书前台展示必须额外以“后端最终格式化后的 `qa_direct` 文本 / action card payload”验收,不能只看内部 handoff payload
|
||
- 若某个样例当前无法直接落测试,至少要在测试里以 `xfail` 或 coverage gap 形式显式占位
|
||
- 任何修复不得回到关键词仓、正则路由或硬编码 semantic split helper
|