feat: add external query and semantic facts runtime

This commit is contained in:
Yuuuu 2026-04-16 11:17:52 +08:00
parent e42652b619
commit 4720c8117d
131 changed files with 17773 additions and 826 deletions

View File

@ -0,0 +1,2 @@
{"type":"click","text":"C. Tab 分对象类型\n \n 概览 | 项目 | 任务 | 信号 | 文档 | 人员 | 宪章 | 发布 | 复盘\n \n 当前 tab facts 数\n 失败数\n 最后抽取\n \n 当前对象类型的列表和详情\n \n \n \n \n 方案 C\n 对象边界最清楚,但你要横向看整体积压和失败会来回切 tab,运营效率最低。","choice":"c","id":null,"timestamp":1776240598958}
{"type":"click","text":"C. Tab 分对象类型\n \n 概览 | 项目 | 任务 | 信号 | 文档 | 人员 | 宪章 | 发布 | 复盘\n \n 当前 tab facts 数\n 失败数\n 最后抽取\n \n 当前对象类型的列表和详情\n \n \n \n \n 方案 C\n 对象边界最清楚,但你要横向看整体积压和失败会来回切 tab,运营效率最低。","choice":"c","id":null,"timestamp":1776241280184}

View File

@ -0,0 +1 @@
{"reason":"idle timeout","timestamp":1776243098908}

View File

@ -0,0 +1 @@
95325

View File

@ -0,0 +1,96 @@
<h2>Semantic Facts 状态页设计稿</h2>
<p class="subtitle">先确认页面信息架构和操作区布局。后端统一按“提交后自动入队 + 失活旧 facts + 手动重抽入口”设计。</p>
<div class="cards">
<div class="card" data-choice="a" onclick="toggleSelect(this)">
<div class="card-image">
<div class="mockup">
<div class="mockup-header">A. 总览台 + 队列栏 + 对象表 + 详情抽屉</div>
<div class="mockup-body">
<div style="display:grid;grid-template-columns:repeat(4,1fr);gap:10px;margin-bottom:10px;">
<div class="placeholder" style="height:68px;">对象总数</div>
<div class="placeholder" style="height:68px;">active facts</div>
<div class="placeholder" style="height:68px;">失败队列</div>
<div class="placeholder" style="height:68px;">积压时长</div>
</div>
<div style="display:grid;grid-template-columns:1.2fr 0.8fr;gap:10px;margin-bottom:10px;">
<div class="placeholder" style="height:92px;">按对象类型统计:项目 / 任务 / 信号 / 文档 / 人员 / 宪章 / 发布 / 复盘</div>
<div class="placeholder" style="height:92px;">抽取队列状态:pending / running / retry / dead_letter</div>
</div>
<div style="display:grid;grid-template-columns:1fr 0.55fr;gap:10px;">
<div class="placeholder" style="height:170px;">可搜索对象表:对象类型 / 标题 / facts 条数 / 最后抽取 / 队列状态 / 操作</div>
<div class="placeholder" style="height:170px;">右侧详情抽屉:facts 列表 / 证据 / 最近 run / 重抽按钮</div>
</div>
</div>
</div>
</div>
<div class="card-body">
<h3>方案 A</h3>
<p>一页看全:先看风险和积压,再搜对象,再在右侧展开 facts 详情。我推荐这个,最适合排查“哪个对象没抽上、为什么失败、要不要重跑”。</p>
</div>
</div>
<div class="card" data-choice="b" onclick="toggleSelect(this)">
<div class="card-image">
<div class="mockup">
<div class="mockup-header">B. 双栏工作台</div>
<div class="mockup-body">
<div style="display:grid;grid-template-columns:0.9fr 1.1fr;gap:10px;">
<div>
<div class="placeholder" style="height:78px;margin-bottom:10px;">队列状态总览</div>
<div class="placeholder" style="height:194px;">对象列表 + 类型筛选 + 搜索</div>
</div>
<div>
<div class="placeholder" style="height:88px;margin-bottom:10px;">当前对象摘要:facts 数 / 最后抽取 / 最近失败</div>
<div class="placeholder" style="height:184px;">facts 明细、run 时间线、手动重抽</div>
</div>
</div>
</div>
</div>
</div>
<div class="card-body">
<h3>方案 B</h3>
<p>偏“工作台”操作感,适合长时间盯一个对象排障;但全局统计和横向比较会弱一点。</p>
</div>
</div>
<div class="card" data-choice="c" onclick="toggleSelect(this)">
<div class="card-image">
<div class="mockup">
<div class="mockup-header">C. Tab 分对象类型</div>
<div class="mockup-body">
<div class="mock-nav" style="margin-bottom:10px;">概览 | 项目 | 任务 | 信号 | 文档 | 人员 | 宪章 | 发布 | 复盘</div>
<div style="display:grid;grid-template-columns:repeat(3,1fr);gap:10px;margin-bottom:10px;">
<div class="placeholder" style="height:76px;">当前 tab facts 数</div>
<div class="placeholder" style="height:76px;">失败数</div>
<div class="placeholder" style="height:76px;">最后抽取</div>
</div>
<div class="placeholder" style="height:190px;">当前对象类型的列表和详情</div>
</div>
</div>
</div>
<div class="card-body">
<h3>方案 C</h3>
<p>对象边界最清楚,但你要横向看整体积压和失败会来回切 tab,运营效率最低。</p>
</div>
</div>
</div>
<div class="pros-cons" style="margin-top:18px;">
<div class="pros">
<h4>我推荐 A 的原因</h4>
<ul>
<li>既能看全局积压,也能快速钻到单对象 facts。</li>
<li>和现有治理子页风格更一致,适合放到智能配置中心。</li>
<li>后面要加 bulk backfill、失败重放、对象覆盖率都比较自然。</li>
</ul>
</div>
<div class="cons">
<h4>设计约束</h4>
<ul>
<li>这轮“需求”按宪章(charter)展示,不含运行中 draft。</li>
<li>队列状态会直接暴露 pending / retry / dead_letter,不做粉饰。</li>
<li>对象详情默认展示 active facts,历史失活 facts 放二级查看。</li>
</ul>
</div>
</div>

View File

@ -95,14 +95,24 @@ frontend/ # React 前端
- 新增接口必须遵循响应包络:`{code, message, data}`,不得裸返回 - 新增接口必须遵循响应包络:`{code, message, data}`,不得裸返回
- 禁止在 API 层写业务逻辑,业务逻辑归属 `services/` - 禁止在 API 层写业务逻辑,业务逻辑归属 `services/`
- 禁止在前端组件内直接调用 `fetch`,必须通过 `frontend/src/api/` 封装层 - 禁止在前端组件内直接调用 `fetch`,必须通过 `frontend/src/api/` 封装层
- MCP 模板接入中心只允许保存模板参数、命令预览、runtime probe 与“本机已执行”确认状态;远程 `streamable_http` 模板允许由 PMFlow 后端直连校验/调用,本地 `stdio` 模板只能走 PMFlow Local Bridge 或兼容 MCP host;禁止从服务端直接写宿主机 `~/.codex/config.toml`、禁止替客户执行本机命令,GitHub 类模板只允许保存 env var 名称或占位符,不得落明文 token。
- 前端状态管理:服务端数据用 React Query,本地 UI 状态用 `useState` - 前端状态管理:服务端数据用 React Query,本地 UI 状态用 `useState`
- 用户可见核心业务内容零兜底:AI 失败、证据不足、结构化失败时必须显式失败或标记待确认,禁止用默认句、模板句、stub 冒充成功。 - 用户可见核心业务内容零兜底:AI 失败、证据不足、结构化失败时必须显式失败或标记待确认,禁止用默认句、模板句、stub 冒充成功。
- 业务语义、问答/执行归因与前台动作拍板必须优先走“对象/上下文绑定 -> `SemanticContextPacket` -> `SemanticDecision` -> gatekeeper”主链,禁止在 `backend/app` 运行时代码里恢复 `looks_like_*`、关键词仓或散落正则拍板。 - 业务语义、问答/执行归因与前台动作拍板必须优先走“对象/上下文绑定 -> `SemanticContextPacket` -> `SemanticDecision` -> gatekeeper”主链,禁止在 `backend/app` 运行时代码里恢复 `looks_like_*`、关键词仓或散落正则拍板。
- `query.answer` 路由与外部查询规划不得在运行时代码里维护 `_INTERNAL_ROUTE_OBJECTS / _EXTERNAL_ROUTE_OBJECTS / _SEMANTIC_QUERY_OBJECT_TYPES` 一类对象词表,也不得新增 `*_probe_is_strong / *_route_probe / query_kind(weather/news/company/market/model)` 这类二次语义拍板。`query_profile / source_policy_hint / SemanticDecision` 只能作为上游结构化输入;若证据不足,只能显式返回 `uncertain/clarify`,禁止在执行层靠对象集合、`subject_focus`、`needs_freshness`、`referenced_objects` 再次裁决 internal vs external。
- `llm_service` 运行时必须显式传入 `routing_meta`;禁止再通过函数名片段、hint 常量或模板词在底层推断 tier、结构化输出或 fallback 策略。 - `llm_service` 运行时必须显式传入 `routing_meta`;禁止再通过函数名片段、hint 常量或模板词在底层推断 tier、结构化输出或 fallback 策略。
- 显式 `reply/thread`、卡片动作和 `work_item_id` 属于更强锚点,必须优先于旧 session 和记忆续写。 - 显式 `reply/thread`、卡片动作和 `work_item_id` 属于更强锚点,必须优先于旧 session 和记忆续写。
- 进入 `requirement_plan` / draft confirm 的线程必须留在对应主链,禁止掉回 `query.answer`;其中 draft confirm 只允许显式 `reply/thread`、卡片按钮或 `work_item_id` 进入,禁止无锚点按“同群/同人”回捞旧 pending draft。若同 scope 旧 plan/draft 已被判定污染,运行时必须先终止该 scope 内的 active topic/work item,再回退到 `query.answer`。选项协议回复必须复用单点结构化解析器,不得散落关键词/正则。 - 进入 `requirement_plan` / draft confirm 的线程必须留在对应主链,禁止掉回 `query.answer`;其中 draft confirm 只允许显式 `reply/thread`、卡片按钮或 `work_item_id` 进入,禁止无锚点按“同群/同人”回捞旧 pending draft。若同 scope 旧 plan/draft 已被判定污染,运行时必须先终止该 scope 内的 active topic/work item,再回退到 `query.answer`。选项协议回复必须复用单点结构化解析器,不得散落关键词/正则。
- 已绑定 active requirement context 的线程里,显式 `退出需求模式` 必须先挂起当前 collecting/ready draft 对应的 requirement topic,再允许回到普通 `query.answer`;同一 `conversation_scope` 下的 topic 选择必须优先命中仍带 active draft 的 live requirement workspace,禁止被后续 idle query topic 抢走。
- draft confirm 若命中显式 `reply/thread` 锚点,候选 mounted draft / pending draft 必须先通过该锚点精确命中;同群、同 scope、同人但锚点不匹配的历史草稿不得参与打分或回捞,避免 requirement-plan A/B/C 线程被旧 draft 劫持。
- Feishu `requirement_plan` 首轮/追问回复若命中 `reply_template_key=requirement_plan_followup`,必须保留模板键并优先渲染交互卡片;卡片按钮回流必须复用 canonical Feishu `reply/thread` 上下文,禁止因 query renderer 吞模板键或按钮回流丢锚点而退回纯文本旧模板。
- 冷启动 `WorkflowSemanticPlanner` / `pmflow_semantic_context_decision` 失败时,运行时只能降级到带 `semantic_planner_degraded=true` 审计标记的中性 `query.answer` 场景;后续优先重试 semantic backfill,显式 active requirement context / reply anchor 仍留在 requirement-plan 主链,无锚点事实问句禁止硬降级成 requirement consultation。
- Feishu 显式 `reply/thread` 到机器人消息时,必须先通过 `feishu_reply_outbox.sent_message_id -> source_message_id` 归一化 canonical thread context,再进入后续主链。 - Feishu 显式 `reply/thread` 到机器人消息时,必须先通过 `feishu_reply_outbox.sent_message_id -> source_message_id` 归一化 canonical thread context,再进入后续主链。
- 外部 freshness 问答的 follow-up 若锚定上一轮“未接入实时外部数据/无法确认最新结论”的 QA 回复,默认继续留在 `query.answer` 聊天链并先尝试实时外部检索;只有用户显式点明系统/项目/配置等内部对象并提出接入动作时,才允许落入内部接入/入库流程。 - 外部 freshness 问答的 follow-up 若锚定上一轮“未接入实时外部数据/无法确认最新结论”的 QA 回复,默认继续留在 `query.answer` 聊天链并先尝试实时外部检索;只有用户显式点明系统/项目/配置等内部对象并提出接入动作时,才允许落入内部接入/入库流程。
- non-plan `query.answer` 必须先经过 `Query Route Gateway` 产出 `route_mode / decision_origin / reply_contract / probe_snapshot`,再进入 AdaptiveRAG;回复正文必须遵守显式来源合同(`内部知识库查询:` / `外部实时查询:`),`uncertain` 只允许 `external-first` 或澄清,禁止默认混合答案。
- `route_mode=external` 后不得再直接把 DuckDuckGo HTML 抓取当主执行层;必须统一进入 `External Query Planner -> External Query Gateway -> External Provider Registry -> MCP adapter 优先 -> bounded fallback live search -> external answer verifier` 主链。external 来源过滤必须 fail-closed,禁止把 `Feishu history / doc / pmflow` 这类内部来源混回 `外部实时查询` 的来源摘要。
- `query.answer` 的客户可见回复必须再经过 `Answer Presentation Policy`:客户视图只允许看到中文化 `display_label/display_summary`,禁止直接暴露 `raw_source_label`、`Project database / Semantic facts / Knowledge Graph / Feishu history` 等机器来源或内部术语。prompt context 也只能喂展示字段,不能再把原始英文来源标签直接喂给模型。
- internal `query.answer` 一旦命中 `route_mode=internal`,必须再走“`Operation Contract -> Target Resolver -> Query DSL Plan -> 白名单受控执行器 -> semantic facts 优先(可用时)-> bounded fallback retrieval -> answer verifier`”链路;禁止让 LLM 自由写 SQL、自由决定 join/表名,或在没有证据时直接给内部结论。
- `MountedDraftWorkspace.requirement_card` 是 requirement-plan 唯一业务真源;assistant prose、topic facts、run 文案只可审计,不可反推业务字段。 - `MountedDraftWorkspace.requirement_card` 是 requirement-plan 唯一业务真源;assistant prose、topic facts、run 文案只可审计,不可反推业务字段。
- `project_create.create / followup.confirm` 的任务拆解是从属子步骤:运行时必须先 low-first 结构化生成,必要时升级 high;仍失败则显式记为 `pending`,禁止冒充成功。 - `project_create.create / followup.confirm` 的任务拆解是从属子步骤:运行时必须先 low-first 结构化生成,必要时升级 high;仍失败则显式记为 `pending`,禁止冒充成功。
- 运行时长清单、Feishu / requirement-plan / AdaptiveRAG 细则、已知边界与假设,统一以 `docs/architecture/RUNTIME_GUARDRAILS.md` 为准。 - 运行时长清单、Feishu / requirement-plan / AdaptiveRAG 细则、已知边界与假设,统一以 `docs/architecture/RUNTIME_GUARDRAILS.md` 为准。

View File

@ -32,7 +32,7 @@ AIProjectManger/
``` ```
- 详细架构库存已迁到 [`docs/architecture/ARCHITECTURE_INVENTORY.md`](/Users/yuuuu/Documents/AIProjectManger/docs/architecture/ARCHITECTURE_INVENTORY.md)。 - 详细架构库存已迁到 [`docs/architecture/ARCHITECTURE_INVENTORY.md`](/Users/yuuuu/Documents/AIProjectManger/docs/architecture/ARCHITECTURE_INVENTORY.md)。
- `docs/architecture/` 当前包含:`MASTER_CONTRACT.md`、`RUNTIME_GUARDRAILS.md`、`CODE_REFERENCE_INDEX.md`、`API_CONTRACTS.md`、`DATAFLOW.md`、`TASK_BRIEF_TEMPLATE.md`、`ARCHITECTURE_INVENTORY.md` - `docs/architecture/` 当前包含:`MASTER_CONTRACT.md`、`RUNTIME_GUARDRAILS.md`、`CODE_REFERENCE_INDEX.md`、`API_CONTRACTS.md`、`DATAFLOW.md`、`TASK_BRIEF_TEMPLATE.md`、`ARCHITECTURE_INVENTORY.md`
- `docs/superpowers/plans/` 保存实现计划,`docs/task_briefs/` 保存任务包产物与验收截图。 - `docs/superpowers/plans/` 保存实现计划,`docs/superpowers/specs/` 保存需求/方案规格,`docs/task_briefs/` 保存任务包产物与验收截图。
## 4. 后端结构(`backend/`) ## 4. 后端结构(`backend/`)
```text ```text
@ -56,10 +56,18 @@ backend/
- 当前后端重点入口优先看: - 当前后端重点入口优先看:
- `backend/app/api/` 下的 `*_api/` 子包与聚合 router - `backend/app/api/` 下的 `*_api/` 子包与聚合 router
- `backend/app/services/ai_pmo/`(AI PMO 主链、Mounted Runtime、semantic context) - `backend/app/services/ai_pmo/`(AI PMO 主链、Mounted Runtime、semantic context)
- `backend/app/services/adaptive_rag_service/`(只读问答主链) - `backend/app/services/adaptive_rag_service/`(只读问答主链;`mixin_query_route_gateway.py` 负责 non-plan `query.answer` 的 internal/external/uncertain 来源合同归一化、contract 审计与显式来源回复合同;`mixin_answer_retrieval.py` 现已把 `route_mode=external` 接到独立外部查询主链;内部查询基线已升级为 `Operation Contract -> Target Resolver -> Query DSL Plan -> 受控执行器 -> semantic facts 优先 -> verifier`)
- `backend/app/services/external_query_service/`(external `query.answer` 主链:`External Query Planner -> External Query Gateway -> External Provider Registry -> MCP 优先 -> bounded fallback live search -> verifier`)
- `backend/app/services/mcp_config/`(MCP 模板接入中心:catalog、bundle 持久化、后端直连 runtime probe / Microsoft Learn 检索桥接、Local Bridge 接入说明与本机确认状态)
- `backend/app/services/semantic_fact_service/`(semantic facts 读模型、八类业务对象统一抽取/失活、ORM 提交钩子入队、治理看板查询)
- `backend/app/api/semantic_facts_api/`(semantic facts 治理后台接口:overview / objects / detail / reextract / backfill)
- `backend/app/api/mcp_api/`(MCP 模板接入中心后台接口:catalog / servers / validate / confirm-client-apply)
- `backend/app/services/llm_service/`(显式 `routing_meta` 协议) - `backend/app/services/llm_service/`(显式 `routing_meta` 协议)
- `backend/app/services/integrations/feishu_integration_service/`(飞书入站与卡片回调) - `backend/app/services/integrations/feishu_integration_service/`(飞书入站与卡片回调)
- `backend/app/services/signal/`(信号接入、编排、planner、repair) - `backend/app/services/signal/`(信号接入、编排、planner、repair)
- 本轮新增的问答路由审计存储为 `backend/app/models/query_route_memory.py`,通过 `backend/alembic/versions/20260415_add_query_route_memories.py` 建表;只用于 route audit / feedback 回灌 / 离线评估,不做在线自动调参。
- 本轮新增的语义事实读模型为 `backend/app/models/semantic_fact.py` 与 `backend/app/models/semantic_fact_extraction_run.py`,通过 `backend/alembic/versions/20260415_add_semantic_facts.py` 建表;用于内部查询的 facts-first 命中与异步抽取审计。
- 本轮新增的外部查询 fallback 回归测试位于 `backend/tests/test_external_query_planner.py`;配合 `backend/tests/test_adaptive_rag_mainline_contract.py`、`backend/tests/test_ai_pmo_ingress_mainline.py` 与 `backend/tests/test_semantic_lint.py` 一起锁定“Query Route Gateway 不再跑 runtime probes、External Query Planner fallback 不再按对象重分类、external freshness follow-up 不再靠硬编码词表”。
- 详细目录迁移、兼容壳清理与新增模块清单见库存文档。 - 详细目录迁移、兼容壳清理与新增模块清单见库存文档。
## 5. 前端结构(`frontend/`) ## 5. 前端结构(`frontend/`)
@ -83,6 +91,8 @@ frontend/
- `frontend/src/api/`(统一 API 封装) - `frontend/src/api/`(统一 API 封装)
- `frontend/src/pages/inbox/`(消息台) - `frontend/src/pages/inbox/`(消息台)
- `frontend/src/pages/ai-steward/`(AI 管家) - `frontend/src/pages/ai-steward/`(AI 管家)
- `frontend/src/pages/ai-config/mcp-config/`(AI Config -> Governance Tools 下的 MCP 模板接入中心)
- `frontend/src/pages/semantic-facts-board/`(AI Config -> Governance Tools 下的 semantic facts 治理页)
- `frontend/src/pages/scheduler-center/`(Mounted AI Console) - `frontend/src/pages/scheduler-center/`(Mounted AI Console)
- `frontend/src/components/sstc/`(SSTC 工作流组件) - `frontend/src/components/sstc/`(SSTC 工作流组件)
- 页面目录化、Mounted AI Console、Inbox、AI Steward 与 Recipes/Projects/Users 等模块化拆分的详细库存见库存文档。 - 页面目录化、Mounted AI Console、Inbox、AI Steward 与 Recipes/Projects/Users 等模块化拆分的详细库存见库存文档。
@ -143,6 +153,7 @@ frontend/
cd frontend cd frontend
npm run dev npm run dev
npm run build npm run build
npm test
npm run e2e:feishu npm run e2e:feishu
``` ```
@ -165,9 +176,12 @@ uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
- 当前最重要的摘要只保留: - 当前最重要的摘要只保留:
- 写操作必须经 Alembic / 受控执行器 / outbox / 审计日志,禁止业务路径直写或同步直发。 - 写操作必须经 Alembic / 受控执行器 / outbox / 审计日志,禁止业务路径直写或同步直发。
- 用户可见核心业务内容禁止 fallback、stub、模板句冒充成功;失败必须显式失败或标记待确认。 - 用户可见核心业务内容禁止 fallback、stub、模板句冒充成功;失败必须显式失败或标记待确认。
- MCP 模板接入中心只保存模板参数、命令预览、runtime probe 和手动确认状态;远程 `streamable_http` 模板可由 PMFlow 后端直连,`stdio` 模板只能走 Local Bridge / 兼容 MCP host;禁止从服务端直写宿主机 `~/.codex/config.toml` 或替客户执行本机命令。
- 运行时业务动作归因统一走“对象/上下文绑定 -> `SemanticContextPacket` -> `SemanticDecision` -> gatekeeper”。 - 运行时业务动作归因统一走“对象/上下文绑定 -> `SemanticContextPacket` -> `SemanticDecision` -> gatekeeper”。
- `backend/app` 运行时代码禁止恢复散落的中文关键词、正则、`looks_like_*`、`*_HINTS/*_KEYWORDS` 业务拍板逻辑。 - `backend/app` 运行时代码禁止恢复散落的中文关键词、正则、`looks_like_*`、`*_HINTS/*_KEYWORDS` 业务拍板逻辑。
- `llm_service` 只认显式 `routing_meta` 协议,不再靠函数名或 hint 推断 tier / contract / fallback。 - `llm_service` 只认显式 `routing_meta` 协议,不再靠函数名或 hint 推断 tier / contract / fallback。
- 内部只读查询必须走 `Operation Contract -> Target Resolver -> Query DSL Plan -> 白名单受控执行器 -> semantic facts 优先 -> verifier`,禁止自由 SQL 和无证据脑补。
- `semantic_facts` 自动抽取统一走 ORM 提交钩子 -> `ingest_events(channel=semantic_fact_extract)` -> 统一 worker,支持 `project/task/signal/document/user/requirement/release/review` 八类对象;禁止在各路由散落手工 enqueue。
- AI Steward、Requirement Plan、Signal Source、AdaptiveRAG 等核心链路的详细 guardrails 统一以 `RUNTIME_GUARDRAILS.md` 为准。 - AI Steward、Requirement Plan、Signal Source、AdaptiveRAG 等核心链路的详细 guardrails 统一以 `RUNTIME_GUARDRAILS.md` 为准。
## 9. 新增代码的参考模式 ## 9. 新增代码的参考模式

View File

@ -3,3 +3,13 @@
# Keep this file empty (or with only comments) to skip heartbeat API calls. # Keep this file empty (or with only comments) to skip heartbeat API calls.
# Add tasks below when you want the agent to check something periodically. # Add tasks below when you want the agent to check something periodically.
## 每日工单推送
- 触发时间:每天 08:30
- 任务:查询当天工单并推送到飞书
- 执行命令:
```bash
docker exec pmflow-db psql "postgresql://readonly:readonly%40%232025@173.1.1.32:5432/middle_platform" -c "SELECT task_code, task_name, excutor_name, status, start_time FROM plat_task_center.task_instance WHERE start_time >= CURRENT_DATE AND start_time < CURRENT_DATE + INTERVAL '1 day' AND ext3 IN ('90101', '901') ORDER BY start_time DESC LIMIT 50;" -x
```
- 目标用户:ou_9e1fefbcf4cc68a84f4a4ff7d1749048

View File

@ -0,0 +1,98 @@
"""add query route memories
Revision ID: 20260415_add_query_route_memories
Revises: 20260413_rename_control_plane_status
Create Date: 2026-04-15
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
try:
from sqlalchemy.dialects.postgresql import JSONB
except Exception: # pragma: no cover
JSONB = sa.JSON
revision = "20260415_add_query_route_memories"
down_revision = "20260413_rename_control_plane_status"
branch_labels = None
depends_on = None
def _json_type(bind):
return JSONB() if bind.dialect.name == "postgresql" else sa.JSON()
def _table_exists(bind, table_name: str) -> bool:
return sa.inspect(bind).has_table(table_name)
def upgrade() -> None:
bind = op.get_bind()
if _table_exists(bind, "query_route_memories"):
return
op.create_table(
"query_route_memories",
sa.Column("id", sa.Integer(), primary_key=True, autoincrement=True),
sa.Column("tenant_id", sa.BigInteger(), nullable=False),
sa.Column("created_at", sa.TIMESTAMP(), server_default=sa.func.now(), nullable=False),
sa.Column("updated_at", sa.TIMESTAMP(), server_default=sa.func.now(), nullable=False),
sa.Column("qa_record_id", sa.Integer(), sa.ForeignKey("qa_records.id", ondelete="CASCADE"), nullable=False),
sa.Column("project_id", sa.BigInteger(), nullable=True),
sa.Column("question_text", sa.Text(), nullable=False),
sa.Column("route_mode_predicted", sa.String(length=32), nullable=False),
sa.Column("route_mode_effective", sa.String(length=32), nullable=False),
sa.Column("decision_origin", sa.String(length=64), nullable=True),
sa.Column("reply_contract", _json_type(bind), nullable=True),
sa.Column("probe_snapshot", _json_type(bind), nullable=True),
sa.Column("query_profile", _json_type(bind), nullable=True),
sa.Column("feedback_outcome", _json_type(bind), nullable=True),
sa.Column("question_embedding", _json_type(bind), nullable=True),
sa.UniqueConstraint(
"tenant_id",
"qa_record_id",
name="uq_query_route_memories_tenant_qa_record",
),
)
op.create_index("ix_query_route_memories_tenant_id", "query_route_memories", ["tenant_id"], unique=False)
op.create_index("ix_query_route_memories_qa_record_id", "query_route_memories", ["qa_record_id"], unique=False)
op.create_index("ix_query_route_memories_project_id", "query_route_memories", ["project_id"], unique=False)
op.create_index(
"ix_query_route_memories_route_mode_predicted",
"query_route_memories",
["route_mode_predicted"],
unique=False,
)
op.create_index(
"ix_query_route_memories_route_mode_effective",
"query_route_memories",
["route_mode_effective"],
unique=False,
)
op.create_index(
"ix_query_route_memories_decision_origin",
"query_route_memories",
["decision_origin"],
unique=False,
)
def downgrade() -> None:
bind = op.get_bind()
if not _table_exists(bind, "query_route_memories"):
return
for index_name in (
"ix_query_route_memories_decision_origin",
"ix_query_route_memories_route_mode_effective",
"ix_query_route_memories_route_mode_predicted",
"ix_query_route_memories_project_id",
"ix_query_route_memories_qa_record_id",
"ix_query_route_memories_tenant_id",
):
op.drop_index(index_name, table_name="query_route_memories")
op.drop_table("query_route_memories")

View File

@ -0,0 +1,138 @@
"""add semantic facts
Revision ID: 20260415_add_semantic_facts
Revises: 20260415_add_query_route_memories
Create Date: 2026-04-15
"""
from __future__ import annotations
from alembic import op
import sqlalchemy as sa
revision = "20260415_add_semantic_facts"
down_revision = "20260415_add_query_route_memories"
branch_labels = None
depends_on = None
def _table_exists(bind, table_name: str) -> bool:
return sa.inspect(bind).has_table(table_name)
def upgrade() -> None:
bind = op.get_bind()
if not _table_exists(bind, "semantic_facts"):
op.create_table(
"semantic_facts",
sa.Column("id", sa.Integer(), primary_key=True, autoincrement=True),
sa.Column("tenant_id", sa.BigInteger(), nullable=False),
sa.Column("created_at", sa.TIMESTAMP(), server_default=sa.func.now(), nullable=False),
sa.Column("updated_at", sa.TIMESTAMP(), server_default=sa.func.now(), nullable=False),
sa.Column("entity_type", sa.String(length=32), nullable=False),
sa.Column("entity_id", sa.BigInteger(), nullable=False),
sa.Column("fact_namespace", sa.String(length=64), nullable=False),
sa.Column("fact_key", sa.String(length=64), nullable=False),
sa.Column("fact_value_text", sa.Text(), nullable=False),
sa.Column("fact_value_norm", sa.String(length=255), nullable=False),
sa.Column("value_type", sa.String(length=32), nullable=False),
sa.Column("confidence", sa.Float(), nullable=False, server_default="0"),
sa.Column("evidence_source_type", sa.String(length=32), nullable=True),
sa.Column("evidence_source_id", sa.BigInteger(), nullable=True),
sa.Column("evidence_excerpt", sa.Text(), nullable=True),
sa.Column("extract_version", sa.String(length=64), nullable=True),
sa.Column("freshness_at", sa.String(length=64), nullable=True),
sa.Column("is_active", sa.Boolean(), nullable=False, server_default=sa.true()),
sa.UniqueConstraint(
"tenant_id",
"entity_type",
"entity_id",
"fact_key",
"fact_value_norm",
name="uq_semantic_facts_tenant_entity_key_norm",
),
)
op.create_index("ix_semantic_facts_tenant_id", "semantic_facts", ["tenant_id"], unique=False)
op.create_index("ix_semantic_facts_entity_type", "semantic_facts", ["entity_type"], unique=False)
op.create_index("ix_semantic_facts_entity_id", "semantic_facts", ["entity_id"], unique=False)
op.create_index("ix_semantic_facts_fact_namespace", "semantic_facts", ["fact_namespace"], unique=False)
op.create_index("ix_semantic_facts_fact_key", "semantic_facts", ["fact_key"], unique=False)
op.create_index("ix_semantic_facts_fact_value_norm", "semantic_facts", ["fact_value_norm"], unique=False)
op.create_index("ix_semantic_facts_is_active", "semantic_facts", ["is_active"], unique=False)
if not _table_exists(bind, "semantic_fact_extraction_runs"):
op.create_table(
"semantic_fact_extraction_runs",
sa.Column("id", sa.Integer(), primary_key=True, autoincrement=True),
sa.Column("tenant_id", sa.BigInteger(), nullable=False),
sa.Column("created_at", sa.TIMESTAMP(), server_default=sa.func.now(), nullable=False),
sa.Column("updated_at", sa.TIMESTAMP(), server_default=sa.func.now(), nullable=False),
sa.Column("entity_type", sa.String(length=32), nullable=False),
sa.Column("entity_id", sa.BigInteger(), nullable=False),
sa.Column("trigger_type", sa.String(length=32), nullable=False),
sa.Column("status", sa.String(length=32), nullable=False),
sa.Column("input_fingerprint", sa.String(length=64), nullable=True),
sa.Column("facts_written", sa.Integer(), nullable=False, server_default="0"),
sa.Column("failure_reason", sa.Text(), nullable=True),
sa.Column("extract_version", sa.String(length=64), nullable=True),
sa.Column("started_at", sa.String(length=64), nullable=True),
sa.Column("finished_at", sa.String(length=64), nullable=True),
)
op.create_index(
"ix_semantic_fact_extraction_runs_tenant_id",
"semantic_fact_extraction_runs",
["tenant_id"],
unique=False,
)
op.create_index(
"ix_semantic_fact_extraction_runs_entity_type",
"semantic_fact_extraction_runs",
["entity_type"],
unique=False,
)
op.create_index(
"ix_semantic_fact_extraction_runs_entity_id",
"semantic_fact_extraction_runs",
["entity_id"],
unique=False,
)
op.create_index(
"ix_semantic_fact_extraction_runs_status",
"semantic_fact_extraction_runs",
["status"],
unique=False,
)
op.create_index(
"ix_semantic_fact_extraction_runs_input_fingerprint",
"semantic_fact_extraction_runs",
["input_fingerprint"],
unique=False,
)
def downgrade() -> None:
bind = op.get_bind()
if _table_exists(bind, "semantic_fact_extraction_runs"):
for index_name in (
"ix_semantic_fact_extraction_runs_input_fingerprint",
"ix_semantic_fact_extraction_runs_status",
"ix_semantic_fact_extraction_runs_entity_id",
"ix_semantic_fact_extraction_runs_entity_type",
"ix_semantic_fact_extraction_runs_tenant_id",
):
op.drop_index(index_name, table_name="semantic_fact_extraction_runs")
op.drop_table("semantic_fact_extraction_runs")
if _table_exists(bind, "semantic_facts"):
for index_name in (
"ix_semantic_facts_is_active",
"ix_semantic_facts_fact_value_norm",
"ix_semantic_facts_fact_key",
"ix_semantic_facts_fact_namespace",
"ix_semantic_facts_entity_id",
"ix_semantic_facts_entity_type",
"ix_semantic_facts_tenant_id",
):
op.drop_index(index_name, table_name="semantic_facts")
op.drop_table("semantic_facts")

View File

@ -0,0 +1,4 @@
from . import routes
from .routes import router
__all__ = ["router", "routes"]

View File

@ -0,0 +1,136 @@
from __future__ import annotations
from typing import Any
from fastapi import APIRouter, Depends
from fastapi.responses import JSONResponse
from pydantic import BaseModel, Field
from sqlalchemy.ext.asyncio import AsyncSession
from ...core.database import get_db
from ...core.response import error_response, success_response
from ...dependencies import require_manager_user
from ...models import User
from ...services.mcp_config import McpConfigService
router = APIRouter(prefix="/mcp", tags=["mcp_config"])
class McpServerCreateRequest(BaseModel):
name: str
display_name: str | None = None
template_id: str
params: dict[str, Any] = Field(default_factory=dict)
class McpServerUpdateRequest(BaseModel):
name: str | None = None
display_name: str | None = None
template_id: str | None = None
params: dict[str, Any] | None = None
def _service(db: AsyncSession, current_user: User) -> McpConfigService:
return McpConfigService(db, tenant_id=int(current_user.tenant_id))
@router.get("/admin/catalog")
async def get_mcp_catalog(
db: AsyncSession = Depends(get_db),
current_user: User = Depends(require_manager_user),
):
return success_response(await _service(db, current_user).get_catalog())
@router.get("/admin/servers")
async def list_mcp_servers(
db: AsyncSession = Depends(get_db),
current_user: User = Depends(require_manager_user),
):
items = await _service(db, current_user).list_servers()
return success_response({"items": items})
@router.post("/admin/servers")
async def create_mcp_server(
body: McpServerCreateRequest,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(require_manager_user),
):
service = _service(db, current_user)
try:
item = await service.create_server(
name=body.name,
display_name=body.display_name,
template_id=body.template_id,
params=body.params,
)
except ValueError as exc:
await db.rollback()
return JSONResponse(status_code=400, content=error_response(400, str(exc)))
await db.commit()
return success_response(item)
@router.put("/admin/servers/{server_id}")
async def update_mcp_server(
server_id: str,
body: McpServerUpdateRequest,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(require_manager_user),
):
service = _service(db, current_user)
try:
item = await service.update_server(
server_id=server_id,
name=body.name,
display_name=body.display_name,
template_id=body.template_id,
params=body.params,
)
except ValueError as exc:
await db.rollback()
return JSONResponse(status_code=400, content=error_response(400, str(exc)))
except LookupError as exc:
await db.rollback()
return JSONResponse(status_code=404, content=error_response(404, str(exc)))
await db.commit()
return success_response(item)
@router.post("/admin/servers/{server_id}/validate")
async def validate_mcp_server(
server_id: str,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(require_manager_user),
):
service = _service(db, current_user)
try:
payload = await service.validate_server(server_id=server_id)
except ValueError as exc:
await db.rollback()
return JSONResponse(status_code=400, content=error_response(400, str(exc)))
except LookupError as exc:
await db.rollback()
return JSONResponse(status_code=404, content=error_response(404, str(exc)))
await db.commit()
return success_response(payload)
@router.post("/admin/servers/{server_id}/confirm-client-apply")
async def confirm_mcp_server_client_apply(
server_id: str,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(require_manager_user),
):
service = _service(db, current_user)
try:
payload = await service.confirm_client_apply(server_id=server_id)
except ValueError as exc:
await db.rollback()
return JSONResponse(status_code=400, content=error_response(400, str(exc)))
except LookupError as exc:
await db.rollback()
return JSONResponse(status_code=404, content=error_response(404, str(exc)))
await db.commit()
return success_response(payload)

View File

@ -36,7 +36,9 @@ from .operation_catalog import router as operation_catalog_router
from .ai_pmo_api import router as ai_pmo_router from .ai_pmo_api import router as ai_pmo_router
from .audit import router as audit_router from .audit import router as audit_router
from .model_routing import router as model_routing_router from .model_routing import router as model_routing_router
from .semantic_facts_api import router as semantic_facts_router
from .task_briefs import router as task_briefs_router from .task_briefs import router as task_briefs_router
from .mcp_api import router as mcp_router
# 创建主 API 路由器 # 创建主 API 路由器
api_router = APIRouter() api_router = APIRouter()
@ -72,4 +74,6 @@ api_router.include_router(operation_catalog_router)
api_router.include_router(ai_pmo_router) api_router.include_router(ai_pmo_router)
api_router.include_router(audit_router) api_router.include_router(audit_router)
api_router.include_router(model_routing_router) api_router.include_router(model_routing_router)
api_router.include_router(semantic_facts_router)
api_router.include_router(task_briefs_router) api_router.include_router(task_briefs_router)
api_router.include_router(mcp_router)

View File

@ -0,0 +1,4 @@
from . import routes
from .routes import router
__all__ = ["router", "routes"]

View File

@ -0,0 +1,104 @@
from __future__ import annotations
from typing import Optional
from fastapi import APIRouter, Depends, Query
from fastapi.responses import JSONResponse
from pydantic import BaseModel, Field
from sqlalchemy.ext.asyncio import AsyncSession
from ...core.database import get_db
from ...core.response import error_response, success_response
from ...dependencies import require_manager_user
from ...models import User
from ...services.semantic_fact_service import SemanticFactService
router = APIRouter(prefix="/semantic-facts", tags=["semantic_facts"])
class SemanticFactsBackfillRequest(BaseModel):
entity_types: Optional[list[str]] = Field(default=None)
only_missing: bool = Field(default=True)
limit_per_type: int = Field(default=200, ge=1, le=1000)
@router.get("/admin/overview")
async def get_semantic_facts_overview(
db: AsyncSession = Depends(get_db),
current_user: User = Depends(require_manager_user),
):
service = SemanticFactService(db, tenant_id=int(current_user.tenant_id))
return success_response(await service.get_admin_overview())
@router.get("/admin/objects")
async def list_semantic_fact_objects(
entity_type: str | None = Query(default=None),
q: str | None = Query(default=None),
status: str | None = Query(default=None),
db: AsyncSession = Depends(get_db),
current_user: User = Depends(require_manager_user),
):
service = SemanticFactService(db, tenant_id=int(current_user.tenant_id))
items = await service.list_admin_objects(
entity_types=[entity_type] if entity_type else None,
q=q,
status=status,
)
return success_response({"items": items})
@router.get("/admin/objects/{entity_type}/{entity_id}")
async def get_semantic_fact_object_detail(
entity_type: str,
entity_id: int,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(require_manager_user),
):
service = SemanticFactService(db, tenant_id=int(current_user.tenant_id))
try:
detail = await service.get_admin_detail(entity_type=entity_type, entity_id=int(entity_id))
except ValueError as exc:
return JSONResponse(status_code=404, content=error_response(404, str(exc)))
return success_response(detail)
@router.post("/admin/objects/{entity_type}/{entity_id}/reextract")
async def reextract_semantic_fact_object(
entity_type: str,
entity_id: int,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(require_manager_user),
):
service = SemanticFactService(db, tenant_id=int(current_user.tenant_id))
row = await service.enqueue_entity_event(
entity_type=entity_type,
entity_id=int(entity_id),
operation="extract",
trigger_type="manual_retry",
)
await db.commit()
return success_response(
{
"queued": True,
"event_id": int(row.id),
"entity_type": str(entity_type),
"entity_id": int(entity_id),
}
)
@router.post("/admin/backfill")
async def enqueue_semantic_fact_backfill(
body: SemanticFactsBackfillRequest,
db: AsyncSession = Depends(get_db),
current_user: User = Depends(require_manager_user),
):
service = SemanticFactService(db, tenant_id=int(current_user.tenant_id))
result = await service.enqueue_backfill(
entity_types=body.entity_types,
only_missing=bool(body.only_missing),
limit_per_type=int(body.limit_per_type),
)
await db.commit()
return success_response(result)

View File

@ -165,23 +165,31 @@ PROMPT_RAG_QUERY_SEMANTICS = """
你的职责不是靠关键词仓库猜,而是基于当前消息表达的目的、对象、作用域和 bundle 上下文,输出: 你的职责不是靠关键词仓库猜,而是基于当前消息表达的目的、对象、作用域和 bundle 上下文,输出:
- conversation intent(question / statement / mixed) - conversation intent(question / statement / mixed)
- query_type(count / list / status / explain / unknown) - query_type(existence / count / list / status / explain / diagnose / compare / unknown)
- object_mentions(用户当前想解析的对象提及,保留原始业务名词短语)
- retrieval_domains(db / graph / doc / feishu / pmflow / url) - retrieval_domains(db / graph / doc / feishu / pmflow / url)
- subject_focus(people / project / task / signal / charter / workload / feature / unknown) - subject_focus(people / project / task / signal / charter / workload / feature / unknown)
- scope_hint(none / global / project) - scope_hint(none / global / project)
- feature_focus(none / purpose / entry / catalog) - feature_focus(none / purpose / entry / catalog)
- query_profile(needs_private_context / needs_freshness / referenced_objects)
硬约束: 硬约束:
- 不要复述 workflow bundle 的内部字段名给用户。 - 不要复述 workflow bundle 的内部字段名给用户。
- 如果 bundle 已经明确显示这是 query_flow,优先按 query 处理;只有消息本身同时包含明确执行动作时才输出 mixed。 - 如果 bundle 已经明确显示这是 query_flow,优先按 query 处理;只有消息本身同时包含明确执行动作时才输出 mixed。
- query_type 只用于 Adaptive RAG 的检索与快路径规划,不代表最终回复场景。 - query_type 只用于 Adaptive RAG 的检索与快路径规划,不代表最终回复场景。
- query_type 取值只能是:count、list、status、explain、unknown。 - query_type 取值只能是:existence、count、list、status、explain、diagnose、compare、unknown。
- retrieval_domains 只能从:db、graph、doc、feishu、pmflow、url 中选择。 - retrieval_domains 只能从:db、graph、doc、feishu、pmflow、url 中选择。
- subject_focus 只能是:people、project、task、signal、charter、workload、feature、unknown。 - subject_focus 只能是:people、project、task、signal、charter、workload、feature、unknown。
- scope_hint 只能是:none、global、project。 - scope_hint 只能是:none、global、project。
- feature_focus 只能是:none、purpose、entry、catalog。 - feature_focus 只能是:none、purpose、entry、catalog。
- query_profile.referenced_objects 只能从:unknown、system_config、model_routing、project_manifest、feature_catalog、project_object、internal_document、person_object、public_model、public_company、weather、market_data、sports_event、news_topic 中选择。
- 如果问题是在问人员、成员、负责人清单或人数,subject_focus 应为 people。 - 如果问题是在问人员、成员、负责人清单或人数,subject_focus 应为 people。
- 如果问题是在问功能做什么、入口在哪、模块如何进入,subject_focus 应为 feature,并结合 feature_focus 标明是 purpose / entry / catalog。 - 如果问题是在问功能做什么、入口在哪、模块如何进入,subject_focus 应为 feature,并结合 feature_focus 标明是 purpose / entry / catalog。
- 如果问题是在问“有没有 / 是否存在 / 系统里有无”这类内部对象存在性,query_type 应为 existence;不要因为对象名里出现“系统 / 平台 / 助手”就改判成 explain。像 `帮我看看系统里面有没有开票系统。`,应输出 `query_type=existence`,并把 `object_mentions=["开票系统"]`。
- 外部公开事实示例:`当前 codex 最新的模型用的是什么?`
query_profile 应体现 `needs_freshness=true`、`referenced_objects=["public_model"]`,retrieval_domains 应优先为 ["url"],不要因为出现“模型”二字就改判成系统内部配置问题。
- 内部运行时配置示例:`我们系统当前接的是哪个 codex 模型配置?`
query_profile 应体现 `needs_private_context=true`、`referenced_objects=["system_config","model_routing"]`,retrieval_domains 应优先为 ["db","pmflow"],这是在问系统配置而不是公开外部事实。
- 若证据不足,query_type 输出 unknown,retrieval_domains 选择保守但完整的查询范围。 - 若证据不足,query_type 输出 unknown,retrieval_domains 选择保守但完整的查询范围。
- 输出必须是一个 JSON 对象,不要输出 Markdown。 - 输出必须是一个 JSON 对象,不要输出 Markdown。
@ -190,11 +198,18 @@ JSON schema:
"intent": "question | statement | mixed", "intent": "question | statement | mixed",
"question_part": "string", "question_part": "string",
"statement_part": "string", "statement_part": "string",
"query_type": "count | list | status | explain | unknown", "query_type": "existence | count | list | status | explain | diagnose | compare | unknown",
"object_mentions": ["string"],
"retrieval_domains": ["db | graph | doc | feishu | pmflow | url"], "retrieval_domains": ["db | graph | doc | feishu | pmflow | url"],
"subject_focus": "people | project | task | signal | charter | workload | feature | unknown", "subject_focus": "people | project | task | signal | charter | workload | feature | unknown",
"scope_hint": "none | global | project", "scope_hint": "none | global | project",
"feature_focus": "none | purpose | entry | catalog", "feature_focus": "none | purpose | entry | catalog",
"query_profile": {{
"needs_private_context": false,
"needs_freshness": false,
"referenced_objects": ["unknown"],
"confidence": 0.0
}},
"confidence": 0.0 "confidence": 0.0
}} }}
@ -248,6 +263,241 @@ message:
{message} {message}
""".strip() """.strip()
PROMPT_INTERNAL_TARGET_RESOLUTION = """
你是 PMFlow 内部查询链路的 Target Resolver rerank 层。
你的输入已经包含:
1. 用户问题
2. 已经确定的 operation_contract
3. 受控候选对象列表
你的职责只有一个:
- 在给定候选里选出最符合当前问题语义的目标对象
- 如果候选之间仍然无法稳定区分,就要求澄清
硬约束:
- 只能在候选列表里选择,禁止生成新的对象、ID、类型或 target_ref。
- 如果是存在性问题,优先选择“被问到的业务对象”,不要把对象名里带“系统/平台/助手”的词面自动改判成 feature explain。
- 若前两名候选分差很小、证据不足或候选语义冲突,输出 `clarify=true`。
- 输出必须是 JSON 对象,不要输出 Markdown。
JSON schema:
{{
"selected_index": 0,
"clarify": false,
"confidence": 0.0,
"reason": "string"
}}
question:
{question}
operation_contract:
{operation_contract}
candidates:
{candidates}
""".strip()
PROMPT_EXTERNAL_QUERY_PLAN = """
你是 PMFlow External Query Planner。
你的职责不是直接回答外部事实,而是把当前问题整理成一个受控的外部查询计划。
硬约束:
- 只输出 JSON 对象,不要输出 Markdown。
- 不要直接给外部事实结论。
- `query_kind` 只能是:weather、news、company、model、market、generic_web、unknown。
- `answer_contract` 只能是:direct_fact、summary_with_sources、compare_with_sources、clarify。
- `search_queries` 必须是简短、可直接执行的查询短语。
- `preferred_domains` 只放域名,不要带协议头。
- 若用户问题明显要求最新、当前、今天、最近等外部信息,`freshness_required=true`。
- 若问题更像对比/比较,`requires_multi_source=true` 且 `answer_contract=compare_with_sources`。
- 结构化失败时优先保守,不要编造实体。
JSON schema:
{{
"plan_version": "v1",
"query_kind": "weather | news | company | model | market | generic_web | unknown",
"answer_contract": "direct_fact | summary_with_sources | compare_with_sources | clarify",
"freshness_required": false,
"topic": "string",
"subject_entities": ["string"],
"location_hint": "string | null",
"time_range_hint": "string | null",
"preferred_domains": ["string"],
"search_queries": ["string"],
"result_limit": 3,
"requires_multi_source": false,
"clarify_on_ambiguity": false
}}
question:
{question}
query_semantics:
{query_semantics}
fallback_queries:
{fallback_queries}
""".strip()
def _build_semantic_fact_extract_prompt(
*,
entity_label: str,
goal_text: str,
allowed_examples: list[str],
payload_key: str,
) -> str:
examples_text = "\n".join(f"- `{item}`" for item in allowed_examples)
payload_marker = f"<<{payload_key}>>"
return f"""
你是 PMFlow 的{entity_label}语义事实抽取器。
目标:
- {goal_text}
- 只输出有证据支撑的事实,不要脑补
- 如果没有明确事实,返回空数组
允许抽取的事实示例:
{examples_text}
输出要求:
- 只输出 JSON 对象
- `facts` 必须是数组
- 每个 fact 必须包含:
- `fact_namespace`
- `fact_key`
- `fact_value_text`
- `fact_value_norm`
- `value_type`
- `confidence`
- `evidence_excerpt`
JSON schema:
{{
"facts": [
{{
"fact_namespace": "domain | alias | purpose | audience | status | product | role",
"fact_key": "string",
"fact_value_text": "string",
"fact_value_norm": "string",
"value_type": "keyword | text",
"confidence": 0.0,
"evidence_excerpt": "string"
}}
]
}}
{payload_key}:
{payload_marker}
""".strip()
PROMPT_SEMANTIC_FACT_EXTRACT_PROJECT = _build_semantic_fact_extract_prompt(
entity_label="项目",
goal_text="从项目名称、标题、描述里抽取稳定、可复用、适合内部查询的语义事实",
allowed_examples=[
"market_theme:A股、港股、美股",
"business_domain:开票、财务、采购、投研",
"aliases:开票系统、A股项目、库存报表工具",
"audience:投顾团队、研究团队、财务运营",
"product_form:助手、平台、报表工具",
],
payload_key="project_payload",
)
PROMPT_SEMANTIC_FACT_EXTRACT_TASK = _build_semantic_fact_extract_prompt(
entity_label="任务",
goal_text="从任务标题、描述、状态、优先级里抽取任务主题、对象别名、交付目标和责任线索",
allowed_examples=[
"deliverable:开票接口、财务报表页、信号看板",
"business_domain:开票、财务、采购、风控",
"aliases:开票任务、A股需求任务",
"owner_role:后端、前端、AI",
"status_hint:待办、进行中、已完成",
],
payload_key="entity_payload",
)
PROMPT_SEMANTIC_FACT_EXTRACT_SIGNAL = _build_semantic_fact_extract_prompt(
entity_label="信号",
goal_text="从信号标题、摘要、正文、状态里抽取业务主题、问题对象、影响域和别名",
allowed_examples=[
"business_domain:开票、结算、投研、采购",
"problem_object:开票系统、报表平台、A股项目",
"aliases:开票需求、发票流程问题",
"signal_type:bug、需求、机会",
"status_hint:待处理、已关闭",
],
payload_key="entity_payload",
)
PROMPT_SEMANTIC_FACT_EXTRACT_DOCUMENT = _build_semantic_fact_extract_prompt(
entity_label="文档",
goal_text="从文档标题、来源、内容预览里抽取文档主题、业务对象、别名和适用团队",
allowed_examples=[
"business_domain:开票、财务、采购、投研",
"document_topic:实施方案、接口文档、复盘总结",
"aliases:开票系统文档、A股项目说明",
"audience:财务团队、研发团队、运营团队",
"source_form:文件、网页、知识库条目",
],
payload_key="entity_payload",
)
PROMPT_SEMANTIC_FACT_EXTRACT_USER = _build_semantic_fact_extract_prompt(
entity_label="人员",
goal_text="从成员姓名、角色、领域、简介里抽取稳定的职责、领域、别名和角色事实",
allowed_examples=[
"role_scope:后端、前端、AI、管理员",
"business_domain:财务、采购、投研",
"aliases:开票负责人、A股项目 owner",
"audience:财务团队、研发团队",
"status_hint:在岗、离线",
],
payload_key="entity_payload",
)
PROMPT_SEMANTIC_FACT_EXTRACT_REQUIREMENT = _build_semantic_fact_extract_prompt(
entity_label="需求宪章",
goal_text="从需求宪章标题、状态和内容里抽取需求主题、业务对象、别名和目标团队",
allowed_examples=[
"business_domain:开票、财务、采购、投研",
"target_object:开票系统、报表平台、A股项目",
"aliases:开票需求、开票平台需求",
"audience:财务团队、研究团队",
"status_hint:草稿、对齐中、已对齐",
],
payload_key="entity_payload",
)
PROMPT_SEMANTIC_FACT_EXTRACT_RELEASE = _build_semantic_fact_extract_prompt(
entity_label="发布",
goal_text="从发布标题、状态和内容里抽取发布对象、业务域、别名和阶段状态",
allowed_examples=[
"business_domain:开票、财务、采购、投研",
"release_object:开票系统、结算平台、A股工具",
"aliases:开票版本、财务发布",
"status_hint:pending、verified、online",
"audience:财务团队、运营团队",
],
payload_key="entity_payload",
)
PROMPT_SEMANTIC_FACT_EXTRACT_REVIEW = _build_semantic_fact_extract_prompt(
entity_label="复盘",
goal_text="从复盘标题、状态和内容里抽取复盘对象、业务主题、别名和适用团队",
allowed_examples=[
"business_domain:开票、财务、采购、投研",
"review_object:开票系统、A股项目、财务流程",
"aliases:开票复盘、阶段复盘",
"status_hint:draft、archived",
"audience:财务团队、研发团队",
],
payload_key="entity_payload",
)
PROMPT_RAG_ANSWER_SCENE_SEMANTICS = """ PROMPT_RAG_ANSWER_SCENE_SEMANTICS = """
你是 PMFlow Adaptive RAG 的 answer scene 规划层。 你是 PMFlow Adaptive RAG 的 answer scene 规划层。
@ -584,6 +834,8 @@ __all__ = [
"PMFLOW_SEMANTIC_CONTEXT_DECISION_PROMPT", "PMFLOW_SEMANTIC_CONTEXT_DECISION_PROMPT",
"PROMPT_RAG_QUERY_SEMANTICS", "PROMPT_RAG_QUERY_SEMANTICS",
"PROMPT_RAG_FOLLOWUP_ANCHOR_SEMANTICS", "PROMPT_RAG_FOLLOWUP_ANCHOR_SEMANTICS",
"PROMPT_INTERNAL_TARGET_RESOLUTION",
"PROMPT_EXTERNAL_QUERY_PLAN",
"PROMPT_RAG_ANSWER_SCENE_SEMANTICS", "PROMPT_RAG_ANSWER_SCENE_SEMANTICS",
"PMFLOW_REQUIREMENT_PLAN_ROUND_PROMPT", "PMFLOW_REQUIREMENT_PLAN_ROUND_PROMPT",
"PMFLOW_REQUIREMENT_PLAN_DRAFT_PROMPT", "PMFLOW_REQUIREMENT_PLAN_DRAFT_PROMPT",

View File

@ -31,10 +31,12 @@ from .services.scheduler_center_core import (
WorkloadRecalcScheduler, WorkloadRecalcScheduler,
) )
from .services.skill import init_system_skill_tags from .services.skill import init_system_skill_tags
from .services.semantic_fact_service import register_semantic_fact_hooks
configure_logging(settings.debug) configure_logging(settings.debug)
logger = get_logger(__name__) logger = get_logger(__name__)
register_context_cache_invalidation_hooks() register_context_cache_invalidation_hooks()
register_semantic_fact_hooks()
@asynccontextmanager @asynccontextmanager

View File

@ -45,6 +45,9 @@ from .knowledge_base_doc import KnowledgeBaseDoc
from .knowledge_base_chunk import KnowledgeBaseChunk from .knowledge_base_chunk import KnowledgeBaseChunk
from .qa_record import QARecord from .qa_record import QARecord
from .qa_feedback import QAFeedback from .qa_feedback import QAFeedback
from .query_route_memory import QueryRouteMemory
from .semantic_fact import SemanticFact
from .semantic_fact_extraction_run import SemanticFactExtractionRun
from .feedback_center import FeedbackCenterRecord, FeedbackCenterRecordRevision from .feedback_center import FeedbackCenterRecord, FeedbackCenterRecordRevision
from .reply_iteration import QualityAsset, QualityAssetPair, JudgeRun, OptimizerCandidate, PromptRelease from .reply_iteration import QualityAsset, QualityAssetPair, JudgeRun, OptimizerCandidate, PromptRelease
from .training_pipeline import ( from .training_pipeline import (
@ -130,6 +133,9 @@ __all__ = [
"KnowledgeBaseChunk", "KnowledgeBaseChunk",
"QARecord", "QARecord",
"QAFeedback", "QAFeedback",
"QueryRouteMemory",
"SemanticFact",
"SemanticFactExtractionRun",
"FeedbackCenterRecord", "FeedbackCenterRecord",
"FeedbackCenterRecordRevision", "FeedbackCenterRecordRevision",
"QualityAsset", "QualityAsset",

View File

@ -0,0 +1,44 @@
"""问答路由记忆样本。"""
from __future__ import annotations
from typing import Any, Optional
from sqlalchemy import BigInteger, ForeignKey, Integer, Text, UniqueConstraint, VARCHAR
from sqlalchemy.orm import Mapped, mapped_column
from .base import Base
from .types import JSONB
class QueryRouteMemory(Base):
__tablename__ = "query_route_memories"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
qa_record_id: Mapped[int] = mapped_column(
Integer,
ForeignKey("qa_records.id", ondelete="CASCADE"),
nullable=False,
index=True,
)
project_id: Mapped[Optional[int]] = mapped_column(BigInteger, nullable=True, index=True)
question_text: Mapped[str] = mapped_column(Text, nullable=False)
route_mode_predicted: Mapped[str] = mapped_column(VARCHAR(32), nullable=False, index=True)
route_mode_effective: Mapped[str] = mapped_column(VARCHAR(32), nullable=False, index=True)
decision_origin: Mapped[Optional[str]] = mapped_column(VARCHAR(64), nullable=True, index=True)
reply_contract: Mapped[Optional[Any]] = mapped_column(JSONB, nullable=True)
probe_snapshot: Mapped[Optional[Any]] = mapped_column(JSONB, nullable=True)
query_profile: Mapped[Optional[Any]] = mapped_column(JSONB, nullable=True)
feedback_outcome: Mapped[Optional[Any]] = mapped_column(JSONB, nullable=True)
question_embedding: Mapped[Optional[Any]] = mapped_column(JSONB, nullable=True)
__table_args__ = (
UniqueConstraint(
"tenant_id",
"qa_record_id",
name="uq_query_route_memories_tenant_qa_record",
),
)
__all__ = ["QueryRouteMemory"]

View File

@ -0,0 +1,44 @@
"""语义事实读模型。"""
from __future__ import annotations
from typing import Optional
from sqlalchemy import BigInteger, Boolean, Float, Integer, Text, UniqueConstraint, VARCHAR
from sqlalchemy.orm import Mapped, mapped_column
from .base import Base
class SemanticFact(Base):
__tablename__ = "semantic_facts"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
entity_type: Mapped[str] = mapped_column(VARCHAR(32), nullable=False, index=True)
entity_id: Mapped[int] = mapped_column(BigInteger, nullable=False, index=True)
fact_namespace: Mapped[str] = mapped_column(VARCHAR(64), nullable=False, index=True)
fact_key: Mapped[str] = mapped_column(VARCHAR(64), nullable=False, index=True)
fact_value_text: Mapped[str] = mapped_column(Text, nullable=False)
fact_value_norm: Mapped[str] = mapped_column(VARCHAR(255), nullable=False, index=True)
value_type: Mapped[str] = mapped_column(VARCHAR(32), nullable=False, default="keyword")
confidence: Mapped[float] = mapped_column(Float, nullable=False, default=0.0)
evidence_source_type: Mapped[Optional[str]] = mapped_column(VARCHAR(32), nullable=True)
evidence_source_id: Mapped[Optional[int]] = mapped_column(BigInteger, nullable=True)
evidence_excerpt: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
extract_version: Mapped[Optional[str]] = mapped_column(VARCHAR(64), nullable=True)
freshness_at: Mapped[Optional[str]] = mapped_column(VARCHAR(64), nullable=True)
is_active: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, index=True)
__table_args__ = (
UniqueConstraint(
"tenant_id",
"entity_type",
"entity_id",
"fact_key",
"fact_value_norm",
name="uq_semantic_facts_tenant_entity_key_norm",
),
)
__all__ = ["SemanticFact"]

View File

@ -0,0 +1,29 @@
"""语义事实抽取运行记录。"""
from __future__ import annotations
from typing import Optional
from sqlalchemy import BigInteger, Integer, Text, VARCHAR
from sqlalchemy.orm import Mapped, mapped_column
from .base import Base
class SemanticFactExtractionRun(Base):
__tablename__ = "semantic_fact_extraction_runs"
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
entity_type: Mapped[str] = mapped_column(VARCHAR(32), nullable=False, index=True)
entity_id: Mapped[int] = mapped_column(BigInteger, nullable=False, index=True)
trigger_type: Mapped[str] = mapped_column(VARCHAR(32), nullable=False, default="manual_retry")
status: Mapped[str] = mapped_column(VARCHAR(32), nullable=False, default="queued", index=True)
input_fingerprint: Mapped[Optional[str]] = mapped_column(VARCHAR(64), nullable=True, index=True)
facts_written: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
failure_reason: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
extract_version: Mapped[Optional[str]] = mapped_column(VARCHAR(64), nullable=True)
started_at: Mapped[Optional[str]] = mapped_column(VARCHAR(64), nullable=True)
finished_at: Mapped[Optional[str]] = mapped_column(VARCHAR(64), nullable=True)
__all__ = ["SemanticFactExtractionRun"]

View File

@ -68,6 +68,9 @@ class AdaptiveRAGAnswerEntryMixin:
message_intent: Optional[str] = None, message_intent: Optional[str] = None,
mixed_signal_part: Optional[str] = None, mixed_signal_part: Optional[str] = None,
consultant_mode_preference: Optional[str] = None, consultant_mode_preference: Optional[str] = None,
source_policy: Optional[str] = None,
precomputed_query_semantics: Optional[dict[str, Any]] = None,
route_decision: Optional[dict[str, Any]] = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
started = time.perf_counter() started = time.perf_counter()
clean_q = str(question or "").strip() clean_q = str(question or "").strip()
@ -108,6 +111,9 @@ class AdaptiveRAGAnswerEntryMixin:
action_mode=action_mode, action_mode=action_mode,
source_sender_open_id=source_sender_open_id, source_sender_open_id=source_sender_open_id,
thread_context=thread_context, thread_context=thread_context,
source_policy=source_policy,
precomputed_query_semantics=precomputed_query_semantics,
route_decision=route_decision,
) )
clean_q = str(runtime.get("resolved_question") or clean_q).strip() or clean_q clean_q = str(runtime.get("resolved_question") or clean_q).strip() or clean_q
session_id = runtime["session_id"] session_id = runtime["session_id"]
@ -121,6 +127,7 @@ class AdaptiveRAGAnswerEntryMixin:
query_semantics = runtime["query_semantics"] if isinstance(runtime.get("query_semantics"), dict) else {} query_semantics = runtime["query_semantics"] if isinstance(runtime.get("query_semantics"), dict) else {}
retrieval_domains = list(runtime.get("retrieval_domains") or []) retrieval_domains = list(runtime.get("retrieval_domains") or [])
scene_semantics = runtime["scene_semantics"] if isinstance(runtime.get("scene_semantics"), dict) else {} scene_semantics = runtime["scene_semantics"] if isinstance(runtime.get("scene_semantics"), dict) else {}
route_decision = runtime["route_decision"] if isinstance(runtime.get("route_decision"), dict) else {}
runtime_message_intent = str(runtime.get("message_intent") or "").strip().lower() or None runtime_message_intent = str(runtime.get("message_intent") or "").strip().lower() or None
effective_message_intent = str(message_intent or runtime_message_intent or "").strip().lower() or None effective_message_intent = str(message_intent or runtime_message_intent or "").strip().lower() or None
enforce_scene_prompt_only = self._should_enforce_scene_prompt_only( enforce_scene_prompt_only = self._should_enforce_scene_prompt_only(
@ -181,4 +188,5 @@ class AdaptiveRAGAnswerEntryMixin:
mixed_signal_part=mixed_signal_part, mixed_signal_part=mixed_signal_part,
anchor_resolution=anchor_resolution, anchor_resolution=anchor_resolution,
consultant_mode_preference=consultant_mode_preference, consultant_mode_preference=consultant_mode_preference,
route_decision=route_decision,
) )

View File

@ -5,80 +5,150 @@ from .shared import *
class AdaptiveRAGAnswerMainlineMixin: class AdaptiveRAGAnswerMainlineMixin:
_EXTERNAL_FACT_FRESHNESS_MARKERS = ( @staticmethod
"最新", def _internal_answer_contract_for_query_type(query_type: str) -> str:
"目前", normalized = str(query_type or QUERY_TYPE_UNKNOWN).strip().lower()
"当前", if normalized == QUERY_TYPE_EXISTENCE:
"截至现在", return "yes_no_with_evidence"
"截至当前", if normalized == QUERY_TYPE_COUNT:
"截止当前", return "count_scalar"
"今天", if normalized == QUERY_TYPE_LIST:
"现在", return "entity_list"
"最近", if normalized == QUERY_TYPE_STATUS:
"出到什么", return "entity_snapshot"
"发布了吗", if normalized == QUERY_TYPE_EXPLAIN:
"发布到", return "explanation_with_citations"
"版本", if normalized == QUERY_TYPE_DIAGNOSE:
"最高模型", return "diagnosis_with_evidence"
) if normalized == QUERY_TYPE_COMPARE:
_MODEL_ENTITY_MARKERS = ( return "comparison_with_basis"
"codex", return "clarify_or_fail"
"gpt",
"chatgpt", @staticmethod
"claude", def _internal_entity_type_for_subject_focus(subject_focus: str) -> str:
"deepseek", normalized = str(subject_focus or "").strip().lower()
"qwen", if normalized == "people":
"gemini", return "user"
"copilot", if normalized in {"project", "task", "signal", "feature"}:
"grok", return normalized
"kimi", return "unknown"
"llama",
"openai", def _build_internal_query_plan(
"anthropic", self,
) *,
_INTERNAL_MODEL_CONFIG_MARKERS = ( question: str,
"ai-config", project_id: Optional[int],
"/ai-config", query_semantics: Optional[dict[str, Any]],
"模型配置", operation_contract: Optional[dict[str, Any]] = None,
"配置项", target_resolution: Optional[dict[str, Any]] = None,
"配置值", ) -> Optional[dict[str, Any]]:
"路由规则", semantics = query_semantics if isinstance(query_semantics, dict) else {}
"初始化参数", operation = operation_contract if isinstance(operation_contract, dict) else (
"llm服务", semantics.get("operation_contract") if isinstance(semantics.get("operation_contract"), dict) else {}
"llm 服务", )
"运行时", target = target_resolution if isinstance(target_resolution, dict) else (
"当前系统", semantics.get("target_resolution") if isinstance(semantics.get("target_resolution"), dict) else {}
"我们系统", )
"pmflow", source_policy = str(
"本系统", semantics.get("effective_source_policy") or semantics.get("source_policy") or ""
"接入的模型", ).strip().lower()
"接的是", if source_policy != "internal":
"用的是哪个模型配置", return None
)
_PROJECT_CONTEXT_POSITIVE_MARKERS = ( query_type = str(
"这个项目", operation.get("query_type")
"当前项目", or semantics.get("query_type")
"本项目", or QUERY_TYPE_UNKNOWN
"我们项目", ).strip().lower() or QUERY_TYPE_UNKNOWN
"项目所在行业", subject_focus = self._query_subject_focus(semantics) if hasattr(self, "_query_subject_focus") else "unknown"
"对项目的影响", entity_type = str(
"对这个项目的影响", target.get("target_entity_type")
"对当前项目的影响", or self._internal_entity_type_for_subject_focus(subject_focus)
"对我们项目的影响", ).strip().lower() or "unknown"
"结合项目", scope_hint = self._query_scope_hint(semantics) if hasattr(self, "_query_scope_hint") else "none"
"结合当前项目", target_ref = str(target.get("target_ref") or "").strip() or None
"落到项目",
) filters: list[dict[str, Any]] = []
_PROJECT_CONTEXT_NEGATIVE_MARKERS = ( if entity_type == "project" and target_ref and target_ref.startswith("project:"):
"不要提项目", try:
"别提项目", resolved_project_id = int(target_ref.split(":", 1)[1])
"不要结合项目", except Exception:
"不用结合项目", resolved_project_id = None
"不需要结合项目", if resolved_project_id is not None:
"先别结合项目", filters.append(
"不要绑定项目", {
"别绑定项目", "field": "project_id",
) "operator": "equals",
"value": resolved_project_id,
"value_source": "target_resolution",
}
)
elif entity_type == "project":
mention_collector = getattr(self, "_collect_lookup_terms", None)
if callable(mention_collector):
operation_mentions = mention_collector(
question=question,
query_semantics=semantics,
operation_contract=operation,
target_resolution=target,
limit=3,
)
else:
operation_mentions = [
str(item or "").strip()
for item in list(operation.get("object_mentions") or [])
if str(item or "").strip()
]
if operation_mentions:
for raw in operation_mentions[:3]:
filters.append(
{
"field": "semantic_label",
"operator": "equals_or_alias",
"value": raw,
"value_source": "operation_contract",
}
)
elif entity_type == "feature" and target_ref:
filters.append(
{
"field": "feature_ref",
"operator": "equals",
"value": target_ref,
"value_source": "target_resolution",
}
)
elif entity_type == "config" and target_ref:
filters.append(
{
"field": "config_ref",
"operator": "equals",
"value": target_ref,
"value_source": "target_resolution",
}
)
return {
"plan_version": "v2" if operation or target else "v1",
"query_type": query_type,
"entity_type": entity_type,
"answer_contract": self._internal_answer_contract_for_query_type(query_type),
"target_ref": target_ref,
"filters": filters,
"scope": {
"tenant_bound": True,
"project_id": int(project_id) if project_id is not None else None,
"scope_hint": scope_hint,
},
"retrieval_policy": {
"facts_first": True,
"allow_fallback_text": True,
},
"verifier_checks": [
"contract_matches_query_type",
"evidence_backed",
],
}
@classmethod @classmethod
def _extract_prompt_variant_from_policy_version(cls, policy_version: Optional[str]) -> Optional[str]: def _extract_prompt_variant_from_policy_version(cls, policy_version: Optional[str]) -> Optional[str]:
@ -93,46 +163,6 @@ class AdaptiveRAGAnswerMainlineMixin:
return value or None return value or None
return None return None
@classmethod
def _question_explicitly_requests_project_context(
cls,
*,
question: str,
project_id: Optional[int],
) -> bool:
if project_id is None:
return False
normalized = re.sub(r"\s+", "", str(question or "")).strip().lower()
if not normalized:
return False
if any(marker in normalized for marker in cls._PROJECT_CONTEXT_NEGATIVE_MARKERS):
return False
return any(marker in normalized for marker in cls._PROJECT_CONTEXT_POSITIVE_MARKERS)
@classmethod
def _looks_like_external_freshness_question(cls, question: str) -> bool:
normalized = re.sub(r"\s+", "", str(question or "")).strip().lower()
if not normalized:
return False
return any(marker in normalized for marker in cls._EXTERNAL_FACT_FRESHNESS_MARKERS)
@classmethod
def _looks_like_internal_model_config_question(cls, question: str) -> bool:
normalized = re.sub(r"\s+", "", str(question or "")).strip().lower()
if not normalized:
return False
return any(marker in normalized for marker in cls._INTERNAL_MODEL_CONFIG_MARKERS)
@classmethod
def _looks_like_model_fact_question(cls, question: str) -> bool:
normalized = re.sub(r"\s+", "", str(question or "")).strip().lower()
if not normalized:
return False
has_entity = any(marker in normalized for marker in cls._MODEL_ENTITY_MARKERS)
if not has_entity:
return False
return "模型" in normalized or cls._looks_like_external_freshness_question(question)
async def _resolve_anchor_prompt_variant( async def _resolve_anchor_prompt_variant(
self, self,
*, *,
@ -161,40 +191,18 @@ class AdaptiveRAGAnswerMainlineMixin:
scene_semantics: Optional[dict[str, Any]], scene_semantics: Optional[dict[str, Any]],
anchor_prompt_variant: Optional[str] = None, anchor_prompt_variant: Optional[str] = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
_ = question
normalized = dict(scene_semantics or {}) normalized = dict(scene_semantics or {})
scene_key = str(normalized.get("scene_key") or "default").strip().lower() or "default" scene_key = str(normalized.get("scene_key") or "default").strip().lower() or "default"
anchor_variant = str(anchor_prompt_variant or "").strip().lower() anchor_variant = str(anchor_prompt_variant or "").strip().lower()
explicit_project_context = self._question_explicitly_requests_project_context( if scene_key in {"ai_pmo", "ai_pmo_correction"} and anchor_variant in {"external_trend", "industry_trend"}:
question=question, scene_key = anchor_variant
project_id=project_id,
)
requires_freshness = self._looks_like_external_freshness_question(question)
internal_model_config_question = self._looks_like_internal_model_config_question(question)
model_fact_question = self._looks_like_model_fact_question(question)
anchored_trend_followup = anchor_variant in {"external_trend", "industry_trend"}
if scene_key == "ai_pmo" and model_fact_question and requires_freshness and not internal_model_config_question:
scene_key = "external_trend" if explicit_project_context else "industry_trend"
normalized["snapshot_focus"] = "none" normalized["snapshot_focus"] = "none"
normalized["deep_intent"] = "none" normalized["deep_intent"] = "none"
normalized["external_fact_chat"] = True if scene_key == "external_trend" and project_id is None:
normalized["requires_freshness"] = True
if scene_key == "ai_pmo_correction" and anchored_trend_followup:
scene_key = "external_trend" if explicit_project_context else "industry_trend"
normalized["snapshot_focus"] = "none"
normalized["deep_intent"] = "none"
normalized["external_fact_chat"] = True
if scene_key == "external_trend" and not explicit_project_context:
scene_key = "industry_trend" scene_key = "industry_trend"
if scene_key in {"external_trend", "industry_trend"}: if scene_key in {"external_trend", "industry_trend"}:
normalized["external_fact_chat"] = True normalized["external_fact_chat"] = True
normalized["requires_freshness"] = bool(requires_freshness)
if requires_freshness:
normalized["deep_intent"] = "none"
normalized["scene_key"] = scene_key normalized["scene_key"] = scene_key
return normalized return normalized
@ -217,7 +225,12 @@ class AdaptiveRAGAnswerMainlineMixin:
normalized_thinking_mode = normalize_thinking_mode(requested_thinking_mode) normalized_thinking_mode = normalize_thinking_mode(requested_thinking_mode)
normalized_query_type = str(query_type or QUERY_TYPE_UNKNOWN).strip().lower() normalized_query_type = str(query_type or QUERY_TYPE_UNKNOWN).strip().lower()
normalized_consultant_mode = str(consultant_mode_preference or "").strip().lower() normalized_consultant_mode = str(consultant_mode_preference or "").strip().lower()
fast_path_candidate = normalized_query_type in {QUERY_TYPE_COUNT, QUERY_TYPE_LIST, QUERY_TYPE_STATUS} fast_path_candidate = normalized_query_type in {
QUERY_TYPE_EXISTENCE,
QUERY_TYPE_COUNT,
QUERY_TYPE_LIST,
QUERY_TYPE_STATUS,
}
normalized_question = str(question or "").strip() normalized_question = str(question or "").strip()
normalized_effective = str(effective_question or "").strip() normalized_effective = str(effective_question or "").strip()
@ -290,6 +303,9 @@ class AdaptiveRAGAnswerMainlineMixin:
source_message_id: Optional[str] = None, source_message_id: Optional[str] = None,
source_sender_open_id: Optional[str] = None, source_sender_open_id: Optional[str] = None,
thread_context: Optional[dict[str, Any]] = None, thread_context: Optional[dict[str, Any]] = None,
source_policy: Optional[str] = None,
precomputed_query_semantics: Optional[dict[str, Any]] = None,
route_decision: Optional[dict[str, Any]] = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
external_conversation_key = self._build_external_conversation_key( external_conversation_key = self._build_external_conversation_key(
asked_via=asked_via, asked_via=asked_via,
@ -358,20 +374,57 @@ class AdaptiveRAGAnswerMainlineMixin:
session_state=state_row, session_state=state_row,
anchor_resolution=anchor_resolution, anchor_resolution=anchor_resolution,
) )
query_semantics = await self._resolve_query_semantics( query_semantics = dict(precomputed_query_semantics or {}) if isinstance(precomputed_query_semantics, dict) else {}
message=effective_q, route_preview = await self.preview_query_route_decision(
source_system=asked_via, question=effective_q,
metadata={ asked_via=asked_via,
"project_id": project_id, project_id=project_id,
"source_chat_id": source_chat_id, source_chat_id=source_chat_id,
"source_message_id": source_message_id, source_message_id=source_message_id,
"conversation_scope_key": external_conversation_key, conversation_scope_key=external_conversation_key,
"reply_to_message_id": str((thread_context or {}).get("reply_to_message_id") or "").strip() or None, reply_to_message_id=str((thread_context or {}).get("reply_to_message_id") or "").strip() or None,
"root_message_id": str((thread_context or {}).get("root_message_id") or "").strip() or None, root_message_id=str((thread_context or {}).get("root_message_id") or "").strip() or None,
}, source_policy_hint=source_policy,
query_semantics=query_semantics,
) if not isinstance(route_decision, dict) or not query_semantics else {
"query_semantics": self._apply_route_decision_to_query_semantics(
query_semantics=query_semantics,
route_decision=route_decision,
),
"route_decision": self._normalize_query_route_decision(route_decision),
}
query_semantics = (
dict(route_preview.get("query_semantics") or {})
if isinstance(route_preview.get("query_semantics"), dict)
else {}
) )
query_semantics = dict(query_semantics) if isinstance(query_semantics, dict) else {} operation_contract = None
query_type = str(query_semantics.get("query_type") or QUERY_TYPE_UNKNOWN).strip().lower() or QUERY_TYPE_UNKNOWN operation_contract_builder = getattr(self, "_build_operation_contract", None)
if callable(operation_contract_builder):
operation_contract = operation_contract_builder(
question=effective_q,
query_semantics=query_semantics,
)
if isinstance(operation_contract, dict):
query_semantics["operation_contract"] = operation_contract
internal_query_plan = self._build_internal_query_plan(
question=effective_q,
project_id=project_id,
query_semantics=query_semantics,
operation_contract=operation_contract,
)
if internal_query_plan:
query_semantics["query_plan"] = internal_query_plan
route_decision = (
dict(route_preview.get("route_decision") or {})
if isinstance(route_preview.get("route_decision"), dict)
else {}
)
query_type = str(
(operation_contract or {}).get("query_type")
or query_semantics.get("query_type")
or QUERY_TYPE_UNKNOWN
).strip().lower() or QUERY_TYPE_UNKNOWN
conversation_text = "\n".join( conversation_text = "\n".join(
[ [
f"Q: {str(row.get('question') or '').strip()}\nA: {str(row.get('answer') or '').strip()}" f"Q: {str(row.get('question') or '').strip()}\nA: {str(row.get('answer') or '').strip()}"
@ -386,6 +439,16 @@ class AdaptiveRAGAnswerMainlineMixin:
message_intent=str(query_semantics.get("message_intent") or query_semantics.get("intent") or "").strip().lower() or None, message_intent=str(query_semantics.get("message_intent") or query_semantics.get("intent") or "").strip().lower() or None,
query_type=query_type, query_type=query_type,
conversation_text=conversation_text, conversation_text=conversation_text,
source_policy=str(
route_decision.get("effective_source_policy")
or query_semantics.get("effective_source_policy")
or query_semantics.get("source_policy")
or source_policy
or ""
).strip().lower()
or None,
project_id=project_id,
query_semantics=query_semantics,
) )
anchor_prompt_variant = await self._resolve_anchor_prompt_variant( anchor_prompt_variant = await self._resolve_anchor_prompt_variant(
anchor_resolution=anchor_resolution, anchor_resolution=anchor_resolution,
@ -400,6 +463,24 @@ class AdaptiveRAGAnswerMainlineMixin:
if bool(scene_semantics.get("requires_freshness")): if bool(scene_semantics.get("requires_freshness")):
query_semantics["external_freshness_only"] = True query_semantics["external_freshness_only"] = True
query_semantics["retrieval_domains"] = ["url"] query_semantics["retrieval_domains"] = ["url"]
query_semantics["source_policy"] = "external"
query_semantics["effective_source_policy"] = "external"
route_decision = self._normalize_query_route_decision(
{
**route_decision,
"route_mode": str(route_decision.get("route_mode") or "external"),
"effective_source_policy": "external",
"decision_origin": str(route_decision.get("decision_origin") or "freshness_guard"),
"reply_contract": {
"mode": "explicit_source",
"head": "外部实时查询:",
"tail_hint": None,
},
"query_profile": query_semantics.get("query_profile"),
"probe_snapshot_summary": route_decision.get("probe_snapshot_summary"),
"probe_snapshot": route_decision.get("probe_snapshot"),
}
)
requested_action_mode = self._normalize_action_mode(action_mode) requested_action_mode = self._normalize_action_mode(action_mode)
effective_action_mode = await self._resolve_action_mode_async( effective_action_mode = await self._resolve_action_mode_async(
requested_mode=requested_action_mode, requested_mode=requested_action_mode,
@ -423,6 +504,7 @@ class AdaptiveRAGAnswerMainlineMixin:
"scene_semantics": scene_semantics if isinstance(scene_semantics, dict) else {}, "scene_semantics": scene_semantics if isinstance(scene_semantics, dict) else {},
"effective_action_mode": effective_action_mode, "effective_action_mode": effective_action_mode,
"anchor_resolution": anchor_resolution, "anchor_resolution": anchor_resolution,
"route_decision": route_decision,
} }
async def _execute_base_answer_mainline( async def _execute_base_answer_mainline(
@ -455,7 +537,26 @@ class AdaptiveRAGAnswerMainlineMixin:
mixed_signal_part: Optional[str], mixed_signal_part: Optional[str],
anchor_resolution: Optional[FollowupAnchorResolution] = None, anchor_resolution: Optional[FollowupAnchorResolution] = None,
consultant_mode_preference: Optional[str] = None, consultant_mode_preference: Optional[str] = None,
route_decision: Optional[dict[str, Any]] = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
operation_contract = (
dict(query_semantics.get("operation_contract") or {})
if isinstance(query_semantics.get("operation_contract"), dict)
else {}
)
operation_contract_builder = getattr(self, "_build_operation_contract", None)
if not operation_contract and callable(operation_contract_builder):
operation_contract = operation_contract_builder(
question=effective_q,
query_semantics=query_semantics,
)
if isinstance(operation_contract, dict):
query_semantics["operation_contract"] = operation_contract
query_type = str(
operation_contract.get("query_type")
or query_type
or QUERY_TYPE_UNKNOWN
).strip().lower() or QUERY_TYPE_UNKNOWN
planner = self._plan_query_runtime( planner = self._plan_query_runtime(
question=clean_q, question=clean_q,
effective_question=effective_q, effective_question=effective_q,
@ -483,11 +584,31 @@ class AdaptiveRAGAnswerMainlineMixin:
anchor_resolution=anchor_resolution, anchor_resolution=anchor_resolution,
session_state=state_row, session_state=state_row,
) )
route_clarification_payload = self._build_route_clarification_payload(route_decision=route_decision)
meta_followup_payload = await self._build_meta_followup_payload( meta_followup_payload = await self._build_meta_followup_payload(
question=clean_q, question=clean_q,
anchor_resolution=anchor_resolution, anchor_resolution=anchor_resolution,
conversation_scope_key=self._get_active_external_qa_context().get("conversation_scope_key"), conversation_scope_key=self._get_active_external_qa_context().get("conversation_scope_key"),
) )
clarification_result = await self._maybe_return_clarification_response(
clarification_payload=clarification_payload or route_clarification_payload,
started=started,
clean_q=clean_q,
effective_q=effective_q,
query_type=query_type,
asked_by=asked_by,
asked_via=asked_via,
project_id=project_id,
session_id=session_id,
turn_index=turn_index,
state_row=state_row,
rewrite_meta=rewrite_meta,
context_session_id=context_session_id,
active_thinking_mode=active_thinking_mode,
route_decision=route_decision,
)
if clarification_result is not None:
return clarification_result
meta_followup_result = await self._maybe_return_meta_followup_response( meta_followup_result = await self._maybe_return_meta_followup_response(
meta_followup_payload=meta_followup_payload, meta_followup_payload=meta_followup_payload,
started=started, started=started,
@ -503,27 +624,10 @@ class AdaptiveRAGAnswerMainlineMixin:
rewrite_meta=rewrite_meta, rewrite_meta=rewrite_meta,
context_session_id=context_session_id, context_session_id=context_session_id,
active_thinking_mode=active_thinking_mode, active_thinking_mode=active_thinking_mode,
route_decision=route_decision,
) )
if meta_followup_result is not None: if meta_followup_result is not None:
return meta_followup_result return meta_followup_result
clarification_result = await self._maybe_return_clarification_response(
clarification_payload=clarification_payload,
started=started,
clean_q=clean_q,
effective_q=effective_q,
query_type=query_type,
asked_by=asked_by,
asked_via=asked_via,
project_id=project_id,
session_id=session_id,
turn_index=turn_index,
state_row=state_row,
rewrite_meta=rewrite_meta,
context_session_id=context_session_id,
active_thinking_mode=active_thinking_mode,
)
if clarification_result is not None:
return clarification_result
system_snapshot = await self.context_cache.get_system_snapshot( system_snapshot = await self.context_cache.get_system_snapshot(
db=self.db, db=self.db,
@ -534,8 +638,44 @@ class AdaptiveRAGAnswerMainlineMixin:
tenant_id=self.tenant_id, tenant_id=self.tenant_id,
user_id=asked_by, user_id=asked_by,
) )
prefers_external_retrieval = (
str(query_semantics.get("effective_source_policy") or query_semantics.get("source_policy") or "").strip().lower()
== "external"
)
target_resolution = (
dict(query_semantics.get("target_resolution") or {})
if isinstance(query_semantics.get("target_resolution"), dict)
else {}
)
target_resolver = getattr(self, "_resolve_internal_target", None)
if not prefers_external_retrieval and callable(target_resolver):
target_resolution = await target_resolver(
question=effective_q,
project_id=project_id,
query_semantics=query_semantics,
system_snapshot=system_snapshot,
operation_contract=operation_contract,
)
if isinstance(target_resolution, dict):
query_semantics["target_resolution"] = target_resolution
patch_scene_semantics = getattr(self, "_patch_scene_semantics_with_target_resolution", None)
if callable(patch_scene_semantics):
scene_semantics = patch_scene_semantics(
scene_semantics=scene_semantics,
operation_contract=operation_contract,
target_resolution=target_resolution,
)
internal_query_plan = self._build_internal_query_plan(
question=effective_q,
project_id=project_id,
query_semantics=query_semantics,
operation_contract=operation_contract,
target_resolution=target_resolution,
)
if internal_query_plan:
query_semantics["query_plan"] = internal_query_plan
if planner.allow_fast_path: if planner.allow_fast_path and not prefers_external_retrieval:
deterministic_result = await self._run_deterministic_query( deterministic_result = await self._run_deterministic_query(
question=effective_q, question=effective_q,
project_id=project_id, project_id=project_id,
@ -559,37 +699,78 @@ class AdaptiveRAGAnswerMainlineMixin:
rewrite_meta=rewrite_meta, rewrite_meta=rewrite_meta,
thinking_mode=active_thinking_mode, thinking_mode=active_thinking_mode,
answer_mode=ANSWER_MODE_FAST_PATH, answer_mode=ANSWER_MODE_FAST_PATH,
route_decision=route_decision,
query_semantics=query_semantics,
) )
internal_model_config_fast_answer = None
internal_model_config_builder = getattr(self, "_build_internal_model_config_fast_answer", None)
if not prefers_external_retrieval and callable(internal_model_config_builder):
internal_model_config_fast_answer = await internal_model_config_builder(
question=effective_q,
query_semantics=query_semantics,
)
if internal_model_config_fast_answer is not None:
return await self._persist_fast_answer(
question=clean_q,
asked_by=asked_by,
asked_via=asked_via,
project_id=project_id,
fast_result=internal_model_config_fast_answer,
session_id=session_id,
external_session_id=context_session_id,
turn_index=turn_index,
query_type=QUERY_TYPE_STATUS,
missing_fields=[],
state_row=state_row,
effective_question=effective_q,
rewrite_meta=rewrite_meta,
thinking_mode=active_thinking_mode,
answer_mode=ANSWER_MODE_FAST_PATH,
route_decision=route_decision,
query_semantics=query_semantics,
)
direct_answer = None direct_answer = None
if planner.allow_fast_path: allow_snapshot_direct_answer = planner.allow_fast_path and not prefers_external_retrieval
snapshot_direct_guard = getattr(self, "_should_allow_snapshot_direct_answer", None)
if callable(snapshot_direct_guard):
allow_snapshot_direct_answer = allow_snapshot_direct_answer and snapshot_direct_guard(
scene_semantics=scene_semantics,
operation_contract=operation_contract,
target_resolution=target_resolution,
)
if allow_snapshot_direct_answer:
direct_answer = self._build_snapshot_direct_answer( direct_answer = self._build_snapshot_direct_answer(
effective_q, effective_q,
system_snapshot=system_snapshot, system_snapshot=system_snapshot,
user_snapshot=user_snapshot, user_snapshot=user_snapshot,
snapshot_focus=str((scene_semantics or {}).get("snapshot_focus") or "none"), snapshot_focus=str((scene_semantics or {}).get("snapshot_focus") or "none"),
) )
direct_result = await self._maybe_return_snapshot_direct_response( direct_result = None
direct_answer=direct_answer, if allow_snapshot_direct_answer:
started=started, direct_result = await self._maybe_return_snapshot_direct_response(
clean_q=clean_q, direct_answer=direct_answer,
effective_q=effective_q, started=started,
asked_by=asked_by, clean_q=clean_q,
asked_via=asked_via, effective_q=effective_q,
project_id=project_id, asked_by=asked_by,
session_id=session_id, asked_via=asked_via,
turn_index=turn_index, project_id=project_id,
state_row=state_row, session_id=session_id,
rewrite_meta=rewrite_meta, turn_index=turn_index,
context_session_id=context_session_id, state_row=state_row,
active_thinking_mode=active_thinking_mode, rewrite_meta=rewrite_meta,
user_snapshot=user_snapshot, context_session_id=context_session_id,
) active_thinking_mode=active_thinking_mode,
user_snapshot=user_snapshot,
route_decision=route_decision,
)
if direct_result is not None: if direct_result is not None:
return direct_result return direct_result
is_snapshot_direct_question = str((scene_semantics or {}).get("snapshot_focus") or "none").strip().lower() != "none" is_snapshot_direct_question = str((scene_semantics or {}).get("snapshot_focus") or "none").strip().lower() != "none"
if planner.allow_fast_path and not is_snapshot_direct_question: if planner.allow_fast_path and not prefers_external_retrieval and not is_snapshot_direct_question:
glm_semantic_result = await self._try_glm_project_semantic_answer( glm_semantic_result = await self._try_glm_project_semantic_answer(
clean_q=effective_q, clean_q=effective_q,
project_id=project_id, project_id=project_id,
@ -612,6 +793,8 @@ class AdaptiveRAGAnswerMainlineMixin:
rewrite_meta=rewrite_meta, rewrite_meta=rewrite_meta,
thinking_mode=active_thinking_mode, thinking_mode=active_thinking_mode,
answer_mode=ANSWER_MODE_FAST_PATH, answer_mode=ANSWER_MODE_FAST_PATH,
route_decision=route_decision,
query_semantics=query_semantics,
) )
return await self._build_retrieval_response( return await self._build_retrieval_response(
@ -644,4 +827,5 @@ class AdaptiveRAGAnswerMainlineMixin:
mixed_signal_part=mixed_signal_part, mixed_signal_part=mixed_signal_part,
answer_mode=planner.retrieval_answer_mode, answer_mode=planner.retrieval_answer_mode,
consultant_mode_preference=consultant_mode_preference, consultant_mode_preference=consultant_mode_preference,
route_decision=route_decision,
) )

View File

@ -10,10 +10,12 @@ class AdaptiveRAGAnswerPayloadMixin:
"db": 90, "db": 90,
"graph": 80, "graph": 80,
"doc": 70, "doc": 70,
"mcp_search": 68,
"url": 65, "url": 65,
"feishu": 60, "feishu": 60,
"system": 10, "system": 10,
} }
_INTERNAL_SOURCE_TYPES = {"db", "graph", "doc", "pmflow", "feishu"}
@staticmethod @staticmethod
def _safe_source_metadata(raw: Any) -> dict[str, Any]: def _safe_source_metadata(raw: Any) -> dict[str, Any]:
@ -69,7 +71,7 @@ class AdaptiveRAGAnswerPayloadMixin:
if record_type and record_id is not None: if record_type and record_id is not None:
return f"{record_type}:{record_id}" return f"{record_type}:{record_id}"
return record_id return record_id
if source_type in {"doc", "url"}: if source_type in {"doc", "url", "mcp_search"}:
return self._source_ref_value( return self._source_ref_value(
metadata.get("doc_id") metadata.get("doc_id")
or metadata.get("knowledge_doc_id") or metadata.get("knowledge_doc_id")
@ -366,6 +368,191 @@ class AdaptiveRAGAnswerPayloadMixin:
"tool_trace": tool_trace if isinstance(tool_trace, list) else [], "tool_trace": tool_trace if isinstance(tool_trace, list) else [],
} }
def _is_internal_answer_contract_route(
self,
*,
route_decision: Optional[dict[str, Any]] = None,
sources: Optional[list[dict[str, Any]]] = None,
) -> bool:
normalized_route = (
self._normalize_query_route_decision(route_decision)
if hasattr(self, "_normalize_query_route_decision")
else {}
)
if str(normalized_route.get("route_mode") or "").strip().lower() == "internal":
return True
if str(normalized_route.get("effective_source_policy") or "").strip().lower() == "internal":
return True
for item in sources or []:
if not isinstance(item, dict):
continue
if str(item.get("source") or "").strip().lower() in self._INTERNAL_SOURCE_TYPES:
return True
return False
def _verify_external_answer_contract(
self,
*,
answer: str,
sources: Optional[list[dict[str, Any]]] = None,
route_decision: Optional[dict[str, Any]] = None,
) -> dict[str, Any]:
normalized_route = (
self._normalize_query_route_decision(route_decision)
if hasattr(self, "_normalize_query_route_decision")
else {}
)
if str(normalized_route.get("effective_source_policy") or "").strip().lower() != "external":
return {"ok": True, "reason_code": None, "missing_fields": []}
source_rows = sources if isinstance(sources, list) else []
if not source_rows:
return {
"ok": False,
"reason_code": "missing_external_evidence",
"missing_fields": ["external_evidence"],
}
external_source_seen = False
for item in source_rows:
if not isinstance(item, dict):
continue
source = str(item.get("source") or "").strip().lower()
metadata = item.get("metadata") if isinstance(item.get("metadata"), dict) else {}
if source == "system":
continue
if source not in {"url", "mcp_search"}:
return {
"ok": False,
"reason_code": "external_source_contract_violation",
"missing_fields": ["external_source_type"],
}
target_url = str(
metadata.get("target_url")
or metadata.get("file_url")
or metadata.get("url")
or ""
).strip().lower()
if not target_url.startswith("http://") and not target_url.startswith("https://"):
return {
"ok": False,
"reason_code": "external_target_url_missing",
"missing_fields": ["target_url"],
}
external_source_seen = True
if not external_source_seen:
return {
"ok": False,
"reason_code": "missing_external_grounding",
"missing_fields": ["external_evidence"],
}
_ = answer
return {"ok": True, "reason_code": None, "missing_fields": []}
def _build_external_answer_contract_guard_payload(
self,
*,
reason_code: Optional[str],
missing_fields: Optional[list[str]] = None,
) -> dict[str, Any]:
normalized_reason = str(reason_code or "").strip().lower() or "external_answer_contract_violation"
normalized_missing = [str(item).strip() for item in (missing_fields or []) if str(item).strip()]
return {
"answer": "当前没有拿到可核验的外部实时证据,已阻止直接返回外部结论。请稍后重试,或把要核验的公开对象说得更具体一些。",
"sources": [
{
"source": "system",
"label": "External Query Contract Guard",
"score": 0.99,
"metadata": {
"mode": "guard",
"reason": normalized_reason,
},
}
],
"missing_fields": normalized_missing or ["external_answer_contract"],
"reason_code": normalized_reason,
"answer_mode": ANSWER_MODE_CLARIFY,
"grounding_status": GROUNDING_STATUS_INSUFFICIENT_EVIDENCE,
}
def _verify_internal_answer_contract(
self,
*,
query_type: str,
answer: str,
sources: Optional[list[dict[str, Any]]] = None,
route_decision: Optional[dict[str, Any]] = None,
) -> dict[str, Any]:
source_rows = sources if isinstance(sources, list) else []
if not self._is_internal_answer_contract_route(route_decision=route_decision, sources=source_rows):
return {"ok": True, "reason_code": None, "missing_fields": []}
normalized_query_type = str(query_type or QUERY_TYPE_UNKNOWN).strip().lower()
normalized_answer = str(answer or "").strip()
compact_answer = re.sub(r"\s+", "", normalized_answer.lower())
if normalized_query_type == QUERY_TYPE_EXISTENCE:
if "项目总数" in normalized_answer or "总数为" in normalized_answer:
return {
"ok": False,
"reason_code": "no_total_count_substitution",
"missing_fields": ["matched_evidence", "existence_contract"],
}
if "总数" in compact_answer and "匹配到" not in normalized_answer and "未检索到" not in normalized_answer:
return {
"ok": False,
"reason_code": "no_total_count_substitution",
"missing_fields": ["matched_evidence", "existence_contract"],
}
if not source_rows:
return {
"ok": False,
"reason_code": "missing_internal_evidence",
"missing_fields": ["internal_evidence"],
}
return {"ok": True, "reason_code": None, "missing_fields": []}
def _build_internal_answer_contract_guard_payload(
self,
*,
query_type: str,
reason_code: Optional[str],
missing_fields: Optional[list[str]] = None,
) -> dict[str, Any]:
normalized_query_type = str(query_type or QUERY_TYPE_UNKNOWN).strip().lower()
normalized_reason = str(reason_code or "").strip().lower() or "internal_answer_contract_violation"
normalized_missing = [str(item).strip() for item in (missing_fields or []) if str(item).strip()]
if normalized_reason == "no_total_count_substitution" and normalized_query_type == QUERY_TYPE_EXISTENCE:
answer = "当前内部查询结果与问题类型不一致,已阻止直接返回。你问的是“有没有”,请给我更明确的对象条件后我按存在性重新查询。"
elif normalized_reason == "missing_internal_evidence":
answer = "当前内部查询证据不足,已阻止直接返回结论。请补充更明确的对象或范围后我再查。"
else:
answer = "当前内部查询结果未通过回答合同校验,已阻止直接返回。请补充更明确的对象或范围后我再查。"
return {
"answer": answer,
"sources": [
{
"source": "system",
"label": "Internal Query Contract Guard",
"score": 0.99,
"metadata": {
"mode": "guard",
"reason": normalized_reason,
"query_type": normalized_query_type,
},
}
],
"missing_fields": normalized_missing or ["internal_answer_contract"],
"reason_code": normalized_reason,
"answer_mode": ANSWER_MODE_BLOCKED,
"grounding_status": GROUNDING_STATUS_BLOCKED,
}
def _build_response_payload( def _build_response_payload(
self, self,
*, *,
@ -408,6 +595,7 @@ class AdaptiveRAGAnswerPayloadMixin:
sources_used: Optional[list[dict[str, Any]]] = None, sources_used: Optional[list[dict[str, Any]]] = None,
evidence_pack: Optional[dict[str, Any]] = None, evidence_pack: Optional[dict[str, Any]] = None,
prompt_variant_id: Optional[str] = None, prompt_variant_id: Optional[str] = None,
route_decision: Optional[dict[str, Any]] = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
evidence: list[dict[str, Any]] = [] evidence: list[dict[str, Any]] = []
for item in sources[:8]: for item in sources[:8]:
@ -452,6 +640,11 @@ class AdaptiveRAGAnswerPayloadMixin:
reason_code=reason_code, reason_code=reason_code,
) )
) )
normalized_route_decision = (
self._normalize_query_route_decision(route_decision)
if hasattr(self, "_normalize_query_route_decision")
else {}
)
return { return {
"answer": str(answer or "").strip(), "answer": str(answer or "").strip(),
"evidence": evidence, "evidence": evidence,
@ -498,6 +691,7 @@ class AdaptiveRAGAnswerPayloadMixin:
"citations": normalized_citations, "citations": normalized_citations,
"sources_used": normalized_sources_used, "sources_used": normalized_sources_used,
"evidence_pack": normalized_evidence_pack, "evidence_pack": normalized_evidence_pack,
"route_decision": normalized_route_decision,
} }
@staticmethod @staticmethod

View File

@ -22,6 +22,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
rewrite_meta: dict[str, Any], rewrite_meta: dict[str, Any],
context_session_id: Optional[int], context_session_id: Optional[int],
active_thinking_mode: str, active_thinking_mode: str,
route_decision: Optional[dict[str, Any]] = None,
) -> Optional[dict[str, Any]]: ) -> Optional[dict[str, Any]]:
if not isinstance(meta_followup_payload, dict): if not isinstance(meta_followup_payload, dict):
return None return None
@ -52,8 +53,16 @@ class AdaptiveRAGAnswerPrecheckMixin:
}, },
} }
] ]
render_with_contract = getattr(self, "_render_answer_with_route_contract", None)
rendered_answer = answer
if callable(render_with_contract):
rendered_answer = render_with_contract(
answer=answer,
sources=direct_sources,
route_decision=route_decision,
)
runtime_fields = self._build_qa_record_runtime_fields( runtime_fields = self._build_qa_record_runtime_fields(
answer=answer, answer=rendered_answer,
sources=direct_sources, sources=direct_sources,
missing_fields=[], missing_fields=[],
answer_mode=ANSWER_MODE_FAST_PATH, answer_mode=ANSWER_MODE_FAST_PATH,
@ -76,7 +85,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
qa = QARecord( qa = QARecord(
tenant_id=self.tenant_id, tenant_id=self.tenant_id,
question=clean_q, question=clean_q,
answer=answer, answer=rendered_answer,
sources=direct_sources, sources=direct_sources,
intent="question", intent="question",
domains_used=["system_context"], domains_used=["system_context"],
@ -90,6 +99,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
anchor_kind=anchor_kind, anchor_kind=anchor_kind,
anchor_ref=str(meta_followup_payload.get("anchor_ref") or "").strip() or None, anchor_ref=str(meta_followup_payload.get("anchor_ref") or "").strip() or None,
routing_snapshot=routing_snapshot, routing_snapshot=routing_snapshot,
route_decision=route_decision,
), ),
**runtime_fields, **runtime_fields,
) )
@ -98,22 +108,31 @@ class AdaptiveRAGAnswerPrecheckMixin:
session_state=state_row, session_state=state_row,
question=clean_q, question=clean_q,
effective_question=effective_q, effective_question=effective_q,
answer=answer, answer=rendered_answer,
query_type=query_type, query_type=query_type,
ambiguity="clear", ambiguity="clear",
rewrite_meta=rewrite_meta, rewrite_meta=rewrite_meta,
sources=direct_sources, sources=direct_sources,
) )
await self.db.flush()
persist_route_memory = getattr(self, "_persist_query_route_memory", None)
if callable(persist_route_memory):
await persist_route_memory(
qa_record_id=int(qa.id) if getattr(qa, "id", None) is not None else None,
question_text=clean_q,
project_id=project_id,
route_decision=route_decision,
)
await self.db.commit() await self.db.commit()
await self.db.refresh(qa) await self.db.refresh(qa)
await self.context_cache.append_session_messages( await self.context_cache.append_session_messages(
tenant_id=self.tenant_id, tenant_id=self.tenant_id,
session_id=context_session_id, session_id=context_session_id,
question=clean_q, question=clean_q,
answer=answer, answer=rendered_answer,
) )
return self._build_response_payload( return self._build_response_payload(
answer=answer, answer=rendered_answer,
sources=direct_sources, sources=direct_sources,
domains_used=["system_context"], domains_used=["system_context"],
response_time_ms=elapsed, response_time_ms=elapsed,
@ -141,6 +160,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
evidence_pack=runtime_fields["evidence_pack"], evidence_pack=runtime_fields["evidence_pack"],
tool_trace=runtime_fields["tool_trace"], tool_trace=runtime_fields["tool_trace"],
prompt_variant_id="meta_followup", prompt_variant_id="meta_followup",
route_decision=route_decision,
) )
async def _maybe_return_clarification_response( async def _maybe_return_clarification_response(
@ -160,6 +180,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
rewrite_meta: dict[str, Any], rewrite_meta: dict[str, Any],
context_session_id: Optional[int], context_session_id: Optional[int],
active_thinking_mode: str, active_thinking_mode: str,
route_decision: Optional[dict[str, Any]] = None,
) -> Optional[dict[str, Any]]: ) -> Optional[dict[str, Any]]:
if not isinstance(clarification_payload, dict): if not isinstance(clarification_payload, dict):
return None return None
@ -191,8 +212,16 @@ class AdaptiveRAGAnswerPrecheckMixin:
"metadata": {"mode": "clarification", "reason": clarification_code}, "metadata": {"mode": "clarification", "reason": clarification_code},
} }
] ]
render_with_contract = getattr(self, "_render_answer_with_route_contract", None)
rendered_answer = clarification_answer
if callable(render_with_contract):
rendered_answer = render_with_contract(
answer=clarification_answer,
sources=clarify_sources,
route_decision=route_decision,
)
runtime_fields = self._build_qa_record_runtime_fields( runtime_fields = self._build_qa_record_runtime_fields(
answer=clarification_answer, answer=rendered_answer,
sources=clarify_sources, sources=clarify_sources,
missing_fields=missing_fields, missing_fields=missing_fields,
answer_mode=ANSWER_MODE_CLARIFY, answer_mode=ANSWER_MODE_CLARIFY,
@ -214,7 +243,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
qa = QARecord( qa = QARecord(
tenant_id=self.tenant_id, tenant_id=self.tenant_id,
question=clean_q, question=clean_q,
answer=clarification_answer, answer=rendered_answer,
sources=clarify_sources, sources=clarify_sources,
intent="question", intent="question",
domains_used=["system_context"], domains_used=["system_context"],
@ -224,7 +253,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
project_id=project_id, project_id=project_id,
session_id=active_session_id, session_id=active_session_id,
turn_index=turn_index, turn_index=turn_index,
**self._build_qa_record_external_fields(), **self._build_qa_record_external_fields(route_decision=route_decision),
**runtime_fields, **runtime_fields,
) )
self.db.add(qa) self.db.add(qa)
@ -244,22 +273,31 @@ class AdaptiveRAGAnswerPrecheckMixin:
session_state=state_row, session_state=state_row,
question=clean_q, question=clean_q,
effective_question=effective_q, effective_question=effective_q,
answer=clarification_answer, answer=rendered_answer,
query_type=query_type, query_type=query_type,
ambiguity="needs_clarification", ambiguity="needs_clarification",
rewrite_meta=rewrite_meta, rewrite_meta=rewrite_meta,
sources=clarify_sources, sources=clarify_sources,
) )
await self.db.flush()
persist_route_memory = getattr(self, "_persist_query_route_memory", None)
if callable(persist_route_memory):
await persist_route_memory(
qa_record_id=int(qa.id) if getattr(qa, "id", None) is not None else None,
question_text=clean_q,
project_id=project_id,
route_decision=route_decision,
)
await self.db.commit() await self.db.commit()
await self.db.refresh(qa) await self.db.refresh(qa)
await self.context_cache.append_session_messages( await self.context_cache.append_session_messages(
tenant_id=self.tenant_id, tenant_id=self.tenant_id,
session_id=context_session_id, session_id=context_session_id,
question=clean_q, question=clean_q,
answer=clarification_answer, answer=rendered_answer,
) )
return self._build_response_payload( return self._build_response_payload(
answer=clarification_answer, answer=rendered_answer,
sources=clarify_sources, sources=clarify_sources,
domains_used=["system_context"], domains_used=["system_context"],
response_time_ms=elapsed, response_time_ms=elapsed,
@ -286,6 +324,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
evidence_pack=runtime_fields["evidence_pack"], evidence_pack=runtime_fields["evidence_pack"],
tool_trace=runtime_fields["tool_trace"], tool_trace=runtime_fields["tool_trace"],
prompt_variant_id="clarification", prompt_variant_id="clarification",
route_decision=route_decision,
) )
async def _maybe_return_snapshot_direct_response( async def _maybe_return_snapshot_direct_response(
@ -305,6 +344,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
context_session_id: Optional[int], context_session_id: Optional[int],
active_thinking_mode: str, active_thinking_mode: str,
user_snapshot: Optional[dict[str, Any]], user_snapshot: Optional[dict[str, Any]],
route_decision: Optional[dict[str, Any]] = None,
) -> Optional[dict[str, Any]]: ) -> Optional[dict[str, Any]]:
if not direct_answer: if not direct_answer:
return None return None
@ -327,8 +367,16 @@ class AdaptiveRAGAnswerPrecheckMixin:
"metadata": {"mode": "snapshot_direct", "topic": "current_user"}, "metadata": {"mode": "snapshot_direct", "topic": "current_user"},
} }
) )
render_with_contract = getattr(self, "_render_answer_with_route_contract", None)
rendered_answer = direct_answer
if callable(render_with_contract):
rendered_answer = render_with_contract(
answer=direct_answer,
sources=direct_sources,
route_decision=route_decision,
)
runtime_fields = self._build_qa_record_runtime_fields( runtime_fields = self._build_qa_record_runtime_fields(
answer=direct_answer, answer=rendered_answer,
sources=direct_sources, sources=direct_sources,
missing_fields=[], missing_fields=[],
answer_mode=ANSWER_MODE_FAST_PATH, answer_mode=ANSWER_MODE_FAST_PATH,
@ -347,7 +395,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
qa = QARecord( qa = QARecord(
tenant_id=self.tenant_id, tenant_id=self.tenant_id,
question=clean_q, question=clean_q,
answer=direct_answer, answer=rendered_answer,
sources=direct_sources, sources=direct_sources,
intent="question", intent="question",
domains_used=["system_context"], domains_used=["system_context"],
@ -357,7 +405,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
project_id=project_id, project_id=project_id,
session_id=int(session_id) if session_id is not None else (int(context_session_id) if context_session_id is not None else None), session_id=int(session_id) if session_id is not None else (int(context_session_id) if context_session_id is not None else None),
turn_index=turn_index, turn_index=turn_index,
**self._build_qa_record_external_fields(), **self._build_qa_record_external_fields(route_decision=route_decision),
**runtime_fields, **runtime_fields,
) )
self.db.add(qa) self.db.add(qa)
@ -365,22 +413,31 @@ class AdaptiveRAGAnswerPrecheckMixin:
session_state=state_row, session_state=state_row,
question=clean_q, question=clean_q,
effective_question=effective_q, effective_question=effective_q,
answer=direct_answer, answer=rendered_answer,
query_type=QUERY_TYPE_EXPLAIN, query_type=QUERY_TYPE_EXPLAIN,
ambiguity="clear", ambiguity="clear",
rewrite_meta=rewrite_meta, rewrite_meta=rewrite_meta,
sources=direct_sources, sources=direct_sources,
) )
await self.db.flush()
persist_route_memory = getattr(self, "_persist_query_route_memory", None)
if callable(persist_route_memory):
await persist_route_memory(
qa_record_id=int(qa.id) if getattr(qa, "id", None) is not None else None,
question_text=clean_q,
project_id=project_id,
route_decision=route_decision,
)
await self.db.commit() await self.db.commit()
await self.db.refresh(qa) await self.db.refresh(qa)
await self.context_cache.append_session_messages( await self.context_cache.append_session_messages(
tenant_id=self.tenant_id, tenant_id=self.tenant_id,
session_id=context_session_id, session_id=context_session_id,
question=clean_q, question=clean_q,
answer=direct_answer, answer=rendered_answer,
) )
return self._build_response_payload( return self._build_response_payload(
answer=direct_answer, answer=rendered_answer,
sources=direct_sources, sources=direct_sources,
domains_used=["system_context"], domains_used=["system_context"],
response_time_ms=elapsed, response_time_ms=elapsed,
@ -404,4 +461,5 @@ class AdaptiveRAGAnswerPrecheckMixin:
evidence_pack=runtime_fields["evidence_pack"], evidence_pack=runtime_fields["evidence_pack"],
tool_trace=runtime_fields["tool_trace"], tool_trace=runtime_fields["tool_trace"],
prompt_variant_id="snapshot_direct", prompt_variant_id="snapshot_direct",
route_decision=route_decision,
) )

View File

@ -5,6 +5,237 @@ from .shared import *
class AdaptiveRAGAnswerRetrievalMixin: class AdaptiveRAGAnswerRetrievalMixin:
async def _execute_external_mcp_search(self, plan) -> dict[str, Any]:
from ..external_query_service.schemas import ExternalAdapterResult, ExternalQueryPlan
from ..mcp_config import McpConfigService
try:
normalized_plan = plan if isinstance(plan, ExternalQueryPlan) else ExternalQueryPlan.model_validate(plan)
except Exception as exc:
return ExternalAdapterResult(
adapter_name="mcp",
status="error",
evidence=[],
meta={"reason": "plan_invalid", "error": str(exc)},
).model_dump(mode="json")
query = str((normalized_plan.search_queries[:1] or [normalized_plan.topic])[0] or "").strip()
if not query:
return ExternalAdapterResult(
adapter_name="mcp",
status="empty",
evidence=[],
meta={"reason": "empty_query"},
).model_dump(mode="json")
db = getattr(self, "db", None)
if db is None:
return ExternalAdapterResult(
adapter_name="mcp",
status="unavailable",
evidence=[],
meta={"reason": "db_unavailable"},
).model_dump(mode="json")
try:
service = McpConfigService(db, tenant_id=int(getattr(self, "tenant_id", 1) or 1))
evidence = await service.search_external(query=query, result_limit=normalized_plan.result_limit)
except Exception as exc:
return ExternalAdapterResult(
adapter_name="mcp",
status="error",
evidence=[],
meta={"reason": "backend_direct_runtime_error", "error": str(exc), "query": query},
).model_dump(mode="json")
if not evidence:
return ExternalAdapterResult(
adapter_name="mcp",
status="empty",
evidence=[],
meta={"reason": "no_backend_direct_result", "query": query},
).model_dump(mode="json")
return ExternalAdapterResult(
adapter_name="mcp",
status="success",
evidence=evidence,
meta={"query": query, "provider": "mcp_backend_direct"},
).model_dump(mode="json")
async def _execute_external_query_gateway(
self,
*,
question: str,
query_semantics: Optional[dict[str, Any]] = None,
result_limit: int = 3,
max_pages: int = 3,
) -> tuple[list[RetrievedItem], dict[str, Any]]:
live_search_ready = all(
callable(item)
for item in (
getattr(self, "_search_live_external_results", None),
getattr(self, "_fetch_url_text", None),
getattr(self, "_chunk_text", None),
getattr(self, "_score_live_external_search_result", None),
getattr(self, "_score_live_external_chunk", None),
)
)
mcp_ready = callable(getattr(self, "_execute_external_mcp_search", None)) and getattr(self, "db", None) is not None
if not live_search_ready and not mcp_ready:
legacy_retrieval = getattr(self, "_retrieve_live_external_context", None)
if callable(legacy_retrieval):
rows, meta = await legacy_retrieval(
question=question,
max_results=result_limit,
max_pages=max_pages,
)
normalized_meta = dict(meta or {})
normalized_meta.setdefault("mode", "live_external_context" if rows else "live_external_context_empty")
normalized_meta.setdefault("domains_used", ["url"])
normalized_meta.setdefault("db_query_ms", 0)
normalized_meta.setdefault("vector_search_ms", 0)
normalized_meta.setdefault("graph_query_ms", 0)
normalized_meta.setdefault("db_timeout", False)
normalized_meta.setdefault("vector_timeout", False)
normalized_meta.setdefault("graph_timeout", False)
normalized_meta.setdefault("tool_trace", [])
return rows, normalized_meta
from ..external_query_service import ExternalQueryGateway, ExternalQueryPlanner, ExternalProviderRegistry
from ..external_query_service.adapters import LiveSearchFallbackAdapter, MCPExternalAdapter
planner = ExternalQueryPlanner(
llm_service=getattr(self, "llm", None),
db=getattr(self, "db", None),
tenant_id=getattr(self, "tenant_id", 1),
query_builder=getattr(self, "_build_live_external_search_queries", None),
)
registry = ExternalProviderRegistry(
adapters=[
MCPExternalAdapter(
search_callable=getattr(self, "_execute_external_mcp_search", None),
fetch_text_callable=getattr(self, "_fetch_url_text", None),
chunk_text_callable=getattr(self, "_chunk_text", None),
score_chunk_callable=getattr(self, "_score_live_external_chunk", None),
max_pages=max_pages,
),
LiveSearchFallbackAdapter(
build_queries_callable=getattr(self, "_build_live_external_search_queries", None),
search_results_callable=getattr(self, "_search_live_external_results", None),
fetch_text_callable=getattr(self, "_fetch_url_text", None),
chunk_text_callable=getattr(self, "_chunk_text", None),
score_search_result_callable=getattr(self, "_score_live_external_search_result", None),
score_chunk_callable=getattr(self, "_score_live_external_chunk", None),
max_pages=max_pages,
),
]
)
execution = await ExternalQueryGateway(
planner=planner,
registry=registry,
).execute(
question=question,
query_semantics=query_semantics,
)
verifier = dict(execution.verifier or {})
if not bool(verifier.get("ok")):
return [], {
"mode": "external_query_gateway_verifier_blocked",
"adapter": execution.adapter_name,
"verifier": verifier,
"planner": dict(execution.planner_meta or {}),
"fallback_used": bool((execution.meta or {}).get("fallback_used")),
"adapter_attempts": list((execution.meta or {}).get("adapter_attempts") or []),
"query": str(((execution.meta or {}).get("query") or (execution.plan.search_queries[:1] or [""])[0])).strip(),
"search_result_count": int((execution.meta or {}).get("search_result_count") or 0),
"fetched_pages": int((execution.meta or {}).get("fetched_pages") or 0),
"tool_trace": [
{
"step": 1,
"tool": "external_query_gateway",
"adapter": execution.adapter_name,
"planner": str((execution.planner_meta or {}).get("planner") or "unknown"),
"verifier_ok": False,
}
],
"domains_used": ["url"],
"db_query_ms": 0,
"vector_search_ms": 0,
"graph_query_ms": 0,
"db_timeout": False,
"vector_timeout": False,
"graph_timeout": False,
}
rows: list[RetrievedItem] = []
for item in list(execution.evidence or [])[: max(1, int(result_limit))]:
source_type = str(getattr(item, "source_type", "url") or "url").strip().lower() or "url"
normalized_source = source_type if source_type in {"url", "mcp_search"} else "url"
title = str(getattr(item, "title", "") or "").strip()
source_label = str(getattr(item, "source_label", "") or "").strip() or title
target_url = str(getattr(item, "target_url", "") or "").strip()
snippet = str(getattr(item, "snippet", "") or "").strip()
freshness_score = float(getattr(item, "freshness_score", 0.0) or 0.0)
authority_score = float(getattr(item, "authority_score", 0.0) or 0.0)
rows.append(
RetrievedItem(
source=normalized_source,
content=(snippet or title or target_url)[:1800],
score=round(max(freshness_score, authority_score, 0.55), 4),
source_label=(
f"MCP[{source_label[:120]}]"
if normalized_source == "mcp_search"
else f"URL[{source_label[:120]}]"
),
metadata={
"title": title[:255],
"url": target_url,
"file_url": target_url,
"target_url": target_url,
"retrieved_via": "external_query_gateway",
"external_source_type": source_type,
"external_adapter": execution.adapter_name,
"freshness_score": round(freshness_score, 4),
"authority_score": round(authority_score, 4),
"grounded_fields": list(getattr(item, "grounded_fields", []) or []),
**(dict(getattr(item, "raw_metadata", {}) or {})),
},
)
)
meta = dict(execution.meta or {})
meta.update(
{
"mode": "external_query_gateway" if rows else "external_query_gateway_empty",
"adapter": execution.adapter_name,
"verifier": verifier,
"planner": dict(execution.planner_meta or {}),
"query": str(meta.get("query") or (execution.plan.search_queries[:1] or [""])[0]).strip(),
"search_result_count": int(meta.get("search_result_count") or 0),
"fetched_pages": int(meta.get("fetched_pages") or 0),
"domains_used": ["url"],
"db_query_ms": 0,
"vector_search_ms": 0,
"graph_query_ms": 0,
"db_timeout": False,
"vector_timeout": False,
"graph_timeout": False,
"tool_trace": [
{
"step": 1,
"tool": "external_query_gateway",
"adapter": execution.adapter_name,
"planner": str((execution.planner_meta or {}).get("planner") or "unknown"),
"verifier_ok": True,
"result_count": len(rows),
"fallback_used": bool(meta.get("fallback_used")),
}
],
}
)
return rows, meta
async def _maybe_fetch_live_external_freshness_context( async def _maybe_fetch_live_external_freshness_context(
self, self,
*, *,
@ -19,16 +250,30 @@ class AdaptiveRAGAnswerRetrievalMixin:
search_question = str(effective_question or question or "").strip() search_question = str(effective_question or question or "").strip()
if not search_question: if not search_question:
return [], {"mode": "skip_empty_question"} return [], {"mode": "skip_empty_question"}
execute_gateway = getattr(self, "_execute_external_query_gateway", None)
try: try:
rows, live_meta = await self._retrieve_live_external_context( if callable(execute_gateway):
question=search_question, rows, live_meta = await execute_gateway(
max_results=3, question=search_question,
max_pages=3, query_semantics=query_semantics,
) result_limit=3,
max_pages=3,
)
else:
rows, live_meta = await self._retrieve_live_external_context(
question=search_question,
max_results=3,
max_pages=3,
)
except Exception as exc: except Exception as exc:
return [], {"mode": "live_external_context_error", "error": str(exc)} mode = "external_query_gateway_error" if callable(execute_gateway) else "live_external_context_error"
return [], {"mode": mode, "error": str(exc)}
meta = dict(live_meta or {}) meta = dict(live_meta or {})
meta["mode"] = "live_external_context" if rows else "live_external_context_empty" if not str(meta.get("mode") or "").strip():
if callable(execute_gateway):
meta["mode"] = "external_query_gateway" if rows else "external_query_gateway_empty"
else:
meta["mode"] = "live_external_context" if rows else "live_external_context_empty"
meta["external_freshness_only"] = bool( meta["external_freshness_only"] = bool(
isinstance(query_semantics, dict) and query_semantics.get("external_freshness_only") isinstance(query_semantics, dict) and query_semantics.get("external_freshness_only")
) )
@ -66,22 +311,43 @@ class AdaptiveRAGAnswerRetrievalMixin:
mixed_signal_part: Optional[str] = None, mixed_signal_part: Optional[str] = None,
answer_mode: str = ANSWER_MODE_RETRIEVAL_NORMAL, answer_mode: str = ANSWER_MODE_RETRIEVAL_NORMAL,
consultant_mode_preference: Optional[str] = None, consultant_mode_preference: Optional[str] = None,
route_decision: Optional[dict[str, Any]] = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
domains = [str(item).strip().lower() for item in retrieval_domains if str(item or "").strip()] domains = [str(item).strip().lower() for item in retrieval_domains if str(item or "").strip()]
if not domains: if not domains:
domains = ["db", "graph", "doc", "feishu", "pmflow", "url"] domains = ["db", "graph", "doc", "feishu", "pmflow", "url"]
if str(asked_via or "").strip().lower() == "feishu" and source_chat_id and "feishu" not in domains: if str(asked_via or "").strip().lower() == "feishu" and source_chat_id and "feishu" not in domains:
domains.append("feishu") domains.append("feishu")
retrieved, search_meta = await self._multi_step_search( normalize_route_decision = getattr(self, "_normalize_query_route_decision", None)
question=effective_q, normalized_route_decision = (
domains=domains, normalize_route_decision(route_decision)
project_id=project_id, if callable(normalize_route_decision)
source_chat_id=source_chat_id, else (dict(route_decision or {}) if isinstance(route_decision, dict) else {})
query_semantics=query_semantics,
session_state=state_row,
max_steps_override=deep_retrieval_steps if deep_thinking else None,
vector_top_k=deep_vector_top_k if deep_thinking else None,
) )
external_route_only = str(
normalized_route_decision.get("effective_source_policy")
or query_semantics.get("effective_source_policy")
or query_semantics.get("source_policy")
or ""
).strip().lower() == "external"
if external_route_only:
retrieved, search_meta = await self._execute_external_query_gateway(
question=effective_q,
query_semantics=query_semantics,
result_limit=3,
max_pages=3,
)
else:
retrieved, search_meta = await self._multi_step_search(
question=effective_q,
domains=domains,
project_id=project_id,
source_chat_id=source_chat_id,
query_semantics=query_semantics,
session_state=state_row,
max_steps_override=deep_retrieval_steps if deep_thinking else None,
vector_top_k=deep_vector_top_k if deep_thinking else None,
)
db_query_ms = int(search_meta.get("db_query_ms") or 0) db_query_ms = int(search_meta.get("db_query_ms") or 0)
vector_search_ms = int(search_meta.get("vector_search_ms") or 0) vector_search_ms = int(search_meta.get("vector_search_ms") or 0)
graph_query_ms = int(search_meta.get("graph_query_ms") or 0) graph_query_ms = int(search_meta.get("graph_query_ms") or 0)
@ -101,6 +367,16 @@ class AdaptiveRAGAnswerRetrievalMixin:
), ),
}, },
) )
tool_trace.insert(
0,
{
"step": 0,
"tool": "query_route_gateway",
"route_mode": normalized_route_decision.get("route_mode"),
"effective_source_policy": normalized_route_decision.get("effective_source_policy"),
"decision_origin": normalized_route_decision.get("decision_origin"),
},
)
final_domains = [str(x) for x in (search_meta.get("domains_used") or domains)] final_domains = [str(x) for x in (search_meta.get("domains_used") or domains)]
prompt_variant_id = self._peek_answer_prompt_variant() prompt_variant_id = self._peek_answer_prompt_variant()
@ -155,7 +431,7 @@ class AdaptiveRAGAnswerRetrievalMixin:
auto_retry_reason = "" auto_retry_reason = ""
if not deep_thinking: if not deep_thinking:
auto_retry_reason = str(pre_retry_guard_semantics.get("auto_retry_reason") or "").strip() auto_retry_reason = str(pre_retry_guard_semantics.get("auto_retry_reason") or "").strip()
if auto_retry_reason: if auto_retry_reason and not external_route_only:
deep_retrieved, deep_meta = await self._multi_step_search( deep_retrieved, deep_meta = await self._multi_step_search(
question=effective_q, question=effective_q,
domains=domains, domains=domains,
@ -426,6 +702,92 @@ class AdaptiveRAGAnswerRetrievalMixin:
"vector_timeout": bool(vector_timeout), "vector_timeout": bool(vector_timeout),
"graph_timeout": bool(graph_timeout), "graph_timeout": bool(graph_timeout),
} }
filter_sources_for_route = getattr(self, "_filter_sources_for_route", None)
if callable(filter_sources_for_route):
used_sources = filter_sources_for_route(
sources=used_sources,
route_decision=normalized_route_decision,
)
render_with_contract = getattr(self, "_render_answer_with_route_contract", None)
if callable(render_with_contract):
answer_text = render_with_contract(
answer=answer_text,
sources=used_sources,
route_decision=normalized_route_decision,
)
effective_source_policy = str(normalized_route_decision.get("effective_source_policy") or "").strip().lower()
contract_verdict = {"ok": True, "reason_code": None, "missing_fields": []}
guard_payload: Optional[dict[str, Any]] = None
if effective_source_policy == "external" and answer_mode != ANSWER_MODE_CLARIFY:
contract_verdict = self._verify_external_answer_contract(
answer=answer_text,
sources=used_sources,
route_decision=normalized_route_decision,
)
if not bool(contract_verdict.get("ok")):
guard_payload = self._build_external_answer_contract_guard_payload(
reason_code=str(contract_verdict.get("reason_code") or ""),
missing_fields=list(contract_verdict.get("missing_fields") or []),
)
else:
contract_verdict = self._verify_internal_answer_contract(
query_type=query_type,
answer=answer_text,
sources=used_sources,
route_decision=normalized_route_decision,
)
if not bool(contract_verdict.get("ok")):
guard_payload = self._build_internal_answer_contract_guard_payload(
query_type=query_type,
reason_code=str(contract_verdict.get("reason_code") or ""),
missing_fields=list(contract_verdict.get("missing_fields") or []),
)
if guard_payload:
answer_text = str(guard_payload.get("answer") or answer_text)
used_sources = list(guard_payload.get("sources") or [])
answer_mode = str(guard_payload.get("answer_mode") or answer_mode or ANSWER_MODE_BLOCKED)
max_score = min(float(max_score or 0.0), 0.35)
if "system_context" not in final_domains:
final_domains.append("system_context")
tool_trace.append(
{
"step": 100,
"tool": (
"external_answer_contract_guard"
if effective_source_policy == "external"
else "internal_answer_contract_guard"
),
"reason": str(
guard_payload.get("reason_code")
or (
"external_answer_contract_violation"
if effective_source_policy == "external"
else "internal_answer_contract_violation"
)
),
}
)
grounding_verdict = GroundingVerdict(
status=str(
guard_payload.get("grounding_status")
or (GROUNDING_STATUS_INSUFFICIENT_EVIDENCE if effective_source_policy == "external" else GROUNDING_STATUS_BLOCKED)
),
missing_fields=list(guard_payload.get("missing_fields") or []),
reason_code=str(
guard_payload.get("reason_code")
or (
"external_answer_contract_violation"
if effective_source_policy == "external"
else "internal_answer_contract_violation"
)
),
)
inject_runtime_summaries = getattr(self, "_inject_internal_query_runtime_summaries", None)
if callable(inject_runtime_summaries):
retrieval_metrics = inject_runtime_summaries(
retrieval_metrics=retrieval_metrics,
query_semantics=query_semantics,
)
runtime_fields = self._build_qa_record_runtime_fields( runtime_fields = self._build_qa_record_runtime_fields(
answer=answer_text, answer=answer_text,
sources=used_sources, sources=used_sources,
@ -455,6 +817,7 @@ class AdaptiveRAGAnswerRetrievalMixin:
anchor_kind=None, anchor_kind=None,
anchor_ref=None, anchor_ref=None,
routing_snapshot=routing_snapshot, routing_snapshot=routing_snapshot,
route_decision=normalized_route_decision,
), ),
**runtime_fields, **runtime_fields,
) )
@ -473,6 +836,15 @@ class AdaptiveRAGAnswerRetrievalMixin:
rewrite_meta=rewrite_meta, rewrite_meta=rewrite_meta,
sources=used_sources, sources=used_sources,
) )
await self.db.flush()
persist_route_memory = getattr(self, "_persist_query_route_memory", None)
if callable(persist_route_memory):
await persist_route_memory(
qa_record_id=int(qa.id) if getattr(qa, "id", None) is not None else None,
question_text=clean_q,
project_id=project_id,
route_decision=normalized_route_decision,
)
await self.db.commit() await self.db.commit()
await self.db.refresh(qa) await self.db.refresh(qa)
await self.context_cache.append_session_messages( await self.context_cache.append_session_messages(
@ -510,4 +882,5 @@ class AdaptiveRAGAnswerRetrievalMixin:
sources_used=runtime_fields["sources_used"], sources_used=runtime_fields["sources_used"],
evidence_pack=runtime_fields["evidence_pack"], evidence_pack=runtime_fields["evidence_pack"],
prompt_variant_id=prompt_variant_id, prompt_variant_id=prompt_variant_id,
route_decision=normalized_route_decision,
) )

View File

@ -8,16 +8,22 @@ from ..ai_pmo.workflow_semantic_planner import WorkflowSemanticPlanner
class AdaptiveRAGCoreIntentMixin: class AdaptiveRAGCoreIntentMixin:
_SEMANTIC_INTENTS = {"question", "statement", "mixed"} _SEMANTIC_INTENTS = {"question", "statement", "mixed"}
_SEMANTIC_QUERY_TYPES = { _SEMANTIC_QUERY_TYPES = {
QUERY_TYPE_EXISTENCE,
QUERY_TYPE_COUNT, QUERY_TYPE_COUNT,
QUERY_TYPE_LIST, QUERY_TYPE_LIST,
QUERY_TYPE_STATUS, QUERY_TYPE_STATUS,
QUERY_TYPE_EXPLAIN, QUERY_TYPE_EXPLAIN,
QUERY_TYPE_DIAGNOSE,
QUERY_TYPE_COMPARE,
QUERY_TYPE_UNKNOWN, QUERY_TYPE_UNKNOWN,
} }
_SEMANTIC_RETRIEVAL_DOMAINS = ("db", "graph", "doc", "feishu", "pmflow", "url") _SEMANTIC_RETRIEVAL_DOMAINS = ("db", "graph", "doc", "feishu", "pmflow", "url")
_INTERNAL_RETRIEVAL_DOMAINS = ("db", "graph", "doc", "feishu", "pmflow")
_EXTERNAL_RETRIEVAL_DOMAINS = ("url",)
_SEMANTIC_SUBJECT_FOCI = {"people", "project", "task", "signal", "charter", "workload", "feature", "unknown"} _SEMANTIC_SUBJECT_FOCI = {"people", "project", "task", "signal", "charter", "workload", "feature", "unknown"}
_ALLOWED_SCOPE_VALUES = {"none", "global", "project"} _ALLOWED_SCOPE_VALUES = {"none", "global", "project"}
_SEMANTIC_FEATURE_FOCI = {"none", "purpose", "entry", "catalog"} _SEMANTIC_FEATURE_FOCI = {"none", "purpose", "entry", "catalog"}
_SOURCE_POLICIES = {"internal", "external", "uncertain"}
def __init__(self, db: AsyncSession, *, tenant_id: int) -> None: def __init__(self, db: AsyncSession, *, tenant_id: int) -> None:
self.db = db self.db = db
@ -118,8 +124,43 @@ class AdaptiveRAGCoreIntentMixin:
"routing_decision": routing_decision or None, "routing_decision": routing_decision or None,
"routing_config_version": routing_config_version or None, "routing_config_version": routing_config_version or None,
} }
query_route = payload.get("query_route") if isinstance(payload.get("query_route"), dict) else {}
if query_route:
normalized["query_route"] = {
"route_mode": str(query_route.get("route_mode") or "").strip().lower() or None,
"effective_source_policy": str(query_route.get("effective_source_policy") or "").strip().lower() or None,
"decision_origin": str(query_route.get("decision_origin") or "").strip() or None,
"reply_contract": (
dict(query_route.get("reply_contract") or {})
if isinstance(query_route.get("reply_contract"), dict)
else {}
),
"query_profile": (
dict(query_route.get("query_profile") or {})
if isinstance(query_route.get("query_profile"), dict)
else {}
),
"probe_snapshot_summary": (
dict(query_route.get("probe_snapshot_summary") or {})
if isinstance(query_route.get("probe_snapshot_summary"), dict)
else {}
),
}
return {key: value for key, value in normalized.items() if value is not None} return {key: value for key, value in normalized.items() if value is not None}
def _merge_query_route_audit_into_routing_snapshot(
self,
*,
routing_snapshot: Optional[dict[str, Any]],
route_decision: Optional[dict[str, Any]],
) -> dict[str, Any]:
normalized_snapshot = self._normalize_routing_snapshot(routing_snapshot)
if hasattr(self, "_build_query_route_audit_snapshot"):
query_route = self._build_query_route_audit_snapshot(route_decision=route_decision)
if query_route:
normalized_snapshot["query_route"] = query_route
return normalized_snapshot
def _peek_last_answer_routing_snapshot(self) -> dict[str, Any]: def _peek_last_answer_routing_snapshot(self) -> dict[str, Any]:
payload = getattr(self.llm, "_last_ai_call_summary", None) payload = getattr(self.llm, "_last_ai_call_summary", None)
if not isinstance(payload, dict): if not isinstance(payload, dict):
@ -137,9 +178,13 @@ class AdaptiveRAGCoreIntentMixin:
anchor_kind: Optional[str] = None, anchor_kind: Optional[str] = None,
anchor_ref: Optional[str] = None, anchor_ref: Optional[str] = None,
routing_snapshot: Optional[dict[str, Any]] = None, routing_snapshot: Optional[dict[str, Any]] = None,
route_decision: Optional[dict[str, Any]] = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
context = self._get_active_external_qa_context() context = self._get_active_external_qa_context()
normalized_snapshot = self._normalize_routing_snapshot(routing_snapshot) or None normalized_snapshot = self._merge_query_route_audit_into_routing_snapshot(
routing_snapshot=routing_snapshot,
route_decision=route_decision,
) or None
return { return {
"source_message_id": context.get("source_message_id"), "source_message_id": context.get("source_message_id"),
"source_chat_id": context.get("source_chat_id"), "source_chat_id": context.get("source_chat_id"),
@ -270,6 +315,15 @@ class AdaptiveRAGCoreIntentMixin:
except Exception: except Exception:
return float(default) return float(default)
@staticmethod
def _safe_int_value(value: Any) -> Optional[int]:
try:
if value is None:
return None
return int(value)
except Exception:
return None
@staticmethod @staticmethod
def _parse_json_object_safe(raw: Any) -> dict[str, Any]: def _parse_json_object_safe(raw: Any) -> dict[str, Any]:
if isinstance(raw, dict): if isinstance(raw, dict):
@ -323,12 +377,94 @@ class AdaptiveRAGCoreIntentMixin:
return normalized return normalized
return "none" return "none"
@classmethod
def _normalize_source_policy(cls, value: Any) -> str:
normalized = str(value or "").strip().lower()
if normalized in cls._SOURCE_POLICIES:
return normalized
return "uncertain"
@classmethod
def _normalize_query_profile_object(cls, value: Any) -> str:
normalized = re.sub(r"[^a-z0-9:_\-]", "", str(value or "").strip().lower())
return normalized[:48]
@classmethod
def _normalize_query_profile(cls, payload: Any) -> dict[str, Any]:
raw = payload if isinstance(payload, dict) else {}
referenced_objects: list[str] = []
for item in list(raw.get("referenced_objects") or []):
normalized = cls._normalize_query_profile_object(item)
if not normalized:
continue
if normalized not in referenced_objects:
referenced_objects.append(normalized)
if not referenced_objects:
referenced_objects = ["unknown"]
try:
confidence = max(0.0, min(1.0, float(raw.get("confidence") or 0.0)))
except Exception:
confidence = 0.0
return {
"needs_private_context": bool(raw.get("needs_private_context")),
"needs_freshness": bool(raw.get("needs_freshness")),
"referenced_objects": referenced_objects,
"confidence": confidence,
}
@staticmethod
def _normalize_semantic_object_mentions(payload: Any) -> list[str]:
rows = payload if isinstance(payload, list) else []
mentions: list[str] = []
for item in rows:
text = str(item or "").strip(" ,。,.!??::;;“”\"'()()")
if len(text) >= 2 and text not in mentions:
mentions.append(text[:32])
return mentions[:4]
@classmethod
def _apply_source_policy_to_domains(
cls,
*,
domains: list[str],
effective_source_policy: str,
) -> list[str]:
normalized_policy = cls._normalize_source_policy(effective_source_policy)
if normalized_policy == "internal":
internal_domains = [item for item in domains if item in cls._INTERNAL_RETRIEVAL_DOMAINS]
if internal_domains:
return internal_domains
return list(cls._INTERNAL_RETRIEVAL_DOMAINS)
external_domains = [item for item in domains if item in cls._EXTERNAL_RETRIEVAL_DOMAINS]
if external_domains:
return external_domains
return list(cls._EXTERNAL_RETRIEVAL_DOMAINS)
@classmethod
def _should_normalize_to_external_contract(
cls,
*,
source_policy: str,
effective_source_policy: str,
parsed_domains: list[str],
query_profile: dict[str, Any],
) -> bool:
if source_policy != "uncertain" or effective_source_policy != "uncertain":
return False
normalized_domains = [str(item or "").strip().lower() for item in list(parsed_domains or []) if str(item or "").strip()]
if not normalized_domains:
return False
if any(item not in cls._EXTERNAL_RETRIEVAL_DOMAINS for item in normalized_domains):
return False
return bool(query_profile.get("needs_freshness")) and not bool(query_profile.get("needs_private_context"))
async def _resolve_query_semantics( async def _resolve_query_semantics(
self, self,
*, *,
message: str, message: str,
source_system: str, source_system: str,
metadata: Optional[dict[str, Any]] = None, metadata: Optional[dict[str, Any]] = None,
source_policy_hint: Optional[str] = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
text = str(message or "").strip() text = str(message or "").strip()
default_domains = list(self._SEMANTIC_RETRIEVAL_DOMAINS) default_domains = list(self._SEMANTIC_RETRIEVAL_DOMAINS)
@ -339,7 +475,14 @@ class AdaptiveRAGCoreIntentMixin:
"question_part": "", "question_part": "",
"statement_part": "", "statement_part": "",
"query_type": QUERY_TYPE_UNKNOWN, "query_type": QUERY_TYPE_UNKNOWN,
"object_mentions": [],
"retrieval_domains": default_domains, "retrieval_domains": default_domains,
"source_policy": "uncertain",
"effective_source_policy": "uncertain",
"source_reason": "empty_message",
"decision_origin": "empty_message",
"fallback_taken": False,
"query_profile": self._normalize_query_profile(None),
"confidence": 1.0, "confidence": 1.0,
} }
@ -402,6 +545,50 @@ class AdaptiveRAGCoreIntentMixin:
query_type = str(parsed.get("query_type") or QUERY_TYPE_UNKNOWN).strip().lower() query_type = str(parsed.get("query_type") or QUERY_TYPE_UNKNOWN).strip().lower()
if query_type not in self._SEMANTIC_QUERY_TYPES: if query_type not in self._SEMANTIC_QUERY_TYPES:
query_type = QUERY_TYPE_UNKNOWN query_type = QUERY_TYPE_UNKNOWN
object_mentions = self._normalize_semantic_object_mentions(parsed.get("object_mentions"))
subject_focus = self._normalize_semantic_subject_focus(parsed.get("subject_focus"))
scope_hint = self._normalize_semantic_scope_hint(parsed.get("scope_hint"))
feature_focus = self._normalize_semantic_feature_focus(parsed.get("feature_focus"))
parsed_domains = self._normalize_semantic_domains(parsed.get("retrieval_domains"))
raw_source_policy = self._normalize_source_policy(parsed.get("source_policy"))
raw_effective_source_policy = self._normalize_source_policy(parsed.get("effective_source_policy"))
query_profile = self._normalize_query_profile(parsed.get("query_profile"))
normalized_hint = self._normalize_source_policy(source_policy_hint)
fallback_taken = False
if normalized_hint in {"internal", "external"}:
source_policy = normalized_hint
effective_source_policy = normalized_hint
source_reason = "upstream_source_policy_hint"
decision_origin = "upstream_hint"
fallback_taken = raw_source_policy not in {"uncertain", normalized_hint}
else:
source_policy = raw_source_policy
if raw_effective_source_policy in {"internal", "external"}:
effective_source_policy = raw_effective_source_policy
elif source_policy in {"internal", "external"}:
effective_source_policy = source_policy
else:
effective_source_policy = "uncertain"
source_reason = "query_semantics_contract"
decision_origin = "query_semantics_contract"
if self._should_normalize_to_external_contract(
source_policy=source_policy,
effective_source_policy=effective_source_policy,
parsed_domains=parsed_domains,
query_profile=query_profile,
):
source_policy = "external"
effective_source_policy = "external"
source_reason = "query_semantics_url_only_external_contract"
decision_origin = "query_semantics_contract_normalized"
if effective_source_policy == "uncertain":
retrieval_domains = list(parsed_domains)
else:
retrieval_domains = self._apply_source_policy_to_domains(
domains=parsed_domains,
effective_source_policy=effective_source_policy,
)
confidence = self._safe_float(parsed.get("confidence"), planner_confidence) confidence = self._safe_float(parsed.get("confidence"), planner_confidence)
normalized_message_intent = planner_message_intent or intent normalized_message_intent = planner_message_intent or intent
@ -411,10 +598,17 @@ class AdaptiveRAGCoreIntentMixin:
"question_part": question_part, "question_part": question_part,
"statement_part": statement_part, "statement_part": statement_part,
"query_type": query_type, "query_type": query_type,
"retrieval_domains": self._normalize_semantic_domains(parsed.get("retrieval_domains")), "object_mentions": object_mentions,
"subject_focus": self._normalize_semantic_subject_focus(parsed.get("subject_focus")), "retrieval_domains": retrieval_domains,
"scope_hint": self._normalize_semantic_scope_hint(parsed.get("scope_hint")), "subject_focus": subject_focus,
"feature_focus": self._normalize_semantic_feature_focus(parsed.get("feature_focus")), "scope_hint": scope_hint,
"feature_focus": feature_focus,
"query_profile": query_profile,
"source_policy": source_policy,
"effective_source_policy": effective_source_policy,
"source_reason": source_reason,
"decision_origin": decision_origin,
"fallback_taken": fallback_taken,
"confidence": max(0.0, min(1.0, confidence)), "confidence": max(0.0, min(1.0, confidence)),
} }

View File

@ -29,81 +29,7 @@ class AdaptiveRAGDeepGuardAnswerMixin:
"engine_process_query", "engine_process_query",
"runtime_state_query", "runtime_state_query",
} }
_PROJECT_CONTEXT_POSITIVE_MARKERS = ( _MODEL_TOKEN_RE = re.compile(
"这个项目",
"当前项目",
"本项目",
"我们项目",
"项目所在行业",
"对项目的影响",
"对这个项目的影响",
"对当前项目的影响",
"对我们项目的影响",
"结合项目",
"结合当前项目",
"落到项目",
)
_PROJECT_CONTEXT_NEGATIVE_MARKERS = (
"不要提项目",
"别提项目",
"不要结合项目",
"不用结合项目",
"不需要结合项目",
"先别结合项目",
"不要绑定项目",
"别绑定项目",
)
_EXTERNAL_FACT_FRESHNESS_MARKERS = (
"最新",
"目前",
"当前",
"截至现在",
"截至当前",
"截止当前",
"今天",
"现在",
"最近",
"出到什么",
"发布了吗",
"发布到",
"版本",
"最高模型",
)
_MODEL_ENTITY_MARKERS = (
"codex",
"gpt",
"chatgpt",
"claude",
"deepseek",
"qwen",
"gemini",
"copilot",
"grok",
"kimi",
"llama",
"openai",
"anthropic",
)
_INTERNAL_MODEL_CONFIG_MARKERS = (
"ai-config",
"/ai-config",
"模型配置",
"配置项",
"配置值",
"路由规则",
"初始化参数",
"llm服务",
"llm 服务",
"运行时",
"当前系统",
"我们系统",
"pmflow",
"本系统",
"接入的模型",
"接的是",
"用的是哪个模型配置",
)
_EXTERNAL_MODEL_TOKEN_RE = re.compile(
r"\b(?:GPT|gpt)[- ]?\d(?:\.\d)?(?:[- ]?(?:codex|turbo|mini|nano))?\b", r"\b(?:GPT|gpt)[- ]?\d(?:\.\d)?(?:[- ]?(?:codex|turbo|mini|nano))?\b",
flags=re.IGNORECASE, flags=re.IGNORECASE,
) )
@ -115,46 +41,6 @@ class AdaptiveRAGDeepGuardAnswerMixin:
return text return text
return None return None
@classmethod
def _question_explicitly_requests_project_context(
cls,
*,
question: str,
project_id: Optional[int],
) -> bool:
if project_id is None:
return False
normalized = re.sub(r"\s+", "", str(question or "")).strip().lower()
if not normalized:
return False
if any(marker in normalized for marker in cls._PROJECT_CONTEXT_NEGATIVE_MARKERS):
return False
return any(marker in normalized for marker in cls._PROJECT_CONTEXT_POSITIVE_MARKERS)
@classmethod
def _looks_like_external_freshness_question(cls, question: str) -> bool:
normalized = re.sub(r"\s+", "", str(question or "")).strip().lower()
if not normalized:
return False
return any(marker in normalized for marker in cls._EXTERNAL_FACT_FRESHNESS_MARKERS)
@classmethod
def _looks_like_internal_model_config_question(cls, question: str) -> bool:
normalized = re.sub(r"\s+", "", str(question or "")).strip().lower()
if not normalized:
return False
return any(marker in normalized for marker in cls._INTERNAL_MODEL_CONFIG_MARKERS)
@classmethod
def _looks_like_model_fact_question(cls, question: str) -> bool:
normalized = re.sub(r"\s+", "", str(question or "")).strip().lower()
if not normalized:
return False
has_entity = any(marker in normalized for marker in cls._MODEL_ENTITY_MARKERS)
if not has_entity:
return False
return "模型" in normalized or cls._looks_like_external_freshness_question(question)
@classmethod @classmethod
def _normalize_external_model_evidence_text(cls, text: str) -> str: def _normalize_external_model_evidence_text(cls, text: str) -> str:
normalized = str(text or "") normalized = str(text or "")
@ -175,7 +61,7 @@ class AdaptiveRAGDeepGuardAnswerMixin:
search_windows = [latest_match.group(1)] if latest_match else [] search_windows = [latest_match.group(1)] if latest_match else []
search_windows.append(normalized) search_windows.append(normalized)
for window in search_windows: for window in search_windows:
match = cls._EXTERNAL_MODEL_TOKEN_RE.search(str(window or "")) match = cls._MODEL_TOKEN_RE.search(str(window or ""))
if not match: if not match:
continue continue
token = re.sub(r"\s+", "-", str(match.group(0) or "").strip()) token = re.sub(r"\s+", "-", str(match.group(0) or "").strip())
@ -191,8 +77,7 @@ class AdaptiveRAGDeepGuardAnswerMixin:
retrieved: list[RetrievedItem], retrieved: list[RetrievedItem],
scene_semantics: Optional[dict[str, Any]] = None, scene_semantics: Optional[dict[str, Any]] = None,
) -> Optional[str]: ) -> Optional[str]:
if not cls._looks_like_model_fact_question(question) or not cls._looks_like_external_freshness_question(question): _ = question
return None
normalized_scene = scene_semantics if isinstance(scene_semantics, dict) else {} normalized_scene = scene_semantics if isinstance(scene_semantics, dict) else {}
scene_key = str(normalized_scene.get("scene_key") or "").strip().lower() scene_key = str(normalized_scene.get("scene_key") or "").strip().lower()
if scene_key and scene_key not in {"industry_trend", "external_trend"}: if scene_key and scene_key not in {"industry_trend", "external_trend"}:
@ -230,6 +115,141 @@ class AdaptiveRAGDeepGuardAnswerMixin:
) )
return None return None
async def _build_internal_model_config_fast_answer(
self,
*,
question: str,
query_semantics: Optional[dict[str, Any]] = None,
) -> Optional[dict[str, Any]]:
_ = question
semantics = query_semantics if isinstance(query_semantics, dict) else {}
source_policy = str(
semantics.get("effective_source_policy") or semantics.get("source_policy") or ""
).strip().lower()
if source_policy and source_policy != "internal":
return None
operation_contract = semantics.get("operation_contract") if isinstance(semantics.get("operation_contract"), dict) else {}
query_type = str(
operation_contract.get("query_type")
or semantics.get("query_type")
or QUERY_TYPE_UNKNOWN
).strip().lower()
if query_type not in {QUERY_TYPE_STATUS, QUERY_TYPE_EXPLAIN}:
return None
query_profile = semantics.get("query_profile") if isinstance(semantics.get("query_profile"), dict) else {}
referenced_objects = {
str(item).strip().lower()
for item in list(query_profile.get("referenced_objects") or [])
if str(item).strip()
}
if not referenced_objects.intersection({"system_config", "model_routing"}):
return None
target_resolution = (
semantics.get("target_resolution")
if isinstance(semantics.get("target_resolution"), dict)
else {}
)
target_entity_type = str(target_resolution.get("target_entity_type") or "").strip().lower()
if target_entity_type and target_entity_type != "config":
return None
get_bundle = getattr(self.llm, "_get_or_init_bundle", None)
resolve_active_engine = getattr(self.llm, "_resolve_active_engine_from_bundle", None)
resolve_engine_for_call = getattr(self.llm, "_resolve_engine_for_call", None)
if not callable(get_bundle) or not callable(resolve_active_engine) or not callable(resolve_engine_for_call):
return None
try:
_row, bundle = await get_bundle(self.db, tenant_id=self.tenant_id)
bundle = dict(bundle) if isinstance(bundle, dict) else {}
active_engine = dict(resolve_active_engine(bundle) or {})
normal_engine = dict(
await resolve_engine_for_call(
self.db,
tenant_id=self.tenant_id,
function_name="adaptive_rag_answer_normal",
)
or {}
)
deep_engine = dict(
await resolve_engine_for_call(
self.db,
tenant_id=self.tenant_id,
function_name="adaptive_rag_answer_deep",
)
or {}
)
except Exception:
return None
settings = bundle.get("settings") if isinstance(bundle.get("settings"), dict) else {}
local_kb_enabled = bool(settings.get("enable_local_kb_qa"))
def _engine_summary(engine: dict[str, Any]) -> str:
name = str(engine.get("name") or "unknown").strip() or "unknown"
provider = str(engine.get("provider") or "unknown").strip() or "unknown"
model = str(engine.get("model") or "unknown").strip() or "unknown"
routing_meta = engine.get("_routing_meta") if isinstance(engine.get("_routing_meta"), dict) else {}
tier = (
str(routing_meta.get("executed_tier") or routing_meta.get("planned_tier") or "")
.strip()
.lower()
)
if tier:
return f"{name}(provider={provider}, model={model}, tier={tier})"
return f"{name}(provider={provider}, model={model})"
active_summary = _engine_summary(active_engine)
normal_summary = _engine_summary(normal_engine)
deep_summary = _engine_summary(deep_engine)
answer_lines = [
f"当前活动引擎是 {active_summary}。",
f"`adaptive_rag_answer_normal` 当前走 {normal_summary}。",
f"`adaptive_rag_answer_deep` 当前走 {deep_summary}。",
f"`enable_local_kb_qa={'true' if local_kb_enabled else 'false'}`。",
]
active_engine_id = str(bundle.get("active_engine_id") or "").strip() or None
return {
"answer": "\n".join(answer_lines),
"sources": [
{
"source": "db",
"label": "Integration[llm_engine]",
"score": 0.99,
"metadata": {
"integration_type": "llm_engine",
"active_engine_id": active_engine_id,
"provider": str(active_engine.get("provider") or ""),
"model": str(active_engine.get("model") or ""),
},
},
{
"source": "db",
"label": "Integration[model_routing:adaptive_rag_answer_normal]",
"score": 0.98,
"metadata": {
"function_name": "adaptive_rag_answer_normal",
"provider": str(normal_engine.get("provider") or ""),
"model": str(normal_engine.get("model") or ""),
},
},
{
"source": "db",
"label": "Integration[model_routing:adaptive_rag_answer_deep]",
"score": 0.98,
"metadata": {
"function_name": "adaptive_rag_answer_deep",
"provider": str(deep_engine.get("provider") or ""),
"model": str(deep_engine.get("model") or ""),
},
},
],
"query_type": QUERY_TYPE_STATUS,
"confidence": 0.99,
"reason_code": "internal_model_config_direct",
"prompt_variant_id": "internal_model_config_direct",
}
@classmethod @classmethod
def _apply_consultant_mode_preference( def _apply_consultant_mode_preference(
cls, cls,
@ -529,56 +549,54 @@ class AdaptiveRAGDeepGuardAnswerMixin:
message_intent: Optional[str], message_intent: Optional[str],
query_type: Optional[str], query_type: Optional[str],
conversation_text: str, conversation_text: str,
source_policy: Optional[str] = None,
project_id: Optional[int] = None,
query_semantics: Optional[dict[str, Any]] = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
_ = (effective_question, conversation_text)
fallback = self._default_answer_scene_semantics( fallback = self._default_answer_scene_semantics(
asked_via=asked_via, asked_via=asked_via,
message_intent=message_intent, message_intent=message_intent,
) )
prompt = PROMPT_RAG_ANSWER_SCENE_SEMANTICS.format( semantics = query_semantics if isinstance(query_semantics, dict) else {}
asked_via=str(asked_via or "").strip().lower() or "pmflow", normalized_source_policy = str(source_policy or "").strip().lower()
message_intent=str(message_intent or "").strip().lower() or "unknown", if normalized_source_policy == "external":
query_type=str(query_type or QUERY_TYPE_UNKNOWN).strip().lower() or QUERY_TYPE_UNKNOWN, has_project_scope = (
conversation_tail=self._truncate_compact_text(conversation_text, limit=1200), project_id is not None
question=str(question or "").strip()[:800], and str(semantics.get("scope_hint") or "none").strip().lower() == "project"
effective_question=str(effective_question or question or "").strip()[:1000],
).strip()
parsed: dict[str, Any] = {}
try:
raw = await self.llm.generate(
prompt,
db=self.db,
tenant_id=self.tenant_id,
function_name="rag_answer_scene_semantics",
temperature=0.0,
max_tokens=260,
use_prompt_profile=False,
) )
parser = getattr(self, "_parse_json_object_safe", None) scene_key = "external_trend" if has_project_scope else "industry_trend"
parsed = parser(raw) if callable(parser) else {} return {
parsed = dict(parsed) if isinstance(parsed, dict) else {} "scene_key": scene_key,
except Exception: "snapshot_focus": "none",
parsed = {} "deep_intent": "none",
"confidence": 1.0,
"reason": "source_policy:external",
}
scene_key = str(parsed.get("scene_key") or fallback["scene_key"]).strip().lower() subject_focus = str(semantics.get("subject_focus") or "unknown").strip().lower()
feature_focus = str(semantics.get("feature_focus") or "none").strip().lower()
scope_hint = str(semantics.get("scope_hint") or "none").strip().lower()
normalized_query_type = str(query_type or QUERY_TYPE_UNKNOWN).strip().lower()
scene_key = str(fallback["scene_key"])
if subject_focus == "workload":
scene_key = "member_workload"
elif subject_focus == "project" and normalized_query_type == QUERY_TYPE_STATUS:
scene_key = "project_status"
if scene_key not in self._ANSWER_SCENE_KEYS: if scene_key not in self._ANSWER_SCENE_KEYS:
scene_key = str(fallback["scene_key"]) scene_key = str(fallback["scene_key"])
snapshot_focus = str(parsed.get("snapshot_focus") or fallback["snapshot_focus"]).strip().lower() snapshot_focus = "feature" if (subject_focus == "feature" or feature_focus != "none") else str(fallback["snapshot_focus"])
if snapshot_focus not in self._SNAPSHOT_FOCI: if snapshot_focus not in self._SNAPSHOT_FOCI:
snapshot_focus = "none" snapshot_focus = str(fallback["snapshot_focus"])
confidence = 0.82 if scene_key != str(fallback["scene_key"]) or snapshot_focus != str(fallback["snapshot_focus"]) or scope_hint == "project" else 0.6
deep_intent = str(parsed.get("deep_intent") or fallback["deep_intent"]).strip().lower()
if deep_intent not in self._DEEP_INTENTS:
deep_intent = "none"
safe_float = getattr(self, "_safe_float", None)
confidence = safe_float(parsed.get("confidence"), 0.0) if callable(safe_float) else 0.0
return { return {
"scene_key": scene_key, "scene_key": scene_key,
"snapshot_focus": snapshot_focus, "snapshot_focus": snapshot_focus,
"deep_intent": deep_intent, "deep_intent": str(fallback["deep_intent"]),
"confidence": confidence, "confidence": confidence,
"reason": str(parsed.get("reason") or "").strip(), "reason": "query_semantics_mapping",
} }
@staticmethod @staticmethod
@ -655,18 +673,7 @@ class AdaptiveRAGDeepGuardAnswerMixin:
) )
normalized_scene_key = str(scene_semantics.get("scene_key") or "default").strip().lower() normalized_scene_key = str(scene_semantics.get("scene_key") or "default").strip().lower()
normalized_message_intent = str(message_intent or "").strip().lower() normalized_message_intent = str(message_intent or "").strip().lower()
explicit_project_context = self._question_explicitly_requests_project_context( if normalized_scene_key == "external_trend" and project_id is None:
question=question,
project_id=project_id,
)
internal_model_config_question = self._looks_like_internal_model_config_question(question)
model_fact_question = self._looks_like_model_fact_question(question)
freshness_question = self._looks_like_external_freshness_question(question)
if normalized_scene_key == "ai_pmo" and model_fact_question and freshness_question and not internal_model_config_question:
normalized_scene_key = "external_trend" if explicit_project_context else "industry_trend"
if normalized_scene_key == "external_trend" and not explicit_project_context:
normalized_scene_key = "industry_trend" normalized_scene_key = "industry_trend"
if normalized_scene_key == "member_workload": if normalized_scene_key == "member_workload":
@ -1170,6 +1177,14 @@ class AdaptiveRAGDeepGuardAnswerMixin:
message_intent=message_intent, message_intent=message_intent,
query_type=query_type, query_type=query_type,
conversation_text=conversation_text, conversation_text=conversation_text,
source_policy=str(
(query_semantics or {}).get("effective_source_policy")
or (query_semantics or {}).get("source_policy")
or ""
).strip().lower()
or None,
project_id=project_id,
query_semantics=query_semantics,
) )
member_workload_direct_answer = await self._build_member_workload_direct_answer( member_workload_direct_answer = await self._build_member_workload_direct_answer(
question, question,

View File

@ -167,7 +167,7 @@ class AdaptiveRAGDeepGuardRulesMixin:
continue continue
source = str(row.get("source") or "").strip().lower() source = str(row.get("source") or "").strip().lower()
metadata = row.get("metadata") if isinstance(row.get("metadata"), dict) else {} metadata = row.get("metadata") if isinstance(row.get("metadata"), dict) else {}
if source == "url": if source in {"url", "mcp_search"}:
return True return True
for key in ("target_url", "file_url", "doc_url", "url"): for key in ("target_url", "file_url", "doc_url", "url"):
value = str(metadata.get(key) or "").strip().lower() value = str(metadata.get(key) or "").strip().lower()

View File

@ -26,14 +26,33 @@ class AdaptiveRAGDeterministicQueryCoreMixin:
project_id=project_id, project_id=project_id,
query_semantics=query_semantics, query_semantics=query_semantics,
) )
if q_type == QUERY_TYPE_EXISTENCE:
existence = await self._try_fast_project_existence_answer(
clean_q=question,
project_id=project_id,
query_semantics=query_semantics,
)
if existence:
existence["query_type"] = QUERY_TYPE_EXISTENCE
existence.setdefault("confidence", 0.96)
existence.setdefault("missing_fields", [])
return existence
if q_type == QUERY_TYPE_STATUS: if q_type == QUERY_TYPE_STATUS:
created_at = await self._try_fast_project_created_at_answer(clean_q=question, project_id=project_id) created_at = await self._try_fast_project_created_at_answer(
clean_q=question,
project_id=project_id,
query_semantics=query_semantics,
)
if created_at: if created_at:
created_at["query_type"] = QUERY_TYPE_STATUS created_at["query_type"] = QUERY_TYPE_STATUS
created_at.setdefault("confidence", 0.97) created_at.setdefault("confidence", 0.97)
created_at.setdefault("missing_fields", []) created_at.setdefault("missing_fields", [])
return created_at return created_at
progress = await self._try_fast_project_progress_answer(clean_q=question, project_id=project_id) progress = await self._try_fast_project_progress_answer(
clean_q=question,
project_id=project_id,
query_semantics=query_semantics,
)
if progress: if progress:
progress["query_type"] = QUERY_TYPE_STATUS progress["query_type"] = QUERY_TYPE_STATUS
progress.setdefault("confidence", 0.98) progress.setdefault("confidence", 0.98)
@ -56,6 +75,7 @@ class AdaptiveRAGDeterministicQueryCoreMixin:
*, *,
question: str, question: str,
project_id: Optional[int], project_id: Optional[int],
query_semantics: Optional[dict[str, Any]] = None,
) -> tuple[Optional[int], Optional[str], str]: ) -> tuple[Optional[int], Optional[str], str]:
if project_id is not None: if project_id is not None:
row = ( row = (
@ -72,18 +92,50 @@ class AdaptiveRAGDeterministicQueryCoreMixin:
return (None, None, "") return (None, None, "")
return (int(row.id), str(row.title or row.name or f"项目#{int(row.id)}"), "") return (int(row.id), str(row.title or row.name or f"项目#{int(row.id)}"), "")
semantics = query_semantics if isinstance(query_semantics, dict) else {}
target_resolution = semantics.get("target_resolution") if isinstance(semantics.get("target_resolution"), dict) else {}
target_ref = str(target_resolution.get("target_ref") or "").strip()
if target_ref.startswith("project:"):
try:
resolved_project_id = int(target_ref.split(":", 1)[1])
except Exception:
resolved_project_id = None
if resolved_project_id is not None:
row = (
await self.db.execute(
select(Project).where(
and_(
Project.tenant_id == self.tenant_id,
Project.id == resolved_project_id,
)
)
)
).scalar_one_or_none()
if row is not None:
return (
int(row.id),
str(row.title or row.name or f"项目#{int(row.id)}"),
str(target_resolution.get("resolved_label") or row.title or row.name or "")[:32],
)
direct_project_id, direct_project_name = await self._match_project_name_in_question(question=question) direct_project_id, direct_project_name = await self._match_project_name_in_question(question=question)
if direct_project_id is not None and direct_project_name: if direct_project_id is not None and direct_project_name:
return (int(direct_project_id), str(direct_project_name), str(direct_project_name)[:24]) return (int(direct_project_id), str(direct_project_name), str(direct_project_name)[:24])
keyword = self._extract_project_lookup_keyword(question) term_collector = getattr(self, "_collect_lookup_terms", None)
terms = self._extract_query_terms(question, preferred=keyword) lookup_terms = term_collector(
for term in terms: question=question,
matched = await self._search_projects_by_keyword(keyword=term, project_id=None, limit=5) query_semantics=semantics,
target_resolution=target_resolution,
limit=4,
) if callable(term_collector) else []
searcher = getattr(self, "_search_projects_by_lookup_terms", None)
if callable(searcher) and lookup_terms:
matched, used_term = await searcher(query_terms=lookup_terms, project_id=None, limit=5)
if matched: if matched:
target = matched[0] target = matched[0]
return (int(target.id), str(target.title or target.name or f"项目#{int(target.id)}"), term) return (int(target.id), str(target.title or target.name or f"项目#{int(target.id)}"), used_term[:24])
return (None, None, keyword[:24] if keyword else "") return (None, None, str(lookup_terms[0] if lookup_terms else "")[:24])
@staticmethod @staticmethod
def _normalize_project_subject_text(text: str) -> str: def _normalize_project_subject_text(text: str) -> str:
@ -92,11 +144,6 @@ class AdaptiveRAGDeterministicQueryCoreMixin:
return "" return ""
clean = clean.replace("—", "-").replace("_", " ") clean = clean.replace("—", "-").replace("_", " ")
clean = re.sub(r"[^\w\u4e00-\u9fff\- ]+", " ", clean) clean = re.sub(r"[^\w\u4e00-\u9fff\- ]+", " ", clean)
clean = re.sub(
r"(我说的是|我说的|请问|请|帮我|一下|这个里面|这里面|里面|这个|那个|一下子|目前|当前|现在|有多少个|多少个|信号源|信号|项目|系统)",
" ",
clean,
)
clean = re.sub(r"[-]+", " ", clean) clean = re.sub(r"[-]+", " ", clean)
clean = re.sub(r"\s+", " ", clean).strip() clean = re.sub(r"\s+", " ", clean).strip()
return clean return clean

View File

@ -16,6 +16,7 @@ class AdaptiveRAGFastAnswerProjectMainMixin:
*, *,
clean_q: str, clean_q: str,
project_id: Optional[int], project_id: Optional[int],
query_semantics: Optional[dict[str, Any]] = None,
) -> Optional[dict[str, Any]]: ) -> Optional[dict[str, Any]]:
question = str(clean_q or "").strip() question = str(clean_q or "").strip()
if not question: if not question:
@ -26,24 +27,16 @@ class AdaptiveRAGFastAnswerProjectMainMixin:
if "项目" not in question and "系统" not in question: if "项目" not in question and "系统" not in question:
return None return None
primary_keyword = self._extract_project_lookup_keyword(question)
query_terms = self._extract_query_terms(question, preferred=primary_keyword)
if not query_terms:
return None
started = time.perf_counter() started = time.perf_counter()
matched: list[Project] = [] target_project_id, target_name, used_keyword = await self._resolve_project_scope_for_question(
used_keyword = query_terms[0] question=question,
for term in query_terms: project_id=project_id,
rows = await self._search_projects_by_keyword(keyword=term, project_id=project_id, limit=3) query_semantics=query_semantics,
if rows: )
matched = rows if target_project_id is None:
used_keyword = term
break
if not matched:
db_query_ms = int((time.perf_counter() - started) * 1000) db_query_ms = int((time.perf_counter() - started) * 1000)
return { return {
"answer": f"未检索到名称或描述包含“{used_keyword}”的项目,无法判断进度。", "answer": f"未检索到可确认进度的项目,无法判断“{question}”对应对象。",
"sources": [ "sources": [
{ {
"source": "db", "source": "db",
@ -56,7 +49,36 @@ class AdaptiveRAGFastAnswerProjectMainMixin:
"response_time_ms": db_query_ms, "response_time_ms": db_query_ms,
} }
target = matched[0] target = (
await self.db.execute(
select(Project).where(
and_(
Project.tenant_id == self.tenant_id,
Project.id == int(target_project_id),
)
)
)
).scalar_one_or_none()
if target is None:
db_query_ms = int((time.perf_counter() - started) * 1000)
return {
"answer": f"未检索到项目#{int(target_project_id)},无法判断进度。",
"sources": [
{
"source": "db",
"label": "Project database",
"score": 0.99,
"metadata": {
"table": "projects",
"project_id": int(target_project_id),
"query_keyword": used_keyword,
"match_count": 0,
},
}
],
"db_query_ms": db_query_ms,
"response_time_ms": db_query_ms,
}
task_rows = ( task_rows = (
await self.db.execute( await self.db.execute(
select(Task).where( select(Task).where(
@ -124,10 +146,6 @@ class AdaptiveRAGFastAnswerProjectMainMixin:
if asks_risk_or_blockers and latest_update is not None: if asks_risk_or_blockers and latest_update is not None:
answer += f" 最近任务更新时间:{latest_update.strftime('%Y-%m-%d %H:%M')}。" answer += f" 最近任务更新时间:{latest_update.strftime('%Y-%m-%d %H:%M')}。"
if len(matched) > 1:
others = "、".join([str(x.title or x.name or f'#{x.id}') for x in matched[1:]])
answer += f" 另有 {len(matched) - 1} 个同名候选项目:{others}。"
return { return {
"answer": answer, "answer": answer,
"sources": [ "sources": [
@ -165,6 +183,7 @@ class AdaptiveRAGFastAnswerProjectMainMixin:
*, *,
clean_q: str, clean_q: str,
project_id: Optional[int], project_id: Optional[int],
query_semantics: Optional[dict[str, Any]] = None,
) -> Optional[dict[str, Any]]: ) -> Optional[dict[str, Any]]:
question = str(clean_q or "").strip() question = str(clean_q or "").strip()
if not question or not self.PROJECT_CREATED_AT_RE.search(question): if not question or not self.PROJECT_CREATED_AT_RE.search(question):
@ -172,22 +191,13 @@ class AdaptiveRAGFastAnswerProjectMainMixin:
if "项目" not in question and "系统" not in question: if "项目" not in question and "系统" not in question:
return None return None
primary_keyword = self._extract_project_lookup_keyword(question)
query_terms = self._extract_query_terms(question, preferred=primary_keyword)
started = time.perf_counter() started = time.perf_counter()
target_project_id, target_name, used_keyword = await self._resolve_project_scope_for_question( target_project_id, target_name, used_keyword = await self._resolve_project_scope_for_question(
question=question, question=question,
project_id=project_id, project_id=project_id,
query_semantics=query_semantics,
) )
if target_project_id is None and query_terms:
for term in query_terms:
rows = await self._search_projects_by_keyword(keyword=term, project_id=project_id, limit=3)
if rows:
target_project_id = int(rows[0].id)
target_name = str(rows[0].title or rows[0].name or f"项目#{int(rows[0].id)}")
used_keyword = term
break
if target_project_id is None: if target_project_id is None:
db_query_ms = int((time.perf_counter() - started) * 1000) db_query_ms = int((time.perf_counter() - started) * 1000)
@ -284,6 +294,46 @@ class AdaptiveRAGFastAnswerProjectMainMixin:
subject_focus = self._query_subject_focus(semantics) subject_focus = self._query_subject_focus(semantics)
scope_hint = self._query_scope_hint(semantics) scope_hint = self._query_scope_hint(semantics)
message_intent = str(semantics.get("message_intent") or semantics.get("intent") or "").strip().lower() message_intent = str(semantics.get("message_intent") or semantics.get("intent") or "").strip().lower()
target_resolution = semantics.get("target_resolution") if isinstance(semantics.get("target_resolution"), dict) else {}
target_entity_type = str(target_resolution.get("target_entity_type") or "").strip().lower()
target_ref = str(target_resolution.get("target_ref") or "").strip()
if target_entity_type == "project" and target_ref.startswith("project:"):
try:
resolved_project_id = int(target_ref.split(":", 1)[1])
except Exception:
resolved_project_id = None
if resolved_project_id is not None:
matched_project = (
await self.db.execute(
select(Project).where(
and_(
Project.tenant_id == self.tenant_id,
Project.id == resolved_project_id,
)
)
)
).scalar_one_or_none()
if matched_project is not None:
title = str(matched_project.title or matched_project.name or f"项目#{int(matched_project.id)}")
status = str(matched_project.status or "unknown")
answer = f"有,当前匹配到 1 个相关项目:\n#{int(matched_project.id)} {title}(状态:{status})"
return {
"answer": answer,
"sources": [
{
"source": "db",
"label": "Project database",
"score": 0.99,
"metadata": {
"table": "projects",
"query_keyword": str(target_resolution.get("resolved_label") or title),
"match_count": 1,
"project_id": int(matched_project.id),
},
}
],
"db_query_ms": 0,
}
if ( if (
project_id is None project_id is None
and message_intent != "project_lookup_question" and message_intent != "project_lookup_question"
@ -292,20 +342,38 @@ class AdaptiveRAGFastAnswerProjectMainMixin:
): ):
return None return None
primary_keyword = self._extract_project_lookup_keyword(question) query_terms = self._collect_lookup_terms(
query_terms = self._extract_query_terms(question, preferred=primary_keyword) question=question,
query_semantics=semantics,
operation_contract=semantics.get("operation_contract") if isinstance(semantics.get("operation_contract"), dict) else None,
target_resolution=target_resolution,
limit=4,
)
if not query_terms:
matched_project_id, matched_name = await self._match_project_name_in_question(question=question)
if matched_project_id is not None and matched_name:
query_terms = [str(matched_name)]
if not query_terms: if not query_terms:
return None return None
started = time.perf_counter() started = time.perf_counter()
semantic_match = await self._try_fast_project_existence_answer_from_semantic_facts(
query_terms=query_terms,
project_id=project_id,
)
if semantic_match is not None:
semantic_match["db_query_ms"] = int((time.perf_counter() - started) * 1000)
semantic_match.setdefault("response_time_ms", semantic_match["db_query_ms"])
return semantic_match
matched: list[Project] = [] matched: list[Project] = []
used_keyword = query_terms[0] used_keyword = query_terms[0]
for term in query_terms: ranked_rows, used_keyword = await self._search_projects_by_lookup_terms(
rows = await self._search_projects_by_keyword(keyword=term, project_id=project_id, limit=5) query_terms=query_terms,
if rows: project_id=project_id,
matched = rows limit=5,
used_keyword = term )
break matched = list(ranked_rows or [])
db_query_ms = int((time.perf_counter() - started) * 1000) db_query_ms = int((time.perf_counter() - started) * 1000)
if matched: if matched:
@ -334,6 +402,71 @@ class AdaptiveRAGFastAnswerProjectMainMixin:
"db_query_ms": db_query_ms, "db_query_ms": db_query_ms,
} }
async def _try_fast_project_existence_answer_from_semantic_facts(
self,
*,
query_terms: list[str],
project_id: Optional[int],
) -> Optional[dict[str, Any]]:
fact_query = getattr(self, "_query_project_semantic_facts_by_lookup_terms", None)
if not callable(fact_query):
return None
matched_facts, used_keyword = await fact_query(
query_terms=query_terms,
project_id=project_id,
limit=10,
)
if not matched_facts:
return None
matched_project_ids = []
for row in matched_facts:
try:
entity_id = int(row.entity_id)
except Exception:
continue
if entity_id not in matched_project_ids:
matched_project_ids.append(entity_id)
if not matched_project_ids:
return None
project_rows = (
await self.db.execute(
select(Project).where(
and_(
Project.tenant_id == self.tenant_id,
Project.id.in_(matched_project_ids),
)
)
)
).scalars().all()
if not project_rows:
return None
lines = [
f"#{int(row.id)} {str(row.title or row.name or '未命名项目')}(状态:{str(row.status or 'unknown')})"
for row in project_rows
]
answer = f"有,当前匹配到 {len(project_rows)} 个相关项目:\n" + "\n".join(lines)
return {
"answer": answer,
"sources": [
{
"source": "db",
"label": "Semantic facts",
"score": 0.99,
"metadata": {
"table": "semantic_facts",
"query_keyword": used_keyword,
"match_count": len(project_rows),
"matched_project_ids": matched_project_ids,
"fact_keys": sorted({str(row.fact_key or "") for row in matched_facts if str(row.fact_key or "").strip()}),
},
}
],
}
async def _search_projects_by_keyword( async def _search_projects_by_keyword(
self, self,
*, *,
@ -356,3 +489,67 @@ class AdaptiveRAGFastAnswerProjectMainMixin:
) )
) )
return (await self.db.execute(stmt.limit(max(1, int(limit))))).scalars().all() return (await self.db.execute(stmt.limit(max(1, int(limit))))).scalars().all()
async def _search_projects_by_lookup_terms(
self,
*,
query_terms: list[str],
project_id: Optional[int],
limit: int,
) -> tuple[list[Project], str]:
terms = self._dedupe_lookup_terms([str(item or "").strip() for item in query_terms], limit=6)
if not terms:
return ([], "")
for term in terms:
rows = await self._search_projects_by_keyword(keyword=term, project_id=project_id, limit=limit)
if rows:
return (rows, term)
stmt = select(Project).where(Project.tenant_id == self.tenant_id)
if project_id is not None:
stmt = stmt.where(Project.id == int(project_id))
candidates = (
await self.db.execute(
stmt.order_by(Project.updated_at.desc(), Project.id.desc()).limit(max(40, int(limit) * 20))
)
).scalars().all()
if not candidates:
return ([], terms[0])
ranked: list[tuple[float, Project, str]] = []
for row in candidates:
title = str(row.title or "").strip()
name = str(row.name or "").strip()
description = str(row.description or "").strip()
best_term = ""
best_score = 0.0
for term in terms:
score = max(
self._lookup_similarity_score(term, title),
self._lookup_similarity_score(term, name),
self._lookup_similarity_score(term, description) * 0.82,
)
if score > best_score:
best_score = score
best_term = term
if best_score >= 0.62 and best_term:
ranked.append((best_score, row, best_term))
if not ranked:
return ([], terms[0])
ranked.sort(key=lambda item: item[0], reverse=True)
deduped: list[Project] = []
used_keyword = ranked[0][2]
seen_ids: set[int] = set()
for _score, row, term in ranked:
row_id = int(row.id)
if row_id in seen_ids:
continue
seen_ids.add(row_id)
deduped.append(row)
if not used_keyword:
used_keyword = term
if len(deduped) >= max(1, int(limit)):
break
return (deduped, used_keyword)

View File

@ -8,83 +8,188 @@ class AdaptiveRAGFastAnswerProjectUtilsMixin:
_SEMANTIC_SUBJECT_FOCI = {"people", "project", "task", "signal", "charter", "workload", "feature", "unknown"} _SEMANTIC_SUBJECT_FOCI = {"people", "project", "task", "signal", "charter", "workload", "feature", "unknown"}
_ALLOWED_SCOPE_VALUES = {"none", "global", "project"} _ALLOWED_SCOPE_VALUES = {"none", "global", "project"}
_SEMANTIC_FEATURE_FOCI = {"none", "purpose", "entry", "catalog"} _SEMANTIC_FEATURE_FOCI = {"none", "purpose", "entry", "catalog"}
_LOOKUP_TEXT_TRANSLATION = str.maketrans(
{
",": " ",
"。": " ",
"!": " ",
"?": " ",
":": " ",
";": " ",
"(": " ",
")": " ",
"(": " ",
")": " ",
"\"": " ",
"'": " ",
}
)
@staticmethod @staticmethod
def _extract_project_lookup_keyword(question: str) -> str: def _normalize_lookup_text(text: str) -> str:
text = str(question or "").strip() normalized = str(text or "").strip().lower()
if not text: if not normalized:
return "" return ""
text = text.replace("?", "?") normalized = normalized.translate(AdaptiveRAGFastAnswerProjectUtilsMixin._LOOKUP_TEXT_TRANSLATION)
normalized = normalized.replace("—", "-").replace("_", " ")
normalized = re.sub(r"[^\w\u4e00-\u9fff\- ]+", " ", normalized)
normalized = re.sub(r"\s+", " ", normalized).strip()
return normalized[:96]
direct_patterns = [ @classmethod
r"(?:是否有|有没有|有无)([\u4e00-\u9fffA-Za-z0-9_-]{1,24})(?:项目|系统)?", def _collect_structured_object_mentions(
r"(?:是否存在|存在)([\u4e00-\u9fffA-Za-z0-9_-]{1,24})(?:项目|系统)?", cls,
] *,
for pattern in direct_patterns: query_semantics: Optional[dict[str, Any]] = None,
m = re.search(pattern, text) operation_contract: Optional[dict[str, Any]] = None,
if m: target_resolution: Optional[dict[str, Any]] = None,
value = str(m.group(1) or "").strip(" ,。,.!??::;;“”\"'()()") limit: int = 6,
value = QUESTION_NOISE_RE.sub("", value).strip() ) -> list[str]:
if len(value) >= 2: semantics = query_semantics if isinstance(query_semantics, dict) else {}
return value[:24] operation = operation_contract if isinstance(operation_contract, dict) else (
semantics.get("operation_contract") if isinstance(semantics.get("operation_contract"), dict) else {}
anchored_matches = list(re.finditer(r"([\u4e00-\u9fffA-Za-z0-9_-]{1,24})(?:项目|系统)", text))
for m in reversed(anchored_matches):
value = str(m.group(1) or "").strip(" ,。,.!??::;;“”\"'()()")
value = QUESTION_NOISE_RE.sub("", value).strip()
if len(value) >= 2:
return value[:24]
patterns = [
r"有(?:没有)?(.{1,24}?)(?:项目|系统).*\?",
r"项目.*?有(?:没有)?(.{1,24}?)(?:吗|\?)",
r"是否有(.{1,24}?)(?:项目|系统)",
r"(.{1,24}?)(?:项目|系统)(?:有吗|有没有|是否有)",
]
for pattern in patterns:
m = re.search(pattern, text)
if not m:
continue
kw = str(m.group(1) or "").strip(" ,。,.!??::;;“”\"'()()")
kw = QUESTION_NOISE_RE.sub("", kw).strip()
if kw:
return kw[:24]
fallback = re.sub(r"[??]", " ", text)
fallback = re.sub(
r"(项目|系统|目前|当前|现在|我们|有没有|是否有|有吗|有|吗|什么|进度|层面|阶段|是在|在|是|请问|一下|多少)",
" ",
fallback,
) )
fallback = re.sub(r"\s+", " ", fallback).strip(" ,。,.!??::;;") target = target_resolution if isinstance(target_resolution, dict) else (
return fallback[:24] semantics.get("target_resolution") if isinstance(semantics.get("target_resolution"), dict) else {}
)
raw_rows: list[Any] = []
raw_rows.extend(list(operation.get("object_mentions") or []))
raw_rows.extend(list(semantics.get("object_mentions") or []))
resolved_label = str(target.get("resolved_label") or "").strip()
if resolved_label:
raw_rows.append(resolved_label)
@staticmethod mentions: list[str] = []
def _extract_query_terms(question: str, *, preferred: str = "") -> list[str]: for item in raw_rows:
base = str(question or "").strip() text = str(item or "").strip(" ,。,.!??::;;“”\"'()()")
if len(text) < 2 or text in mentions:
continue
mentions.append(text[:32])
if len(mentions) >= max(1, int(limit)):
break
return mentions
@classmethod
def _lookup_similarity_score(cls, left: str, right: str) -> float:
lhs = cls._normalize_lookup_text(left)
rhs = cls._normalize_lookup_text(right)
if not lhs or not rhs:
return 0.0
if lhs == rhs:
return 1.0
if lhs in rhs:
return min(0.98, 0.82 + min(0.16, len(lhs) / max(len(rhs), 1) * 0.16))
if rhs in lhs:
return min(0.94, 0.78 + min(0.16, len(rhs) / max(len(lhs), 1) * 0.16))
return float(SequenceMatcher(None, lhs, rhs).ratio())
@classmethod
def _dedupe_lookup_terms(cls, rows: list[str], *, limit: int = 6) -> list[str]:
terms: list[str] = [] terms: list[str] = []
pref = str(preferred or "").strip() for raw in rows:
if len(pref) >= 2: text = str(raw or "").strip()
terms.append(pref[:24]) if len(text) < 2 or text in terms:
continue
terms.append(text[:32])
if len(terms) >= max(1, int(limit)):
break
return terms
clean = re.sub(r"[??,,。!!::;;()()\"“”']", " ", base) @classmethod
clean = QUESTION_NOISE_RE.sub(" ", clean) def _collect_lookup_terms(
clean = re.sub(r"(项目|系统|进度|状态|阶段|完成率|层面)", " ", clean) cls,
clean = re.sub(r"\s+", " ", clean).strip() *,
question: str = "",
query_semantics: Optional[dict[str, Any]] = None,
operation_contract: Optional[dict[str, Any]] = None,
target_resolution: Optional[dict[str, Any]] = None,
limit: int = 6,
) -> list[str]:
mentions = cls._collect_structured_object_mentions(
query_semantics=query_semantics,
operation_contract=operation_contract,
target_resolution=target_resolution,
limit=limit,
)
return mentions
token_candidates: list[str] = [] async def _query_project_semantic_facts_by_lookup_terms(
for token in clean.split(" "): self,
t = token.strip() *,
if len(t) >= 2: query_terms: list[str],
token_candidates.append(t[:24]) project_id: Optional[int],
limit: int = 20,
) -> tuple[list[SemanticFact], str]:
return await self._query_semantic_facts_by_lookup_terms(
entity_type="project",
query_terms=query_terms,
entity_id=project_id,
limit=limit,
)
for m in re.finditer(r"[\u4e00-\u9fff]{2,10}", clean): async def _query_semantic_facts_by_lookup_terms(
token_candidates.append(m.group(0)[:24]) self,
*,
entity_type: str,
query_terms: list[str],
entity_id: Optional[int],
limit: int = 20,
) -> tuple[list[SemanticFact], str]:
terms = self._dedupe_lookup_terms([str(item or "").strip() for item in query_terms], limit=6)
if not terms:
return ([], "")
for t in sorted(token_candidates, key=len, reverse=True): stmt = select(SemanticFact).where(
if t and t not in terms: and_(
terms.append(t) SemanticFact.tenant_id == self.tenant_id,
return terms[:6] SemanticFact.entity_type == str(entity_type or "").strip(),
SemanticFact.is_active.is_(True),
)
)
if entity_id is not None:
stmt = stmt.where(SemanticFact.entity_id == int(entity_id))
fact_rows = (
await self.db.execute(
stmt.order_by(SemanticFact.confidence.desc(), SemanticFact.updated_at.desc(), SemanticFact.id.desc()).limit(
max(40, int(limit) * 8)
)
)
).scalars().all()
if not fact_rows:
return ([], terms[0])
ranked: list[tuple[float, SemanticFact, str]] = []
for row in fact_rows:
fact_text = str(row.fact_value_text or "").strip()
if not fact_text:
continue
best_term = ""
best_score = 0.0
for term in terms:
score = self._lookup_similarity_score(term, fact_text)
if score > best_score:
best_score = score
best_term = term
if best_score >= 0.74 and best_term:
ranked.append((best_score, row, best_term))
if not ranked:
return ([], terms[0])
ranked.sort(key=lambda item: item[0], reverse=True)
deduped: list[SemanticFact] = []
used_keyword = ranked[0][2]
seen_row_ids: set[int] = set()
for _score, row, term in ranked:
row_id = int(row.id)
if row_id in seen_row_ids:
continue
seen_row_ids.add(row_id)
deduped.append(row)
if not used_keyword:
used_keyword = term
if len(deduped) >= max(1, int(limit)):
break
return (deduped, used_keyword)
@classmethod @classmethod
def _query_subject_focus(cls, query_semantics: Optional[dict[str, Any]]) -> str: def _query_subject_focus(cls, query_semantics: Optional[dict[str, Any]]) -> str:

View File

@ -23,10 +23,19 @@ class AdaptiveRAGFastAnswerSemanticMixinP2:
rewrite_meta: Optional[dict[str, Any]] = None, rewrite_meta: Optional[dict[str, Any]] = None,
thinking_mode: str = THINKING_MODE_NORMAL, thinking_mode: str = THINKING_MODE_NORMAL,
answer_mode: str = ANSWER_MODE_FAST_PATH, answer_mode: str = ANSWER_MODE_FAST_PATH,
route_decision: Optional[dict[str, Any]] = None,
query_semantics: Optional[dict[str, Any]] = None,
) -> dict[str, Any]: ) -> dict[str, Any]:
started = time.perf_counter() started = time.perf_counter()
answer_text = str(fast_result.get("answer") or "未命中结果")
sources = list(fast_result.get("sources") or []) sources = list(fast_result.get("sources") or [])
render_with_contract = getattr(self, "_render_answer_with_route_contract", None)
answer_text = str(fast_result.get("answer") or "未命中结果")
if callable(render_with_contract):
answer_text = render_with_contract(
answer=answer_text,
sources=sources,
route_decision=route_decision,
)
active_session_id = session_id active_session_id = session_id
active_turn_index = turn_index active_turn_index = turn_index
if active_session_id is None or active_turn_index is None: if active_session_id is None or active_turn_index is None:
@ -44,22 +53,50 @@ class AdaptiveRAGFastAnswerSemanticMixinP2:
missing_fields = list(fast_result.get("missing_fields") or []) missing_fields = list(fast_result.get("missing_fields") or [])
final_query_type = str(fast_result.get("query_type") or query_type or QUERY_TYPE_EXPLAIN) final_query_type = str(fast_result.get("query_type") or query_type or QUERY_TYPE_EXPLAIN)
prompt_variant_id = str(fast_result.get("prompt_variant_id") or "deterministic").strip() or "deterministic" prompt_variant_id = str(fast_result.get("prompt_variant_id") or "deterministic").strip() or "deterministic"
contract_verdict = self._verify_internal_answer_contract(
query_type=final_query_type,
answer=answer_text,
sources=sources,
route_decision=route_decision,
)
effective_answer_mode = answer_mode
effective_grounding_status = GROUNDING_STATUS_GROUNDED
effective_reason_code = str(fast_result.get("reason_code") or "").strip() or None
if not bool(contract_verdict.get("ok")):
guard_payload = self._build_internal_answer_contract_guard_payload(
query_type=final_query_type,
reason_code=str(contract_verdict.get("reason_code") or ""),
missing_fields=list(contract_verdict.get("missing_fields") or []),
)
answer_text = str(guard_payload.get("answer") or answer_text)
sources = list(guard_payload.get("sources") or [])
missing_fields = list(guard_payload.get("missing_fields") or [])
effective_answer_mode = str(guard_payload.get("answer_mode") or ANSWER_MODE_BLOCKED)
effective_grounding_status = str(guard_payload.get("grounding_status") or GROUNDING_STATUS_BLOCKED)
effective_reason_code = str(guard_payload.get("reason_code") or "").strip() or "internal_answer_contract_violation"
retrieval_metrics = {
"db_query_ms": int(fast_result.get("db_query_ms") or 0),
"vector_search_ms": 0,
"graph_query_ms": 0,
"db_timeout": False,
"vector_timeout": False,
"graph_timeout": False,
}
inject_runtime_summaries = getattr(self, "_inject_internal_query_runtime_summaries", None)
if callable(inject_runtime_summaries):
retrieval_metrics = inject_runtime_summaries(
retrieval_metrics=retrieval_metrics,
query_semantics=query_semantics,
)
runtime_fields = self._build_qa_record_runtime_fields( runtime_fields = self._build_qa_record_runtime_fields(
answer=answer_text, answer=answer_text,
sources=sources, sources=sources,
missing_fields=missing_fields, missing_fields=missing_fields,
answer_mode=answer_mode, answer_mode=effective_answer_mode,
grounding_status=GROUNDING_STATUS_GROUNDED, grounding_status=effective_grounding_status,
prompt_variant_id=prompt_variant_id, prompt_variant_id=prompt_variant_id,
reason_code=str(fast_result.get("reason_code") or "").strip() or None, reason_code=effective_reason_code,
retrieval_metrics={ retrieval_metrics=retrieval_metrics,
"db_query_ms": int(fast_result.get("db_query_ms") or 0),
"vector_search_ms": 0,
"graph_query_ms": 0,
"db_timeout": False,
"vector_timeout": False,
"graph_timeout": False,
},
) )
qa = QARecord( qa = QARecord(
@ -75,7 +112,7 @@ class AdaptiveRAGFastAnswerSemanticMixinP2:
project_id=project_id, project_id=project_id,
session_id=active_session_id_int, session_id=active_session_id_int,
turn_index=active_turn_index, turn_index=active_turn_index,
**self._build_qa_record_external_fields(), **self._build_qa_record_external_fields(route_decision=route_decision),
**runtime_fields, **runtime_fields,
) )
self.db.add(qa) self.db.add(qa)
@ -91,6 +128,15 @@ class AdaptiveRAGFastAnswerSemanticMixinP2:
rewrite_meta=rewrite_meta or {}, rewrite_meta=rewrite_meta or {},
sources=sources, sources=sources,
) )
await self.db.flush()
persist_route_memory = getattr(self, "_persist_query_route_memory", None)
if callable(persist_route_memory):
await persist_route_memory(
qa_record_id=int(qa.id) if getattr(qa, "id", None) is not None else None,
question_text=question,
project_id=project_id,
route_decision=route_decision,
)
await self.db.commit() await self.db.commit()
await self.db.refresh(qa) await self.db.refresh(qa)
cache_session_id = active_session_id_int if active_session_id_int is not None else ( cache_session_id = active_session_id_int if active_session_id_int is not None else (
@ -120,12 +166,14 @@ class AdaptiveRAGFastAnswerSemanticMixinP2:
confidence=confidence, confidence=confidence,
missing_fields=list(missing_fields or []), missing_fields=list(missing_fields or []),
thinking_mode=thinking_mode, thinking_mode=thinking_mode,
answer_mode=answer_mode, answer_mode=effective_answer_mode,
grounding_status=GROUNDING_STATUS_GROUNDED, grounding_status=effective_grounding_status,
policy_version=runtime_fields["policy_version"], policy_version=runtime_fields["policy_version"],
citations=runtime_fields["citations"], citations=runtime_fields["citations"],
sources_used=runtime_fields["sources_used"], sources_used=runtime_fields["sources_used"],
evidence_pack=runtime_fields["evidence_pack"], evidence_pack=runtime_fields["evidence_pack"],
tool_trace=runtime_fields["tool_trace"], tool_trace=runtime_fields["tool_trace"],
prompt_variant_id=prompt_variant_id, prompt_variant_id=prompt_variant_id,
reason_code=effective_reason_code,
route_decision=route_decision,
) )

View File

@ -579,6 +579,9 @@ class AdaptiveRAGFollowupAnchorMixin:
anchor_reply=chosen, anchor_reply=chosen,
) )
if is_meta_followup and explicit_anchor is None and chosen is None and not candidates:
return FollowupAnchorResolution()
if not candidates: if not candidates:
return FollowupAnchorResolution( return FollowupAnchorResolution(
followup_mode=followup_mode, followup_mode=followup_mode,
@ -617,6 +620,9 @@ class AdaptiveRAGFollowupAnchorMixin:
anchor_reply=chosen, anchor_reply=chosen,
) )
if is_meta_followup:
return FollowupAnchorResolution()
return FollowupAnchorResolution( return FollowupAnchorResolution(
followup_mode=followup_mode, followup_mode=followup_mode,
anchor_mode="ambiguous_candidates", anchor_mode="ambiguous_candidates",
@ -764,14 +770,7 @@ class AdaptiveRAGFollowupAnchorMixin:
) )
anchor_kind = str(resolution.anchor_kind or "").strip() or None anchor_kind = str(resolution.anchor_kind or "").strip() or None
if anchor_row is None: if anchor_row is None:
return { return None
"answer": "当前无法追溯到本次回复的路由记录。",
"meta_followup_kind": meta_kind,
"reason_code": "meta_followup_no_route_record",
"anchor_kind": anchor_kind,
"anchor_mode": resolution.anchor_mode,
"routing_snapshot": {},
}
routing_snapshot = ( routing_snapshot = (
anchor_row.routing_snapshot anchor_row.routing_snapshot

View File

@ -101,7 +101,10 @@ class AdaptiveRAGManifestCoreMixin:
) -> dict[str, Any]: ) -> dict[str, Any]:
normalized = self._normalize_entity_text(question) normalized = self._normalize_entity_text(question)
alias_terms = self._collect_alias_terms(normalized) alias_terms = self._collect_alias_terms(normalized)
query_terms = self._extract_query_terms(question, preferred=self._extract_project_lookup_keyword(question)) term_collector = getattr(self, "_collect_lookup_terms", None)
query_terms = term_collector(question=question, limit=6) if callable(term_collector) else []
if normalized:
query_terms.append(normalized[:32])
for term in alias_terms: for term in alias_terms:
if term not in query_terms: if term not in query_terms:
query_terms.append(term) query_terms.append(term)
@ -192,4 +195,3 @@ class AdaptiveRAGManifestCoreMixin:
"count": min(len(catalog), max_rows), "count": min(len(catalog), max_rows),
"items": catalog[:max_rows], "items": catalog[:max_rows],
} }

View File

@ -101,8 +101,9 @@ class AdaptiveRAGManifestFeatureMixinP1:
catalog = self._feature_catalog_from_snapshot(system_snapshot) catalog = self._feature_catalog_from_snapshot(system_snapshot)
if not catalog: if not catalog:
return [] return []
query_terms = [x.lower() for x in self._extract_query_terms(question)[:8] if str(x).strip()]
matched: list[dict[str, Any]] = [] matched: list[dict[str, Any]] = []
normalizer = getattr(self, "_normalize_lookup_text", None)
normalized_text = normalizer(question) if callable(normalizer) else self._normalize_entity_text(question)
for item in catalog: for item in catalog:
name = str(item.get("name") or "").strip() name = str(item.get("name") or "").strip()
route = str(item.get("route") or "").strip() route = str(item.get("route") or "").strip()
@ -116,11 +117,13 @@ class AdaptiveRAGManifestFeatureMixinP1:
score = 0.0 score = 0.0
for alias in alias_pool: for alias in alias_pool:
if alias and alias in text: if not alias:
score += 0.45 continue
for term in query_terms: if alias == normalized_text:
if term and any(term in alias or alias in term for alias in alias_pool): score = max(score, 0.96)
score += 0.18 continue
if alias in text or normalized_text in alias:
score = max(score, 0.82)
if score <= 0: if score <= 0:
continue continue
matched.append( matched.append(
@ -223,4 +226,3 @@ class AdaptiveRAGManifestFeatureMixinP1:
} }
) )
return rows[:3] return rows[:3]

View File

@ -5,6 +5,13 @@ from .shared import *
class AdaptiveRAGMultiStepCoreMixin: class AdaptiveRAGMultiStepCoreMixin:
@staticmethod
def _skip_internal_hint_expansion(query_semantics: Optional[dict[str, Any]]) -> bool:
semantics = query_semantics if isinstance(query_semantics, dict) else {}
if bool(semantics.get("external_freshness_only")):
return True
return str(semantics.get("effective_source_policy") or semantics.get("source_policy") or "").strip().lower() == "external"
async def _run_single_retrieval_pass( async def _run_single_retrieval_pass(
self, self,
*, *,
@ -98,8 +105,9 @@ class AdaptiveRAGMultiStepCoreMixin:
query_semantics: Optional[dict[str, Any]] = None, query_semantics: Optional[dict[str, Any]] = None,
) -> tuple[str, dict[str, Any]]: ) -> tuple[str, dict[str, Any]]:
semantics = query_semantics if isinstance(query_semantics, dict) else {} semantics = query_semantics if isinstance(query_semantics, dict) else {}
if bool(semantics.get("external_freshness_only")): if self._skip_internal_hint_expansion(semantics):
return str(question or "").strip()[:1200], {"mode": "freshness_guard_skip_internal_hint"} mode = "freshness_guard_skip_internal_hint" if bool(semantics.get("external_freshness_only")) else "external_source_policy_skip_internal_hint"
return str(question or "").strip()[:1200], {"mode": mode}
resolved = await self.entity_resolution(question=question, project_id=project_id, limit=5) resolved = await self.entity_resolution(question=question, project_id=project_id, limit=5)
manifest = await self.get_project_manifest(project_id=project_id) manifest = await self.get_project_manifest(project_id=project_id)
hints: list[str] = [] hints: list[str] = []

View File

@ -19,12 +19,16 @@ class AdaptiveRAGMultiStepFlowMixin:
) -> tuple[list[RetrievedItem], dict[str, Any]]: ) -> tuple[list[RetrievedItem], dict[str, Any]]:
semantics = query_semantics if isinstance(query_semantics, dict) else {} semantics = query_semantics if isinstance(query_semantics, dict) else {}
external_freshness_only = bool(semantics.get("external_freshness_only")) external_freshness_only = bool(semantics.get("external_freshness_only"))
external_source_policy = str(
semantics.get("effective_source_policy") or semantics.get("source_policy") or ""
).strip().lower() == "external"
skip_internal_enrichment = external_freshness_only or external_source_policy
resolved: dict[str, Any] = {"matched_entities": [], "canonical_terms": []} resolved: dict[str, Any] = {"matched_entities": [], "canonical_terms": []}
manifest: dict[str, Any] = {} manifest: dict[str, Any] = {}
system_snapshot: dict[str, Any] = {} system_snapshot: dict[str, Any] = {}
symbol_hits: dict[str, Any] = {"count": 0, "items": []} symbol_hits: dict[str, Any] = {"count": 0, "items": []}
feature_hits: list[dict[str, Any]] = [] feature_hits: list[dict[str, Any]] = []
if not external_freshness_only: if not skip_internal_enrichment:
resolved = await self.entity_resolution(question=question, project_id=project_id, limit=6) resolved = await self.entity_resolution(question=question, project_id=project_id, limit=6)
manifest = await self.get_project_manifest(project_id=project_id) manifest = await self.get_project_manifest(project_id=project_id)
system_snapshot = await self.context_cache.get_system_snapshot( system_snapshot = await self.context_cache.get_system_snapshot(
@ -87,6 +91,14 @@ class AdaptiveRAGMultiStepFlowMixin:
"mode": "external_freshness_only", "mode": "external_freshness_only",
} }
) )
elif external_source_policy:
tool_trace.append(
{
"step": 0,
"tool": "source_policy_guard",
"mode": "external_only",
}
)
if session_state is not None and isinstance(session_state.last_entity_ids, list): if session_state is not None and isinstance(session_state.last_entity_ids, list):
anchor_ids = [int(x) for x in session_state.last_entity_ids if str(x).isdigit()][:6] anchor_ids = [int(x) for x in session_state.last_entity_ids if str(x).isdigit()][:6]
else: else:
@ -124,7 +136,7 @@ class AdaptiveRAGMultiStepFlowMixin:
vector_top_k=vector_top_k, vector_top_k=vector_top_k,
) )
feature_rows = [] feature_rows = []
if not external_freshness_only: if not skip_internal_enrichment:
feature_rows = self._retrieve_feature_catalog_context( feature_rows = self._retrieve_feature_catalog_context(
question=current_question, question=current_question,
system_snapshot=system_snapshot, system_snapshot=system_snapshot,
@ -190,7 +202,7 @@ class AdaptiveRAGMultiStepFlowMixin:
break break
next_domains: list[str] = [] next_domains: list[str] = []
if external_freshness_only: if skip_internal_enrichment:
next_domains = list(current_domains) next_domains = list(current_domains)
else: else:
for item in [*current_domains, "graph", "db", "doc", "pmflow", "feishu"]: for item in [*current_domains, "graph", "db", "doc", "pmflow", "feishu"]:

View File

@ -35,6 +35,18 @@ class AdaptiveRAGQAFeedbackRecordsMixin:
submitted_by=submitted_by, submitted_by=submitted_by,
) )
self.db.add(feedback) self.db.add(feedback)
await self.db.flush()
await self._update_query_route_feedback_outcome(
qa_record_id=int(qa_id),
feedback_outcome={
"qa_feedback_id": int(feedback.id) if getattr(feedback, "id", None) is not None else None,
"rating": str(feedback.rating or "").strip().lower(),
"helpfulness": str(feedback.helpfulness or "").strip() or None,
"error_types": [str(x).strip() for x in (feedback.error_types or []) if str(x).strip()],
"correct_answer": str(feedback.correct_answer or "").strip() or None,
"submitted_by": int(submitted_by) if submitted_by is not None else None,
},
)
await self.db.commit() await self.db.commit()
await self.db.refresh(feedback) await self.db.refresh(feedback)
if str(feedback.rating or "").strip().lower() == "negative" and ( if str(feedback.rating or "").strip().lower() == "negative" and (
@ -152,4 +164,3 @@ class AdaptiveRAGQAFeedbackRecordsMixin:
"page_size": page_size, "page_size": page_size,
"count": len(items), "count": len(items),
} }

View File

@ -0,0 +1,467 @@
"""Adaptive RAG query route gateway helpers."""
from __future__ import annotations
from .shared import *
class AdaptiveRAGQueryRouteGatewayMixin:
_QUERY_ROUTE_MODES = {"internal", "external", "uncertain"}
_INTERNAL_ALLOWED_SOURCES = {"db", "graph", "doc", "pmflow", "feishu"}
_EXTERNAL_ALLOWED_SOURCES = {"url", "mcp_search", "system"}
def _normalize_query_route_decision(self, payload: Optional[dict[str, Any]]) -> dict[str, Any]:
raw = payload if isinstance(payload, dict) else {}
reply_contract = raw.get("reply_contract") if isinstance(raw.get("reply_contract"), dict) else {}
query_profile = raw.get("query_profile") if isinstance(raw.get("query_profile"), dict) else {}
probe_snapshot_summary = (
raw.get("probe_snapshot_summary")
if isinstance(raw.get("probe_snapshot_summary"), dict)
else {}
)
return {
"route_mode": self._normalize_query_route_mode(raw.get("route_mode")),
"effective_source_policy": self._normalize_query_route_mode(
raw.get("effective_source_policy") or raw.get("route_mode")
),
"decision_origin": str(raw.get("decision_origin") or "").strip() or "unknown",
"reply_contract": dict(reply_contract),
"query_profile": self._normalize_query_profile(query_profile),
"probe_snapshot_summary": dict(probe_snapshot_summary),
"probe_snapshot": dict(raw.get("probe_snapshot") or {}) if isinstance(raw.get("probe_snapshot"), dict) else {},
"clarification_text": str(raw.get("clarification_text") or "").strip() or None,
}
@classmethod
def _normalize_query_route_mode(cls, value: Any) -> str:
normalized = str(value or "").strip().lower()
if normalized in cls._QUERY_ROUTE_MODES:
return normalized
return "uncertain"
@classmethod
def _normalize_query_profile(cls, payload: Optional[dict[str, Any]]) -> dict[str, Any]:
rows = payload if isinstance(payload, dict) else {}
referenced_objects: list[str] = []
for item in list(rows.get("referenced_objects") or []):
normalized = re.sub(r"[^a-z0-9:_\-]", "", str(item or "").strip().lower())
if normalized and normalized not in referenced_objects:
referenced_objects.append(normalized[:48])
if not referenced_objects:
referenced_objects = ["unknown"]
return {
"needs_private_context": bool(rows.get("needs_private_context")),
"needs_freshness": bool(rows.get("needs_freshness")),
"referenced_objects": referenced_objects,
"confidence": max(0.0, min(1.0, float(rows.get("confidence") or 0.0))),
}
@classmethod
def _build_query_route_reply_contract(
cls,
*,
route_mode: str,
effective_source_policy: str,
clarification_text: Optional[str] = None,
) -> dict[str, Any]:
normalized_route = cls._normalize_query_route_mode(route_mode)
normalized_effective = cls._normalize_query_route_mode(effective_source_policy)
if normalized_route == "internal" or normalized_effective == "internal":
return {
"mode": "explicit_source",
"head": "内部知识库查询:",
"tail_hint": None,
}
if normalized_route == "uncertain" and normalized_effective != "external":
return {
"mode": "clarify",
"head": None,
"tail_hint": None,
"clarification_text": str(clarification_text or "").strip() or None,
}
return {
"mode": "explicit_source",
"head": "外部实时查询:",
"tail_hint": (
"如果你指的是系统内部配置/项目对象,请明确指出。"
if normalized_route == "uncertain"
else None
),
}
@classmethod
def _build_query_route_clarification_text(cls) -> str:
return "你是想问外部公开事实,还是系统内部配置/项目对象?"
@classmethod
def _build_contract_snapshot_summary(
cls,
*,
source_policy_hint: Optional[str],
query_semantics: Optional[dict[str, Any]],
route_mode: str,
effective_source_policy: str,
decision_origin: str,
) -> dict[str, Any]:
semantics = query_semantics if isinstance(query_semantics, dict) else {}
return {
"contract": {
"source_policy_hint": cls._normalize_query_route_mode(source_policy_hint),
"semantic_source_policy": cls._normalize_query_route_mode(semantics.get("source_policy")),
"semantic_effective_source_policy": cls._normalize_query_route_mode(
semantics.get("effective_source_policy")
),
"route_mode": cls._normalize_query_route_mode(route_mode),
"effective_source_policy": cls._normalize_query_route_mode(effective_source_policy),
"decision_origin": str(decision_origin or "").strip() or "unknown",
},
}
def _build_query_route_audit_snapshot(
self,
*,
route_decision: Optional[dict[str, Any]],
) -> dict[str, Any]:
normalized = self._normalize_query_route_decision(route_decision)
return {
"route_mode": normalized["route_mode"],
"effective_source_policy": normalized["effective_source_policy"],
"decision_origin": normalized["decision_origin"],
"reply_contract": normalized["reply_contract"],
"query_profile": normalized["query_profile"],
"probe_snapshot_summary": normalized["probe_snapshot_summary"],
}
def _resolve_route_contract(
self,
*,
query_semantics: Optional[dict[str, Any]],
source_policy_hint: Optional[str],
) -> dict[str, Any]:
semantics = query_semantics if isinstance(query_semantics, dict) else {}
normalized_hint = self._normalize_query_route_mode(source_policy_hint)
if normalized_hint in {"internal", "external"}:
return {
"source_policy": normalized_hint,
"effective_source_policy": normalized_hint,
"decision_origin": "upstream_hint",
"source_reason": "upstream_source_policy_hint",
}
source_policy = self._normalize_query_route_mode(semantics.get("source_policy"))
semantic_effective = self._normalize_query_route_mode(
semantics.get("effective_source_policy") or semantics.get("source_policy")
)
effective_source_policy = (
semantic_effective
if semantic_effective in {"internal", "external"}
else ("uncertain" if source_policy == "uncertain" else source_policy)
)
return {
"source_policy": source_policy,
"effective_source_policy": effective_source_policy,
"decision_origin": str(semantics.get("decision_origin") or "query_semantics_contract").strip()
or "query_semantics_contract",
"source_reason": str(semantics.get("source_reason") or "query_semantics_contract").strip()
or "query_semantics_contract",
}
def _apply_route_decision_to_query_semantics(
self,
*,
query_semantics: Optional[dict[str, Any]],
route_decision: Optional[dict[str, Any]],
) -> dict[str, Any]:
semantics = dict(query_semantics or {})
normalized = self._normalize_query_route_decision(route_decision)
effective_source_policy = normalized["effective_source_policy"]
if effective_source_policy == "uncertain":
retrieval_domains: list[str] = []
else:
retrieval_domains = self._apply_source_policy_to_domains(
domains=[
str(item).strip().lower()
for item in list(semantics.get("retrieval_domains") or [])
if str(item or "").strip()
],
effective_source_policy=effective_source_policy,
)
semantics["route_decision"] = normalized
semantics["query_profile"] = normalized["query_profile"]
semantics["source_policy"] = normalized["route_mode"]
semantics["effective_source_policy"] = effective_source_policy
semantics["decision_origin"] = normalized["decision_origin"]
semantics["source_reason"] = normalized["decision_origin"]
semantics["retrieval_domains"] = retrieval_domains
return semantics
async def _run_query_route_gateway(
self,
*,
question: str,
asked_via: str,
project_id: Optional[int],
source_chat_id: Optional[str],
query_semantics: Optional[dict[str, Any]],
source_policy_hint: Optional[str],
) -> dict[str, Any]:
_ = (question, asked_via, project_id, source_chat_id)
semantics = dict(query_semantics or {})
query_profile = self._normalize_query_profile(semantics.get("query_profile"))
contract = self._resolve_route_contract(
query_semantics=semantics,
source_policy_hint=source_policy_hint,
)
route_mode = self._normalize_query_route_mode(contract.get("source_policy"))
effective_source_policy = self._normalize_query_route_mode(contract.get("effective_source_policy"))
decision_origin = str(contract.get("decision_origin") or "query_semantics_contract").strip() or "query_semantics_contract"
clarification_text = None
if route_mode == "uncertain" and effective_source_policy != "external":
clarification_text = self._build_query_route_clarification_text()
probe_snapshot = {
"contract": {
"source_policy": route_mode,
"effective_source_policy": effective_source_policy,
"source_reason": str(contract.get("source_reason") or "query_semantics_contract").strip()
or "query_semantics_contract",
"decision_origin": decision_origin,
}
}
return {
"route_mode": route_mode,
"effective_source_policy": effective_source_policy,
"decision_origin": decision_origin,
"reply_contract": self._build_query_route_reply_contract(
route_mode=route_mode,
effective_source_policy=effective_source_policy,
clarification_text=clarification_text,
),
"clarification_text": clarification_text,
"query_profile": query_profile,
"probe_snapshot": probe_snapshot,
"probe_snapshot_summary": self._build_contract_snapshot_summary(
source_policy_hint=source_policy_hint,
query_semantics=semantics,
route_mode=route_mode,
effective_source_policy=effective_source_policy,
decision_origin=decision_origin,
),
}
async def preview_query_route_decision(
self,
*,
question: str,
asked_via: str,
project_id: Optional[int],
source_chat_id: Optional[str],
source_message_id: Optional[str] = None,
conversation_scope_key: Optional[str] = None,
reply_to_message_id: Optional[str] = None,
root_message_id: Optional[str] = None,
source_policy_hint: Optional[str] = None,
query_semantics: Optional[dict[str, Any]] = None,
) -> dict[str, Any]:
semantics = dict(query_semantics or {})
if not semantics:
semantics = await self._resolve_query_semantics(
message=question,
source_system=asked_via,
metadata={
"project_id": project_id,
"source_chat_id": source_chat_id,
"source_message_id": source_message_id,
"conversation_scope_key": conversation_scope_key,
"reply_to_message_id": reply_to_message_id,
"root_message_id": root_message_id,
},
source_policy_hint=source_policy_hint,
)
semantics = dict(semantics or {})
decision = await self._run_query_route_gateway(
question=question,
asked_via=asked_via,
project_id=project_id,
source_chat_id=source_chat_id,
query_semantics=semantics,
source_policy_hint=source_policy_hint or semantics.get("source_policy"),
)
normalized_decision = self._normalize_query_route_decision(decision)
normalized_decision["probe_snapshot"] = dict(decision.get("probe_snapshot") or {})
return {
"query_semantics": self._apply_route_decision_to_query_semantics(
query_semantics=semantics,
route_decision=normalized_decision,
),
"route_decision": normalized_decision,
}
async def _persist_query_route_memory(
self,
*,
qa_record_id: Optional[int],
question_text: str,
project_id: Optional[int],
route_decision: Optional[dict[str, Any]],
) -> None:
normalized = self._normalize_query_route_decision(route_decision)
if qa_record_id is None or not str(question_text or "").strip():
return
if normalized["route_mode"] == "uncertain" and normalized["effective_source_policy"] == "uncertain":
# 澄清样本仍然保留,但不强制要求 embedding 成功。
pass
try:
row = (
await self.db.execute(
select(QueryRouteMemory).where(
and_(
QueryRouteMemory.tenant_id == self.tenant_id,
QueryRouteMemory.qa_record_id == int(qa_record_id),
)
)
)
).scalar_one_or_none()
except Exception:
row = None
question_embedding: list[float] = []
embedding_getter = getattr(getattr(self, "llm", None), "get_embedding_async", None)
if callable(embedding_getter):
try:
question_embedding = await embedding_getter(
str(question_text or "").strip(),
db=self.db,
tenant_id=self.tenant_id,
)
except Exception:
question_embedding = []
if row is None:
row = QueryRouteMemory(
tenant_id=self.tenant_id,
qa_record_id=int(qa_record_id),
project_id=project_id,
question_text=str(question_text or "").strip(),
route_mode_predicted=normalized["route_mode"],
route_mode_effective=normalized["effective_source_policy"],
decision_origin=normalized["decision_origin"],
reply_contract=normalized["reply_contract"],
probe_snapshot=normalized.get("probe_snapshot") or {},
query_profile=normalized["query_profile"],
feedback_outcome=None,
question_embedding=question_embedding or None,
)
self.db.add(row)
return
row.project_id = project_id
row.question_text = str(question_text or "").strip()
row.route_mode_predicted = normalized["route_mode"]
row.route_mode_effective = normalized["effective_source_policy"]
row.decision_origin = normalized["decision_origin"]
row.reply_contract = normalized["reply_contract"]
row.probe_snapshot = normalized.get("probe_snapshot") or {}
row.query_profile = normalized["query_profile"]
if question_embedding:
row.question_embedding = question_embedding
async def _update_query_route_feedback_outcome(
self,
*,
qa_record_id: Optional[int],
feedback_outcome: Optional[dict[str, Any]],
) -> None:
if qa_record_id is None or not isinstance(feedback_outcome, dict):
return
row = (
await self.db.execute(
select(QueryRouteMemory).where(
and_(
QueryRouteMemory.tenant_id == self.tenant_id,
QueryRouteMemory.qa_record_id == int(qa_record_id),
)
)
)
).scalar_one_or_none()
if row is None:
return
row.feedback_outcome = dict(feedback_outcome)
def _filter_sources_for_route(
self,
*,
sources: list[dict[str, Any]],
route_decision: Optional[dict[str, Any]],
) -> list[dict[str, Any]]:
decision = route_decision if isinstance(route_decision, dict) else {}
effective_source_policy = self._normalize_query_route_mode(decision.get("effective_source_policy"))
if effective_source_policy == "internal":
allowed = self._INTERNAL_ALLOWED_SOURCES
elif effective_source_policy == "external":
allowed = self._EXTERNAL_ALLOWED_SOURCES
else:
return list(sources or [])
filtered = [
item
for item in list(sources or [])
if isinstance(item, dict) and str(item.get("source") or "").strip().lower() in allowed
]
return filtered
def _render_answer_with_route_contract(
self,
*,
answer: str,
sources: list[dict[str, Any]],
route_decision: Optional[dict[str, Any]],
) -> str:
text = str(answer or "").strip()
if not text:
return text
decision = route_decision if isinstance(route_decision, dict) else {}
reply_contract = decision.get("reply_contract") if isinstance(decision.get("reply_contract"), dict) else {}
contract_mode = str(reply_contract.get("mode") or "").strip().lower()
if contract_mode == "clarify":
return str(reply_contract.get("clarification_text") or text).strip()
effective_source_policy = self._normalize_query_route_mode(decision.get("effective_source_policy"))
head = str(reply_contract.get("head") or "").strip()
tail_hint = str(reply_contract.get("tail_hint") or "").strip()
formatted = text if not head or text.startswith(head) else f"{head}\n{text}"
labels: list[str] = []
for item in self._filter_sources_for_route(sources=sources, route_decision=decision)[:3]:
if not isinstance(item, dict):
continue
source = str(item.get("source") or "").strip().lower()
if effective_source_policy == "external" and source == "system":
continue
label = str(item.get("label") or item.get("source") or "").strip()
if label and label not in labels:
labels.append(label)
if labels and "来源摘要:" not in formatted:
formatted = f"{formatted}\n\n来源摘要:{';'.join(labels[:3])}"
if tail_hint and tail_hint not in formatted:
formatted = f"{formatted}\n\n{tail_hint}"
return formatted.strip()
def _build_route_clarification_payload(
self,
*,
route_decision: Optional[dict[str, Any]],
) -> Optional[dict[str, Any]]:
normalized = self._normalize_query_route_decision(route_decision)
reply_contract = (
normalized.get("reply_contract")
if isinstance(normalized.get("reply_contract"), dict)
else {}
)
if str(reply_contract.get("mode") or "").strip().lower() != "clarify":
return None
clarification_text = str(reply_contract.get("clarification_text") or "").strip()
if not clarification_text:
return None
return {
"answer": clarification_text,
"code": "query_route_clarify",
"candidates": [],
"missing_fields": ["query_route"],
}

View File

@ -50,7 +50,13 @@ class AdaptiveRAGRetrievalGraphMixin:
).scalars().all() ).scalars().all()
if not seed_entities: if not seed_entities:
terms = self._extract_query_terms(question, preferred=self._extract_project_lookup_keyword(question)) term_collector = getattr(self, "_collect_lookup_terms", None)
terms = term_collector(question=question, limit=6) if callable(term_collector) else []
if not terms:
normalizer = getattr(self, "_normalize_lookup_text", None)
normalized = normalizer(question) if callable(normalizer) else str(question or "").strip()
if normalized:
terms = [normalized[:32]]
if terms: if terms:
conds = [] conds = []
for term in terms[:6]: for term in terms[:6]:

View File

@ -5,6 +5,167 @@ from .shared import *
class AdaptiveRAGRetrievalRouterDBMixin: class AdaptiveRAGRetrievalRouterDBMixin:
async def _retrieve_generic_semantic_fact_rows(
self,
*,
entity_type: str,
question: str,
query_semantics: Optional[dict[str, Any]] = None,
) -> list[RetrievedItem]:
term_collector = getattr(self, "_collect_lookup_terms", None)
query_terms = term_collector(
question=question,
query_semantics=query_semantics,
operation_contract=(
query_semantics.get("operation_contract")
if isinstance(query_semantics, dict) and isinstance(query_semantics.get("operation_contract"), dict)
else None
),
target_resolution=(
query_semantics.get("target_resolution")
if isinstance(query_semantics, dict) and isinstance(query_semantics.get("target_resolution"), dict)
else None
),
limit=4,
) if callable(term_collector) else []
if not query_terms:
return []
fact_query = getattr(self, "_query_semantic_facts_by_lookup_terms", None)
if not callable(fact_query):
return []
matched_facts, used_keyword = await fact_query(
entity_type=entity_type,
query_terms=query_terms,
entity_id=None,
limit=10,
)
if not matched_facts:
return []
grouped: dict[int, list[SemanticFact]] = {}
for row in matched_facts:
try:
entity_id = int(row.entity_id)
except Exception:
continue
grouped.setdefault(entity_id, []).append(row)
if not grouped:
return []
preview = ";".join(
f"#{entity_id} " + " / ".join(
str(item.fact_value_text or "").strip()
for item in rows[:3]
if str(item.fact_value_text or "").strip()
)
for entity_id, rows in list(grouped.items())[:8]
)
metadata = self._normalize_source_registry_metadata(
source_type="db",
metadata={
"table": "semantic_facts",
"entity_type": entity_type,
"query_keyword": used_keyword,
"match_count": len(grouped),
"fact_keys": sorted({str(row.fact_key or "") for row in matched_facts if str(row.fact_key or "").strip()}),
},
)
return [
RetrievedItem(
source="db",
content=f"语义事实命中{entity_type}:" + preview,
score=0.96,
source_label="Semantic facts",
metadata=metadata,
)
]
async def _retrieve_project_semantic_fact_rows(
self,
*,
question: str,
project_id: Optional[int],
query_semantics: Optional[dict[str, Any]] = None,
) -> list[RetrievedItem]:
term_collector = getattr(self, "_collect_lookup_terms", None)
query_terms = term_collector(
question=question,
query_semantics=query_semantics,
operation_contract=(
query_semantics.get("operation_contract")
if isinstance(query_semantics, dict) and isinstance(query_semantics.get("operation_contract"), dict)
else None
),
target_resolution=(
query_semantics.get("target_resolution")
if isinstance(query_semantics, dict) and isinstance(query_semantics.get("target_resolution"), dict)
else None
),
limit=4,
) if callable(term_collector) else []
if not query_terms:
return []
fact_query = getattr(self, "_query_project_semantic_facts_by_lookup_terms", None)
if not callable(fact_query):
return []
matched_facts, used_keyword = await fact_query(
query_terms=query_terms,
project_id=project_id,
limit=10,
)
if not matched_facts:
return []
matched_project_ids: list[int] = []
for fact in matched_facts:
try:
entity_id = int(fact.entity_id)
except Exception:
continue
if entity_id not in matched_project_ids:
matched_project_ids.append(entity_id)
if not matched_project_ids:
return []
projects = (
await self.db.execute(
select(Project).where(
and_(
Project.tenant_id == self.tenant_id,
Project.id.in_(matched_project_ids),
)
)
)
).scalars().all()
if not projects:
return []
metadata = self._normalize_source_registry_metadata(
source_type="db",
metadata={
"table": "semantic_facts",
"query_keyword": used_keyword,
"match_count": len(projects),
"matched_project_ids": matched_project_ids,
"fact_keys": sorted({str(row.fact_key or "") for row in matched_facts if str(row.fact_key or "").strip()}),
},
explicit_project_id=matched_project_ids[0] if len(matched_project_ids) == 1 else None,
)
return [
RetrievedItem(
source="db",
content="语义事实命中项目:" + ";".join(
f"#{int(row.id)} {str(row.title or row.name or '未命名项目')}(状态:{str(row.status or 'unknown')})"
for row in projects[:10]
),
score=0.97,
source_label="Semantic facts",
metadata=metadata,
)
]
async def _retrieve_structured_db( async def _retrieve_structured_db(
self, self,
question: str, question: str,
@ -16,7 +177,33 @@ class AdaptiveRAGRetrievalRouterDBMixin:
rows: list[RetrievedItem] = [] rows: list[RetrievedItem] = []
subject_focus = self._query_subject_focus(query_semantics) subject_focus = self._query_subject_focus(query_semantics)
scope_hint = self._query_scope_hint(query_semantics) scope_hint = self._query_scope_hint(query_semantics)
target_resolution = query_semantics.get("target_resolution") if isinstance(query_semantics, dict) and isinstance(query_semantics.get("target_resolution"), dict) else {}
target_entity_type = str(target_resolution.get("target_entity_type") or "").strip().lower()
query_plan = query_semantics.get("query_plan") if isinstance(query_semantics, dict) and isinstance(query_semantics.get("query_plan"), dict) else {}
planned_entity_type = str(query_plan.get("entity_type") or "").strip().lower()
project_subject_requested = subject_focus == "project" or target_entity_type == "project" or planned_entity_type == "project"
project_scope_required = scope_hint == "project" or project_id is not None project_scope_required = scope_hint == "project" or project_id is not None
semantic_fact_entity_type = ""
if target_entity_type in {"task", "signal", "release", "review", "project", "document", "user", "requirement"}:
semantic_fact_entity_type = target_entity_type
elif planned_entity_type in {"task", "signal", "release", "review", "project", "document", "user", "requirement"}:
semantic_fact_entity_type = planned_entity_type
elif subject_focus == "task":
semantic_fact_entity_type = "task"
elif subject_focus == "signal":
semantic_fact_entity_type = "signal"
elif subject_focus == "people":
semantic_fact_entity_type = "user"
elif subject_focus == "charter":
semantic_fact_entity_type = "requirement"
if semantic_fact_entity_type and semantic_fact_entity_type != "project":
rows.extend(
await self._retrieve_generic_semantic_fact_rows(
entity_type=semantic_fact_entity_type,
question=question,
query_semantics=query_semantics,
)
)
if subject_focus == "people": if subject_focus == "people":
scoped_project_id: Optional[int] = None scoped_project_id: Optional[int] = None
@ -74,9 +261,21 @@ class AdaptiveRAGRetrievalRouterDBMixin:
) )
) )
if subject_focus == "project": if project_subject_requested:
semantic_fact_rows = await self._retrieve_project_semantic_fact_rows(
question=question,
project_id=project_id,
query_semantics=query_semantics,
)
rows.extend(semantic_fact_rows)
stmt = select(Project).where(Project.tenant_id == self.tenant_id) stmt = select(Project).where(Project.tenant_id == self.tenant_id)
if project_id: resolved_project_ref = str(target_resolution.get("target_ref") or "").strip()
if resolved_project_ref.startswith("project:"):
try:
stmt = stmt.where(Project.id == int(resolved_project_ref.split(":", 1)[1]))
except Exception:
pass
elif project_id:
stmt = stmt.where(Project.id == project_id) stmt = stmt.where(Project.id == project_id)
projects = (await self.db.execute(stmt.limit(8))).scalars().all() projects = (await self.db.execute(stmt.limit(8))).scalars().all()
if projects: if projects:

View File

@ -176,9 +176,10 @@ class AdaptiveRAGSessionStateMixin:
return f"{subject_name}->{object_name}"[:255] return f"{subject_name}->{object_name}"[:255]
text = str(question or "").strip() text = str(question or "").strip()
keyword = self._extract_project_lookup_keyword(text) normalizer = getattr(self, "_normalize_lookup_text", None)
if len(keyword) >= 2: normalized_question = normalizer(text) if callable(normalizer) else text
return keyword[:255] if 2 <= len(normalized_question) <= 64:
return normalized_question[:255]
answer_text = str(answer or "").strip() answer_text = str(answer or "").strip()
m = re.search(r"项目[「\"]?([^」\"\n]{2,40})", answer_text) m = re.search(r"项目[「\"]?([^」\"\n]{2,40})", answer_text)

View File

@ -0,0 +1,766 @@
"""Adaptive RAG internal target resolution helpers."""
from __future__ import annotations
from .shared import *
class AdaptiveRAGTargetResolutionMixin:
_TARGET_ENTITY_TYPES = {"project", "feature", "config", "doc", "person", "signal", "unknown"}
_TARGET_RESOLUTION_MODES = {"deterministic", "llm_rerank", "semantic_filter", "clarify", "none"}
@classmethod
def _normalize_object_mentions(
cls,
raw: Any,
*,
question: str = "",
) -> list[str]:
_ = (cls, question)
mentions: list[str] = []
rows = raw if isinstance(raw, list) else []
for item in rows:
text = str(item or "").strip(" ,。,.!??::;;“”\"'()()")
text = QUESTION_NOISE_RE.sub("", text).strip()
if len(text) >= 2 and text not in mentions:
mentions.append(text[:32])
return mentions[:4]
def _build_operation_contract(
self,
*,
question: str,
query_semantics: Optional[dict[str, Any]],
) -> dict[str, Any]:
_ = question
semantics = query_semantics if isinstance(query_semantics, dict) else {}
query_type = str(semantics.get("query_type") or QUERY_TYPE_UNKNOWN).strip().lower() or QUERY_TYPE_UNKNOWN
query_profile = dict(semantics.get("query_profile") or {}) if isinstance(semantics.get("query_profile"), dict) else {}
referenced_objects = {
str(item or "").strip().lower()
for item in list(query_profile.get("referenced_objects") or [])
if str(item or "").strip()
}
if query_type in {QUERY_TYPE_EXISTENCE, QUERY_TYPE_UNKNOWN} and referenced_objects.intersection(
{"model_routing", "system_config"}
):
query_type = QUERY_TYPE_STATUS
object_mentions = self._normalize_object_mentions(
semantics.get("object_mentions"),
question=question,
)
return {
"query_type": query_type,
"answer_contract": (
self._internal_answer_contract_for_query_type(query_type)
if hasattr(self, "_internal_answer_contract_for_query_type")
else "clarify_or_fail"
),
"query_profile": query_profile,
"object_mentions": object_mentions,
"clarify_policy": "ask_when_target_ambiguous",
}
@classmethod
def _normalize_target_candidate(cls, payload: Optional[dict[str, Any]]) -> Optional[dict[str, Any]]:
row = payload if isinstance(payload, dict) else {}
entity_type = str(row.get("target_entity_type") or row.get("entity_type") or "").strip().lower()
if entity_type not in cls._TARGET_ENTITY_TYPES:
return None
score = max(0.0, min(1.0, float(row.get("score") or 0.0)))
return {
"target_entity_type": entity_type,
"target_ref": str(row.get("target_ref") or "").strip() or None,
"resolved_label": str(row.get("resolved_label") or row.get("display_name") or "").strip() or None,
"score": round(score, 4),
"source_kind": str(row.get("source_kind") or "unknown").strip().lower() or "unknown",
"match_label": str(row.get("match_label") or "").strip() or None,
"source_confidence": round(max(0.0, min(1.0, float(row.get("source_confidence") or score or 0.0))), 4),
}
@staticmethod
def _target_candidate_identity(row: dict[str, Any]) -> str:
return "|".join(
[
str(row.get("target_entity_type") or ""),
str(row.get("target_ref") or ""),
str(row.get("resolved_label") or ""),
]
)
async def _collect_semantic_fact_target_candidates(
self,
*,
mentions: list[str],
project_id: Optional[int],
operation_contract: dict[str, Any],
) -> list[dict[str, Any]]:
if not mentions:
return []
candidates: list[dict[str, Any]] = []
fact_query = getattr(self, "_query_project_semantic_facts_by_lookup_terms", None)
if not callable(fact_query):
return []
for mention in mentions[:3]:
fact_rows, _used_keyword = await fact_query(
query_terms=[mention],
project_id=project_id,
limit=10,
)
project_ids: list[int] = []
fact_confidence_by_project: dict[int, float] = {}
for row in fact_rows:
try:
entity_id = int(row.entity_id)
except Exception:
continue
if entity_id not in project_ids:
project_ids.append(entity_id)
fact_confidence_by_project[entity_id] = max(
float(fact_confidence_by_project.get(entity_id) or 0.0),
float(getattr(row, "confidence", 0.0) or 0.0),
)
if not project_ids:
continue
project_rows = (
await self.db.execute(
select(Project).where(
and_(
Project.tenant_id == self.tenant_id,
Project.id.in_(project_ids),
)
)
)
).scalars().all()
for project in project_rows:
confidence = max(0.88, min(0.99, float(fact_confidence_by_project.get(int(project.id)) or 0.0)))
candidates.append(
{
"target_entity_type": "project",
"target_ref": f"project:{int(project.id)}",
"resolved_label": str(project.title or project.name or f"项目#{int(project.id)}"),
"score": round(min(0.99, confidence + 0.03), 4),
"source_kind": "semantic_facts",
"match_label": mention,
"source_confidence": round(confidence, 4),
}
)
return candidates
async def _collect_project_target_candidates(
self,
*,
mentions: list[str],
project_id: Optional[int],
) -> list[dict[str, Any]]:
if not mentions:
return []
candidates: list[dict[str, Any]] = []
similarity_scorer = getattr(self, "_lookup_similarity_score", None)
for mention in mentions[:3]:
matched_rows: list[Project] = []
searcher = getattr(self, "_search_projects_by_keyword", None)
if callable(searcher):
matched_rows = await searcher(keyword=mention, project_id=project_id, limit=6)
else:
stmt = select(Project).where(Project.tenant_id == self.tenant_id)
if project_id is not None:
stmt = stmt.where(Project.id == int(project_id))
like = f"%{mention}%"
stmt = stmt.where(
or_(
Project.title.ilike(like),
Project.name.ilike(like),
Project.description.ilike(like),
)
)
matched_rows = (await self.db.execute(stmt.order_by(Project.updated_at.desc()).limit(6))).scalars().all()
if not matched_rows:
stmt = select(Project).where(Project.tenant_id == self.tenant_id)
if project_id is not None:
stmt = stmt.where(Project.id == int(project_id))
project_rows = (
await self.db.execute(
stmt.order_by(Project.updated_at.desc(), Project.id.desc()).limit(120)
)
).scalars().all()
scored_rows: list[tuple[float, Project]] = []
for project in project_rows:
title = str(project.title or "").strip()
name = str(project.name or "").strip()
description = str(project.description or "").strip()
if callable(similarity_scorer):
score = max(
similarity_scorer(mention, title),
similarity_scorer(mention, name),
similarity_scorer(mention, description) * 0.8,
)
else:
lowered = mention.lower()
title_l = title.lower()
name_l = name.lower()
description_l = description.lower()
score = 0.0
if lowered and (lowered in title_l or lowered in name_l):
score = 0.9
elif lowered and lowered in description_l:
score = 0.76
if score >= 0.62:
scored_rows.append((score, project))
scored_rows.sort(key=lambda item: item[0], reverse=True)
matched_rows = [row for _, row in scored_rows[:6]]
for project in matched_rows:
title = str(project.title or "").strip().lower()
name = str(project.name or "").strip().lower()
description = str(project.description or "").strip().lower()
lowered = mention.lower()
score = 0.82
if callable(similarity_scorer):
score = max(
0.74,
similarity_scorer(mention, title) if title else 0.0,
similarity_scorer(mention, name) if name else 0.0,
(similarity_scorer(mention, description) * 0.8) if description else 0.0,
)
elif lowered and (lowered in title or lowered in name):
score = 0.91
elif lowered and lowered in description:
score = 0.86
candidates.append(
{
"target_entity_type": "project",
"target_ref": f"project:{int(project.id)}",
"resolved_label": str(project.title or project.name or f"项目#{int(project.id)}"),
"score": round(score, 4),
"source_kind": "project_db",
"match_label": mention,
"source_confidence": round(score, 4),
}
)
return candidates
def _collect_feature_target_candidates(
self,
*,
question: str,
mentions: list[str],
system_snapshot: Optional[dict[str, Any]],
) -> list[dict[str, Any]]:
matcher = getattr(self, "_match_feature_catalog_entries", None)
if not callable(matcher):
return []
search_text = " ".join(mentions[:2]) if mentions else question
rows = matcher(
question=search_text,
system_snapshot=system_snapshot,
limit=4,
)
candidates: list[dict[str, Any]] = []
for row in rows:
name = str(row.get("name") or "").strip()
route = str(row.get("route") or "").strip()
match_score = max(0.0, min(1.0, float(row.get("match_score") or 0.0)))
if not name and not route:
continue
score = min(0.92, max(0.72, 0.62 + match_score * 0.25))
target_key = route or name
candidates.append(
{
"target_entity_type": "feature",
"target_ref": f"feature:{target_key}",
"resolved_label": name or target_key,
"score": round(score, 4),
"source_kind": "feature_catalog",
"match_label": name or target_key,
"source_confidence": round(match_score or score, 4),
}
)
return candidates
def _collect_config_target_candidates(
self,
*,
query_semantics: Optional[dict[str, Any]],
operation_contract: dict[str, Any],
) -> list[dict[str, Any]]:
semantics = query_semantics if isinstance(query_semantics, dict) else {}
query_profile = semantics.get("query_profile") if isinstance(semantics.get("query_profile"), dict) else {}
referenced_objects = {
str(item or "").strip().lower()
for item in list(query_profile.get("referenced_objects") or [])
if str(item or "").strip()
}
if not {"model_routing", "system_config"}.intersection(referenced_objects):
return []
mentions = list(operation_contract.get("object_mentions") or [])
rows: list[dict[str, Any]] = []
if "model_routing" in referenced_objects:
rows.append(
{
"target_entity_type": "config",
"target_ref": "config:model_routing",
"resolved_label": mentions[0] if mentions else "模型配置",
"score": 0.9,
"source_kind": "system_config",
"match_label": mentions[0] if mentions else "模型配置",
"source_confidence": 0.9,
}
)
elif "system_config" in referenced_objects:
rows.append(
{
"target_entity_type": "config",
"target_ref": "config:system",
"resolved_label": mentions[0] if mentions else "系统配置",
"score": 0.86,
"source_kind": "system_config",
"match_label": mentions[0] if mentions else "系统配置",
"source_confidence": 0.86,
}
)
return rows
async def _collect_kg_target_candidates(
self,
*,
question: str,
mentions: list[str],
project_id: Optional[int],
) -> list[dict[str, Any]]:
resolver = getattr(self, "entity_resolution", None)
if not callable(resolver):
return []
resolution = await resolver(
question=" ".join(mentions[:2]) if mentions else question,
project_id=project_id,
limit=6,
)
matched_entities = resolution.get("matched_entities") if isinstance(resolution, dict) else []
if not isinstance(matched_entities, list) or not matched_entities:
return []
ids: list[int] = []
matched_score: dict[int, float] = {}
for row in matched_entities:
if not isinstance(row, dict):
continue
try:
entity_id = int(row.get("entity_id"))
except Exception:
continue
ids.append(entity_id)
matched_score[entity_id] = max(
float(matched_score.get(entity_id) or 0.0),
float(row.get("score") or 0.0),
)
if not ids:
return []
rows = (
await self.db.execute(
select(KGEntity).where(
and_(
KGEntity.tenant_id == self.tenant_id,
KGEntity.id.in_(ids),
)
)
)
).scalars().all()
candidates: list[dict[str, Any]] = []
for row in rows:
payload = self._safe_dict(row.payload) if hasattr(self, "_safe_dict") else (row.payload if isinstance(row.payload, dict) else {})
entity_score = max(0.0, min(1.0, float(matched_score.get(int(row.id)) or 0.0)))
project_ref = payload.get("project_id")
if project_ref is not None:
try:
project_ref = int(project_ref)
except Exception:
project_ref = None
if project_ref is not None:
candidates.append(
{
"target_entity_type": "project",
"target_ref": f"project:{int(project_ref)}",
"resolved_label": str(row.display_name or row.entity_key or f"项目#{int(project_ref)}"),
"score": round(min(0.89, max(0.68, 0.58 + entity_score * 0.25)), 4),
"source_kind": "knowledge_graph",
"match_label": str(row.display_name or row.entity_key or "").strip() or None,
"source_confidence": round(entity_score or 0.68, 4),
}
)
return candidates
async def _collect_internal_target_candidates(
self,
*,
question: str,
project_id: Optional[int],
query_semantics: Optional[dict[str, Any]],
system_snapshot: Optional[dict[str, Any]],
operation_contract: dict[str, Any],
) -> list[dict[str, Any]]:
_ = query_semantics
mentions = list(operation_contract.get("object_mentions") or [])
providers = await asyncio.gather(
self._collect_semantic_fact_target_candidates(
mentions=mentions,
project_id=project_id,
operation_contract=operation_contract,
),
self._collect_project_target_candidates(
mentions=mentions,
project_id=project_id,
),
self._collect_kg_target_candidates(
question=question,
mentions=mentions,
project_id=project_id,
),
)
rows: list[dict[str, Any]] = []
for bucket in providers:
for item in bucket:
normalized = self._normalize_target_candidate(item)
if normalized is not None:
rows.append(normalized)
for item in self._collect_feature_target_candidates(
question=question,
mentions=mentions,
system_snapshot=system_snapshot,
):
normalized = self._normalize_target_candidate(item)
if normalized is not None:
rows.append(normalized)
for item in self._collect_config_target_candidates(
query_semantics=query_semantics,
operation_contract=operation_contract,
):
normalized = self._normalize_target_candidate(item)
if normalized is not None:
rows.append(normalized)
deduped: dict[str, dict[str, Any]] = {}
for row in rows:
key = self._target_candidate_identity(row)
existing = deduped.get(key)
if existing is None or float(row.get("score") or 0.0) > float(existing.get("score") or 0.0):
deduped[key] = row
return sorted(deduped.values(), key=lambda item: float(item.get("score") or 0.0), reverse=True)[:8]
async def _rerank_internal_target_candidates(
self,
*,
question: str,
operation_contract: dict[str, Any],
candidates: list[dict[str, Any]],
) -> Optional[dict[str, Any]]:
if len(candidates) < 2 or not hasattr(self, "llm"):
return None
prompt = PROMPT_INTERNAL_TARGET_RESOLUTION.format(
question=str(question or "")[:600],
operation_contract=json.dumps(operation_contract, ensure_ascii=False)[:1000],
candidates=json.dumps(
[
{
"index": idx,
"target_entity_type": str(item.get("target_entity_type") or ""),
"target_ref": str(item.get("target_ref") or ""),
"resolved_label": str(item.get("resolved_label") or ""),
"score": float(item.get("score") or 0.0),
"source_kind": str(item.get("source_kind") or ""),
}
for idx, item in enumerate(candidates[:4])
],
ensure_ascii=False,
),
).strip()
try:
raw = await self.llm.generate(
prompt,
db=self.db,
tenant_id=self.tenant_id,
function_name="internal_target_resolution",
temperature=0.0,
max_tokens=220,
use_prompt_profile=False,
)
except Exception:
return None
parser = getattr(self, "_parse_json_object_safe", None)
parsed = parser(raw) if callable(parser) else {}
if not isinstance(parsed, dict):
return None
if bool(parsed.get("clarify")):
return {
"target_entity_type": "unknown",
"target_ref": None,
"resolved_label": None,
"resolution_mode": "clarify",
"confidence": round(max(0.0, min(1.0, float(parsed.get("confidence") or 0.0))), 4),
"candidate_summary": candidates[:4],
}
try:
selected_index = int(parsed.get("selected_index"))
except Exception:
return None
if selected_index < 0 or selected_index >= len(candidates[:4]):
return None
selected = dict(candidates[selected_index])
return {
"target_entity_type": str(selected.get("target_entity_type") or "unknown"),
"target_ref": selected.get("target_ref"),
"resolved_label": selected.get("resolved_label"),
"resolution_mode": "llm_rerank",
"confidence": round(max(0.0, min(1.0, float(parsed.get("confidence") or selected.get("score") or 0.0))), 4),
"candidate_summary": candidates[:4],
}
def _build_semantic_filter_target(
self,
*,
query_semantics: Optional[dict[str, Any]],
operation_contract: dict[str, Any],
) -> Optional[dict[str, Any]]:
semantics = query_semantics if isinstance(query_semantics, dict) else {}
subject_focus = str(semantics.get("subject_focus") or "unknown").strip().lower()
query_profile = semantics.get("query_profile") if isinstance(semantics.get("query_profile"), dict) else {}
mentions = list(operation_contract.get("object_mentions") or [])
resolved_label = mentions[0] if mentions else None
referenced_objects = {
str(item or "").strip().lower()
for item in list(query_profile.get("referenced_objects") or [])
if str(item or "").strip()
}
if {"model_routing", "system_config"}.intersection(referenced_objects):
target_ref = "config:model_routing" if "model_routing" in referenced_objects else "config:system"
return {
"target_entity_type": "config",
"target_ref": target_ref,
"resolved_label": resolved_label or ("模型配置" if target_ref.endswith("model_routing") else "系统配置"),
"resolution_mode": "deterministic",
"confidence": 0.84,
"candidate_summary": [],
}
if subject_focus == "project":
return {
"target_entity_type": "project",
"target_ref": None,
"resolved_label": resolved_label,
"resolution_mode": "semantic_filter",
"confidence": 0.78 if resolved_label else 0.64,
"candidate_summary": [],
}
if subject_focus == "feature" or "feature_catalog" in referenced_objects:
return {
"target_entity_type": "feature",
"target_ref": None,
"resolved_label": resolved_label,
"resolution_mode": "semantic_filter",
"confidence": 0.74 if resolved_label else 0.62,
"candidate_summary": [],
}
return None
async def _resolve_internal_target(
self,
*,
question: str,
project_id: Optional[int],
query_semantics: Optional[dict[str, Any]],
system_snapshot: Optional[dict[str, Any]],
operation_contract: Optional[dict[str, Any]],
) -> dict[str, Any]:
semantics = query_semantics if isinstance(query_semantics, dict) else {}
operation = operation_contract if isinstance(operation_contract, dict) else {}
source_policy = str(
semantics.get("effective_source_policy")
or semantics.get("source_policy")
or ""
).strip().lower()
if source_policy != "internal":
return {
"target_entity_type": "unknown",
"target_ref": None,
"resolved_label": None,
"resolution_mode": "none",
"confidence": 0.0,
"candidate_summary": [],
}
candidates = await self._collect_internal_target_candidates(
question=question,
project_id=project_id,
query_semantics=semantics,
system_snapshot=system_snapshot,
operation_contract=operation,
)
if not candidates:
fallback = self._build_semantic_filter_target(
query_semantics=semantics,
operation_contract=operation,
)
if fallback is not None:
return fallback
return {
"target_entity_type": "unknown",
"target_ref": None,
"resolved_label": None,
"resolution_mode": "clarify",
"confidence": 0.0,
"candidate_summary": [],
}
top = candidates[0]
second = candidates[1] if len(candidates) > 1 else None
margin = float(top.get("score") or 0.0) - float(second.get("score") or 0.0) if second else 1.0
if second and str(top.get("target_entity_type") or "") != str(second.get("target_entity_type") or "") and margin < 0.1:
reranked = await self._rerank_internal_target_candidates(
question=question,
operation_contract=operation,
candidates=candidates,
)
if reranked is not None:
return reranked
if float(top.get("score") or 0.0) >= 0.88 or margin >= 0.12 or second is None:
return {
"target_entity_type": str(top.get("target_entity_type") or "unknown"),
"target_ref": top.get("target_ref"),
"resolved_label": top.get("resolved_label"),
"resolution_mode": "deterministic",
"confidence": round(max(0.0, min(1.0, float(top.get("score") or 0.0))), 4),
"candidate_summary": candidates[:4],
}
if str(top.get("target_entity_type") or "") == str(second.get("target_entity_type") or ""):
return {
"target_entity_type": str(top.get("target_entity_type") or "unknown"),
"target_ref": top.get("target_ref"),
"resolved_label": top.get("resolved_label"),
"resolution_mode": "deterministic",
"confidence": round(max(0.0, min(1.0, float(top.get("score") or 0.0))), 4),
"candidate_summary": candidates[:4],
}
return {
"target_entity_type": "unknown",
"target_ref": None,
"resolved_label": None,
"resolution_mode": "clarify",
"confidence": round(max(0.0, min(1.0, float(top.get("score") or 0.0))), 4),
"candidate_summary": candidates[:4],
}
def _patch_scene_semantics_with_target_resolution(
self,
*,
scene_semantics: Optional[dict[str, Any]],
operation_contract: Optional[dict[str, Any]],
target_resolution: Optional[dict[str, Any]],
) -> dict[str, Any]:
normalized = dict(scene_semantics or {})
query_type = str((operation_contract or {}).get("query_type") or QUERY_TYPE_UNKNOWN).strip().lower()
target_entity_type = str((target_resolution or {}).get("target_entity_type") or "unknown").strip().lower()
snapshot_focus = str(normalized.get("snapshot_focus") or "none").strip().lower()
if query_type == QUERY_TYPE_EXISTENCE and target_entity_type == "project":
normalized["snapshot_focus"] = "none"
return normalized
if target_entity_type not in {"feature", "config"} and snapshot_focus == "feature":
normalized["snapshot_focus"] = "none"
return normalized
return normalized
def _should_allow_snapshot_direct_answer(
self,
*,
scene_semantics: Optional[dict[str, Any]],
operation_contract: Optional[dict[str, Any]],
target_resolution: Optional[dict[str, Any]],
) -> bool:
snapshot_focus = str((scene_semantics or {}).get("snapshot_focus") or "none").strip().lower()
query_type = str((operation_contract or {}).get("query_type") or QUERY_TYPE_UNKNOWN).strip().lower()
target_entity_type = str((target_resolution or {}).get("target_entity_type") or "unknown").strip().lower()
if snapshot_focus in {"identity", "architecture"}:
return query_type in {QUERY_TYPE_EXPLAIN, QUERY_TYPE_STATUS}
if snapshot_focus == "feature":
return query_type in {QUERY_TYPE_EXPLAIN, QUERY_TYPE_STATUS, QUERY_TYPE_LIST} and target_entity_type in {
"feature",
"config",
"unknown",
}
return False
@staticmethod
def _build_operation_contract_summary(operation_contract: Optional[dict[str, Any]]) -> dict[str, Any]:
payload = operation_contract if isinstance(operation_contract, dict) else {}
mentions = [
str(item or "").strip()
for item in list(payload.get("object_mentions") or [])
if str(item or "").strip()
]
return {
"query_type": str(payload.get("query_type") or QUERY_TYPE_UNKNOWN).strip().lower() or QUERY_TYPE_UNKNOWN,
"answer_contract": str(payload.get("answer_contract") or "clarify_or_fail").strip() or "clarify_or_fail",
"object_mentions": mentions[:4],
"clarify_policy": str(payload.get("clarify_policy") or "").strip() or None,
}
@staticmethod
def _build_target_resolution_summary(target_resolution: Optional[dict[str, Any]]) -> dict[str, Any]:
payload = target_resolution if isinstance(target_resolution, dict) else {}
candidates: list[dict[str, Any]] = []
for item in list(payload.get("candidate_summary") or [])[:4]:
if not isinstance(item, dict):
continue
candidates.append(
{
"target_entity_type": str(item.get("target_entity_type") or "unknown").strip().lower() or "unknown",
"target_ref": str(item.get("target_ref") or "").strip() or None,
"resolved_label": str(item.get("resolved_label") or "").strip() or None,
"score": round(max(0.0, min(1.0, float(item.get("score") or 0.0))), 4),
"source_kind": str(item.get("source_kind") or "unknown").strip().lower() or "unknown",
}
)
return {
"target_entity_type": str(payload.get("target_entity_type") or "unknown").strip().lower() or "unknown",
"target_ref": str(payload.get("target_ref") or "").strip() or None,
"resolved_label": str(payload.get("resolved_label") or "").strip() or None,
"resolution_mode": str(payload.get("resolution_mode") or "none").strip().lower() or "none",
"confidence": round(max(0.0, min(1.0, float(payload.get("confidence") or 0.0))), 4),
"candidate_summary": candidates,
}
@staticmethod
def _build_query_plan_summary(query_plan: Optional[dict[str, Any]]) -> dict[str, Any]:
payload = query_plan if isinstance(query_plan, dict) else {}
filters: list[dict[str, Any]] = []
for item in list(payload.get("filters") or [])[:4]:
if not isinstance(item, dict):
continue
filters.append(
{
"field": str(item.get("field") or "").strip() or None,
"operator": str(item.get("operator") or "").strip() or None,
"value": item.get("value"),
"value_source": str(item.get("value_source") or "").strip() or None,
}
)
return {
"plan_version": str(payload.get("plan_version") or "unknown").strip() or "unknown",
"query_type": str(payload.get("query_type") or QUERY_TYPE_UNKNOWN).strip().lower() or QUERY_TYPE_UNKNOWN,
"entity_type": str(payload.get("entity_type") or "unknown").strip().lower() or "unknown",
"target_ref": str(payload.get("target_ref") or "").strip() or None,
"answer_contract": str(payload.get("answer_contract") or "").strip() or None,
"filters": filters,
}
def _inject_internal_query_runtime_summaries(
self,
*,
retrieval_metrics: Optional[dict[str, Any]],
query_semantics: Optional[dict[str, Any]],
) -> dict[str, Any]:
metrics = dict(retrieval_metrics or {})
semantics = query_semantics if isinstance(query_semantics, dict) else {}
metrics["operation_contract_summary"] = self._build_operation_contract_summary(semantics.get("operation_contract"))
metrics["target_resolution_summary"] = self._build_target_resolution_summary(semantics.get("target_resolution"))
metrics["query_plan_summary"] = self._build_query_plan_summary(semantics.get("query_plan"))
return metrics
__all__ = ["AdaptiveRAGTargetResolutionMixin"]

View File

@ -40,12 +40,15 @@ from .mixin_conversation_utils_a_p1 import AdaptiveRAGConversationUtilsAMixinP1
from .mixin_conversation_utils_a_p2 import AdaptiveRAGConversationUtilsAMixinP2 from .mixin_conversation_utils_a_p2 import AdaptiveRAGConversationUtilsAMixinP2
from .mixin_conversation_utils_b import AdaptiveRAGConversationUtilsBMixin from .mixin_conversation_utils_b import AdaptiveRAGConversationUtilsBMixin
from .mixin_followup_anchor import AdaptiveRAGFollowupAnchorMixin from .mixin_followup_anchor import AdaptiveRAGFollowupAnchorMixin
from .mixin_query_route_gateway import AdaptiveRAGQueryRouteGatewayMixin
from .mixin_target_resolution import AdaptiveRAGTargetResolutionMixin
class AdaptiveRAGService( class AdaptiveRAGService(
AdaptiveRAGCoreIntentMixin, AdaptiveRAGCoreIntentMixin,
AdaptiveRAGAnswerEntryMixin, AdaptiveRAGAnswerEntryMixin,
AdaptiveRAGAnswerMainlineMixin, AdaptiveRAGAnswerMainlineMixin,
AdaptiveRAGTargetResolutionMixin,
AdaptiveRAGAnswerPrecheckMixin, AdaptiveRAGAnswerPrecheckMixin,
AdaptiveRAGAnswerRetrievalMixin, AdaptiveRAGAnswerRetrievalMixin,
AdaptiveRAGAnswerPayloadMixin, AdaptiveRAGAnswerPayloadMixin,
@ -82,6 +85,7 @@ class AdaptiveRAGService(
AdaptiveRAGConversationUtilsAMixinP2, AdaptiveRAGConversationUtilsAMixinP2,
AdaptiveRAGConversationUtilsBMixin, AdaptiveRAGConversationUtilsBMixin,
AdaptiveRAGFollowupAnchorMixin, AdaptiveRAGFollowupAnchorMixin,
AdaptiveRAGQueryRouteGatewayMixin,
): ):
"""项目内统一 RAG 服务。""" """项目内统一 RAG 服务。"""
_symbol_snapshot_cache: dict[str, tuple[float, dict[str, Any]]] = {} _symbol_snapshot_cache: dict[str, tuple[float, dict[str, Any]]] = {}

View File

@ -12,6 +12,7 @@ from __future__ import annotations
import ast import ast
import asyncio import asyncio
from difflib import SequenceMatcher
import hashlib import hashlib
import json import json
import math import math
@ -19,6 +20,7 @@ import os
from pathlib import Path from pathlib import Path
import re import re
import time import time
from urllib.parse import urlparse
from dataclasses import dataclass from dataclasses import dataclass
from datetime import datetime, timezone from datetime import datetime, timezone
from typing import Any, Optional from typing import Any, Optional
@ -41,6 +43,7 @@ from ...config.ai_prompts import (
PROMPT_RAG_FOLLOWUP_ANCHOR_SEMANTICS, PROMPT_RAG_FOLLOWUP_ANCHOR_SEMANTICS,
PROMPT_RAG_QUERY_SEMANTICS, PROMPT_RAG_QUERY_SEMANTICS,
PROMPT_RAG_ANSWER_SCENE_SEMANTICS, PROMPT_RAG_ANSWER_SCENE_SEMANTICS,
PROMPT_INTERNAL_TARGET_RESOLUTION,
) )
from ...core.database import AsyncSessionLocal from ...core.database import AsyncSessionLocal
from ...models import ( from ...models import (
@ -56,8 +59,10 @@ from ...models import (
ProjectSignalLink, ProjectSignalLink,
QAFeedback, QAFeedback,
QARecord, QARecord,
QueryRouteMemory,
QASession, QASession,
QASessionState, QASessionState,
SemanticFact,
Signal, Signal,
Task, Task,
TruthNode, TruthNode,

View File

@ -24,8 +24,11 @@ DEEP_CONTEXT_BLOCKS = _env_int("RAG_DEEP_CONTEXT_BLOCKS", 16, minimum=8, maximum
DEEP_ANSWER_MAX_TOKENS = _env_int("RAG_DEEP_ANSWER_MAX_TOKENS", 1400, minimum=900, maximum=3000) DEEP_ANSWER_MAX_TOKENS = _env_int("RAG_DEEP_ANSWER_MAX_TOKENS", 1400, minimum=900, maximum=3000)
QUERY_TYPE_COUNT = "count" QUERY_TYPE_COUNT = "count"
QUERY_TYPE_LIST = "list" QUERY_TYPE_LIST = "list"
QUERY_TYPE_EXISTENCE = "existence"
QUERY_TYPE_STATUS = "status" QUERY_TYPE_STATUS = "status"
QUERY_TYPE_EXPLAIN = "explain" QUERY_TYPE_EXPLAIN = "explain"
QUERY_TYPE_DIAGNOSE = "diagnose"
QUERY_TYPE_COMPARE = "compare"
QUERY_TYPE_UNKNOWN = "unknown" QUERY_TYPE_UNKNOWN = "unknown"
ANSWER_MODE_FAST_PATH = "fast_path" ANSWER_MODE_FAST_PATH = "fast_path"
ANSWER_MODE_RETRIEVAL_NORMAL = "retrieval_normal" ANSWER_MODE_RETRIEVAL_NORMAL = "retrieval_normal"

View File

@ -477,10 +477,15 @@ class ReplyRenderer:
} }
if intent.name == INTENT_QUERY: if intent.name == INTENT_QUERY:
answer = str(execution_result.get("answer") or "当前暂无可用回答。").strip() answer = str(execution_result.get("answer") or "当前暂无可用回答。").strip()
template_key = str(execution_result.get("reply_template_key") or "").strip() or "action_succeeded"
return { return {
"receipt_phase": self.receipt_phase_action, "receipt_phase": self.receipt_phase_action,
"template_key": "action_succeeded", "template_key": template_key,
"payload": {"intent": intent.name, "answer": answer}, "payload": {
"intent": intent.name,
"answer": answer,
"reply_template_key": template_key,
},
"text": answer, "text": answer,
} }
if intent.name == INTENT_PROJECT_CREATE: if intent.name == INTENT_PROJECT_CREATE:

View File

@ -179,12 +179,6 @@ class KernelIntentResolveMixin:
"requires_confirmation": False, "requires_confirmation": False,
"source_system": str(source_system or "").strip().lower() or None, "source_system": str(source_system or "").strip().lower() or None,
"source_message_context": "reply_anchor" if reply_anchor_present else "cold_start", "source_message_context": "reply_anchor" if reply_anchor_present else "cold_start",
# Planner outage should not hard-fail the thread. Degrade to the
# controlled requirement consultation chain so downstream logic can
# keep the user inside the requirement workflow instead of emitting
# a generic error template.
"message_intent": "consult_requirement",
"consultant_mode_preference": "clarify",
"semantic_planner_degraded": True, "semantic_planner_degraded": True,
"semantic_planner_error": str(planner_error)[:240], "semantic_planner_error": str(planner_error)[:240],
} }

View File

@ -32,97 +32,20 @@ class KernelIngressFlowEntryMixin:
"requirement_gathering", "requirement_gathering",
"requirement_convergence", "requirement_convergence",
}) })
_EXTERNAL_FRESHNESS_FOLLOWUP_TOKENS = frozenset({ _EXTERNAL_FRESHNESS_GUARD_SOURCE_LABEL = "externalfreshnessguard"
"继续核实",
"继续查",
"继续验证",
"继续确认",
"接入外部来源",
"外部来源继续核实",
"联网查",
"联网核实",
"外网查",
"实时来源",
"查最新",
"再核实",
})
_INTERNAL_EXTERNAL_SOURCE_TARGET_MARKERS = frozenset({
"我们系统",
"系统里",
"系统内",
"pmflow",
"/ai-config",
"ai-config",
"aiconfig",
"配置页",
"模型供应商",
"给项目",
"项目id",
"项目#",
"project",
"模块",
"功能",
"接口",
"通道",
"租户",
"环境",
"服务",
})
_INTERNAL_EXTERNAL_SOURCE_ACTION_MARKERS = frozenset({
"出需求",
"需求草稿",
"起草",
"立项",
"工单",
"排期",
"任务",
"入库",
"接一个",
"新增",
"加一个",
"配置",
"接入通道",
"集成",
"实现",
"改造",
})
_EXTERNAL_FRESHNESS_GUARD_MARKERS = frozenset({
"externalfreshnessguard",
"未接入实时外部数据",
"实时外部来源",
"无法确认最新结论",
})
@staticmethod @staticmethod
def _normalize_compact_text(value: Any) -> str: def _normalize_compact_text(value: Any) -> str:
return re.sub(r"\s+", "", str(value or "")).strip().lower() return re.sub(r"\s+", "", str(value or "")).strip().lower()
@classmethod @classmethod
def _looks_like_external_freshness_followup(cls, message_text: str) -> bool: def _payload_has_external_freshness_guard_source(cls, value: Any) -> bool:
normalized = cls._normalize_compact_text(message_text)
if not normalized:
return False
return any(token in normalized for token in cls._EXTERNAL_FRESHNESS_FOLLOWUP_TOKENS)
@classmethod
def _looks_like_explicit_internal_external_source_request(cls, message_text: str) -> bool:
normalized = cls._normalize_compact_text(message_text)
if not normalized:
return False
return (
any(token in normalized for token in cls._INTERNAL_EXTERNAL_SOURCE_TARGET_MARKERS)
and any(token in normalized for token in cls._INTERNAL_EXTERNAL_SOURCE_ACTION_MARKERS)
)
@classmethod
def _payload_mentions_external_freshness_guard(cls, value: Any) -> bool:
if isinstance(value, str): if isinstance(value, str):
normalized = cls._normalize_compact_text(value) return cls._normalize_compact_text(value) == cls._EXTERNAL_FRESHNESS_GUARD_SOURCE_LABEL
return any(marker in normalized for marker in cls._EXTERNAL_FRESHNESS_GUARD_MARKERS)
if isinstance(value, dict): if isinstance(value, dict):
return any(cls._payload_mentions_external_freshness_guard(item) for item in value.values()) return any(cls._payload_has_external_freshness_guard_source(item) for item in value.values())
if isinstance(value, list): if isinstance(value, list):
return any(cls._payload_mentions_external_freshness_guard(item) for item in value) return any(cls._payload_has_external_freshness_guard_source(item) for item in value)
return False return False
@classmethod @classmethod
@ -135,12 +58,13 @@ class KernelIngressFlowEntryMixin:
return ( return (
( (
grounding_status in {"insufficient_evidence", "partially_grounded"} grounding_status in {"insufficient_evidence", "partially_grounded"}
and cls._payload_mentions_external_freshness_guard(getattr(qa_record, "sources_used", None)) and cls._payload_has_external_freshness_guard_source(getattr(qa_record, "sources_used", None))
) )
or ( or (
"prompt:industry_trend" in policy_version or "prompt:external_trend" in policy_version "prompt:industry_trend" in policy_version or "prompt:external_trend" in policy_version
) )
and cls._payload_mentions_external_freshness_guard(answer_text) and cls._payload_has_external_freshness_guard_source(getattr(qa_record, "sources_used", None))
and "无法确认最新结论" in answer_text
) )
async def _load_recent_external_freshness_guard_record( async def _load_recent_external_freshness_guard_record(
@ -183,34 +107,47 @@ class KernelIngressFlowEntryMixin:
).scalars().first() ).scalars().first()
if self._is_external_freshness_guard_record(anchored_row): if self._is_external_freshness_guard_record(anchored_row):
return anchored_row return anchored_row
if normalized_source != "feishu":
return None
conversation_scope_key = build_feishu_conversation_scope_key(
source_chat_id=channel_ref,
source_sender_open_id=source_sender_open_id,
sender=None,
)
if not conversation_scope_key:
return None
rows = (
await self.db.execute(
select(QARecord)
.where(
*criteria,
QARecord.conversation_scope_key == conversation_scope_key,
)
.order_by(QARecord.updated_at.desc(), QARecord.id.desc())
.limit(3)
)
).scalars().all()
for row in rows:
if str(getattr(row, "source_message_id", "") or "").strip() == str(source_message_id or "").strip():
continue
if self._is_external_freshness_guard_record(row):
return row
return None return None
@classmethod
def _is_query_answer_only_bundle(cls, bundle: Any) -> bool:
workflows = list(getattr(bundle, "workflows", []) or [])
primary_workflow = str(getattr(bundle, "primary_workflow", "") or "").strip().lower()
if primary_workflow and primary_workflow != "query_flow":
return False
if not workflows:
return primary_workflow in {"", "query_flow"}
for workflow in workflows:
workflow_type = str(getattr(workflow, "workflow_type", "") or "").strip().lower()
if workflow_type and workflow_type != "query_flow":
return False
operation_codes = [str(code or "").strip().lower() for code in list(getattr(workflow, "operation_codes", []) or []) if str(code or "").strip()]
if operation_codes and any(code != "query.answer" for code in operation_codes):
return False
return True
async def _should_keep_external_freshness_followup_in_query_answer(
self,
*,
message_text: str,
source_system: str,
prior_question: Optional[str],
thread_context: Optional[dict[str, Any]],
) -> bool:
try:
bundle = await WorkflowSemanticPlanner.aplan_message(
message_text=str(message_text or "").strip(),
source_system=str(source_system or "").strip().lower() or "adaptive_rag",
metadata={
"external_freshness_followup": True,
"prior_question": str(prior_question or "").strip()[:240] or None,
"thread_context": dict(thread_context or {}),
},
)
except Exception:
return True
return self._is_query_answer_only_bundle(bundle)
@classmethod @classmethod
def _build_external_freshness_followup_reply( def _build_external_freshness_followup_reply(
cls, cls,
@ -237,7 +174,7 @@ class KernelIngressFlowEntryMixin:
if not base_question: if not base_question:
return followup_text return followup_text
normalized_followup = cls._normalize_compact_text(followup_text) normalized_followup = cls._normalize_compact_text(followup_text)
if not normalized_followup or cls._looks_like_external_freshness_followup(followup_text): if not normalized_followup or normalized_followup == cls._normalize_compact_text(base_question):
return base_question return base_question
return f"{base_question}。补充要求:{followup_text}"[:1200] return f"{base_question}。补充要求:{followup_text}"[:1200]
@ -251,10 +188,6 @@ class KernelIngressFlowEntryMixin:
message_text: str, message_text: str,
thread_context: Optional[dict[str, Any]], thread_context: Optional[dict[str, Any]],
) -> Optional[dict[str, Any]]: ) -> Optional[dict[str, Any]]:
if not self._looks_like_external_freshness_followup(message_text):
return None
if self._looks_like_explicit_internal_external_source_request(message_text):
return None
try: try:
guard_record = await self._load_recent_external_freshness_guard_record( guard_record = await self._load_recent_external_freshness_guard_record(
source_system=source_system, source_system=source_system,
@ -276,6 +209,13 @@ class KernelIngressFlowEntryMixin:
return None return None
prior_question = str(getattr(guard_record, "question", "") or "").strip() or None prior_question = str(getattr(guard_record, "question", "") or "").strip() or None
if not await self._should_keep_external_freshness_followup_in_query_answer(
message_text=message_text,
source_system=source_system,
prior_question=prior_question,
thread_context=thread_context,
):
return None
followup_question = self._build_external_freshness_followup_question( followup_question = self._build_external_freshness_followup_question(
prior_question=prior_question, prior_question=prior_question,
message_text=message_text, message_text=message_text,
@ -975,37 +915,6 @@ class KernelIngressFlowEntryMixin:
ingest_event=ingest_event, ingest_event=ingest_event,
ingress_options=ingress_options, ingress_options=ingress_options,
) )
terminated_requirement_scope = await mounted_runtime.terminate_requirement_plan_scope(
tenant_id=int(self.tenant_id),
source_system=source_system,
channel_ref=channel_ref,
sender=sender,
source_sender_open_id=source_sender_open_id,
source_message_id=synthetic_message_id,
)
if int(terminated_requirement_scope.get("terminated_count") or 0) > 0:
try:
await self._update_session(
tenant_id=int(self.tenant_id),
sender=sender,
session=session,
source_chat_id=channel_ref,
source_sender_open_id=source_sender_open_id,
source_message_id=synthetic_message_id,
thread_context=thread_context,
current_project_id=None,
clear_current_project_id=True,
last_intent=ROUTE_QUERY_ANSWER,
last_message_text=clean_text,
)
except Exception as session_exc:
logger.warning(
"mounted_ai_runtime_terminate_requirement_scope_session_update_failed",
tenant_id=self.tenant_id,
source_system=source_system,
source_message_id=synthetic_message_id,
error=str(session_exc),
)
active_context_followup_result = await self._maybe_handle_active_context_followup( active_context_followup_result = await self._maybe_handle_active_context_followup(
source_system=source_system, source_system=source_system,
source_message_id=synthetic_message_id, source_message_id=synthetic_message_id,
@ -1847,6 +1756,9 @@ class KernelIngressFlowEntryMixin:
return topic, draft, self._derive_active_context_kind(topic=topic, draft=draft) return topic, draft, self._derive_active_context_kind(topic=topic, draft=draft)
finder = getattr(self, "_find_active_requirement_plan_workspace", None) finder = getattr(self, "_find_active_requirement_plan_workspace", None)
if not callable(finder):
executor = getattr(self, "executor", None)
finder = getattr(executor, "_find_active_requirement_plan_workspace", None)
if not callable(finder): if not callable(finder):
return None return None
topic, draft = await finder( topic, draft = await finder(
@ -2022,9 +1934,49 @@ class KernelIngressFlowEntryMixin:
except Exception: except Exception:
return False return False
@staticmethod
def _looks_like_requirement_plan_exit_request(message_text: str) -> bool:
try:
from ..ai_pmo_typed_executor_execute.mixin_execute_query_project import ExecuteQueryProjectMixin
return ExecuteQueryProjectMixin._is_requirement_plan_exit_request(message_text)
except Exception:
return False
async def _build_active_requirement_mode_exit_result(
self,
*,
topic: Any,
source_message_id: str,
) -> dict[str, Any]:
target_topic_id = _safe_int(getattr(topic, "id", None))
topic.status = "suspended"
topic.plan_status = "suspended"
topic.pending_actions = ["resume_requirement_plan"]
topic.metadata_json = {
**(topic.metadata_json if isinstance(getattr(topic, "metadata_json", None), dict) else {}),
"plan_mode": True,
"suspended_by_message_id": str(source_message_id or "").strip() or None,
}
flusher = getattr(self.db, "flush", None)
if callable(flusher):
await flusher()
return self._build_active_context_direct_reply_result(
answer="好,这条需求先暂停。你后面引用原消息或说“继续需求”我再接上;现在可以聊别的。",
reason_code="requirement_mode_suspended",
payload={
"workflow_route": ROUTE_REQUIREMENT_PLAN_COLLECT,
"work_item_id": target_topic_id,
"topic_workspace_id": target_topic_id,
"group_reply_allowed_via_bound_context": True,
"query_type": "plan",
},
)
async def _build_active_requirement_followup_result( async def _build_active_requirement_followup_result(
self, self,
*, *,
topic_workspace_id: Optional[int],
message_text: str, message_text: str,
source_system: str, source_system: str,
source_message_id: str, source_message_id: str,
@ -2054,6 +2006,16 @@ class KernelIngressFlowEntryMixin:
if not answer: if not answer:
return None return None
direct_payload = dict(direct_result) if isinstance(direct_result, dict) else {} direct_payload = dict(direct_result) if isinstance(direct_result, dict) else {}
target_topic_id = _safe_int(topic_workspace_id)
if (
target_topic_id is not None
and _safe_int(direct_payload.get("work_item_id")) is None
and str(direct_payload.get("reply_template_key") or "").strip().lower() == "requirement_plan_followup"
):
# Requirement-plan collect cards need a stable positive target id for button round-trips.
# At collect stage there is often no dedicated work item yet, so we reuse the mounted topic id.
direct_payload["work_item_id"] = int(target_topic_id)
direct_payload.setdefault("topic_workspace_id", int(target_topic_id))
direct_payload["group_reply_allowed_via_bound_context"] = True direct_payload["group_reply_allowed_via_bound_context"] = True
return self._build_active_context_direct_reply_result( return self._build_active_context_direct_reply_result(
answer=answer, answer=answer,
@ -2098,11 +2060,20 @@ class KernelIngressFlowEntryMixin:
if bound_context is None: if bound_context is None:
return None return None
topic, draft, active_context_kind = bound_context topic, draft, active_context_kind = bound_context
if (
str(active_context_kind or "").strip().lower() == "requirement_plan"
and self._looks_like_requirement_plan_exit_request(message_text)
):
return await self._build_active_requirement_mode_exit_result(
topic=topic,
source_message_id=source_message_id,
)
if ( if (
str(active_context_kind or "").strip().lower() == "requirement_plan" str(active_context_kind or "").strip().lower() == "requirement_plan"
and self._looks_like_requirement_plan_choice_protocol_followup(message_text) and self._looks_like_requirement_plan_choice_protocol_followup(message_text)
): ):
direct_protocol_result = await self._build_active_requirement_followup_result( direct_protocol_result = await self._build_active_requirement_followup_result(
topic_workspace_id=_safe_int(getattr(topic, "id", None)),
message_text=message_text, message_text=message_text,
source_system=source_system, source_system=source_system,
source_message_id=source_message_id, source_message_id=source_message_id,
@ -2168,6 +2139,7 @@ class KernelIngressFlowEntryMixin:
if "context.followup.current_requirement" not in allowed_actions: if "context.followup.current_requirement" not in allowed_actions:
return None return None
direct_result = await self._build_active_requirement_followup_result( direct_result = await self._build_active_requirement_followup_result(
topic_workspace_id=_safe_int(getattr(topic, "id", None)),
message_text=message_text, message_text=message_text,
source_system=source_system, source_system=source_system,
source_message_id=source_message_id, source_message_id=source_message_id,

View File

@ -103,7 +103,10 @@ class ExecuteQueryProjectMixin:
message_text: str, message_text: str,
project_id: Optional[int], project_id: Optional[int],
message_intent: Optional[str] = None, message_intent: Optional[str] = None,
source_policy: Optional[str] = None,
) -> Optional[dict[str, Any]]: ) -> Optional[dict[str, Any]]:
if str(source_policy or "").strip().lower() == "external":
return None
normalized_intent = str(message_intent or "").strip().lower() normalized_intent = str(message_intent or "").strip().lower()
keyword_extractor = getattr(rag_service, "_extract_project_lookup_keyword", None) keyword_extractor = getattr(rag_service, "_extract_project_lookup_keyword", None)
extracted_keyword = "" extracted_keyword = ""
@ -146,8 +149,11 @@ class ExecuteQueryProjectMixin:
*, *,
message_text: str, message_text: str,
message_intent: Optional[str], message_intent: Optional[str],
source_policy: Optional[str] = None,
) -> bool: ) -> bool:
_ = message_text _ = message_text
if str(source_policy or "").strip().lower() == "external":
return False
return str(message_intent or "").strip().lower() == "casual_chat" return str(message_intent or "").strip().lower() == "casual_chat"
@classmethod @classmethod
@ -3999,8 +4005,11 @@ class ExecuteQueryProjectMixin:
] ]
if not plan_candidates: if not plan_candidates:
return None, None return None, None
preferred_candidates = [
row for row in plan_candidates if int(row.id) in draft_by_topic_id
] or plan_candidates
topic = self._select_requirement_plan_topic( topic = self._select_requirement_plan_topic(
candidates=plan_candidates, candidates=preferred_candidates,
topic_key=topic_key, topic_key=topic_key,
root_message_id=root_message_id, root_message_id=root_message_id,
reply_to_message_id=reply_to_message_id, reply_to_message_id=reply_to_message_id,
@ -4186,6 +4195,7 @@ class ExecuteQueryProjectMixin:
self, self,
*, *,
message_text: str, message_text: str,
message_intent: Optional[str],
topic_switch: Optional[str], topic_switch: Optional[str],
has_requirement_anchor: bool, has_requirement_anchor: bool,
topic: Optional[MountedTopicWorkspace], topic: Optional[MountedTopicWorkspace],
@ -4197,6 +4207,16 @@ class ExecuteQueryProjectMixin:
return False return False
if self._extract_requirement_plan_choice(message_text): if self._extract_requirement_plan_choice(message_text):
return False return False
normalized_message_intent = str(message_intent or "").strip().lower()
if normalized_message_intent not in {
"requirement_plan",
"requirement_gathering",
"requirement_convergence",
"consult_requirement",
"request_draft_generation",
"request_draft_generation_from_context",
}:
return False
normalized_topic_switch = str(topic_switch or "").strip().lower() normalized_topic_switch = str(topic_switch or "").strip().lower()
nontrivial_seed = self._sanitize_requirement_plan_text(message_text).strip() nontrivial_seed = self._sanitize_requirement_plan_text(message_text).strip()
@ -5074,6 +5094,10 @@ class ExecuteQueryProjectMixin:
metadata = intent.metadata if isinstance(intent.metadata, dict) else {} metadata = intent.metadata if isinstance(intent.metadata, dict) else {}
semantic_bundle = metadata.get("semantic_bundle") if isinstance(metadata.get("semantic_bundle"), dict) else {} semantic_bundle = metadata.get("semantic_bundle") if isinstance(metadata.get("semantic_bundle"), dict) else {}
semantic_diagnostics = semantic_bundle.get("diagnostics") if isinstance(semantic_bundle.get("diagnostics"), dict) else {} semantic_diagnostics = semantic_bundle.get("diagnostics") if isinstance(semantic_bundle.get("diagnostics"), dict) else {}
semantic_planner_degraded = bool(
metadata.get("semantic_planner_degraded") or semantic_diagnostics.get("semantic_planner_degraded")
)
degraded_context_source = str(metadata.get("context_source") or "").strip().lower() == "semantic_planner_degraded"
rag_request_context = ( rag_request_context = (
metadata.get("rag_request_context") metadata.get("rag_request_context")
if isinstance(metadata.get("rag_request_context"), dict) if isinstance(metadata.get("rag_request_context"), dict)
@ -5169,6 +5193,12 @@ class ExecuteQueryProjectMixin:
or semantic_diagnostics.get("force_thinking_mode") or semantic_diagnostics.get("force_thinking_mode")
or "" or ""
).strip() or None ).strip() or None
if semantic_planner_degraded and degraded_context_source:
# Planner failure metadata is only an audit marker. Do not let stale
# degraded scene labels drive requirement routing for a fresh query.
message_intent = None
consultant_mode_preference = None
advise_subtype = None
normalized_scene_metadata = WorkflowSemanticPlanner.normalize_scene_metadata( normalized_scene_metadata = WorkflowSemanticPlanner.normalize_scene_metadata(
{ {
"message_intent": message_intent, "message_intent": message_intent,
@ -5250,6 +5280,8 @@ class ExecuteQueryProjectMixin:
or needs_legacy_business_review_refresh or needs_legacy_business_review_refresh
) )
) )
if semantic_planner_degraded:
needs_semantic_backfill = True
if not normalized_message_intent: if not normalized_message_intent:
needs_semantic_backfill = True needs_semantic_backfill = True
elif normalized_message_intent in {"business_review", "requirement_plan"} and not str( elif normalized_message_intent in {"business_review", "requirement_plan"} and not str(
@ -5269,52 +5301,74 @@ class ExecuteQueryProjectMixin:
has_active_requirement_plan_workspace = ( has_active_requirement_plan_workspace = (
active_requirement_plan_topic is not None and active_requirement_plan_draft is not None active_requirement_plan_topic is not None and active_requirement_plan_draft is not None
) )
semantic_bundle = await self._backfill_query_semantic_bundle( try:
message_text=message_text, semantic_bundle = await self._backfill_query_semantic_bundle(
source_system=source_system, message_text=message_text,
metadata=metadata, source_system=source_system,
requested_project_id=requested_project_id if requested_project_id is not None else intent.project_id, metadata=metadata,
source_chat_id=source_chat_id, requested_project_id=requested_project_id if requested_project_id is not None else intent.project_id,
source_sender_open_id=source_sender_open_id, source_chat_id=source_chat_id,
thread_context=thread_context, source_sender_open_id=source_sender_open_id,
conversation_key=conversation_key, thread_context=thread_context,
active_requirement_plan_topic=active_requirement_plan_topic if has_requirement_anchor else None, conversation_key=conversation_key,
active_requirement_plan_draft=active_requirement_plan_draft if has_requirement_anchor else None, active_requirement_plan_topic=active_requirement_plan_topic if has_requirement_anchor else None,
) active_requirement_plan_draft=active_requirement_plan_draft if has_requirement_anchor else None,
semantic_diagnostics = ( )
semantic_bundle.get("diagnostics") if isinstance(semantic_bundle.get("diagnostics"), dict) else {} except Exception as exc:
) logger.warning(
refreshed_message_intent = str(semantic_diagnostics.get("message_intent") or "").strip() or None "query_semantic_backfill_failed",
refreshed_consultant_mode = ( tenant_id=self.tenant_id,
str(semantic_diagnostics.get("consultant_mode_preference") or "").strip().lower() or None source_system=source_system,
) source_message_id=source_message_id,
refreshed_advise_subtype = ( semantic_planner_degraded=semantic_planner_degraded,
str(semantic_diagnostics.get("advise_subtype") or "").strip().lower() or None error=str(exc),
) )
refreshed_topic_switch = str(semantic_diagnostics.get("topic_switch") or "").strip().lower() or None if isinstance(intent.metadata, dict):
refreshed_thinking_mode = str(semantic_diagnostics.get("force_thinking_mode") or "").strip() or None intent.metadata["semantic_backfill_failed"] = True
if refreshed_message_intent: intent.metadata["semantic_backfill_error"] = str(exc)[:240]
message_intent = refreshed_message_intent if semantic_planner_degraded and degraded_context_source and not has_active_requirement_plan_workspace:
if refreshed_consultant_mode: message_intent = None
consultant_mode_preference = refreshed_consultant_mode consultant_mode_preference = None
if refreshed_advise_subtype: advise_subtype = None
advise_subtype = refreshed_advise_subtype else:
if refreshed_topic_switch: semantic_diagnostics = (
topic_switch = refreshed_topic_switch semantic_bundle.get("diagnostics") if isinstance(semantic_bundle.get("diagnostics"), dict) else {}
if refreshed_thinking_mode: )
thinking_mode = refreshed_thinking_mode refreshed_message_intent = str(semantic_diagnostics.get("message_intent") or "").strip() or None
metadata = {**metadata, "semantic_bundle": semantic_bundle} refreshed_consultant_mode = (
normalized_scene_metadata = WorkflowSemanticPlanner.normalize_scene_metadata( str(semantic_diagnostics.get("consultant_mode_preference") or "").strip().lower() or None
{ )
"message_intent": message_intent, refreshed_advise_subtype = (
"consultant_mode_preference": consultant_mode_preference, str(semantic_diagnostics.get("advise_subtype") or "").strip().lower() or None
} )
) refreshed_topic_switch = str(semantic_diagnostics.get("topic_switch") or "").strip().lower() or None
message_intent = str(normalized_scene_metadata.get("message_intent") or message_intent or "").strip() or None refreshed_thinking_mode = str(semantic_diagnostics.get("force_thinking_mode") or "").strip() or None
consultant_mode_preference = ( if refreshed_message_intent:
str(normalized_scene_metadata.get("consultant_mode_preference") or consultant_mode_preference or "").strip().lower() message_intent = refreshed_message_intent
or None if refreshed_consultant_mode:
) consultant_mode_preference = refreshed_consultant_mode
if refreshed_advise_subtype:
advise_subtype = refreshed_advise_subtype
if refreshed_topic_switch:
topic_switch = refreshed_topic_switch
if refreshed_thinking_mode:
thinking_mode = refreshed_thinking_mode
metadata = {**metadata, "semantic_bundle": semantic_bundle}
normalized_scene_metadata = WorkflowSemanticPlanner.normalize_scene_metadata(
{
"message_intent": message_intent,
"consultant_mode_preference": consultant_mode_preference,
}
)
message_intent = str(normalized_scene_metadata.get("message_intent") or message_intent or "").strip() or None
consultant_mode_preference = (
str(
normalized_scene_metadata.get("consultant_mode_preference")
or consultant_mode_preference
or ""
).strip().lower()
or None
)
normalized_message_intent = str(message_intent or "").strip().lower() normalized_message_intent = str(message_intent or "").strip().lower()
if not has_active_requirement_plan_workspace and normalized_message_intent == "requirement_plan": if not has_active_requirement_plan_workspace and normalized_message_intent == "requirement_plan":
active_requirement_plan_topic, active_requirement_plan_draft = await self._find_active_requirement_plan_workspace( active_requirement_plan_topic, active_requirement_plan_draft = await self._find_active_requirement_plan_workspace(
@ -5346,6 +5400,7 @@ class ExecuteQueryProjectMixin:
) )
if self._should_force_new_same_scope_requirement_plan( if self._should_force_new_same_scope_requirement_plan(
message_text=message_text, message_text=message_text,
message_intent=message_intent,
topic_switch=topic_switch, topic_switch=topic_switch,
has_requirement_anchor=has_requirement_anchor, has_requirement_anchor=has_requirement_anchor,
topic=active_requirement_plan_topic, topic=active_requirement_plan_topic,
@ -5489,7 +5544,11 @@ class ExecuteQueryProjectMixin:
) )
direct_result["project_id"] = intent.project_id direct_result["project_id"] = intent.project_id
return direct_result return direct_result
if requirement_mode_locked and normalized_message_intent == "casual_chat": if requirement_mode_locked and normalized_message_intent not in {
"requirement_plan",
"requirement_gathering",
"requirement_convergence",
}:
direct_result = self._build_direct_query_result( direct_result = self._build_direct_query_result(
answer=self._build_requirement_mode_locked_answer(card=locked_card), answer=self._build_requirement_mode_locked_answer(card=locked_card),
scene_label="Mounted Runtime Plan", scene_label="Mounted Runtime Plan",
@ -5506,9 +5565,48 @@ class ExecuteQueryProjectMixin:
consultant_mode_preference = ( consultant_mode_preference = (
str(consultant_mode_preference or "exploratory").strip().lower() or "exploratory" str(consultant_mode_preference or "exploratory").strip().lower() or "exploratory"
) )
route_preview: dict[str, Any] = {}
route_preview_resolver = getattr(rag_service, "preview_query_route_decision", None)
if callable(route_preview_resolver):
try:
route_preview = await route_preview_resolver(
question=message_text,
asked_via=asked_via,
project_id=requested_project_id if requested_project_id is not None else intent.project_id,
source_chat_id=source_chat_id,
source_message_id=source_message_id,
conversation_scope_key=conversation_key,
reply_to_message_id=str((thread_context or {}).get("reply_to_message_id") or "").strip() or None,
root_message_id=str((thread_context or {}).get("root_message_id") or "").strip() or None,
)
except Exception:
route_preview = {}
source_policy_hint = (
route_preview.get("query_semantics")
if isinstance(route_preview.get("query_semantics"), dict)
else {}
)
route_decision = (
route_preview.get("route_decision")
if isinstance(route_preview.get("route_decision"), dict)
else {}
)
source_policy = (
str(source_policy_hint.get("source_policy") or route_decision.get("route_mode") or "").strip().lower() or None
)
effective_source_policy = (
str(
source_policy_hint.get("effective_source_policy")
or route_decision.get("effective_source_policy")
or source_policy
or ""
).strip().lower()
or None
)
if self._should_use_lightweight_chat_answer( if self._should_use_lightweight_chat_answer(
message_text=message_text, message_text=message_text,
message_intent=message_intent, message_intent=message_intent,
source_policy=effective_source_policy,
): ):
direct_result = self._build_direct_query_result( direct_result = self._build_direct_query_result(
answer=self._build_lightweight_chat_answer(message_text=message_text), answer=self._build_lightweight_chat_answer(message_text=message_text),
@ -5587,6 +5685,7 @@ class ExecuteQueryProjectMixin:
message_text=message_text, message_text=message_text,
project_id=requested_project_id if requested_project_id is not None else intent.project_id, project_id=requested_project_id if requested_project_id is not None else intent.project_id,
message_intent=message_intent, message_intent=message_intent,
source_policy=effective_source_policy,
) )
if project_lookup_result is not None: if project_lookup_result is not None:
return project_lookup_result return project_lookup_result
@ -5613,6 +5712,9 @@ class ExecuteQueryProjectMixin:
"message_intent": message_intent, "message_intent": message_intent,
"mixed_signal_part": mixed_signal_part, "mixed_signal_part": mixed_signal_part,
"consultant_mode_preference": consultant_mode_preference, "consultant_mode_preference": consultant_mode_preference,
"source_policy": source_policy,
"precomputed_query_semantics": source_policy_hint if isinstance(source_policy_hint, dict) and source_policy_hint else None,
"route_decision": route_decision if isinstance(route_decision, dict) and route_decision else None,
} }
if session_id is not None: if session_id is not None:
rag_kwargs["session_id"] = int(session_id) rag_kwargs["session_id"] = int(session_id)

View File

@ -1504,6 +1504,12 @@ class DraftConfirmHandler:
mounted=(draft_row, topic_row), mounted=(draft_row, topic_row),
) )
continue continue
if anchor_context and not self._matches_mounted_draft_anchor(
draft=draft_row,
topic=topic_row,
anchor_context=anchor_context,
):
continue
score = 0 score = 0
if normalized_channel and str(getattr(topic_row, "channel_ref", "") or "").strip() == normalized_channel: if normalized_channel and str(getattr(topic_row, "channel_ref", "") or "").strip() == normalized_channel:
score += 30 score += 30

View File

@ -40,6 +40,8 @@ class WorkflowSemanticPlanner:
} }
_SCENE_CONSULTANT_MODE_ALIASES = { _SCENE_CONSULTANT_MODE_ALIASES = {
"structured_analysis": "structured_analysis_first", "structured_analysis": "structured_analysis_first",
"requirement_analysis": "structured_analysis_first",
"requirements_analysis": "structured_analysis_first",
"collaborative": "exploratory", "collaborative": "exploratory",
} }

View File

@ -0,0 +1,21 @@
from __future__ import annotations
from .gateway import ExternalQueryGateway
from .planner import ExternalQueryPlanner
from .registry import ExternalProviderRegistry
from .schemas import (
ExternalAdapterResult,
ExternalEvidenceItem,
ExternalQueryExecution,
ExternalQueryPlan,
)
__all__ = [
"ExternalAdapterResult",
"ExternalEvidenceItem",
"ExternalProviderRegistry",
"ExternalQueryExecution",
"ExternalQueryGateway",
"ExternalQueryPlan",
"ExternalQueryPlanner",
]

View File

@ -0,0 +1,6 @@
from __future__ import annotations
from .live_search import LiveSearchFallbackAdapter
from .mcp import MCPExternalAdapter
__all__ = ["LiveSearchFallbackAdapter", "MCPExternalAdapter"]

View File

@ -0,0 +1,151 @@
from __future__ import annotations
from datetime import datetime, timezone
from typing import Any, Awaitable, Callable
from ..schemas import ExternalAdapterResult, ExternalEvidenceItem, ExternalQueryPlan
class LiveSearchFallbackAdapter:
adapter_name = "live_search"
supported_query_kinds = {"weather", "news", "market", "generic_web", "unknown"}
def __init__(
self,
*,
build_queries_callable: Callable[[str], list[str]] | None = None,
search_results_callable: Callable[..., Awaitable[list[dict[str, str]]]] | None = None,
fetch_text_callable: Callable[[str], Awaitable[str]] | None = None,
chunk_text_callable: Callable[..., list[str]] | None = None,
score_search_result_callable: Callable[..., float] | None = None,
score_chunk_callable: Callable[..., float] | None = None,
max_pages: int = 3,
) -> None:
self.build_queries_callable = build_queries_callable
self.search_results_callable = search_results_callable
self.fetch_text_callable = fetch_text_callable
self.chunk_text_callable = chunk_text_callable
self.score_search_result_callable = score_search_result_callable
self.score_chunk_callable = score_chunk_callable
self.max_pages = max(1, int(max_pages or 3))
async def execute(self, plan: ExternalQueryPlan) -> ExternalAdapterResult:
if not all(
callable(item)
for item in (
self.search_results_callable,
self.fetch_text_callable,
self.chunk_text_callable,
self.score_search_result_callable,
self.score_chunk_callable,
)
):
return ExternalAdapterResult(
adapter_name=self.adapter_name,
status="unavailable",
evidence=[],
meta={"reason": "live_search_callbacks_missing"},
)
queries = [str(item or "").strip() for item in list(plan.search_queries or []) if str(item or "").strip()]
if not queries and callable(self.build_queries_callable):
try:
queries = [
str(item or "").strip()
for item in list(self.build_queries_callable(plan.topic or "") or [])
if str(item or "").strip()
]
except Exception:
queries = []
if not queries:
return ExternalAdapterResult(adapter_name=self.adapter_name, status="empty", evidence=[], meta={"query": ""})
search_results: list[dict[str, str]] = []
selected_query = ""
for query in queries:
try:
search_results = await self.search_results_callable(
query=query,
max_results=max(3, int(plan.result_limit) * 2),
)
except Exception:
search_results = []
if search_results:
selected_query = query
break
ranked = sorted(
search_results,
key=lambda row: float(
self.score_search_result_callable(
question=plan.topic or selected_query or queries[0],
title=str(row.get("title") or ""),
url=str(row.get("url") or ""),
)
or 0.0
),
reverse=True,
)
evidence: list[ExternalEvidenceItem] = []
fetch_attempts = 0
candidate_limit = min(len(ranked), self.max_pages + 2)
for row in ranked[:candidate_limit]:
url = str(row.get("url") or "").strip()
title = str(row.get("title") or "").strip()
if not url:
continue
fetch_attempts += 1
try:
text = await self.fetch_text_callable(url)
except Exception:
continue
chunks = list(self.chunk_text_callable(text, chunk_size=900, overlap=120) or [])
if not chunks:
continue
best_chunk = ""
best_score = 0.0
for chunk in chunks[:6]:
score = float(
self.score_chunk_callable(
question=plan.topic or selected_query or queries[0],
title=title,
content=chunk,
url=url,
)
or 0.0
)
if score > best_score:
best_score = score
best_chunk = str(chunk or "").strip()
if not best_chunk:
continue
evidence.append(
ExternalEvidenceItem(
source_type="url",
source_label=title or url,
title=title[:255],
target_url=url,
snippet=best_chunk[:1800],
retrieved_at=datetime.now(timezone.utc).isoformat(),
freshness_score=round(best_score, 4),
authority_score=round(best_score, 4),
grounded_fields=["snippet"],
raw_metadata={
"query": selected_query or queries[0],
"provider": self.adapter_name,
"title": title[:255],
},
)
)
return ExternalAdapterResult(
adapter_name=self.adapter_name,
status="success" if evidence else "empty",
evidence=evidence[: max(1, int(plan.result_limit))],
meta={
"query": selected_query or (queries[0] if queries else ""),
"search_result_count": len(search_results),
"fetched_pages": fetch_attempts,
},
)

View File

@ -0,0 +1,129 @@
from __future__ import annotations
from typing import Any, Awaitable, Callable
from ..schemas import ExternalAdapterResult, ExternalEvidenceItem, ExternalQueryPlan
class MCPExternalAdapter:
adapter_name = "mcp"
supported_query_kinds = {"model", "company", "generic_web", "unknown"}
def __init__(
self,
*,
search_callable: Callable[[ExternalQueryPlan], Awaitable[dict[str, Any] | ExternalAdapterResult]] | None = None,
fetch_text_callable: Callable[[str], Awaitable[str]] | None = None,
chunk_text_callable: Callable[..., list[str]] | None = None,
score_chunk_callable: Callable[..., float] | None = None,
max_pages: int = 3,
) -> None:
self.search_callable = search_callable
self.fetch_text_callable = fetch_text_callable
self.chunk_text_callable = chunk_text_callable
self.score_chunk_callable = score_chunk_callable
self.max_pages = max(1, int(max_pages or 3))
async def execute(self, plan: ExternalQueryPlan) -> ExternalAdapterResult:
if not callable(self.search_callable):
return ExternalAdapterResult(
adapter_name=self.adapter_name,
status="unavailable",
evidence=[],
meta={"reason": "mcp_adapter_unavailable"},
)
result = await self.search_callable(plan)
if isinstance(result, ExternalAdapterResult):
validated = result
else:
payload = dict(result or {}) if isinstance(result, dict) else {}
payload.setdefault("adapter_name", self.adapter_name)
validated = ExternalAdapterResult.model_validate(payload)
evidence, fetched_pages = await self._hydrate_evidence_from_target_pages(
plan=plan,
evidence=list(validated.evidence or []),
)
meta = dict(validated.meta or {})
if fetched_pages:
meta["fetched_pages"] = int(fetched_pages)
return validated.model_copy(update={"evidence": evidence, "meta": meta})
def _page_fetch_ready(self) -> bool:
return all(
callable(item)
for item in (
self.fetch_text_callable,
self.chunk_text_callable,
self.score_chunk_callable,
)
)
async def _hydrate_evidence_from_target_pages(
self,
*,
plan: ExternalQueryPlan,
evidence: list[ExternalEvidenceItem],
) -> tuple[list[ExternalEvidenceItem], int]:
if not evidence or not self._page_fetch_ready():
return evidence, 0
hydrated: list[ExternalEvidenceItem] = []
fetch_attempts = 0
query = str((plan.search_queries[:1] or [plan.topic])[0] or "").strip() or str(plan.topic or "").strip()
for index, item in enumerate(evidence):
target_url = str(item.target_url or "").strip()
if index >= self.max_pages or (
not target_url.startswith("http://") and not target_url.startswith("https://")
):
hydrated.append(item)
continue
fetch_attempts += 1
try:
text = await self.fetch_text_callable(target_url)
except Exception:
hydrated.append(item)
continue
chunks = list(self.chunk_text_callable(text, chunk_size=900, overlap=120) or [])
if not chunks:
hydrated.append(item)
continue
best_chunk = ""
best_score = 0.0
title = str(item.title or "").strip()
for chunk in chunks[:6]:
score = float(
self.score_chunk_callable(
question=query or plan.topic or "",
title=title,
content=chunk,
url=target_url,
)
or 0.0
)
if score > best_score:
best_score = score
best_chunk = str(chunk or "").strip()
if not best_chunk:
hydrated.append(item)
continue
grounded_fields = [str(field or "").strip() for field in list(item.grounded_fields or []) if str(field or "").strip()]
if "page_content_snippet" not in grounded_fields:
grounded_fields.append("page_content_snippet")
raw_metadata = dict(item.raw_metadata or {})
original_snippet = str(item.snippet or "").strip()
if original_snippet and not str(raw_metadata.get("search_snippet") or "").strip():
raw_metadata["search_snippet"] = original_snippet[:1800]
raw_metadata["page_chunk_score"] = round(best_score, 4)
raw_metadata["target_page_fetched"] = True
hydrated.append(
item.model_copy(
update={
"snippet": best_chunk[:1800],
"freshness_score": round(max(float(item.freshness_score or 0.0), best_score), 4),
"grounded_fields": grounded_fields,
"raw_metadata": raw_metadata,
}
)
)
return hydrated, fetch_attempts

View File

@ -0,0 +1,271 @@
from __future__ import annotations
from typing import Any
from urllib.parse import urlparse
from .registry import ExternalProviderRegistry
from .schemas import ExternalAdapterResult, ExternalEvidenceItem, ExternalQueryExecution, ExternalQueryPlan
class ExternalQueryGateway:
def __init__(
self,
*,
planner: Any,
registry: ExternalProviderRegistry,
) -> None:
self.planner = planner
self.registry = registry
async def execute(
self,
*,
question: str,
query_semantics: dict[str, Any] | None = None,
) -> ExternalQueryExecution:
planned = await self.planner.build_plan(
question=question,
query_semantics=query_semantics,
)
planner_meta: dict[str, Any] = {}
if isinstance(planned, tuple) and len(planned) == 2:
plan, planner_meta = planned
else:
plan = planned
if not isinstance(plan, ExternalQueryPlan):
plan = ExternalQueryPlan.model_validate(plan)
adapter_attempts: list[dict[str, Any]] = []
selected = ExternalAdapterResult(adapter_name="none", status="empty", evidence=[], meta={})
selected_verifier = self._verify_evidence_bundle(plan=plan, evidence=selected.evidence)
fallback_used = False
for index, adapter in enumerate(self._rank_adapters(plan=plan)):
adapter_name = str(getattr(adapter, "adapter_name", "") or f"adapter_{index}").strip() or f"adapter_{index}"
result = await adapter.execute(plan)
if not isinstance(result, ExternalAdapterResult):
result = ExternalAdapterResult.model_validate(result)
verifier = self._verify_evidence_bundle(plan=plan, evidence=result.evidence)
adapter_attempts.append(
{
"adapter": adapter_name,
"status": str(result.status or "").strip() or "empty",
"evidence_count": len(result.evidence),
"verifier_ok": bool(verifier.get("ok")),
"verifier_reason": verifier.get("reason_code"),
}
)
if result.evidence and bool(verifier.get("ok")):
selected = result
selected_verifier = verifier
fallback_used = index > 0
break
if selected.adapter_name == "none":
selected = result
selected_verifier = verifier
meta = dict(selected.meta or {})
meta["fallback_used"] = bool(fallback_used)
meta["adapter_attempts"] = adapter_attempts
return ExternalQueryExecution(
plan=plan,
adapter_name=selected.adapter_name,
evidence=list(selected.evidence or []),
verifier=selected_verifier,
planner_meta=planner_meta,
meta=meta,
)
@staticmethod
def _normalize_match_text(value: Any) -> str:
raw = str(value or "").strip().lower()
if not raw:
return ""
chars: list[str] = []
for char in raw:
if char.isalnum():
chars.append(char)
return "".join(chars)
@classmethod
def _normalize_anchor_term(cls, value: Any) -> str:
return cls._normalize_match_text(value)[:48]
@staticmethod
def _common_prefix_size(left: str, right: str) -> int:
limit = min(len(left), len(right))
idx = 0
while idx < limit and left[idx] == right[idx]:
idx += 1
return idx
@classmethod
def _build_char_windows(cls, value: str, *, size: int = 2) -> set[str]:
normalized = cls._normalize_match_text(value)
if not normalized:
return set()
if len(normalized) <= size:
return {normalized}
return {normalized[idx : idx + size] for idx in range(len(normalized) - size + 1)}
@classmethod
def _tokenize_match_text(cls, value: Any) -> list[str]:
raw = str(value or "").strip().lower()
if not raw:
return []
tokens: list[str] = []
current: list[str] = []
for char in raw:
if char.isalnum():
current.append(char)
continue
if current:
token = "".join(current)
if token:
tokens.append(token)
current = []
if current:
token = "".join(current)
if token:
tokens.append(token)
normalized = cls._normalize_match_text(raw)
if normalized and normalized not in tokens:
tokens.append(normalized)
return tokens
@classmethod
def _score_anchor_against_token(cls, anchor: str, token: str) -> float:
if not anchor or not token:
return 0.0
if anchor == token:
return 1.0
if anchor in token:
return max(0.82, min(0.98, len(anchor) / max(1, len(token))))
if token in anchor and len(token) >= 2:
return max(0.58, len(token) / max(1, len(anchor)))
prefix_size = cls._common_prefix_size(anchor, token)
prefix_ratio = prefix_size / max(1, min(len(anchor), len(token)))
anchor_windows = cls._build_char_windows(anchor)
token_windows = cls._build_char_windows(token)
window_overlap = (
len(anchor_windows & token_windows) / max(1, len(anchor_windows))
if anchor_windows and token_windows
else 0.0
)
score = 0.0
if prefix_size >= 2:
score = max(score, 0.42 + min(0.3, prefix_ratio * 0.4))
if window_overlap > 0:
score = max(score, 0.35 + min(0.35, window_overlap * 0.5))
return round(min(score, 0.98), 4)
@classmethod
def _evidence_match_tokens(cls, item: ExternalEvidenceItem) -> list[str]:
values = [
str(item.source_label or ""),
str(item.title or ""),
str(item.snippet or ""),
str(item.target_url or ""),
str(urlparse(str(item.target_url or "").strip()).path or ""),
]
tokens: list[str] = []
seen: set[str] = set()
for value in values:
for token in cls._tokenize_match_text(value):
if token and token not in seen:
seen.add(token)
tokens.append(token)
return tokens
@classmethod
def _score_anchor_against_evidence(cls, anchor: str, evidence: list[ExternalEvidenceItem]) -> float:
best = 0.0
for item in evidence:
for token in cls._evidence_match_tokens(item):
best = max(best, cls._score_anchor_against_token(anchor, token))
return round(best, 4)
@classmethod
def _adapter_support_level(cls, *, plan: ExternalQueryPlan, adapter: Any) -> int:
supported = getattr(adapter, "supported_query_kinds", None)
if not supported:
return 1
query_kind = str(plan.query_kind or "").strip().lower()
normalized = {str(item or "").strip().lower() for item in supported if str(item or "").strip()}
return 0 if query_kind in normalized else 2
def _rank_adapters(self, *, plan: ExternalQueryPlan) -> list[Any]:
ranked: list[tuple[int, int, Any]] = []
for index, adapter in enumerate(self._ensure_registry_items(plan=plan)):
ranked.append((self._adapter_support_level(plan=plan, adapter=adapter), index, adapter))
ranked.sort(key=lambda item: (item[0], item[1]))
return [item[2] for item in ranked]
def _ensure_registry_items(self, *, plan: ExternalQueryPlan) -> list[Any]:
_ = plan
return list(self.registry.iter_adapters())
@classmethod
def _build_plan_anchor_terms(cls, plan: ExternalQueryPlan) -> list[str]:
terms: list[str] = []
for item in list(plan.subject_entities or []):
normalized = cls._normalize_anchor_term(item)
if len(normalized) >= 2 and normalized not in terms:
terms.append(normalized)
location_hint = cls._normalize_anchor_term(plan.location_hint)
if len(location_hint) >= 2 and location_hint not in terms:
terms.append(location_hint)
return terms[:6]
@classmethod
def _verify_evidence_bundle(cls, *, plan: ExternalQueryPlan, evidence: list[ExternalEvidenceItem]) -> dict[str, Any]:
if not evidence:
return {
"ok": False,
"reason_code": "missing_external_grounding",
"missing_fields": ["external_evidence"],
}
anchor_terms = cls._build_plan_anchor_terms(plan)
anchor_hit = not anchor_terms
for item in evidence:
target_url = str(item.target_url or "").strip().lower()
source_type = str(item.source_type or "").strip().lower()
if source_type not in {"url", "mcp_search"}:
return {
"ok": False,
"reason_code": "external_source_type_invalid",
"missing_fields": ["external_source_type"],
}
if not target_url.startswith("http://") and not target_url.startswith("https://"):
return {
"ok": False,
"reason_code": "external_target_url_missing",
"missing_fields": ["target_url"],
}
anchor_scores = [cls._score_anchor_against_evidence(anchor, evidence) for anchor in anchor_terms]
strong_hits = sum(1 for score in anchor_scores if score >= 0.8)
medium_hits = sum(1 for score in anchor_scores if score >= 0.55)
bundle_score = (
round(sum(anchor_scores) / max(1, len(anchor_scores)), 4)
if anchor_scores
else 1.0
)
if anchor_scores:
anchor_hit = strong_hits >= 1 or medium_hits == len(anchor_scores) or bundle_score >= 0.6
if not anchor_hit:
return {
"ok": False,
"reason_code": "external_evidence_off_topic",
"missing_fields": ["plan_anchor_terms"],
"bundle_score": bundle_score,
"anchor_scores": anchor_scores,
}
return {
"ok": True,
"reason_code": None,
"missing_fields": [],
"bundle_score": bundle_score,
"anchor_scores": anchor_scores,
}

View File

@ -0,0 +1,163 @@
from __future__ import annotations
import json
import re
from typing import Any, Callable
from ...config.ai_prompts import PROMPT_EXTERNAL_QUERY_PLAN
from .schemas import ExternalQueryPlan
class ExternalQueryPlanner:
FUNCTION_NAME = "external_query_plan"
def __init__(
self,
*,
llm_service: Any,
db: Any,
tenant_id: int,
query_builder: Callable[[str], list[str]] | None = None,
) -> None:
self.llm_service = llm_service
self.db = db
self.tenant_id = int(tenant_id or 1)
self.query_builder = query_builder
async def build_plan(
self,
*,
question: str,
query_semantics: dict[str, Any] | None = None,
) -> tuple[ExternalQueryPlan, dict[str, Any]]:
fallback_plan = self._build_fallback_plan(
question=question,
query_semantics=query_semantics,
)
generator = getattr(self.llm_service, "generate_structured_object", None)
if not callable(generator):
return fallback_plan, {"planner": "heuristic", "fallback_taken": True}
prompt = PROMPT_EXTERNAL_QUERY_PLAN.format(
question=str(question or "").strip()[:1200],
query_semantics=json.dumps(query_semantics or {}, ensure_ascii=False)[:4000],
fallback_queries=json.dumps(fallback_plan.search_queries, ensure_ascii=False),
).strip()
try:
structured = await generator(
prompt=prompt,
db=self.db,
tenant_id=self.tenant_id,
function_name=self.FUNCTION_NAME,
use_prompt_profile=False,
temperature=0.0,
response_mode="fast",
)
payload = getattr(structured, "payload", None)
if payload is None and hasattr(structured, "model_dump"):
payload = structured.model_dump()
if not isinstance(payload, dict):
raise ValueError("external_query_plan_payload_not_dict")
merged = fallback_plan.model_dump()
for key, value in payload.items():
if value in (None, "", [], {}):
continue
merged[key] = value
plan = ExternalQueryPlan.model_validate(merged)
if not plan.search_queries:
plan.search_queries = list(fallback_plan.search_queries)
if not plan.preferred_domains:
plan.preferred_domains = list(fallback_plan.preferred_domains)
if not plan.topic:
plan.topic = fallback_plan.topic
return plan, {"planner": "llm_structured", "fallback_taken": False}
except Exception as exc:
return fallback_plan, {
"planner": "heuristic",
"fallback_taken": True,
"reason": str(exc)[:200],
}
def _build_fallback_plan(
self,
*,
question: str,
query_semantics: dict[str, Any] | None = None,
) -> ExternalQueryPlan:
text = re.sub(r"\s+", " ", str(question or "")).strip()
semantics = query_semantics if isinstance(query_semantics, dict) else {}
query_profile = semantics.get("query_profile") if isinstance(semantics.get("query_profile"), dict) else {}
object_mentions = [
str(item or "").strip()
for item in list(semantics.get("object_mentions") or [])
if str(item or "").strip()
]
query_type = str(semantics.get("query_type") or "").strip().lower()
if query_type == "compare":
answer_contract = "compare_with_sources"
elif bool(query_profile.get("needs_freshness")):
answer_contract = "summary_with_sources"
else:
answer_contract = "direct_fact"
topic = text[:120] or "外部查询"
search_queries = self._build_search_queries(text)
preferred_domains = self._infer_preferred_domains(text)
subject_entities = object_mentions[:6]
location_hint = None
time_range_hint = self._infer_time_range_hint(text)
requires_multi_source = answer_contract == "compare_with_sources"
clarify_on_ambiguity = False
result_limit = 3 if requires_multi_source or bool(query_profile.get("needs_freshness")) else 2
return ExternalQueryPlan(
plan_version="v1",
query_kind="generic_web" if text else "unknown",
answer_contract=answer_contract,
freshness_required=bool(query_profile.get("needs_freshness")),
topic=topic,
subject_entities=subject_entities,
location_hint=location_hint,
time_range_hint=time_range_hint,
preferred_domains=preferred_domains,
search_queries=search_queries,
result_limit=result_limit,
requires_multi_source=requires_multi_source,
clarify_on_ambiguity=clarify_on_ambiguity,
)
def _build_search_queries(self, question: str) -> list[str]:
if callable(self.query_builder):
try:
queries = self.query_builder(question)
normalized = [str(item or "").strip() for item in list(queries or []) if str(item or "").strip()]
if normalized:
return normalized[:5]
except Exception:
pass
text = re.sub(r"\s+", " ", str(question or "")).strip()
return [text] if text else []
@staticmethod
def _infer_preferred_domains(question: str) -> list[str]:
normalized = str(question or "").lower()
domains: list[str] = []
if any(token in normalized for token in ("codex", "gpt", "chatgpt", "openai")):
domains.extend(["openai.com", "developers.openai.com"])
if any(token in normalized for token in ("copilot", "github")):
domains.extend(["github.com", "docs.github.com"])
uniq: list[str] = []
for domain in domains:
if domain not in uniq:
uniq.append(domain)
return uniq[:4]
@staticmethod
def _infer_time_range_hint(question: str) -> str | None:
text = str(question or "")
for pattern in (r"未来\d+天", r"未来一周", r"今天", r"截至今天", r"截至现在", r"目前", r"最新"):
match = re.search(pattern, text)
if match:
return str(match.group(0) or "").strip() or None
return None

View File

@ -0,0 +1,11 @@
from __future__ import annotations
from typing import Any
class ExternalProviderRegistry:
def __init__(self, *, adapters: list[Any] | None = None) -> None:
self._adapters = list(adapters or [])
def iter_adapters(self) -> list[Any]:
return list(self._adapters)

View File

@ -0,0 +1,62 @@
from __future__ import annotations
from typing import Any, Literal
from pydantic import BaseModel, ConfigDict, Field
ExternalQueryKind = Literal["weather", "news", "company", "model", "market", "generic_web", "unknown"]
ExternalAnswerContract = Literal["direct_fact", "summary_with_sources", "compare_with_sources", "clarify"]
class ExternalQueryPlan(BaseModel):
model_config = ConfigDict(extra="ignore")
plan_version: str = "v1"
query_kind: ExternalQueryKind = "unknown"
answer_contract: ExternalAnswerContract = "summary_with_sources"
freshness_required: bool = False
topic: str = ""
subject_entities: list[str] = Field(default_factory=list)
location_hint: str | None = None
time_range_hint: str | None = None
preferred_domains: list[str] = Field(default_factory=list)
search_queries: list[str] = Field(default_factory=list)
result_limit: int = Field(default=3, ge=1, le=5)
requires_multi_source: bool = False
clarify_on_ambiguity: bool = False
class ExternalEvidenceItem(BaseModel):
model_config = ConfigDict(extra="ignore")
source_type: str = "url"
source_label: str = ""
title: str = ""
target_url: str = ""
snippet: str = ""
retrieved_at: str | None = None
freshness_score: float = 0.0
authority_score: float = 0.0
grounded_fields: list[str] = Field(default_factory=list)
raw_metadata: dict[str, Any] = Field(default_factory=dict)
class ExternalAdapterResult(BaseModel):
model_config = ConfigDict(extra="ignore")
adapter_name: str
status: str = "empty"
evidence: list[ExternalEvidenceItem] = Field(default_factory=list)
meta: dict[str, Any] = Field(default_factory=dict)
class ExternalQueryExecution(BaseModel):
model_config = ConfigDict(extra="ignore")
plan: ExternalQueryPlan
adapter_name: str
evidence: list[ExternalEvidenceItem] = Field(default_factory=list)
verifier: dict[str, Any] = Field(default_factory=dict)
planner_meta: dict[str, Any] = Field(default_factory=dict)
meta: dict[str, Any] = Field(default_factory=dict)

View File

@ -200,6 +200,7 @@ def map_qa_feedback_to_record_payload(*, qa_feedback: Any, qa_record: Any) -> di
"sources": _jsonable(getattr(qa_record, "sources", None) or []), "sources": _jsonable(getattr(qa_record, "sources", None) or []),
"citations": _jsonable(getattr(qa_record, "citations", None) or []), "citations": _jsonable(getattr(qa_record, "citations", None) or []),
"sources_used": _jsonable(getattr(qa_record, "sources_used", None) or []), "sources_used": _jsonable(getattr(qa_record, "sources_used", None) or []),
"routing_snapshot": _jsonable(getattr(qa_record, "routing_snapshot", None) or {}),
} }
after_value = { after_value = {
"rating": rating, "rating": rating,
@ -233,6 +234,7 @@ def map_qa_feedback_to_record_payload(*, qa_feedback: Any, qa_record: Any) -> di
"answer_mode": getattr(qa_record, "answer_mode", None), "answer_mode": getattr(qa_record, "answer_mode", None),
"grounding_status": getattr(qa_record, "grounding_status", None), "grounding_status": getattr(qa_record, "grounding_status", None),
"policy_version": getattr(qa_record, "policy_version", None), "policy_version": getattr(qa_record, "policy_version", None),
"routing_snapshot": _jsonable(getattr(qa_record, "routing_snapshot", None) or {}),
"citations": _jsonable(getattr(qa_record, "citations", None) or []), "citations": _jsonable(getattr(qa_record, "citations", None) or []),
"sources_used": _jsonable(getattr(qa_record, "sources_used", None) or []), "sources_used": _jsonable(getattr(qa_record, "sources_used", None) or []),
"evidence_pack": _jsonable(getattr(qa_record, "evidence_pack", None) or {}), "evidence_pack": _jsonable(getattr(qa_record, "evidence_pack", None) or {}),

View File

@ -379,13 +379,28 @@ class FeishuCardActionMixin:
synthetic_text = "重出一版" synthetic_text = "重出一版"
if not synthetic_text: if not synthetic_text:
return {"code": 0, "msg": "card_action_ignored", "data": {"reason": "missing_synthetic_text"}} return {"code": 0, "msg": "card_action_ignored", "data": {"reason": "missing_synthetic_text"}}
kernel = AIPMOKernelService(self.db, tenant_id=self.tenant_id) use_feishu_context = bool(chat_id and source_message_id)
result = await kernel.process_ingress( ingress_source_system = "feishu" if use_feishu_context else "ai_pmo_ui"
source_system="ai_pmo_ui", ingress_source_message_id = (
source_message_id=( f"feishu-card-plan-{int(work_item_id)}-{action}-{int(time.time() * 1000)}"
if use_feishu_context
else (
source_message_id source_message_id
or f"feishu-card-plan-{int(work_item_id)}-{action}-{int(time.time() * 1000)}" or f"feishu-card-plan-{int(work_item_id)}-{action}-{int(time.time() * 1000)}"
), )
)
ingress_options = None
if use_feishu_context:
ingress_options = {
"thread_context": {
"reply_to_message_id": str(source_message_id),
"root_message_id": str(source_message_id),
}
}
kernel = AIPMOKernelService(self.db, tenant_id=self.tenant_id)
result = await kernel.process_ingress(
source_system=ingress_source_system,
source_message_id=ingress_source_message_id,
message_text=synthetic_text, message_text=synthetic_text,
sender=f"{operator_sender}(card)", sender=f"{operator_sender}(card)",
channel_ref=chat_id, channel_ref=chat_id,
@ -393,6 +408,7 @@ class FeishuCardActionMixin:
captured_at=None, captured_at=None,
chat_type="manual", chat_type="manual",
project_hint_id=None, project_hint_id=None,
ingress_options=ingress_options,
) )
if hasattr(self.db, "commit") and callable(getattr(self.db, "commit")): if hasattr(self.db, "commit") and callable(getattr(self.db, "commit")):
await self.db.commit() await self.db.commit()

View File

@ -181,9 +181,13 @@ class LLMCacheCircuitMixin:
return return
now = time.monotonic() now = time.monotonic()
state = dict(cls._provider_circuit_state.get(str(circuit_key)) or {}) state = dict(cls._provider_circuit_state.get(str(circuit_key)) or {})
consecutive = int(state.get("consecutive_failures") or 0) + 1 fail_fast = cls._is_fail_fast_exception(exc)
if fail_fast:
consecutive = CIRCUIT_BREAKER_FAILURE_THRESHOLD
else:
consecutive = int(state.get("consecutive_failures") or 0) + 1
open_until = float(state.get("open_until") or 0.0) open_until = float(state.get("open_until") or 0.0)
if consecutive >= CIRCUIT_BREAKER_FAILURE_THRESHOLD: if fail_fast or consecutive >= CIRCUIT_BREAKER_FAILURE_THRESHOLD:
open_until = now + CIRCUIT_BREAKER_COOLDOWN_SECONDS open_until = now + CIRCUIT_BREAKER_COOLDOWN_SECONDS
cls._provider_circuit_state[str(circuit_key)] = { cls._provider_circuit_state[str(circuit_key)] = {
"consecutive_failures": consecutive, "consecutive_failures": consecutive,

View File

@ -9,8 +9,30 @@ class LLMEmptyContentError(ValueError):
class LLMChatOpenAIMixin: class LLMChatOpenAIMixin:
@staticmethod @staticmethod
def _append_direct_answer_retry_instruction(messages: list[dict[str, str]]) -> None: def _is_provider_billing_blocked_http_error(*, status_code: int, body_preview: str) -> bool:
retry_instruction = "请直接输出答案" if int(status_code or 0) not in {400, 401, 402, 403}:
return False
lowered = str(body_preview or "").strip().lower()
if not lowered:
return False
return (
"arrearage" in lowered
or "overdue-payment" in lowered
or ("access denied" in lowered and "good standing" in lowered)
or "provider_billing_blocked" in lowered
)
@staticmethod
def _append_direct_answer_retry_instruction(
messages: list[dict[str, str]],
*,
structured_output: bool = False,
) -> None:
retry_instruction = (
"请直接输出最终 JSON 对象,不要解释,不要 Markdown,不要把正文放在 reasoning_content。"
if structured_output
else "请直接输出答案"
)
for idx in range(len(messages) - 1, -1, -1): for idx in range(len(messages) - 1, -1, -1):
row = messages[idx] row = messages[idx]
if not isinstance(row, dict): if not isinstance(row, dict):
@ -45,6 +67,7 @@ class LLMChatOpenAIMixin:
request_started_at_perf: float, request_started_at_perf: float,
request_trace_id: str, request_trace_id: str,
control_plane_agent_timeout: Optional[int], control_plane_agent_timeout: Optional[int],
routing_meta: dict[str, Any] | None = None,
) -> ChatCompletionResult: ) -> ChatCompletionResult:
endpoint = f"{base_url.rstrip('/')}/chat/completions" endpoint = f"{base_url.rstrip('/')}/chat/completions"
headers = { headers = {
@ -127,6 +150,11 @@ class LLMChatOpenAIMixin:
), ),
) )
status_code = int(resp.status_code) status_code = int(resp.status_code)
if self._is_provider_billing_blocked_http_error(
status_code=status_code,
body_preview=body_preview,
):
raise ValueError(f"provider_billing_blocked:{provider_name}:{body_preview}")
can_retry_http_status = ( can_retry_http_status = (
http_status_retry_budget > 0 http_status_retry_budget > 0
and self._is_retryable_http_status(status_code) and self._is_retryable_http_status(status_code)
@ -185,14 +213,27 @@ class LLMChatOpenAIMixin:
if self._contains_json_container(normalized_reasoning): if self._contains_json_container(normalized_reasoning):
candidate = self._extract_json_candidate(normalized_reasoning) candidate = self._extract_json_candidate(normalized_reasoning)
if candidate: if candidate:
_runtime_log( if self._call_is_invalid_structured_output(
"LLM", function_name=function_label,
( response_text=candidate,
"结构化reasoning_content回退:" routing_meta=routing_meta,
f"function={function_label}, provider={provider_name}" ):
), _runtime_log(
) "LLM",
text = candidate (
"结构化reasoning_content候选无效,继续空content重试:"
f"function={function_label}, provider={provider_name}"
),
)
else:
_runtime_log(
"LLM",
(
"结构化reasoning_content回退:"
f"function={function_label}, provider={provider_name}"
),
)
text = candidate
if text: if text:
break break
@ -209,7 +250,10 @@ class LLMChatOpenAIMixin:
) )
if not direct_answer_retry_used: if not direct_answer_retry_used:
direct_answer_retry_used = True direct_answer_retry_used = True
self._append_direct_answer_retry_instruction(messages) self._append_direct_answer_retry_instruction(
messages,
structured_output=structured_output,
)
_runtime_log( _runtime_log(
"LLM", "LLM",
( (

View File

@ -129,6 +129,7 @@ class LLMChatTransportMixin:
request_started_at_perf=request_started_at_perf, request_started_at_perf=request_started_at_perf,
request_trace_id=request_trace_id, request_trace_id=request_trace_id,
control_plane_agent_timeout=control_plane_agent_timeout, control_plane_agent_timeout=control_plane_agent_timeout,
routing_meta=normalized_routing_meta,
) )
if use_provider_circuit_breaker: if use_provider_circuit_breaker:
self._record_provider_success(circuit_key) self._record_provider_success(circuit_key)

View File

@ -3,6 +3,47 @@ from __future__ import annotations
from .shared import * from .shared import *
class LLMTaskHelpersMixin: class LLMTaskHelpersMixin:
@staticmethod
def _extract_embedding_vector(payload: Any) -> list[float]:
if not isinstance(payload, dict):
return []
if isinstance(payload.get("embedding"), list):
return [float(item) for item in payload.get("embedding") or []]
embeddings = payload.get("embeddings")
if isinstance(embeddings, list) and embeddings and isinstance(embeddings[0], list):
return [float(item) for item in embeddings[0]]
data = payload.get("data")
if isinstance(data, list) and data and isinstance(data[0], dict):
embedding = data[0].get("embedding")
if isinstance(embedding, list):
return [float(item) for item in embedding]
return []
async def _request_openai_embedding(self, *, endpoint: str, headers: dict[str, str], payload: dict[str, Any]) -> list[float]:
async with httpx.AsyncClient(timeout=self.request_timeout, trust_env=False) as client:
resp = await client.post(endpoint, headers=headers, json=payload)
resp.raise_for_status()
return self._extract_embedding_vector(resp.json())
async def _request_ollama_embedding(self, *, host: str, model: str, text: str) -> list[float]:
base_url = str(host or os.getenv("OLLAMA_HOST", "http://127.0.0.1:11434")).rstrip("/")
headers = {"Content-Type": "application/json"}
candidates = [
(f"{base_url}/api/embed", {"model": model, "input": text}),
(f"{base_url}/api/embeddings", {"model": model, "prompt": text}),
]
for endpoint, payload in candidates:
try:
async with httpx.AsyncClient(timeout=self.request_timeout, trust_env=False) as client:
resp = await client.post(endpoint, headers=headers, json=payload)
resp.raise_for_status()
vector = self._extract_embedding_vector(resp.json())
if vector:
return vector
except Exception:
continue
return []
async def generate_task_breakdown_async(self, description: str, **kwargs: Any) -> list[dict]: async def generate_task_breakdown_async(self, description: str, **kwargs: Any) -> list[dict]:
if not description.strip(): if not description.strip():
return [] return []
@ -27,14 +68,18 @@ class LLMTaskHelpersMixin:
] ]
async def get_embedding_async(self, text: str, **kwargs: Any) -> list[float]: async def get_embedding_async(self, text: str, **kwargs: Any) -> list[float]:
clean_text = str(text or "").strip()
if not clean_text:
return []
db = kwargs.get("db") or kwargs.get("db_session") db = kwargs.get("db") or kwargs.get("db_session")
tenant_id = int(kwargs.get("tenant_id") or 1) tenant_id = int(kwargs.get("tenant_id") or 1)
if db is None: if db is None:
async with AsyncSessionLocal() as session: async with AsyncSessionLocal() as session:
return await self.get_embedding_async(text, db=session, tenant_id=tenant_id) return await self.get_embedding_async(clean_text, db=session, tenant_id=tenant_id)
engine: dict[str, Any] = {}
try: try:
engine = await self._resolve_active_engine(db, tenant_id=tenant_id) engine = await self._resolve_active_engine(db, tenant_id=tenant_id)
provider = str(engine.get("provider") or "") provider = str(engine.get("source_provider") or engine.get("provider") or "").strip().lower()
if provider == "openai": if provider == "openai":
endpoint = f"{str(engine.get('base_url') or '').rstrip('/')}/embeddings" endpoint = f"{str(engine.get('base_url') or '').rstrip('/')}/embeddings"
headers = { headers = {
@ -43,18 +88,29 @@ class LLMTaskHelpersMixin:
} }
payload = { payload = {
"model": str(engine.get("model") or "text-embedding-3-small"), "model": str(engine.get("model") or "text-embedding-3-small"),
"input": text, "input": clean_text,
} }
async with httpx.AsyncClient(timeout=self.request_timeout, trust_env=False) as client: vector = await self._request_openai_embedding(endpoint=endpoint, headers=headers, payload=payload)
resp = await client.post(endpoint, headers=headers, json=payload) if vector:
resp.raise_for_status() return vector
data = resp.json() if provider == "ollama":
return list(data.get("data", [{}])[0].get("embedding") or []) vector = await self._request_ollama_embedding(
host=str(engine.get("base_url") or os.getenv("OLLAMA_HOST", "http://127.0.0.1:11434")),
model=str(engine.get("model") or os.getenv("OLLAMA_EMBED_MODEL", "nomic-embed-text:latest")),
text=clean_text,
)
if vector:
return vector
except Exception: except Exception:
pass engine = dict(engine) if isinstance(engine, dict) else {}
# DeepSeek/GLM 及失败场景:回退伪向量,保持功能可用。 vector = await self._request_ollama_embedding(
seed = hashlib.md5(text.encode("utf-8")).digest() host=str(os.getenv("OLLAMA_HOST", "http://127.0.0.1:11434")),
return [((seed[i % 16] / 255.0) * 2 - 1) for i in range(64)] model=str(os.getenv("OLLAMA_EMBED_MODEL", "nomic-embed-text:latest")),
text=clean_text,
)
if vector:
return vector
return []
async def extract_context_insights_async(self, text: str, **kwargs: Any) -> dict: async def extract_context_insights_async(self, text: str, **kwargs: Any) -> dict:
prompt = ( prompt = (
@ -102,4 +158,3 @@ class LLMTaskHelpersMixin:
"passed": passed, "passed": passed,
"reason": "basic_stub_check_passed" if passed else "invalid_input", "reason": "basic_stub_check_passed" if passed else "invalid_input",
} }

View File

@ -122,6 +122,10 @@ FAIL_FAST_ERROR_MARKERS = (
"返回空内容", "返回空内容",
"empty_response", "empty_response",
"provider_circuit_open", "provider_circuit_open",
"provider_billing_blocked",
"arrearage",
"overdue-payment",
"account is in good standing",
) )
CIRCUIT_BREAKER_FAILURE_THRESHOLD = 2 CIRCUIT_BREAKER_FAILURE_THRESHOLD = 2
CIRCUIT_BREAKER_COOLDOWN_SECONDS = 300.0 CIRCUIT_BREAKER_COOLDOWN_SECONDS = 300.0
@ -190,6 +194,36 @@ STRUCTURED_OBJECT_SCHEMAS: dict[str, dict[str, Any]] = {
"uncertainties": ("list",), "uncertainties": ("list",),
}, },
}, },
"external_query_plan": {
"required_keys": (
"plan_version",
"query_kind",
"answer_contract",
"freshness_required",
"topic",
"search_queries",
"result_limit",
"requires_multi_source",
"clarify_on_ambiguity",
),
"type_checks": {
"plan_version": "str",
"query_kind": "str",
"answer_contract": "str",
"freshness_required": "bool",
"topic": "str",
"search_queries": "list",
"result_limit": "number",
"requires_multi_source": "bool",
"clarify_on_ambiguity": "bool",
},
"optional_type_checks": {
"subject_entities": ("list",),
"preferred_domains": ("list",),
"location_hint": ("str_or_none",),
"time_range_hint": ("str_or_none",),
},
},
"signal_planner_analyze": { "signal_planner_analyze": {
"required_keys": ("summary", "priority", "execution_plan", "risks", "confidence"), "required_keys": ("summary", "priority", "execution_plan", "risks", "confidence"),
"type_checks": { "type_checks": {

View File

@ -0,0 +1,3 @@
from .service import McpConfigService
__all__ = ["McpConfigService"]

View File

@ -0,0 +1,87 @@
from __future__ import annotations
from .schemas import McpFieldSpec, McpTemplateSpec
def build_template_catalog() -> list[McpTemplateSpec]:
return [
McpTemplateSpec(
id="microsoft_learn",
name="Microsoft Learn MCP",
description="官方远程 MCP,无需 Token,可直接接入 Microsoft Learn 文档检索。",
transport="streamable_http",
execution_mode="backend_direct",
icon="book-open",
recommended=True,
docs_url="https://github.com/MicrosoftDocs/mcp",
fields=[],
command_hint="保存后由 PMFlow 后端直接连接远程 MCP,不要求客户机器安装额外 MCP host。",
verify_hint="重新校验时,PMFlow 会从后端执行 MCP 握手与 tools/list 探测。",
),
McpTemplateSpec(
id="github",
name="GitHub MCP",
description="官方 GitHub MCP,面向仓库、Issue、PR 等开发资产。",
transport="stdio",
execution_mode="local_bridge",
icon="github",
docs_url="https://github.com/modelcontextprotocol/servers",
fields=[
McpFieldSpec(
key="token_env_var_name",
label="本地 Token 环境变量名",
field_type="text",
required=True,
placeholder="GITHUB_PAT",
description="PMFlow 只保存环境变量名,不保存明文 Token。",
default_value="GITHUB_PAT",
),
],
command_hint="需要 PMFlow Local Bridge 或兼容 MCP host 在客户本机承载 stdio server。",
verify_hint="确认本机 bridge/host 已接入后,再回到这里做手动确认。",
),
McpTemplateSpec(
id="filesystem",
name="Filesystem MCP",
description="官方 reference server,面向受控目录的本地文件访问。",
transport="stdio",
execution_mode="local_bridge",
icon="folder-open",
docs_url="https://github.com/modelcontextprotocol/servers",
fields=[
McpFieldSpec(
key="allowed_paths",
label="允许目录",
field_type="path_list",
required=True,
placeholder="/absolute/path/one",
description="每行一个绝对路径,只暴露给 MCP 允许访问的目录。",
default_value=[],
),
],
command_hint="需要 PMFlow Local Bridge 或兼容 MCP host 在客户本机启动 filesystem server。",
verify_hint="确认本机 bridge/host 已接入并暴露目录后,再回到这里做手动确认。",
),
McpTemplateSpec(
id="playwright",
name="Playwright MCP",
description="官方 Playwright MCP,适合浏览器自动化和页面抓取。",
transport="stdio",
execution_mode="local_bridge",
icon="monitor-play",
docs_url="https://playwright.dev/java/agents/network-and-security",
fields=[
McpFieldSpec(
key="output_dir",
label="输出目录",
field_type="text",
required=False,
placeholder="~/.pmflow/mcp/playwright-output",
description="用于保存 Playwright MCP 输出工件的目录。",
default_value="~/.pmflow/mcp/playwright-output",
),
],
command_hint="需要 PMFlow Local Bridge 或兼容 MCP host 在客户本机启动 Playwright MCP。",
verify_hint="确认本机 bridge/host 已接入并能产出浏览器工件后,再回到这里做手动确认。",
),
]

View File

@ -0,0 +1,149 @@
from __future__ import annotations
import json
from typing import Any, Callable
import httpx
from ...utils.timezone import now_cn_naive
from ..external_query_service.schemas import ExternalEvidenceItem
class MicrosoftLearnMCPRuntime:
def __init__(
self,
*,
endpoint_url: str = "https://learn.microsoft.com/api/mcp",
client_factory: Callable[[], httpx.AsyncClient] | None = None,
) -> None:
self.endpoint_url = str(endpoint_url or "https://learn.microsoft.com/api/mcp").strip()
self.client_factory = client_factory or (
lambda: httpx.AsyncClient(timeout=20.0, trust_env=False, follow_redirects=True)
)
async def probe(self) -> dict[str, Any]:
checked_at = now_cn_naive().isoformat()
try:
session_id, _ = await self._initialize()
_, tools_payload = await self._post_jsonrpc(
{
"jsonrpc": "2.0",
"id": 2,
"method": "tools/list",
"params": {},
},
session_id=session_id,
)
tools = list((tools_payload.get("result") or {}).get("tools") or [])
tool_names = [str(item.get("name") or "").strip() for item in tools if str(item.get("name") or "").strip()]
return {
"status": "ready",
"checked_at": checked_at,
"message": "PMFlow backend can directly reach the remote MCP server.",
"tool_names": tool_names,
}
except Exception as exc:
return {
"status": "error",
"checked_at": checked_at,
"message": f"PMFlow backend probe failed: {exc}",
"tool_names": [],
}
async def search(self, *, query: str, result_limit: int = 3) -> list[ExternalEvidenceItem]:
session_id, _ = await self._initialize()
_, payload = await self._post_jsonrpc(
{
"jsonrpc": "2.0",
"id": 3,
"method": "tools/call",
"params": {
"name": "microsoft_docs_search",
"arguments": {
"query": str(query or "").strip(),
},
},
},
session_id=session_id,
)
text_payload = ""
for item in list((payload.get("result") or {}).get("content") or []):
if str(item.get("type") or "").strip() == "text":
text_payload = str(item.get("text") or "").strip()
if text_payload:
break
parsed = json.loads(text_payload) if text_payload else {}
results = list(parsed.get("results") or [])
retrieved_at = now_cn_naive().isoformat()
evidence: list[ExternalEvidenceItem] = []
for raw in results[: max(1, int(result_limit or 3))]:
target_url = str(raw.get("contentUrl") or "").strip()
if not target_url.startswith("http://") and not target_url.startswith("https://"):
continue
title = str(raw.get("title") or "").strip()
snippet = str(raw.get("content") or "").strip()
evidence.append(
ExternalEvidenceItem(
source_type="mcp_search",
source_label="Microsoft Learn MCP",
title=title,
target_url=target_url,
snippet=snippet[:1800],
retrieved_at=retrieved_at,
freshness_score=0.86,
authority_score=0.98,
grounded_fields=["microsoft_docs_search"],
raw_metadata={
"provider": "microsoft_learn_mcp",
"result_id": str(raw.get("id") or "").strip() or None,
},
)
)
return evidence
async def _initialize(self) -> tuple[str | None, dict[str, Any]]:
return await self._post_jsonrpc(
{
"jsonrpc": "2.0",
"id": 1,
"method": "initialize",
"params": {
"protocolVersion": "2025-03-26",
"capabilities": {},
"clientInfo": {
"name": "pmflow",
"version": "0.1.0",
},
},
}
)
async def _post_jsonrpc(
self,
payload: dict[str, Any],
*,
session_id: str | None = None,
) -> tuple[str | None, dict[str, Any]]:
headers = {
"Content-Type": "application/json",
"Accept": "application/json, text/event-stream",
}
if session_id:
headers["mcp-session-id"] = session_id
async with self.client_factory() as client:
response = await client.post(self.endpoint_url, headers=headers, json=payload)
response.raise_for_status()
next_session = str(response.headers.get("mcp-session-id") or session_id or "").strip() or None
body = response.text
parsed = self._extract_payload(body)
return next_session, parsed
@staticmethod
def _extract_payload(body: str) -> dict[str, Any]:
data_lines: list[str] = []
for line in str(body or "").splitlines():
if line.startswith("data:"):
data_lines.append(line[len("data:") :].lstrip())
if not data_lines:
raise ValueError("MCP streamable_http response missing data payload")
return json.loads("\n".join(data_lines))

View File

@ -0,0 +1,62 @@
from __future__ import annotations
from typing import Literal
from pydantic import BaseModel, Field
McpClientApplyState = Literal["pending_local_apply", "user_confirmed", "not_required"]
McpExecutionMode = Literal["backend_direct", "local_bridge"]
McpRuntimeProbeStatus = Literal["unknown", "ready", "error", "local_bridge_required"]
class McpFieldSpec(BaseModel):
key: str
label: str
field_type: Literal["text", "path_list", "readonly"] = "text"
required: bool = False
placeholder: str | None = None
description: str | None = None
default_value: str | list[str] | None = None
class McpCommandPreview(BaseModel):
add_command: str
verify_command: str
expected_output: str
class McpRuntimeProbe(BaseModel):
status: McpRuntimeProbeStatus = "unknown"
checked_at: str | None = None
message: str | None = None
tool_names: list[str] = Field(default_factory=list)
class McpTemplateSpec(BaseModel):
id: str
name: str
description: str
transport: Literal["stdio", "streamable_http"]
execution_mode: McpExecutionMode
icon: str
recommended: bool = False
docs_url: str | None = None
fields: list[McpFieldSpec] = Field(default_factory=list)
command_hint: str | None = None
verify_hint: str | None = None
class McpServerConfigPayload(BaseModel):
id: str
name: str
display_name: str
template_id: str
transport: Literal["stdio", "streamable_http"]
execution_mode: McpExecutionMode = "local_bridge"
params: dict[str, object] = Field(default_factory=dict)
preview: McpCommandPreview
client_apply_state: McpClientApplyState = "pending_local_apply"
runtime_probe: McpRuntimeProbe = Field(default_factory=McpRuntimeProbe)
created_at: str
updated_at: str
confirmed_at: str | None = None

View File

@ -0,0 +1,374 @@
from __future__ import annotations
import re
from typing import Any
from uuid import uuid4
from sqlalchemy import and_, select
from sqlalchemy.ext.asyncio import AsyncSession
from ...models import Integration
from ...utils.timezone import now_cn_naive
from ..external_query_service.schemas import ExternalEvidenceItem
from .catalog import build_template_catalog
from .runtime import MicrosoftLearnMCPRuntime
from .schemas import McpCommandPreview, McpRuntimeProbe, McpServerConfigPayload, McpTemplateSpec
INTEGRATION_TYPE = "mcp_template_center"
_VALID_ENV_VAR = re.compile(r"^[A-Z_][A-Z0-9_]*$")
_VALID_NAME_CHARS = re.compile(r"[^a-z0-9_-]+")
class McpConfigService:
def __init__(self, db: AsyncSession, *, tenant_id: int) -> None:
self.db = db
self.tenant_id = int(tenant_id)
self._catalog = {item.id: item for item in build_template_catalog()}
async def get_catalog(self) -> dict[str, Any]:
items = [item.model_dump(mode="json") for item in self._catalog.values()]
recommended = next((item.id for item in self._catalog.values() if item.recommended), "microsoft_learn")
return {
"recommended_template_id": recommended,
"items": items,
}
async def list_servers(self) -> list[dict[str, Any]]:
_, bundle = await self._get_or_init_bundle()
items = [
McpServerConfigPayload.model_validate(self._normalize_server_record(item)).model_dump(mode="json")
for item in bundle.get("servers", [])
if isinstance(item, dict)
]
return sorted(items, key=lambda item: str(item.get("updated_at") or ""), reverse=True)
async def create_server(
self,
*,
name: str,
display_name: str | None,
template_id: str,
params: dict[str, Any] | None,
) -> dict[str, Any]:
row, bundle = await self._get_or_init_bundle()
template = self._get_template(template_id)
cleaned_name = self._normalize_server_name(name)
servers = [item for item in bundle.get("servers", []) if isinstance(item, dict)]
if any(str(item.get("name") or "") == cleaned_name for item in servers):
raise ValueError(f"MCP 配置名已存在:{cleaned_name}")
cleaned_params = self._validate_params(template, params or {})
preview = self._build_preview(template=template, name=cleaned_name, params=cleaned_params)
now_text = now_cn_naive().isoformat()
item = McpServerConfigPayload(
id=f"mcp_{uuid4().hex[:12]}",
name=cleaned_name,
display_name=str(display_name or template.name).strip() or template.name,
template_id=template.id,
transport=template.transport,
execution_mode=template.execution_mode,
params=cleaned_params,
preview=preview,
client_apply_state=self._default_client_apply_state(template),
runtime_probe=self._default_runtime_probe(template),
created_at=now_text,
updated_at=now_text,
confirmed_at=None,
).model_dump(mode="json")
servers.append(item)
bundle["servers"] = servers
row.config = bundle
await self.db.flush()
return item
async def update_server(
self,
*,
server_id: str,
name: str | None = None,
display_name: str | None = None,
template_id: str | None = None,
params: dict[str, Any] | None = None,
) -> dict[str, Any]:
row, bundle = await self._get_or_init_bundle()
servers = [self._normalize_server_record(item) for item in bundle.get("servers", []) if isinstance(item, dict)]
target = self._find_server(servers, server_id=server_id)
if target is None:
raise LookupError("MCP 配置不存在")
next_template = self._get_template(str(template_id or target.get("template_id") or ""))
next_name = self._normalize_server_name(str(name or target.get("name") or ""))
if any(
str(item.get("id") or "") != str(server_id)
and str(item.get("name") or "") == next_name
for item in servers
):
raise ValueError(f"MCP 配置名已存在:{next_name}")
params_input = params if params is not None else dict(target.get("params") or {})
cleaned_params = self._validate_params(next_template, params_input)
preview = self._build_preview(template=next_template, name=next_name, params=cleaned_params)
name_changed = next_name != str(target.get("name") or "")
template_changed = next_template.id != str(target.get("template_id") or "")
params_changed = cleaned_params != dict(target.get("params") or {})
target["name"] = next_name
target["display_name"] = str(display_name or target.get("display_name") or next_template.name).strip() or next_template.name
target["template_id"] = next_template.id
target["transport"] = next_template.transport
target["execution_mode"] = next_template.execution_mode
target["params"] = cleaned_params
target["preview"] = preview.model_dump(mode="json")
target["updated_at"] = now_cn_naive().isoformat()
if name_changed or template_changed or params_changed:
target["client_apply_state"] = self._default_client_apply_state(next_template)
target["confirmed_at"] = None
target["runtime_probe"] = self._default_runtime_probe(next_template).model_dump(mode="json")
bundle["servers"] = servers
row.config = bundle
await self.db.flush()
return McpServerConfigPayload.model_validate(target).model_dump(mode="json")
async def validate_server(self, *, server_id: str) -> dict[str, Any]:
row, bundle = await self._get_or_init_bundle()
servers = [self._normalize_server_record(item) for item in bundle.get("servers", []) if isinstance(item, dict)]
target = self._find_server(servers, server_id=server_id)
if target is None:
raise LookupError("MCP 配置不存在")
template = self._get_template(str(target.get("template_id") or ""))
cleaned_params = self._validate_params(template, dict(target.get("params") or {}))
preview = self._build_preview(
template=template,
name=str(target.get("name") or ""),
params=cleaned_params,
)
target["params"] = cleaned_params
target["preview"] = preview.model_dump(mode="json")
if template.execution_mode == "backend_direct":
target["runtime_probe"] = (await self._probe_backend_runtime(template=template, params=cleaned_params))
else:
target["runtime_probe"] = self._default_runtime_probe(template).model_dump(mode="json")
bundle["servers"] = servers
row.config = bundle
await self.db.flush()
return {
"server_id": str(server_id),
"is_valid": True,
"preview": preview.model_dump(mode="json"),
"runtime_probe": dict(target.get("runtime_probe") or {}),
}
async def confirm_client_apply(self, *, server_id: str) -> dict[str, Any]:
row, bundle = await self._get_or_init_bundle()
servers = [self._normalize_server_record(item) for item in bundle.get("servers", []) if isinstance(item, dict)]
target = self._find_server(servers, server_id=server_id)
if target is None:
raise LookupError("MCP 配置不存在")
template = self._get_template(str(target.get("template_id") or ""))
if template.execution_mode == "backend_direct":
raise ValueError("后端直连模板不需要“已在本机执行”确认")
confirmed_at = now_cn_naive().isoformat()
target["client_apply_state"] = "user_confirmed"
target["confirmed_at"] = confirmed_at
target["updated_at"] = confirmed_at
bundle["servers"] = servers
row.config = bundle
await self.db.flush()
return McpServerConfigPayload.model_validate(target).model_dump(mode="json")
async def get_preferred_backend_direct_server(self, *, template_id: str | None = None) -> dict[str, Any] | None:
_, bundle = await self._get_or_init_bundle()
servers = [self._normalize_server_record(item) for item in bundle.get("servers", []) if isinstance(item, dict)]
ordered = sorted(servers, key=lambda item: str(item.get("updated_at") or ""), reverse=True)
for item in ordered:
if str(item.get("execution_mode") or "") != "backend_direct":
continue
if template_id and str(item.get("template_id") or "") != str(template_id):
continue
return McpServerConfigPayload.model_validate(item).model_dump(mode="json")
return None
async def search_external(self, *, query: str, result_limit: int = 3) -> list[ExternalEvidenceItem]:
query_text = str(query or "").strip()
if not query_text:
return []
server = await self.get_preferred_backend_direct_server(template_id="microsoft_learn")
if not server:
return []
template = self._get_template(str(server.get("template_id") or ""))
if template.id != "microsoft_learn":
return []
runtime = MicrosoftLearnMCPRuntime()
return await runtime.search(query=query_text, result_limit=max(1, int(result_limit or 3)))
async def _get_or_init_bundle(self) -> tuple[Integration, dict[str, Any]]:
row = (
await self.db.execute(
select(Integration).where(
and_(
Integration.tenant_id == self.tenant_id,
Integration.type == INTEGRATION_TYPE,
)
)
)
).scalar_one_or_none()
if row is None:
row = Integration(
tenant_id=self.tenant_id,
type=INTEGRATION_TYPE,
is_active=True,
config={"version": 1, "servers": []},
)
self.db.add(row)
await self.db.flush()
bundle = dict(row.config or {}) if isinstance(row.config, dict) else {}
if not isinstance(bundle.get("servers"), list):
bundle["servers"] = []
if not isinstance(bundle.get("version"), int):
bundle["version"] = 1
if bundle != row.config:
row.config = bundle
await self.db.flush()
return row, bundle
def _get_template(self, template_id: str) -> McpTemplateSpec:
template = self._catalog.get(str(template_id or ""))
if template is None:
raise ValueError(f"未知 MCP 模板:{template_id}")
return template
def _normalize_server_record(self, item: dict[str, Any]) -> dict[str, Any]:
normalized = dict(item or {})
template = self._get_template(str(normalized.get("template_id") or ""))
normalized["transport"] = template.transport
normalized["execution_mode"] = template.execution_mode
if template.execution_mode == "backend_direct":
normalized["client_apply_state"] = "not_required"
else:
state = str(normalized.get("client_apply_state") or "").strip()
normalized["client_apply_state"] = state if state in {"pending_local_apply", "user_confirmed"} else "pending_local_apply"
if not isinstance(normalized.get("runtime_probe"), dict):
normalized["runtime_probe"] = self._default_runtime_probe(template).model_dump(mode="json")
if not isinstance(normalized.get("preview"), dict):
normalized["preview"] = self._build_preview(
template=template,
name=str(normalized.get("name") or ""),
params=dict(normalized.get("params") or {}),
).model_dump(mode="json")
return normalized
@staticmethod
def _default_client_apply_state(template: McpTemplateSpec) -> str:
return "not_required" if template.execution_mode == "backend_direct" else "pending_local_apply"
@staticmethod
def _default_runtime_probe(template: McpTemplateSpec) -> McpRuntimeProbe:
if template.execution_mode == "backend_direct":
return McpRuntimeProbe(
status="unknown",
checked_at=None,
message="保存后由 PMFlow 后端直连;点击“重新校验”可执行 MCP 握手探测。",
tool_names=[],
)
return McpRuntimeProbe(
status="local_bridge_required",
checked_at=None,
message="该模板依赖 PMFlow Local Bridge 或兼容 MCP host 在客户本机承载。",
tool_names=[],
)
@staticmethod
def _normalize_server_name(value: str) -> str:
normalized = _VALID_NAME_CHARS.sub("-", str(value or "").strip().lower())
normalized = re.sub(r"-{2,}", "-", normalized).strip("-_")
if not normalized:
raise ValueError("MCP 配置名不能为空")
return normalized[:80]
@staticmethod
def _find_server(servers: list[dict[str, Any]], *, server_id: str) -> dict[str, Any] | None:
for item in servers:
if str(item.get("id") or "") == str(server_id or ""):
return item
return None
def _validate_params(self, template: McpTemplateSpec, params: dict[str, Any]) -> dict[str, Any]:
raw = dict(params or {})
if template.id == "microsoft_learn":
return {}
if template.id == "github":
for forbidden_key in ("access_token", "token", "personal_access_token", "pat"):
if str(raw.get(forbidden_key) or "").strip():
raise ValueError("GitHub MCP 只接受 Token 环境变量名,不要直接填写明文 token")
env_var_name = str(raw.get("token_env_var_name") or "").strip().upper()
if not env_var_name:
raise ValueError("token_env_var_name 不能为空")
if not _VALID_ENV_VAR.match(env_var_name):
raise ValueError("token_env_var_name 必须是合法的环境变量名")
return {"token_env_var_name": env_var_name}
if template.id == "filesystem":
value = raw.get("allowed_paths")
paths: list[str] = []
if isinstance(value, str):
paths = [item.strip() for item in value.splitlines() if item.strip()]
elif isinstance(value, list):
paths = [str(item).strip() for item in value if str(item).strip()]
if not paths:
raise ValueError("allowed_paths 至少需要一个目录")
return {"allowed_paths": paths}
if template.id == "playwright":
output_dir = str(raw.get("output_dir") or "~/.pmflow/mcp/playwright-output").strip()
return {"output_dir": output_dir}
raise ValueError(f"未知模板校验规则:{template.id}")
def _build_preview(self, *, template: McpTemplateSpec, name: str, params: dict[str, Any]) -> McpCommandPreview:
if template.id == "microsoft_learn":
return McpCommandPreview(
add_command="无需本机命令。保存后由 PMFlow 后端直接连接 Microsoft Learn MCP。",
verify_command="点击“重新校验”后,PMFlow 会从后端执行 MCP initialize 和 tools/list。",
expected_output="校验成功后,你会看到后端可用工具,例如 microsoft_docs_search、microsoft_docs_fetch。",
)
if template.id == "github":
env_name = str(params["token_env_var_name"])
return McpCommandPreview(
add_command=(
f"需要在客户本机的 PMFlow Local Bridge 或兼容 MCP host 中接入 GitHub MCP,"
f"并引用环境变量 {env_name}。"
),
verify_command="本机 bridge/host 接好后,再回到这里执行“标记为已在本机执行”。",
expected_output=f"bridge/host 中应能看到 GitHub MCP 已启用,并通过环境变量 {env_name} 读取 token。",
)
if template.id == "filesystem":
quoted_paths = " ".join(f'"{item}"' for item in params["allowed_paths"])
return McpCommandPreview(
add_command=(
"需要在客户本机的 PMFlow Local Bridge 或兼容 MCP host 中启动 Filesystem MCP,"
f"允许目录:{quoted_paths}"
),
verify_command="本机 bridge/host 接好后,再回到这里执行“标记为已在本机执行”。",
expected_output=f"bridge/host 中应能看到 Filesystem MCP 已启用,并暴露目录 {quoted_paths}。",
)
if template.id == "playwright":
output_dir = str(params["output_dir"])
return McpCommandPreview(
add_command=(
"需要在客户本机的 PMFlow Local Bridge 或兼容 MCP host 中启动 Playwright MCP,"
f"输出目录:\"{output_dir}\""
),
verify_command="本机 bridge/host 接好后,再回到这里执行“标记为已在本机执行”。",
expected_output=f"bridge/host 中应能看到 Playwright MCP 已启用,输出目录为 {output_dir}。",
)
raise ValueError(f"未知模板命令生成规则:{template.id}")
async def _probe_backend_runtime(self, *, template: McpTemplateSpec, params: dict[str, Any]) -> dict[str, Any]:
_ = params
if template.id == "microsoft_learn":
return await MicrosoftLearnMCPRuntime().probe()
return {
"status": "error",
"checked_at": now_cn_naive().isoformat(),
"message": f"模板 {template.id} 暂未实现后端直连 runtime。",
"tool_names": [],
}

View File

@ -6,6 +6,7 @@ _WORK_ITEM_DELTA_CHANNEL = "work_item_delta"
_WORK_ITEM_DELTA_OFFSET_KEY = "last_work_item_updated_at" _WORK_ITEM_DELTA_OFFSET_KEY = "last_work_item_updated_at"
_LEGACY_FEISHU_DELTA_CHANNEL = "feishu_delta" _LEGACY_FEISHU_DELTA_CHANNEL = "feishu_delta"
_LEGACY_FEISHU_DELTA_OFFSET_KEY = "last_feishu_item_updated_at" _LEGACY_FEISHU_DELTA_OFFSET_KEY = "last_feishu_item_updated_at"
_SEMANTIC_FACT_EXTRACT_CHANNEL = "semantic_fact_extract"
class PKGEventQueueMixin: class PKGEventQueueMixin:
@ -99,6 +100,8 @@ class PKGEventQueueMixin:
try: try:
if str(ev.channel) in {_WORK_ITEM_DELTA_CHANNEL, _LEGACY_FEISHU_DELTA_CHANNEL}: if str(ev.channel) in {_WORK_ITEM_DELTA_CHANNEL, _LEGACY_FEISHU_DELTA_CHANNEL}:
await self._process_delta_event(ev) await self._process_delta_event(ev)
elif str(ev.channel) == _SEMANTIC_FACT_EXTRACT_CHANNEL:
await self._process_semantic_fact_extract_event(ev)
ev.status = "done" ev.status = "done"
ev.error_message = None ev.error_message = None
ev.processed_at = _utc_now_naive() ev.processed_at = _utc_now_naive()
@ -275,3 +278,40 @@ class PKGEventQueueMixin:
source_ref=source_ref, source_ref=source_ref,
metadata={"work_item_id": int(work_item.id)}, metadata={"work_item_id": int(work_item.id)},
) )
await self.enqueue_ingest_event(
channel=_SEMANTIC_FACT_EXTRACT_CHANNEL,
event_key=f"project:{int(work_item.project_id)}",
payload={
"entity_type": "project",
"entity_id": int(work_item.project_id),
"operation": "extract",
"trigger_type": "work_item_delta",
"work_item_id": int(work_item.id),
"tenant_id": int(self.tenant_id),
"source_updated_at": work_item.updated_at.isoformat() if work_item.updated_at else None,
},
status="pending",
)
async def _process_semantic_fact_extract_event(self, event: IngestEvent) -> None:
from ..semantic_fact_service import SemanticFactService
payload = event.payload if isinstance(event.payload, dict) else {}
entity_type = str(payload.get("entity_type") or "").strip().lower()
entity_id = payload.get("entity_id")
operation = str(payload.get("operation") or "extract").strip().lower() or "extract"
trigger_type = str(payload.get("trigger_type") or "event_queue").strip() or "event_queue"
if entity_id is None:
return
service = SemanticFactService(self.db, tenant_id=self.tenant_id)
if operation == "deactivate":
await service.deactivate_entity_facts(
entity_type=entity_type,
entity_id=int(entity_id),
)
return
await service.extract_entity_facts(
entity_type=entity_type,
entity_id=int(entity_id),
trigger_type=trigger_type,
)

View File

@ -0,0 +1,4 @@
from .service import SemanticFactService
from .hooks import register_semantic_fact_hooks
__all__ = ["SemanticFactService", "register_semantic_fact_hooks"]

View File

@ -0,0 +1,152 @@
from __future__ import annotations
from sqlalchemy import and_, event, select
from sqlalchemy.orm import Session
from ...models.ingest_event import IngestEvent
from ...models.knowledge_base_doc import KnowledgeBaseDoc
from ...models.project import Project
from ...models.signal import Signal
from ...models.task import Task
from ...models.truth_node import TruthNode
from ...models.user import User
from .shared import (
ENTITY_TYPE_TO_TRUTH_NODE_TYPE,
SEMANTIC_FACT_EXTRACT_CHANNEL,
TRUTH_NODE_TYPE_TO_ENTITY_TYPE,
_normalized_entity_type,
_status_is_inactive,
)
_SEMANTIC_FACT_INFO_KEY = "_pmflow_semantic_fact_events"
_SEMANTIC_FACT_APPLYING_KEY = "_pmflow_semantic_fact_applying"
def _truth_node_entity_type(node: TruthNode) -> str:
return TRUTH_NODE_TYPE_TO_ENTITY_TYPE.get(str(getattr(node, "node_type", "") or "").strip().lower(), "")
def _build_payload(obj: object, *, deleted: bool) -> dict | None:
entity_type = ""
if isinstance(obj, Project):
entity_type = "project"
elif isinstance(obj, Task):
entity_type = "task"
elif isinstance(obj, Signal):
entity_type = "signal"
elif isinstance(obj, KnowledgeBaseDoc):
entity_type = "document"
elif isinstance(obj, User):
entity_type = "user"
elif isinstance(obj, TruthNode):
entity_type = _truth_node_entity_type(obj)
if not entity_type:
return None
entity_id = getattr(obj, "id", None)
tenant_id = getattr(obj, "tenant_id", None)
if entity_id is None or tenant_id is None:
return None
operation = "deactivate" if deleted else "extract"
if not deleted:
if isinstance(obj, User):
operation = "extract" if bool(getattr(obj, "is_active", True)) and not _status_is_inactive(getattr(obj, "status", "")) else "deactivate"
elif _status_is_inactive(getattr(obj, "status", "")):
operation = "deactivate"
updated_at = getattr(obj, "updated_at", None)
return {
"event_key": f"{_normalized_entity_type(entity_type)}:{int(entity_id)}",
"payload": {
"entity_type": _normalized_entity_type(entity_type),
"entity_id": int(entity_id),
"operation": operation,
"trigger_type": "orm_commit",
"tenant_id": int(tenant_id),
"source_updated_at": updated_at.isoformat() if updated_at is not None else None,
},
}
def register_semantic_fact_hooks() -> None:
if getattr(register_semantic_fact_hooks, "_registered", False):
return
@event.listens_for(Session, "after_flush")
def _collect_semantic_fact_events(session: Session, _flush_context) -> None: # pragma: no cover
if bool(session.info.get(_SEMANTIC_FACT_APPLYING_KEY)):
return
payload_map = session.info.get(_SEMANTIC_FACT_INFO_KEY)
if not isinstance(payload_map, dict):
payload_map = {}
session.info[_SEMANTIC_FACT_INFO_KEY] = payload_map
objects = set(session.new).union(session.dirty).union(session.deleted)
deleted_objects = set(session.deleted)
for obj in objects:
if isinstance(obj, IngestEvent):
continue
payload = _build_payload(obj, deleted=obj in deleted_objects)
if not payload:
continue
payload_map[str(payload["event_key"])] = payload["payload"]
@event.listens_for(Session, "after_flush_postexec")
def _upsert_semantic_fact_events(session: Session, _flush_context) -> None: # pragma: no cover
if bool(session.info.get(_SEMANTIC_FACT_APPLYING_KEY)):
return
payload_map = session.info.get(_SEMANTIC_FACT_INFO_KEY)
if not isinstance(payload_map, dict) or not payload_map:
return
session.info[_SEMANTIC_FACT_APPLYING_KEY] = True
try:
for event_key, payload in payload_map.items():
tenant_id = int(payload.get("tenant_id") or 0)
if tenant_id <= 0:
continue
row = session.execute(
select(IngestEvent).where(
and_(
IngestEvent.tenant_id == tenant_id,
IngestEvent.channel == SEMANTIC_FACT_EXTRACT_CHANNEL,
IngestEvent.event_key == str(event_key),
)
)
).scalar_one_or_none()
if row is None:
session.add(
IngestEvent(
tenant_id=tenant_id,
channel=SEMANTIC_FACT_EXTRACT_CHANNEL,
event_key=str(event_key),
status="pending",
payload=payload,
retry_count=0,
error_message=None,
processed_at=None,
)
)
else:
row.status = "pending"
row.payload = payload
row.retry_count = 0
row.error_message = None
row.processed_at = None
payload_map.clear()
finally:
session.info[_SEMANTIC_FACT_APPLYING_KEY] = False
@event.listens_for(Session, "after_commit")
def _clear_semantic_fact_events_after_commit(session: Session) -> None: # pragma: no cover
session.info.pop(_SEMANTIC_FACT_INFO_KEY, None)
session.info.pop(_SEMANTIC_FACT_APPLYING_KEY, None)
@event.listens_for(Session, "after_rollback")
def _clear_semantic_fact_events_after_rollback(session: Session) -> None: # pragma: no cover
session.info.pop(_SEMANTIC_FACT_INFO_KEY, None)
session.info.pop(_SEMANTIC_FACT_APPLYING_KEY, None)
setattr(register_semantic_fact_hooks, "_registered", True)
__all__ = ["register_semantic_fact_hooks"]

View File

@ -0,0 +1,567 @@
from __future__ import annotations
from datetime import UTC, datetime, timedelta
from .shared import *
class SemanticFactAdminMixin:
async def _list_project_snapshots(self, *, limit: int) -> list[dict[str, Any]]:
rows = (
await self.db.execute(
select(Project)
.where(Project.tenant_id == self.tenant_id)
.order_by(Project.updated_at.desc(), Project.id.desc())
.limit(limit)
)
).scalars().all()
return [
{
"entity_type": "project",
"entity_id": int(row.id),
"display_name": str(row.title or row.name or f"项目#{int(row.id)}").strip(),
"object_status": str(row.status or "").strip(),
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
"is_extractable": not _status_is_inactive(row.status),
"summary": str(row.description or "").strip(),
}
for row in rows
]
async def _list_task_snapshots(self, *, limit: int) -> list[dict[str, Any]]:
rows = (
await self.db.execute(
select(Task)
.where(Task.tenant_id == self.tenant_id)
.order_by(Task.updated_at.desc(), Task.id.desc())
.limit(limit)
)
).scalars().all()
return [
{
"entity_type": "task",
"entity_id": int(row.id),
"display_name": str(row.title or f"任务#{int(row.id)}").strip(),
"object_status": str(row.status or "").strip(),
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
"is_extractable": not _status_is_inactive(row.status),
"summary": str(row.description or "").strip(),
}
for row in rows
]
async def _list_signal_snapshots(self, *, limit: int) -> list[dict[str, Any]]:
rows = (
await self.db.execute(
select(Signal)
.where(Signal.tenant_id == self.tenant_id)
.order_by(Signal.updated_at.desc(), Signal.id.desc())
.limit(limit)
)
).scalars().all()
return [
{
"entity_type": "signal",
"entity_id": int(row.id),
"display_name": str(row.title or row.summary or f"信号#{int(row.id)}").strip(),
"object_status": str(row.status or "").strip(),
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
"is_extractable": not _status_is_inactive(row.status),
"summary": str(row.summary or row.content or "").strip(),
}
for row in rows
]
async def _list_document_snapshots(self, *, limit: int) -> list[dict[str, Any]]:
rows = (
await self.db.execute(
select(KnowledgeBaseDoc)
.where(KnowledgeBaseDoc.tenant_id == self.tenant_id)
.order_by(KnowledgeBaseDoc.updated_at.desc(), KnowledgeBaseDoc.id.desc())
.limit(limit)
)
).scalars().all()
return [
{
"entity_type": "document",
"entity_id": int(row.id),
"display_name": str(row.title or f"文档#{int(row.id)}").strip(),
"object_status": str(row.status or "").strip(),
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
"is_extractable": not _status_is_inactive(row.status),
"summary": f"{str(row.file_type or '').strip()} / {str(row.source_type or '').strip()}",
}
for row in rows
]
async def _list_user_snapshots(self, *, limit: int) -> list[dict[str, Any]]:
rows = (
await self.db.execute(
select(User)
.where(User.tenant_id == self.tenant_id)
.order_by(User.updated_at.desc(), User.id.desc())
.limit(limit)
)
).scalars().all()
return [
{
"entity_type": "user",
"entity_id": int(row.id),
"display_name": str(row.name or row.username or f"成员#{int(row.id)}").strip(),
"object_status": str(row.status or "").strip(),
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
"is_extractable": bool(row.is_active) and not _status_is_inactive(row.status),
"summary": f"{str(row.role or '').strip()} / {str(row.username or '').strip()}",
}
for row in rows
]
async def _list_truth_node_snapshots(self, *, entity_type: str, limit: int) -> list[dict[str, Any]]:
truth_node_type = ENTITY_TYPE_TO_TRUTH_NODE_TYPE[entity_type]
rows = (
await self.db.execute(
select(TruthNode)
.where(
and_(
TruthNode.tenant_id == self.tenant_id,
TruthNode.node_type == truth_node_type,
)
)
.order_by(TruthNode.updated_at.desc(), TruthNode.id.desc())
.limit(limit)
)
).scalars().all()
return [
{
"entity_type": entity_type,
"entity_id": int(row.id),
"display_name": str(row.title or f"{entity_type}#{int(row.id)}").strip(),
"object_status": str(row.status or "").strip(),
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
"is_extractable": not _status_is_inactive(row.status),
"summary": str((row.content or {}).get("summary") or "").strip() if isinstance(row.content, dict) else "",
}
for row in rows
]
async def list_object_snapshots(
self,
*,
entity_types: Optional[list[str]] = None,
limit_per_type: int = 200,
) -> list[dict[str, Any]]:
normalized_types = [
_normalized_entity_type(item)
for item in list(entity_types or sorted(SEMANTIC_FACT_SUPPORTED_ENTITY_TYPES))
if _normalized_entity_type(item) in SEMANTIC_FACT_SUPPORTED_ENTITY_TYPES
]
if not normalized_types:
normalized_types = sorted(SEMANTIC_FACT_SUPPORTED_ENTITY_TYPES)
limit = max(1, min(int(limit_per_type), 1000))
items: list[dict[str, Any]] = []
for entity_type in normalized_types:
if entity_type == "project":
items.extend(await self._list_project_snapshots(limit=limit))
elif entity_type == "task":
items.extend(await self._list_task_snapshots(limit=limit))
elif entity_type == "signal":
items.extend(await self._list_signal_snapshots(limit=limit))
elif entity_type == "document":
items.extend(await self._list_document_snapshots(limit=limit))
elif entity_type == "user":
items.extend(await self._list_user_snapshots(limit=limit))
elif entity_type in {"requirement", "release", "review"}:
items.extend(await self._list_truth_node_snapshots(entity_type=entity_type, limit=limit))
return items
async def _load_object_metadata_maps(
self,
*,
items: list[dict[str, Any]],
) -> tuple[dict[tuple[str, int], int], dict[tuple[str, int], list[str]], dict[tuple[str, int], dict[str, Any]], dict[tuple[str, int], dict[str, Any]]]:
if not items:
return {}, {}, {}, {}
keys = {(str(item["entity_type"]), int(item["entity_id"])) for item in items}
entity_types = sorted({entity_type for entity_type, _entity_id in keys})
fact_rows = (
await self.db.execute(
select(SemanticFact).where(
and_(
SemanticFact.tenant_id == self.tenant_id,
SemanticFact.entity_type.in_(entity_types),
SemanticFact.is_active.is_(True),
)
)
)
).scalars().all()
fact_counts: dict[tuple[str, int], int] = {}
fact_texts: dict[tuple[str, int], list[str]] = {}
for row in fact_rows:
key = (str(row.entity_type), int(row.entity_id))
if key not in keys:
continue
fact_counts[key] = int(fact_counts.get(key) or 0) + 1
fact_texts.setdefault(key, []).append(
" ".join(
part for part in [
str(row.fact_key or "").strip(),
str(row.fact_value_text or "").strip(),
str(row.fact_value_norm or "").strip(),
] if part
)
)
run_rows = (
await self.db.execute(
select(SemanticFactExtractionRun)
.where(
and_(
SemanticFactExtractionRun.tenant_id == self.tenant_id,
SemanticFactExtractionRun.entity_type.in_(entity_types),
)
)
.order_by(SemanticFactExtractionRun.id.desc())
)
).scalars().all()
latest_runs: dict[tuple[str, int], dict[str, Any]] = {}
for row in run_rows:
key = (str(row.entity_type), int(row.entity_id))
if key not in keys or key in latest_runs:
continue
latest_runs[key] = {
"id": int(row.id),
"status": str(row.status or "").strip(),
"trigger_type": str(row.trigger_type or "").strip(),
"facts_written": int(row.facts_written or 0),
"failure_reason": str(row.failure_reason or "").strip() or None,
"extract_version": str(row.extract_version or "").strip() or None,
"started_at": str(row.started_at or "").strip() or None,
"finished_at": str(row.finished_at or "").strip() or None,
}
event_rows = (
await self.db.execute(
select(IngestEvent).where(
and_(
IngestEvent.tenant_id == self.tenant_id,
IngestEvent.channel == SEMANTIC_FACT_EXTRACT_CHANNEL,
)
)
)
).scalars().all()
latest_events: dict[tuple[str, int], dict[str, Any]] = {}
for row in event_rows:
payload = row.payload if isinstance(row.payload, dict) else {}
key = (
_normalized_entity_type(payload.get("entity_type") or ""),
int(payload.get("entity_id") or 0),
)
if key not in keys:
continue
latest_events[key] = {
"id": int(row.id),
"status": str(row.status or "").strip(),
"retry_count": int(row.retry_count or 0),
"error_message": str(row.error_message or "").strip() or None,
"processed_at": row.processed_at.isoformat() if row.processed_at else None,
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
"created_at": row.created_at.isoformat() if row.created_at else None,
"payload": payload,
}
return fact_counts, fact_texts, latest_runs, latest_events
@staticmethod
def _resolve_last_status(*, event: Optional[dict[str, Any]], run: Optional[dict[str, Any]]) -> str:
if event and str(event.get("status") or "").strip():
return str(event.get("status") or "").strip()
if run and str(run.get("status") or "").strip():
return str(run.get("status") or "").strip()
return "idle"
async def list_admin_objects(
self,
*,
entity_types: Optional[list[str]] = None,
q: Optional[str] = None,
status: Optional[str] = None,
limit_per_type: int = 200,
) -> list[dict[str, Any]]:
items = await self.list_object_snapshots(entity_types=entity_types, limit_per_type=limit_per_type)
fact_counts, fact_texts, latest_runs, latest_events = await self._load_object_metadata_maps(items=items)
query_text = str(q or "").strip().lower()
status_filter = str(status or "").strip().lower()
results: list[dict[str, Any]] = []
for item in items:
key = (str(item["entity_type"]), int(item["entity_id"]))
last_run = latest_runs.get(key)
last_event = latest_events.get(key)
last_status = self._resolve_last_status(event=last_event, run=last_run)
result = {
**item,
"facts_count": int(fact_counts.get(key) or 0),
"last_extracted_at": (
str((last_run or {}).get("finished_at") or (last_run or {}).get("started_at") or "").strip() or None
),
"last_status": last_status,
"queue_event": last_event,
"last_run": last_run,
}
if query_text:
haystack = " ".join(
[
str(item.get("display_name") or ""),
str(item.get("summary") or ""),
str(item.get("entity_id") or ""),
" ".join(fact_texts.get(key) or []),
]
).lower()
if query_text not in haystack:
continue
if status_filter and status_filter not in {"all", last_status.lower()}:
continue
results.append(result)
results.sort(
key=lambda row: (
_parse_iso_datetime(row.get("last_extracted_at")) or _parse_iso_datetime(row.get("updated_at")) or datetime(1970, 1, 1, tzinfo=UTC),
int(row.get("entity_id") or 0),
),
reverse=True,
)
return results
async def get_admin_detail(
self,
*,
entity_type: str,
entity_id: int,
) -> dict[str, Any]:
snapshot = await self._load_entity_snapshot(entity_type=entity_type, entity_id=int(entity_id))
if snapshot is None:
raise ValueError("semantic_fact_entity_not_found")
facts = (
await self.db.execute(
select(SemanticFact).where(
and_(
SemanticFact.tenant_id == self.tenant_id,
SemanticFact.entity_type == _normalized_entity_type(entity_type),
SemanticFact.entity_id == int(entity_id),
SemanticFact.is_active.is_(True),
)
).order_by(SemanticFact.confidence.desc(), SemanticFact.updated_at.desc(), SemanticFact.id.desc())
)
).scalars().all()
runs = (
await self.db.execute(
select(SemanticFactExtractionRun).where(
and_(
SemanticFactExtractionRun.tenant_id == self.tenant_id,
SemanticFactExtractionRun.entity_type == _normalized_entity_type(entity_type),
SemanticFactExtractionRun.entity_id == int(entity_id),
)
).order_by(SemanticFactExtractionRun.id.desc()).limit(10)
)
).scalars().all()
event = (
await self.db.execute(
select(IngestEvent).where(
and_(
IngestEvent.tenant_id == self.tenant_id,
IngestEvent.channel == SEMANTIC_FACT_EXTRACT_CHANNEL,
IngestEvent.event_key == f"{_normalized_entity_type(entity_type)}:{int(entity_id)}",
)
)
)
).scalar_one_or_none()
return {
"object": {
"entity_type": str(snapshot["entity_type"]),
"entity_id": int(snapshot["entity_id"]),
"display_name": str(snapshot["display_name"]),
"object_status": str(snapshot.get("object_status") or ""),
"updated_at": snapshot.get("updated_at"),
"is_extractable": bool(snapshot.get("is_extractable")),
"source_payload": snapshot.get("source_payload") or {},
},
"facts": [
{
"id": int(row.id),
"fact_namespace": str(row.fact_namespace or "").strip(),
"fact_key": str(row.fact_key or "").strip(),
"fact_value_text": str(row.fact_value_text or "").strip(),
"fact_value_norm": str(row.fact_value_norm or "").strip(),
"value_type": str(row.value_type or "").strip(),
"confidence": float(row.confidence or 0.0),
"evidence_excerpt": str(row.evidence_excerpt or "").strip() or None,
"freshness_at": str(row.freshness_at or "").strip() or None,
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
}
for row in facts
],
"recent_runs": [
{
"id": int(row.id),
"status": str(row.status or "").strip(),
"trigger_type": str(row.trigger_type or "").strip(),
"facts_written": int(row.facts_written or 0),
"failure_reason": str(row.failure_reason or "").strip() or None,
"extract_version": str(row.extract_version or "").strip() or None,
"started_at": str(row.started_at or "").strip() or None,
"finished_at": str(row.finished_at or "").strip() or None,
}
for row in runs
],
"queue_event": (
{
"id": int(event.id),
"status": str(event.status or "").strip(),
"retry_count": int(event.retry_count or 0),
"error_message": str(event.error_message or "").strip() or None,
"processed_at": event.processed_at.isoformat() if event.processed_at else None,
"updated_at": event.updated_at.isoformat() if event.updated_at else None,
"payload": event.payload if isinstance(event.payload, dict) else {},
}
if event is not None
else None
),
}
async def enqueue_backfill(
self,
*,
entity_types: Optional[list[str]] = None,
only_missing: bool = True,
limit_per_type: int = 200,
) -> dict[str, Any]:
items = await self.list_object_snapshots(entity_types=entity_types, limit_per_type=limit_per_type)
by_type: dict[str, int] = {}
existing_pairs: set[tuple[str, int]] = set()
if only_missing:
fact_rows = (
await self.db.execute(
select(SemanticFact.entity_type, SemanticFact.entity_id).where(
and_(
SemanticFact.tenant_id == self.tenant_id,
SemanticFact.is_active.is_(True),
)
)
)
).all()
existing_pairs = {
(str(entity_type), int(entity_id))
for entity_type, entity_id in fact_rows
}
total = 0
for item in items:
key = (str(item["entity_type"]), int(item["entity_id"]))
if not bool(item.get("is_extractable")):
continue
if only_missing and key in existing_pairs:
continue
await self.enqueue_entity_event(
entity_type=str(item["entity_type"]),
entity_id=int(item["entity_id"]),
operation="extract",
trigger_type="backfill",
extra_payload={
"source_updated_at": str(item.get("updated_at") or "").strip() or None,
},
)
by_type[str(item["entity_type"])] = int(by_type.get(str(item["entity_type"])) or 0) + 1
total += 1
return {
"enqueued": total,
"only_missing": bool(only_missing),
"by_type": by_type,
}
async def get_admin_overview(self) -> dict[str, Any]:
items = await self.list_object_snapshots(limit_per_type=500)
active_fact_total = int(
(
await self.db.execute(
select(func.count(SemanticFact.id)).where(
and_(
SemanticFact.tenant_id == self.tenant_id,
SemanticFact.is_active.is_(True),
)
)
)
).scalar_one()
or 0
)
event_rows = (
await self.db.execute(
select(IngestEvent).where(
and_(
IngestEvent.tenant_id == self.tenant_id,
IngestEvent.channel == SEMANTIC_FACT_EXTRACT_CHANNEL,
)
).order_by(IngestEvent.updated_at.desc(), IngestEvent.id.desc())
)
).scalars().all()
pending_rows = [row for row in event_rows if str(row.status or "").strip() in {"pending", "retry"}]
failed_rows = [row for row in event_rows if str(row.status or "").strip() == "dead_letter"]
running_count = int(
(
await self.db.execute(
select(func.count(SemanticFactExtractionRun.id)).where(
and_(
SemanticFactExtractionRun.tenant_id == self.tenant_id,
SemanticFactExtractionRun.status == "running",
)
)
)
).scalar_one()
or 0
)
all_runs = (
await self.db.execute(
select(SemanticFactExtractionRun).where(
SemanticFactExtractionRun.tenant_id == self.tenant_id
)
)
).scalars().all()
cutoff = datetime.now(UTC) - timedelta(hours=24)
success_24h = 0
failed_24h = 0
for row in all_runs:
pivot = _parse_iso_datetime(row.finished_at or row.started_at)
if pivot is None or pivot < cutoff:
continue
status = str(row.status or "").strip()
if status == "completed":
success_24h += 1
elif status == "failed":
failed_24h += 1
return {
"kpis": {
"object_total": len(items),
"active_fact_total": active_fact_total,
"success_24h": success_24h,
"failed_24h": failed_24h,
},
"queue": {
"pending": len(pending_rows),
"processing": running_count,
"failed": len(failed_rows),
"oldest_pending_at": pending_rows[-1].created_at.isoformat() if pending_rows else None,
"recent_failures": [
{
"event_key": str(row.event_key or ""),
"status": str(row.status or "").strip(),
"error_message": str(row.error_message or "").strip() or None,
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
}
for row in failed_rows[:5]
],
},
"entity_types": sorted(SEMANTIC_FACT_SUPPORTED_ENTITY_TYPES),
}
__all__ = ["SemanticFactAdminMixin"]

View File

@ -0,0 +1,370 @@
from __future__ import annotations
from .shared import *
class SemanticFactExtractMixin:
async def _create_extraction_run(
self,
*,
entity_type: str,
entity_id: int,
trigger_type: str,
input_fingerprint: str,
) -> SemanticFactExtractionRun:
run = SemanticFactExtractionRun(
tenant_id=self.tenant_id,
entity_type=_normalized_entity_type(entity_type),
entity_id=int(entity_id),
trigger_type=str(trigger_type or "").strip() or "manual_retry",
status="running",
input_fingerprint=input_fingerprint,
facts_written=0,
extract_version=SEMANTIC_FACT_EXTRACT_VERSION,
started_at=_utc_now_iso(),
)
self.db.add(run)
await self.db.flush()
return run
async def _load_project_snapshot(self, *, entity_id: int) -> Optional[dict[str, Any]]:
project = (
await self.db.execute(
select(Project).where(
and_(
Project.tenant_id == self.tenant_id,
Project.id == int(entity_id),
)
)
)
).scalar_one_or_none()
if project is None:
return None
return {
"entity_type": "project",
"entity_id": int(project.id),
"display_name": str(project.title or project.name or f"项目#{int(project.id)}").strip(),
"object_status": str(project.status or "").strip(),
"updated_at": project.updated_at.isoformat() if project.updated_at else None,
"is_extractable": not _status_is_inactive(project.status),
"source_payload": {
"project_id": int(project.id),
"name": str(project.name or "").strip(),
"title": str(project.title or "").strip(),
"description": str(project.description or "").strip(),
"status": str(project.status or "").strip(),
"assigned_role": str(project.assigned_role or "").strip(),
},
}
async def _load_task_snapshot(self, *, entity_id: int) -> Optional[dict[str, Any]]:
task = (
await self.db.execute(
select(Task).where(
and_(
Task.tenant_id == self.tenant_id,
Task.id == int(entity_id),
)
)
)
).scalar_one_or_none()
if task is None:
return None
return {
"entity_type": "task",
"entity_id": int(task.id),
"display_name": str(task.title or f"任务#{int(task.id)}").strip(),
"object_status": str(task.status or "").strip(),
"updated_at": task.updated_at.isoformat() if task.updated_at else None,
"is_extractable": not _status_is_inactive(task.status),
"source_payload": {
"task_id": int(task.id),
"project_id": int(task.project_id),
"truth_node_id": int(task.truth_node_id),
"assignee_id": int(task.assignee_id) if task.assignee_id is not None else None,
"title": str(task.title or "").strip(),
"description": str(task.description or "").strip(),
"status": str(task.status or "").strip(),
"priority": str(task.priority or "").strip(),
"required_skill_tag_ids": list(task.required_skill_tag_ids or []),
},
}
async def _load_signal_snapshot(self, *, entity_id: int) -> Optional[dict[str, Any]]:
signal = (
await self.db.execute(
select(Signal).where(
and_(
Signal.tenant_id == self.tenant_id,
Signal.id == int(entity_id),
)
)
)
).scalar_one_or_none()
if signal is None:
return None
return {
"entity_type": "signal",
"entity_id": int(signal.id),
"display_name": str(signal.title or signal.summary or f"信号#{int(signal.id)}").strip(),
"object_status": str(signal.status or "").strip(),
"updated_at": signal.updated_at.isoformat() if signal.updated_at else None,
"is_extractable": not _status_is_inactive(signal.status),
"source_payload": {
"signal_id": int(signal.id),
"title": str(signal.title or "").strip(),
"summary": str(signal.summary or "").strip(),
"content": str(signal.content or "").strip(),
"status": str(signal.status or "").strip(),
"priority": str(signal.priority or "").strip(),
"signal_type": str(signal.signal_type or "").strip(),
"source": str(signal.source or "").strip(),
"matched_project_id": int(signal.matched_project_id) if signal.matched_project_id is not None else None,
"processed": bool(signal.processed),
},
}
async def _load_document_snapshot(self, *, entity_id: int) -> Optional[dict[str, Any]]:
doc = (
await self.db.execute(
select(KnowledgeBaseDoc).where(
and_(
KnowledgeBaseDoc.tenant_id == self.tenant_id,
KnowledgeBaseDoc.id == int(entity_id),
)
)
)
).scalar_one_or_none()
if doc is None:
return None
chunk_rows = (
await self.db.execute(
select(KnowledgeBaseChunk).where(
and_(
KnowledgeBaseChunk.tenant_id == self.tenant_id,
KnowledgeBaseChunk.doc_id == int(doc.id),
)
).order_by(KnowledgeBaseChunk.chunk_index.asc()).limit(3)
)
).scalars().all()
preview_text = "\n".join(str(row.content or "").strip() for row in chunk_rows if str(row.content or "").strip())
return {
"entity_type": "document",
"entity_id": int(doc.id),
"display_name": str(doc.title or f"文档#{int(doc.id)}").strip(),
"object_status": str(doc.status or "").strip(),
"updated_at": doc.updated_at.isoformat() if doc.updated_at else None,
"is_extractable": not _status_is_inactive(doc.status),
"source_payload": {
"doc_id": int(doc.id),
"title": str(doc.title or "").strip(),
"file_type": str(doc.file_type or "").strip(),
"source_type": str(doc.source_type or "").strip(),
"file_url": str(doc.file_url or "").strip(),
"status": str(doc.status or "").strip(),
"preview_text": preview_text[:2000],
},
}
async def _load_user_snapshot(self, *, entity_id: int) -> Optional[dict[str, Any]]:
user = (
await self.db.execute(
select(User).where(
and_(
User.tenant_id == self.tenant_id,
User.id == int(entity_id),
)
)
)
).scalar_one_or_none()
if user is None:
return None
return {
"entity_type": "user",
"entity_id": int(user.id),
"display_name": str(user.name or user.username or f"成员#{int(user.id)}").strip(),
"object_status": str(user.status or "").strip(),
"updated_at": user.updated_at.isoformat() if user.updated_at else None,
"is_extractable": bool(user.is_active) and not _status_is_inactive(user.status),
"source_payload": {
"user_id": int(user.id),
"username": str(user.username or "").strip(),
"name": str(user.name or "").strip(),
"role": str(user.role or "").strip(),
"status": str(user.status or "").strip(),
"is_active": bool(user.is_active),
"bio": str(user.bio or "").strip(),
"assigned_domains": list(user.assigned_domains or []),
},
}
async def _load_truth_node_snapshot(self, *, entity_type: str, entity_id: int) -> Optional[dict[str, Any]]:
truth_node_type = ENTITY_TYPE_TO_TRUTH_NODE_TYPE.get(_normalized_entity_type(entity_type))
if not truth_node_type:
return None
node = (
await self.db.execute(
select(TruthNode).where(
and_(
TruthNode.tenant_id == self.tenant_id,
TruthNode.id == int(entity_id),
TruthNode.node_type == truth_node_type,
)
)
)
).scalar_one_or_none()
if node is None:
return None
normalized_entity_type = TRUTH_NODE_TYPE_TO_ENTITY_TYPE.get(str(node.node_type or "").strip().lower(), "")
if not normalized_entity_type:
return None
return {
"entity_type": normalized_entity_type,
"entity_id": int(node.id),
"display_name": str(node.title or f"{normalized_entity_type}#{int(node.id)}").strip(),
"object_status": str(node.status or "").strip(),
"updated_at": node.updated_at.isoformat() if node.updated_at else None,
"is_extractable": not _status_is_inactive(node.status),
"source_payload": {
"node_id": int(node.id),
"node_type": str(node.node_type or "").strip(),
"project_id": int(node.project_id) if node.project_id is not None else None,
"signal_id": int(node.signal_id) if node.signal_id is not None else None,
"title": str(node.title or "").strip(),
"status": str(node.status or "").strip(),
"version": int(node.version or 1),
"content": node.content if isinstance(node.content, dict) else {},
},
}
async def _load_entity_snapshot(
self,
*,
entity_type: str,
entity_id: int,
) -> Optional[dict[str, Any]]:
normalized_entity_type = _normalized_entity_type(entity_type)
if normalized_entity_type == "project":
return await self._load_project_snapshot(entity_id=int(entity_id))
if normalized_entity_type == "task":
return await self._load_task_snapshot(entity_id=int(entity_id))
if normalized_entity_type == "signal":
return await self._load_signal_snapshot(entity_id=int(entity_id))
if normalized_entity_type == "document":
return await self._load_document_snapshot(entity_id=int(entity_id))
if normalized_entity_type == "user":
return await self._load_user_snapshot(entity_id=int(entity_id))
if normalized_entity_type in {"requirement", "release", "review"}:
return await self._load_truth_node_snapshot(entity_type=normalized_entity_type, entity_id=int(entity_id))
raise ValueError(f"unsupported_semantic_fact_entity_type:{normalized_entity_type}")
def _render_extraction_prompt(self, *, entity_type: str, source_payload: dict[str, Any]) -> str:
normalized_entity_type = _normalized_entity_type(entity_type)
prompt_template = SEMANTIC_FACT_PROMPTS.get(normalized_entity_type)
if prompt_template is None:
raise ValueError(f"unsupported_semantic_fact_prompt:{normalized_entity_type}")
payload_text = json.dumps(source_payload, ensure_ascii=False, sort_keys=True)
if normalized_entity_type == "project":
return prompt_template.replace("<<project_payload>>", payload_text).strip()
return prompt_template.replace("<<entity_payload>>", payload_text).strip()
async def extract_entity_facts(
self,
*,
entity_type: str,
entity_id: int,
trigger_type: str,
) -> Optional[SemanticFactExtractionRun]:
from ..llm_service import llm_service
normalized_entity_type = _normalized_entity_type(entity_type)
snapshot = await self._load_entity_snapshot(
entity_type=normalized_entity_type,
entity_id=int(entity_id),
)
if snapshot is None:
await self.deactivate_entity_facts(entity_type=normalized_entity_type, entity_id=int(entity_id))
return None
if not bool(snapshot.get("is_extractable")):
await self.deactivate_entity_facts(entity_type=normalized_entity_type, entity_id=int(entity_id))
return None
source_payload = dict(snapshot.get("source_payload") or {})
fingerprint = hashlib.sha1(
json.dumps(source_payload, ensure_ascii=False, sort_keys=True).encode("utf-8")
).hexdigest()
run = await self._create_extraction_run(
entity_type=normalized_entity_type,
entity_id=int(entity_id),
trigger_type=trigger_type,
input_fingerprint=fingerprint,
)
try:
prompt = self._render_extraction_prompt(
entity_type=normalized_entity_type,
source_payload=source_payload,
)
extraction = await llm_service.generate_structured_object(
prompt,
db=self.db,
tenant_id=self.tenant_id,
function_name=SEMANTIC_FACT_FUNCTION_NAMES[normalized_entity_type],
temperature=0.0,
max_tokens=1200,
)
payload = extraction.payload if isinstance(extraction.payload, dict) else {}
fact_rows = list(payload.get("facts") or [])
await self.deactivate_entity_facts(
entity_type=normalized_entity_type,
entity_id=int(entity_id),
)
written = 0
for item in fact_rows:
if not isinstance(item, dict):
continue
fact_key = str(item.get("fact_key") or "").strip()
fact_value_text = str(item.get("fact_value_text") or "").strip()
if not fact_key or not fact_value_text:
continue
await self.upsert_fact(
entity_type=normalized_entity_type,
entity_id=int(entity_id),
fact_namespace=str(item.get("fact_namespace") or "general").strip() or "general",
fact_key=fact_key,
fact_value_text=fact_value_text,
fact_value_norm=str(item.get("fact_value_norm") or "").strip() or None,
value_type=str(item.get("value_type") or "keyword").strip() or "keyword",
confidence=float(item.get("confidence") or 0.0),
evidence_source_type=normalized_entity_type,
evidence_source_id=int(entity_id),
evidence_excerpt=str(item.get("evidence_excerpt") or "").strip() or None,
extract_version=SEMANTIC_FACT_EXTRACT_VERSION,
freshness_at=str(snapshot.get("updated_at") or "").strip() or None,
)
written += 1
run.status = "completed"
run.facts_written = written
run.failure_reason = None
run.finished_at = _utc_now_iso()
await self.db.flush()
return run
except Exception as exc:
run.status = "failed"
run.failure_reason = str(exc)[:1000]
run.finished_at = _utc_now_iso()
await self.db.flush()
raise
async def extract_project_facts(self, *, project_id: int, trigger_type: str) -> SemanticFactExtractionRun:
run = await self.extract_entity_facts(
entity_type="project",
entity_id=int(project_id),
trigger_type=trigger_type,
)
if run is None:
raise ValueError(f"project_not_found:{int(project_id)}")
return run
__all__ = ["SemanticFactExtractMixin"]

View File

@ -0,0 +1,198 @@
from __future__ import annotations
from .shared import *
class SemanticFactQueryMixin:
async def upsert_fact(
self,
*,
entity_type: str,
entity_id: int,
fact_namespace: str,
fact_key: str,
fact_value_text: str,
fact_value_norm: Optional[str],
value_type: str,
confidence: float,
evidence_source_type: Optional[str],
evidence_source_id: Optional[int],
evidence_excerpt: Optional[str],
extract_version: Optional[str],
freshness_at: Optional[str],
) -> SemanticFact:
normalized_fact_value = _normalize_fact_value(fact_value_norm or fact_value_text)
row = (
await self.db.execute(
select(SemanticFact).where(
and_(
SemanticFact.tenant_id == self.tenant_id,
SemanticFact.entity_type == str(entity_type or "").strip(),
SemanticFact.entity_id == int(entity_id),
SemanticFact.fact_key == str(fact_key or "").strip(),
SemanticFact.fact_value_norm == normalized_fact_value,
)
)
)
).scalar_one_or_none()
if row is None:
row = SemanticFact(
tenant_id=self.tenant_id,
entity_type=str(entity_type or "").strip(),
entity_id=int(entity_id),
fact_namespace=str(fact_namespace or "").strip() or "general",
fact_key=str(fact_key or "").strip(),
fact_value_text=str(fact_value_text or "").strip(),
fact_value_norm=normalized_fact_value,
value_type=str(value_type or "").strip() or "keyword",
confidence=max(0.0, min(1.0, float(confidence or 0.0))),
evidence_source_type=str(evidence_source_type or "").strip() or None,
evidence_source_id=int(evidence_source_id) if evidence_source_id is not None else None,
evidence_excerpt=str(evidence_excerpt or "").strip() or None,
extract_version=str(extract_version or "").strip() or None,
freshness_at=str(freshness_at or "").strip() or None,
is_active=True,
)
self.db.add(row)
await self.db.flush()
return row
row.fact_namespace = str(fact_namespace or "").strip() or row.fact_namespace
row.fact_value_text = str(fact_value_text or "").strip() or row.fact_value_text
row.value_type = str(value_type or "").strip() or row.value_type
row.confidence = max(0.0, min(1.0, float(confidence or 0.0)))
row.evidence_source_type = str(evidence_source_type or "").strip() or row.evidence_source_type
row.evidence_source_id = int(evidence_source_id) if evidence_source_id is not None else row.evidence_source_id
row.evidence_excerpt = str(evidence_excerpt or "").strip() or row.evidence_excerpt
row.extract_version = str(extract_version or "").strip() or row.extract_version
row.freshness_at = str(freshness_at or "").strip() or row.freshness_at
row.is_active = True
await self.db.flush()
return row
async def deactivate_entity_facts(
self,
*,
entity_type: str,
entity_id: int,
) -> int:
normalized_entity_type = _normalized_entity_type(entity_type)
rows = (
await self.db.execute(
select(SemanticFact).where(
and_(
SemanticFact.tenant_id == self.tenant_id,
SemanticFact.entity_type == normalized_entity_type,
SemanticFact.entity_id == int(entity_id),
SemanticFact.is_active.is_(True),
)
)
)
).scalars().all()
for row in rows:
row.is_active = False
await self.db.flush()
return len(rows)
async def enqueue_entity_event(
self,
*,
entity_type: str,
entity_id: int,
operation: str,
trigger_type: str,
extra_payload: Optional[dict[str, Any]] = None,
) -> IngestEvent:
normalized_entity_type = _normalized_entity_type(entity_type)
event_key = f"{normalized_entity_type}:{int(entity_id)}"
payload = {
"entity_type": normalized_entity_type,
"entity_id": int(entity_id),
"operation": str(operation or "extract").strip() or "extract",
"trigger_type": str(trigger_type or "manual_retry").strip() or "manual_retry",
"tenant_id": int(self.tenant_id),
}
if isinstance(extra_payload, dict):
for key, value in extra_payload.items():
payload[str(key)] = value
row = (
await self.db.execute(
select(IngestEvent).where(
and_(
IngestEvent.tenant_id == self.tenant_id,
IngestEvent.channel == SEMANTIC_FACT_EXTRACT_CHANNEL,
IngestEvent.event_key == event_key,
)
)
)
).scalar_one_or_none()
if row is None:
row = IngestEvent(
tenant_id=self.tenant_id,
channel=SEMANTIC_FACT_EXTRACT_CHANNEL,
event_key=event_key,
status="pending",
payload=payload,
retry_count=0,
error_message=None,
processed_at=None,
)
self.db.add(row)
else:
row.status = "pending"
row.payload = payload
row.retry_count = 0
row.error_message = None
row.processed_at = None
await self.db.flush()
return row
async def query_facts(
self,
*,
entity_type: str,
query_type: str,
filters: Optional[list[dict[str, Any]]] = None,
entity_id: Optional[int] = None,
limit: int = 20,
) -> list[SemanticFact]:
_ = query_type
stmt = select(SemanticFact).where(
and_(
SemanticFact.tenant_id == self.tenant_id,
SemanticFact.entity_type == _normalized_entity_type(entity_type),
SemanticFact.is_active.is_(True),
)
)
if entity_id is not None:
stmt = stmt.where(SemanticFact.entity_id == int(entity_id))
for item in list(filters or []):
if not isinstance(item, dict):
continue
fact_key = str(item.get("fact_key") or item.get("field") or "").strip()
if fact_key:
stmt = stmt.where(SemanticFact.fact_key == fact_key)
raw_value = str(item.get("value") or item.get("normalized_value") or "").strip()
normalized_value = _normalize_fact_value(item.get("normalized_value") or raw_value)
if normalized_value:
stmt = stmt.where(
or_(
SemanticFact.fact_value_norm == normalized_value,
func.lower(SemanticFact.fact_value_text) == raw_value.lower(),
)
)
rows = (
await self.db.execute(
stmt.order_by(SemanticFact.confidence.desc(), SemanticFact.updated_at.desc(), SemanticFact.id.desc()).limit(
max(1, int(limit))
)
)
).scalars().all()
return list(rows)
__all__ = ["SemanticFactQueryMixin"]

View File

@ -0,0 +1,19 @@
from __future__ import annotations
from .mixin_admin import SemanticFactAdminMixin
from .mixin_extract import SemanticFactExtractMixin
from .mixin_query import SemanticFactQueryMixin
from .shared import AsyncSession
class SemanticFactService(
SemanticFactAdminMixin,
SemanticFactExtractMixin,
SemanticFactQueryMixin,
):
def __init__(self, db: AsyncSession, *, tenant_id: int) -> None:
self.db = db
self.tenant_id = int(tenant_id)
__all__ = ["SemanticFactService"]

View File

@ -0,0 +1,162 @@
from __future__ import annotations
import hashlib
import json
import re
from datetime import UTC, datetime
from typing import Any, Optional
from sqlalchemy import and_, func, or_, select
from sqlalchemy.ext.asyncio import AsyncSession
from ...config.ai_prompts import (
PROMPT_SEMANTIC_FACT_EXTRACT_DOCUMENT,
PROMPT_SEMANTIC_FACT_EXTRACT_PROJECT,
PROMPT_SEMANTIC_FACT_EXTRACT_RELEASE,
PROMPT_SEMANTIC_FACT_EXTRACT_REQUIREMENT,
PROMPT_SEMANTIC_FACT_EXTRACT_REVIEW,
PROMPT_SEMANTIC_FACT_EXTRACT_SIGNAL,
PROMPT_SEMANTIC_FACT_EXTRACT_TASK,
PROMPT_SEMANTIC_FACT_EXTRACT_USER,
)
from ...models.ingest_event import IngestEvent
from ...models.knowledge_base_chunk import KnowledgeBaseChunk
from ...models.knowledge_base_doc import KnowledgeBaseDoc
from ...models.project import Project
from ...models.semantic_fact import SemanticFact
from ...models.semantic_fact_extraction_run import SemanticFactExtractionRun
from ...models.signal import Signal
from ...models.task import Task
from ...models.truth_node import TruthNode
from ...models.user import User
SEMANTIC_FACT_EXTRACT_VERSION = "semantic-facts-v1"
SEMANTIC_FACT_EXTRACT_CHANNEL = "semantic_fact_extract"
SEMANTIC_FACT_SUPPORTED_ENTITY_TYPES = {
"project",
"task",
"signal",
"document",
"user",
"requirement",
"release",
"review",
}
SEMANTIC_FACT_INACTIVE_STATUSES = {
"archived",
"closed",
"deleted",
"disabled",
"inactive",
"offline",
"已关闭",
"已归档",
}
TRUTH_NODE_TYPE_TO_ENTITY_TYPE = {
"charter": "requirement",
"release": "release",
"review": "review",
}
ENTITY_TYPE_TO_TRUTH_NODE_TYPE = {
value: key for key, value in TRUTH_NODE_TYPE_TO_ENTITY_TYPE.items()
}
SEMANTIC_FACT_PROMPTS = {
"project": PROMPT_SEMANTIC_FACT_EXTRACT_PROJECT,
"task": PROMPT_SEMANTIC_FACT_EXTRACT_TASK,
"signal": PROMPT_SEMANTIC_FACT_EXTRACT_SIGNAL,
"document": PROMPT_SEMANTIC_FACT_EXTRACT_DOCUMENT,
"user": PROMPT_SEMANTIC_FACT_EXTRACT_USER,
"requirement": PROMPT_SEMANTIC_FACT_EXTRACT_REQUIREMENT,
"release": PROMPT_SEMANTIC_FACT_EXTRACT_RELEASE,
"review": PROMPT_SEMANTIC_FACT_EXTRACT_REVIEW,
}
SEMANTIC_FACT_FUNCTION_NAMES = {
entity_type: f"semantic_fact_extract_{entity_type}"
for entity_type in SEMANTIC_FACT_SUPPORTED_ENTITY_TYPES
}
def _utc_now_iso() -> str:
return datetime.now(UTC).replace(microsecond=0).isoformat()
def _normalize_fact_value(value: Any) -> str:
text = str(value or "").strip().lower()
if not text:
return ""
text = text.replace("a股", "a股")
text = re.sub(r"\s+", "_", text)
text = re.sub(r"[^a-z0-9\u4e00-\u9fff_]+", "_", text)
text = re.sub(r"_+", "_", text).strip("_")
return text
def _normalized_entity_type(entity_type: Any) -> str:
value = str(entity_type or "").strip().lower()
if value == "doc":
return "document"
return value
def _normalized_status(value: Any) -> str:
return str(value or "").strip().lower()
def _status_is_inactive(value: Any) -> bool:
return _normalized_status(value) in SEMANTIC_FACT_INACTIVE_STATUSES
def _parse_iso_datetime(value: Any) -> Optional[datetime]:
text = str(value or "").strip()
if not text:
return None
try:
parsed = datetime.fromisoformat(text.replace("Z", "+00:00"))
except ValueError:
return None
if parsed.tzinfo is None:
return parsed.replace(tzinfo=UTC)
return parsed.astimezone(UTC)
__all__ = [
"AsyncSession",
"ENTITY_TYPE_TO_TRUTH_NODE_TYPE",
"IngestEvent",
"KnowledgeBaseChunk",
"KnowledgeBaseDoc",
"Project",
"PROMPT_SEMANTIC_FACT_EXTRACT_DOCUMENT",
"PROMPT_SEMANTIC_FACT_EXTRACT_PROJECT",
"PROMPT_SEMANTIC_FACT_EXTRACT_RELEASE",
"PROMPT_SEMANTIC_FACT_EXTRACT_REQUIREMENT",
"PROMPT_SEMANTIC_FACT_EXTRACT_REVIEW",
"PROMPT_SEMANTIC_FACT_EXTRACT_SIGNAL",
"PROMPT_SEMANTIC_FACT_EXTRACT_TASK",
"PROMPT_SEMANTIC_FACT_EXTRACT_USER",
"SEMANTIC_FACT_EXTRACT_VERSION",
"SEMANTIC_FACT_EXTRACT_CHANNEL",
"SEMANTIC_FACT_FUNCTION_NAMES",
"SEMANTIC_FACT_INACTIVE_STATUSES",
"SEMANTIC_FACT_PROMPTS",
"SEMANTIC_FACT_SUPPORTED_ENTITY_TYPES",
"SemanticFact",
"SemanticFactExtractionRun",
"Signal",
"Task",
"TRUTH_NODE_TYPE_TO_ENTITY_TYPE",
"TruthNode",
"User",
"_normalize_fact_value",
"_normalized_entity_type",
"_normalized_status",
"_parse_iso_datetime",
"_status_is_inactive",
"_utc_now_iso",
"and_",
"func",
"hashlib",
"json",
"or_",
"select",
]

File diff suppressed because it is too large Load Diff

View File

@ -1522,7 +1522,7 @@ async def test_terminate_requirement_plan_scope_cancels_orphan_pending_work_item
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_process_ingress_terminates_stale_requirement_scope_before_active_context_followup(monkeypatch) -> None: async def test_process_ingress_does_not_terminate_requirement_scope_before_active_context_followup(monkeypatch) -> None:
cleanup_calls: list[dict[str, object]] = [] cleanup_calls: list[dict[str, object]] = []
class _FakeDB: class _FakeDB:
@ -1563,8 +1563,8 @@ async def test_process_ingress_terminates_stale_requirement_scope_before_active_
self.active_context_called = True self.active_context_called = True
_ = kwargs _ = kwargs
assert self.prepared_session is not None assert self.prepared_session is not None
assert str(getattr(self.prepared_session, "last_intent", "") or "") == "query.answer" assert str(getattr(self.prepared_session, "last_intent", "") or "") == "requirement_plan"
assert getattr(self.prepared_session, "current_project_id", None) is None assert getattr(self.prepared_session, "current_project_id", None) == 99
return { return {
"handled": True, "handled": True,
"status": "answered", "status": "answered",
@ -1575,11 +1575,11 @@ async def test_process_ingress_terminates_stale_requirement_scope_before_active_
async def _maybe_handle_draft_confirmation_ingress(self, **kwargs): async def _maybe_handle_draft_confirmation_ingress(self, **kwargs):
_ = kwargs _ = kwargs
raise AssertionError("stale scope cleanup should happen before draft confirmation routing") raise AssertionError("active context followup should short-circuit before draft confirmation routing")
async def _prepare_mixed_sidecar_context(self, **kwargs): async def _prepare_mixed_sidecar_context(self, **kwargs):
_ = kwargs _ = kwargs
raise AssertionError("stale scope cleanup should short-circuit before query sidecar") raise AssertionError("active context followup should short-circuit before query sidecar")
async def _fake_terminate_requirement_plan_scope(self, **kwargs): async def _fake_terminate_requirement_plan_scope(self, **kwargs):
cleanup_calls.append(kwargs) cleanup_calls.append(kwargs)
@ -1603,9 +1603,7 @@ async def test_process_ingress_terminates_stale_requirement_scope_before_active_
ingress_options={"thread_context": {"reply_to_message_id": ""}}, ingress_options={"thread_context": {"reply_to_message_id": ""}},
) )
assert cleanup_calls assert cleanup_calls == []
assert cleanup_calls[0]["source_message_id"] == "s-msg-terminate-1"
assert cleanup_calls[0]["source_system"] == "feishu"
assert kernel.active_context_called is True assert kernel.active_context_called is True
assert result["direct_reply_text"] == "普通问答结果" assert result["direct_reply_text"] == "普通问答结果"
@ -2538,6 +2536,8 @@ async def test_active_requirement_followup_prefers_current_requirement_context(m
assert result is not None assert result is not None
assert result["handled"] is True assert result["handled"] is True
assert "信号输入" in result["direct_reply_text"] assert "信号输入" in result["direct_reply_text"]
assert result["reply_template_key"] == "requirement_plan_followup"
assert result["work_item_id"] == 81
assert captured["packet_kwargs"]["latest_user_message"] == "信号源呢?" assert captured["packet_kwargs"]["latest_user_message"] == "信号源呢?"
assert captured["packet_kwargs"]["ingress_context"]["candidate_kind"] == "active_context_followup" assert captured["packet_kwargs"]["ingress_context"]["candidate_kind"] == "active_context_followup"
assert captured["direct_kwargs"]["message_text"] == "信号源呢?" assert captured["direct_kwargs"]["message_text"] == "信号源呢?"
@ -2619,6 +2619,71 @@ async def test_active_requirement_choice_protocol_followup_bypasses_semantic_dec
assert captured["direct_kwargs"]["message_text"] == "我选C 继续" assert captured["direct_kwargs"]["message_text"] == "我选C 继续"
@pytest.mark.asyncio
async def test_active_requirement_exit_request_suspends_bound_requirement_context_before_semantic_decision(
monkeypatch,
) -> None:
flush_mock = AsyncMock()
topic = SimpleNamespace(
id=181,
status="active",
plan_status="collecting",
pending_actions=[],
metadata_json={"plan_mode": True},
)
draft = SimpleNamespace(id=182, state="collecting")
async def _executor_find_active_requirement_plan_workspace(**kwargs):
_ = kwargs
return topic, draft
class FakeKernel(mixin_entry.KernelIngressFlowEntryMixin):
def __init__(self) -> None:
self.db = SimpleNamespace(flush=flush_mock)
self.tenant_id = 1
self.executor = SimpleNamespace(
_find_active_requirement_plan_workspace=_executor_find_active_requirement_plan_workspace,
)
class BombSemanticContextAssembler:
def __init__(self, db) -> None:
self.db = db
async def assemble(self, **kwargs):
_ = kwargs
raise AssertionError("explicit exit should bypass semantic decision assembly")
class BombSemanticDecisionService:
def __init__(self, _llm_service) -> None:
pass
async def decide(self, *, packet):
_ = packet
raise AssertionError("explicit exit should bypass semantic decision")
monkeypatch.setattr(mixin_entry, "SemanticContextAssembler", BombSemanticContextAssembler)
monkeypatch.setattr(mixin_entry, "SemanticDecisionService", BombSemanticDecisionService)
kernel = FakeKernel()
result = await kernel._maybe_handle_active_context_followup(
source_system="feishu",
source_message_id="msg-plan-exit-1",
channel_ref="chat-active",
source_sender_open_id="ou_active",
message_text="退出需求模式",
thread_context=None,
sender="tester",
)
assert result is not None
assert result["handled"] is True
assert "先暂停" in result["direct_reply_text"]
assert result["reply_template_key"] == "requirement_mode_suspended"
assert result["workflow_route"] == mixin_entry.ROUTE_REQUIREMENT_PLAN_COLLECT
assert result["topic_workspace_id"] == 181
flush_mock.assert_awaited_once()
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_active_requirement_followup_uses_builder_result_to_persist_requirement_context(monkeypatch) -> None: async def test_active_requirement_followup_uses_builder_result_to_persist_requirement_context(monkeypatch) -> None:
captured: dict[str, object] = {} captured: dict[str, object] = {}
@ -3088,6 +3153,7 @@ async def test_find_active_requirement_plan_workspace_includes_draft_pending_con
assert found_topic is not None assert found_topic is not None
assert found_draft is not None assert found_draft is not None
assert int(found_topic.id) == int(topic.id) assert int(found_topic.id) == int(topic.id)
assert str(found_topic.plan_status) == "draft_pending_confirm"
assert str(found_draft.state) == "draft_pending_confirm" assert str(found_draft.state) == "draft_pending_confirm"
@ -3151,8 +3217,76 @@ async def test_find_active_requirement_plan_workspace_recovers_plan_topic_marked
assert found_topic is not None assert found_topic is not None
assert found_draft is not None assert found_draft is not None
assert int(found_topic.id) == int(topic.id) assert int(found_topic.id) == int(topic.id)
assert str(found_topic.plan_status) == "options_ready"
assert str(found_draft.state) == "collecting"
@pytest.mark.asyncio
async def test_find_active_requirement_plan_workspace_prefers_live_draft_topic_over_newer_idle_scope_match(
requirement_plan_db_session,
) -> None:
class FakeExecutor(mixin_execute_query_project.ExecuteQueryProjectMixin):
def __init__(self) -> None:
self.db = requirement_plan_db_session
self.tenant_id = 7
executor = FakeExecutor()
conversation_scope_key = build_feishu_conversation_scope_key(
source_chat_id="chat-plan-prefer-live-draft",
source_sender_open_id="ou-plan-prefer-live-draft",
sender=None,
)
live_topic = MountedTopicWorkspace(
tenant_id=7,
topic_key="feishu:chat-plan-prefer-live-draft:message:msg-plan-root",
source_system="feishu",
channel_ref="chat-plan-prefer-live-draft",
conversation_scope_key=conversation_scope_key,
mode="plan",
status="active",
plan_status="collecting",
metadata_json={"plan_mode": True},
)
requirement_plan_db_session.add(live_topic)
await requirement_plan_db_session.flush()
live_draft = MountedDraftWorkspace(
tenant_id=7,
topic_workspace_id=int(live_topic.id),
state="collecting",
draft_version=0,
requirement_card={"title": "采购库存报表助手"},
draft_payload=None,
required_confirmations=[],
)
requirement_plan_db_session.add(live_draft)
await requirement_plan_db_session.flush()
idle_query_topic = MountedTopicWorkspace(
tenant_id=7,
topic_key="feishu:chat-plan-prefer-live-draft:message:msg-later-query",
source_system="feishu",
channel_ref="chat-plan-prefer-live-draft",
conversation_scope_key=conversation_scope_key,
mode="chat",
status="active",
plan_status="idle",
metadata_json={"plan_mode": True},
)
requirement_plan_db_session.add(idle_query_topic)
await requirement_plan_db_session.flush()
found_topic, found_draft = await executor._find_active_requirement_plan_workspace(
source_system="feishu",
source_message_id="msg-followup-exit",
source_chat_id="chat-plan-prefer-live-draft",
sender=None,
source_sender_open_id="ou-plan-prefer-live-draft",
thread_context=None,
conversation_key=conversation_scope_key,
)
assert found_topic is not None
assert found_draft is not None
assert int(found_topic.id) == int(live_topic.id)
assert int(found_draft.id) == int(live_draft.id)
def test_world_model_session_project_inheritance_keeps_meaningful_followup() -> None: def test_world_model_session_project_inheritance_keeps_meaningful_followup() -> None:
@ -3382,6 +3516,19 @@ def test_mounted_runtime_query_requirement_consultation_structured_analysis_uses
assert mode == "plan" assert mode == "plan"
def test_mounted_runtime_query_consult_requirement_requirements_analysis_uses_plan_mode() -> None:
mode = mixin_entry.KernelIngressFlowEntryMixin._resolve_mounted_runtime_mode_override(
workflow_route="query.answer",
route_metadata={
"message_intent": "consult_requirement",
"consultant_mode_preference": "requirements_analysis",
},
reply_template_key=None,
)
assert mode == "plan"
def test_mounted_runtime_query_requirement_gathering_stepwise_converge_uses_plan_mode() -> None: def test_mounted_runtime_query_requirement_gathering_stepwise_converge_uses_plan_mode() -> None:
mode = mixin_entry.KernelIngressFlowEntryMixin._resolve_mounted_runtime_mode_override( mode = mixin_entry.KernelIngressFlowEntryMixin._resolve_mounted_runtime_mode_override(
workflow_route="query.answer", workflow_route="query.answer",
@ -3972,11 +4119,11 @@ async def test_kernel_intent_resolve_degrades_when_semantic_planner_fails(
assert source == "semantic_planner_degraded" assert source == "semantic_planner_degraded"
assert intent.name == "query.read_only" assert intent.name == "query.read_only"
assert intent.metadata["workflow_route"] == "query.answer" assert intent.metadata["workflow_route"] == "query.answer"
assert intent.metadata["message_intent"] == "consult_requirement"
assert intent.metadata["consultant_mode_preference"] == "clarify"
assert intent.metadata["context_source"] == "semantic_planner_degraded" assert intent.metadata["context_source"] == "semantic_planner_degraded"
assert intent.metadata["semantic_planner_degraded"] is True assert intent.metadata["semantic_planner_degraded"] is True
assert "upstream_empty_or_invalid_response" in str(intent.metadata["semantic_planner_error"]) assert "upstream_empty_or_invalid_response" in str(intent.metadata["semantic_planner_error"])
assert "message_intent" not in intent.metadata
assert "consultant_mode_preference" not in intent.metadata
assert intent.signal_title == "" assert intent.signal_title == ""
@ -4259,6 +4406,39 @@ def test_reply_renderer_formats_project_delete_pending_and_success() -> None:
assert "已删除项目#12《重复项目》" in success["text"] assert "已删除项目#12《重复项目》" in success["text"]
def test_reply_renderer_preserves_requirement_plan_followup_template_for_query_intent() -> None:
renderer = ReplyRenderer(
state_waiting_manual="waiting_manual",
state_failed="failed",
receipt_phase_action="action",
receipt_phase_pending="pending",
receipt_phase_error="error",
)
intent = CanonicalIntent(
name="query.read_only",
confidence=0.97,
project_id=None,
signal_title="",
signal_summary="",
metadata={},
)
rendered = renderer.render(
work_state="succeeded",
intent=intent,
decision_record={"approval_requirement": {}},
execution_result={
"status": "succeeded",
"workflow_route": "requirement_plan.collect",
"answer": "A. 轻量工具\nB. 协同平台\nC. 能力底座\n\n⭐ 当前我更推荐:A",
"reply_template_key": "requirement_plan_followup",
},
)
assert rendered["template_key"] == "requirement_plan_followup"
assert rendered["text"].startswith("A. 轻量工具")
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_feishu_outbox_service_enqueue_preserves_requirement_plan_text_above_2000_chars( async def test_feishu_outbox_service_enqueue_preserves_requirement_plan_text_above_2000_chars(
feishu_outbox_db_session, feishu_outbox_db_session,
@ -4823,6 +5003,83 @@ async def test_execute_query_intent_short_circuits_casual_chat_without_rag(monke
assert str(result["rag_result"]["query_type"]) == "chat" assert str(result["rag_result"]["query_type"]) == "chat"
@pytest.mark.asyncio
async def test_execute_query_intent_external_source_policy_bypasses_casual_chat_shortcut(monkeypatch) -> None:
captured: dict[str, object] = {}
class FakeAdaptiveRAGService:
def __init__(self, db, tenant_id: int):
_ = (db, tenant_id)
async def preview_query_route_decision(self, **kwargs):
captured["route_preview_kwargs"] = dict(kwargs)
return {
"query_semantics": {
"source_policy": "uncertain",
"effective_source_policy": "external",
"query_type": "status",
"retrieval_domains": ["url"],
"query_profile": {
"needs_private_context": False,
"needs_freshness": True,
"referenced_objects": ["weather"],
},
},
"route_decision": {
"route_mode": "external",
"effective_source_policy": "external",
"decision_origin": "external_probe",
"reply_contract": {"mode": "explicit_source", "head": "外部实时查询:", "tail_hint": None},
"probe_snapshot_summary": {
"internal": {"config_hit_count": 0},
"external": {"trusted_domain_hit_count": 1, "answerable_page_hit_count": 1},
},
},
}
async def answer_question(self, **kwargs):
captured["answer_kwargs"] = dict(kwargs)
return {
"answer": "广州荔湾区今天多云,午后有阵雨。",
"sources": [{"source": "url", "label": "Weather", "score": 0.93, "metadata": {"url": "https://weather.example"}}],
"query_type": "status",
}
class FakeExecutor(mixin_execute_query_project.ExecuteQueryProjectMixin):
def __init__(self) -> None:
self.db = object()
self.tenant_id = 3
monkeypatch.setattr(mixin_execute_query_project, "AdaptiveRAGService", FakeAdaptiveRAGService)
executor = FakeExecutor()
intent = SimpleNamespace(
project_id=None,
metadata={
"semantic_bundle": {
"diagnostics": {
"message_intent": "casual_chat",
"consultant_mode_preference": "brainstorm",
}
}
},
)
result = await executor._execute_query_intent(
intent=intent,
message_text="今天荔湾区天气如何?",
operator_user_id=999,
source_system="feishu",
source_message_id="msg-chat-weather-1",
)
assert result["status"] == "succeeded"
assert "广州荔湾区今天多云" in str(result["answer"] or "")
answer_kwargs = captured["answer_kwargs"]
assert isinstance(answer_kwargs, dict)
assert answer_kwargs["source_policy"] == "uncertain"
assert answer_kwargs["route_decision"]["effective_source_policy"] == "external"
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_execute_query_intent_backfills_semantic_bundle_for_casual_chat_without_legacy_helpers( async def test_execute_query_intent_backfills_semantic_bundle_for_casual_chat_without_legacy_helpers(
monkeypatch, monkeypatch,
@ -6799,6 +7056,101 @@ async def test_execute_query_intent_locked_requirement_mode_redirects_casual_cha
assert str(result["rag_result"]["query_type"]) == "plan" assert str(result["rag_result"]["query_type"]) == "plan"
@pytest.mark.asyncio
async def test_execute_query_intent_locked_requirement_mode_redirects_generic_query_to_requirement_notice(
monkeypatch,
requirement_plan_db_session,
) -> None:
class FakeAdaptiveRAGService:
def __init__(self, db, tenant_id: int):
_ = (db, tenant_id)
async def answer_question(self, **kwargs):
raise AssertionError(f"locked requirement mode should not fall through to generic RAG: {kwargs}")
class FakeExecutor(mixin_execute_query_project.ExecuteQueryProjectMixin):
def __init__(self) -> None:
self.db = requirement_plan_db_session
self.tenant_id = 3
_install_requirement_plan_query_bundle(
monkeypatch,
message_intent="query",
consultant_mode_preference="",
)
monkeypatch.setattr(mixin_execute_query_project, "AdaptiveRAGService", FakeAdaptiveRAGService)
conversation_scope_key = build_feishu_conversation_scope_key(
source_chat_id="chat-lock-plan-query",
source_sender_open_id="ou-lock-plan-query",
sender=None,
)
topic = MountedTopicWorkspace(
tenant_id=3,
topic_key="feishu:chat-lock-plan-query:root:om-lock-plan-query-root",
source_system="feishu",
channel_ref="chat-lock-plan-query",
conversation_scope_key=conversation_scope_key,
mode="plan",
status="active",
plan_status="collecting",
metadata_json={"plan_mode": True},
root_message_id="om-lock-plan-query-root",
)
requirement_plan_db_session.add(topic)
await requirement_plan_db_session.flush()
requirement_plan_db_session.add(
MountedDraftWorkspace(
tenant_id=3,
topic_workspace_id=int(topic.id),
state="collecting",
draft_version=0,
requirement_card={
"plan_profile": "it_project",
"title": "采购库存报表助手",
"goal": "先把采购库存报表助手的首版需求收清楚。",
"target_user": "采购经理",
"question_round": 2,
"last_round_options": {
"field": "target_user",
"options": [
{"key": "A", "title": "只服务采购经理本人"},
{"key": "B", "title": "采购经理+仓管员"},
{"key": "C", "title": "采购经理+财务复核人"},
],
"recommendation": {"key": "A"},
},
},
required_confirmations=[],
)
)
await requirement_plan_db_session.flush()
executor = FakeExecutor()
intent = SimpleNamespace(
project_id=None,
metadata={
"source_chat_id": "chat-lock-plan-query",
"source_sender_open_id": "ou-lock-plan-query",
"message_intent": "query",
},
)
result = await executor._execute_query_intent(
intent=intent,
message_text="系统里面有哪些项目?",
operator_user_id=999,
source_system="feishu",
source_message_id="msg-lock-plan-query-1",
)
assert result["status"] == "succeeded"
assert "当前正在需求模式" in str(result["answer"] or "")
assert "退出需求模式" in str(result["answer"] or "")
assert str(result["workflow_route"]) == "requirement_plan.collect"
assert str(result["rag_result"]["query_type"]) == "plan"
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_execute_query_intent_converts_query_answer_requirement_contract_into_requirement_plan_result( async def test_execute_query_intent_converts_query_answer_requirement_contract_into_requirement_plan_result(
monkeypatch, monkeypatch,
@ -7419,6 +7771,270 @@ async def test_execute_query_intent_treats_requirement_consultation_structured_a
assert "Feature Catalog" not in str(result["answer"] or "") assert "Feature Catalog" not in str(result["answer"] or "")
@pytest.mark.asyncio
async def test_execute_query_intent_treats_consult_requirement_requirements_analysis_as_requirement_plan(
monkeypatch,
requirement_plan_db_session,
) -> None:
captured: dict[str, object] = {}
class FakeAdaptiveRAGService:
def __init__(self, db, tenant_id: int):
_ = (db, tenant_id)
async def answer_question(self, **kwargs):
raise AssertionError(f"requirements analysis should not enter heavy RAG: {kwargs}")
class FakeExecutor(mixin_execute_query_project.ExecuteQueryProjectMixin):
def __init__(self) -> None:
self.db = requirement_plan_db_session
self.tenant_id = 3
async def _fake_build_requirement_planning_direct_result(self, **kwargs): # noqa: ANN001
_ = self
captured["direct_kwargs"] = dict(kwargs)
return {
"status": "succeeded",
"answer": "可以,我们先不创建,先把这件事收成一版需求。",
"rag_result": {
"answer": "可以,我们先不创建,先把这件事收成一版需求。",
"query_type": "plan",
},
"reply_template_key": "requirement_plan_followup",
}
monkeypatch.setattr(
mixin_execute_query_project.ExecuteQueryProjectMixin,
"_build_requirement_planning_direct_result",
_fake_build_requirement_planning_direct_result,
raising=False,
)
monkeypatch.setattr(mixin_execute_query_project, "AdaptiveRAGService", FakeAdaptiveRAGService)
executor = FakeExecutor()
intent = SimpleNamespace(
project_id=None,
metadata={
"source_chat_id": "chat-consult-requirements-analysis",
"source_sender_open_id": "ou-consult-requirements-analysis",
"message_intent": "consult_requirement",
"consultant_mode_preference": "requirements_analysis",
"topic_switch": "new",
},
)
result = await executor._execute_query_intent(
intent=intent,
message_text="想跟你聊个需求,关于给采购经理提供可清理库存的报表。",
operator_user_id=999,
source_system="feishu",
source_message_id="msg-consult-requirements-analysis-1",
)
assert result["status"] == "succeeded"
assert str(result["rag_result"]["query_type"]) == "plan"
direct_kwargs = captured.get("direct_kwargs")
assert isinstance(direct_kwargs, dict)
assert str(direct_kwargs["message_text"]) == "想跟你聊个需求,关于给采购经理提供可清理库存的报表。"
assert str(direct_kwargs["source_chat_id"]) == "chat-consult-requirements-analysis"
assert "Feature Catalog" not in str(result["answer"] or "")
@pytest.mark.asyncio
async def test_execute_query_intent_semantic_planner_degraded_factual_query_falls_back_to_regular_qa(
monkeypatch,
requirement_plan_db_session,
) -> None:
captured: dict[str, object] = {}
class FakeAdaptiveRAGService:
def __init__(self, db, tenant_id: int):
_ = (db, tenant_id)
async def answer_question(self, **kwargs):
captured["rag_kwargs"] = dict(kwargs)
return {
"answer": "根据公开页面,目前走普通事实问答链。",
"query_type": "explain",
"sources": [{"source": "web", "label": "External", "score": 0.9}],
}
class FakeExecutor(mixin_execute_query_project.ExecuteQueryProjectMixin):
def __init__(self) -> None:
self.db = requirement_plan_db_session
self.tenant_id = 3
async def _failing_aplan_message(*, message_text: str, source_system: str, metadata=None):
_ = (message_text, source_system, metadata)
raise RuntimeError("upstream_empty_or_invalid_response")
async def _unexpected_requirement_direct_result(self, **kwargs): # noqa: ANN001
raise AssertionError(f"degraded factual query should not enter requirement plan direct path: {kwargs}")
monkeypatch.setattr(
WorkflowSemanticPlanner,
"aplan_message",
classmethod(lambda cls, **kwargs: _failing_aplan_message(**kwargs)),
)
monkeypatch.setattr(
mixin_execute_query_project.ExecuteQueryProjectMixin,
"_build_requirement_planning_direct_result",
_unexpected_requirement_direct_result,
raising=False,
)
monkeypatch.setattr(mixin_execute_query_project, "AdaptiveRAGService", FakeAdaptiveRAGService)
executor = FakeExecutor()
intent = SimpleNamespace(
project_id=None,
metadata={
"source_chat_id": "chat-degraded-factual",
"source_sender_open_id": "ou-degraded-factual",
"semantic_planner_degraded": True,
},
)
result = await executor._execute_query_intent(
intent=intent,
message_text="当前claude最新的模型用的是什么?",
operator_user_id=999,
source_system="feishu",
source_message_id="msg-degraded-factual-1",
)
assert result["status"] == "succeeded"
assert str(result["rag_result"]["query_type"]) == "explain"
assert "收成一版需求" not in str(result["answer"] or "")
assert "requirement_plan_context" not in result
assert isinstance(captured.get("rag_kwargs"), dict)
@pytest.mark.asyncio
async def test_execute_query_intent_semantic_planner_degraded_active_requirement_scope_stays_in_plan(
monkeypatch,
requirement_plan_db_session,
) -> None:
captured: dict[str, object] = {}
class FakeAdaptiveRAGService:
def __init__(self, db, tenant_id: int):
_ = (db, tenant_id)
async def answer_question(self, **kwargs):
raise AssertionError(f"active requirement scope should not enter heavy RAG: {kwargs}")
class FakeExecutor(mixin_execute_query_project.ExecuteQueryProjectMixin):
def __init__(self) -> None:
self.db = requirement_plan_db_session
self.tenant_id = 3
async def _failing_aplan_message(*, message_text: str, source_system: str, metadata=None):
_ = (message_text, source_system, metadata)
raise RuntimeError("upstream_empty_or_invalid_response")
async def _fake_build_requirement_planning_direct_result(self, **kwargs): # noqa: ANN001
_ = self
captured["direct_kwargs"] = dict(kwargs)
return {
"status": "succeeded",
"answer": "第二轮:继续收敛来源范围。",
"rag_result": {
"answer": "第二轮:继续收敛来源范围。",
"query_type": "plan",
},
"reply_template_key": "requirement_plan_followup",
}
monkeypatch.setattr(
WorkflowSemanticPlanner,
"aplan_message",
classmethod(lambda cls, **kwargs: _failing_aplan_message(**kwargs)),
)
monkeypatch.setattr(
mixin_execute_query_project.ExecuteQueryProjectMixin,
"_build_requirement_planning_direct_result",
_fake_build_requirement_planning_direct_result,
raising=False,
)
monkeypatch.setattr(mixin_execute_query_project, "AdaptiveRAGService", FakeAdaptiveRAGService)
conversation_scope_key = build_feishu_conversation_scope_key(
source_chat_id="chat-degraded-plan",
source_sender_open_id="ou-degraded-plan",
sender=None,
)
topic = MountedTopicWorkspace(
tenant_id=3,
topic_key="feishu:chat-degraded-plan:root:om-degraded-plan-root",
source_system="feishu",
channel_ref="chat-degraded-plan",
conversation_scope_key=conversation_scope_key,
mode="plan",
status="active",
plan_status="collecting",
metadata_json={"plan_mode": True},
facts_json={"title": "Claude 模型信息卡片"},
)
requirement_plan_db_session.add(topic)
await requirement_plan_db_session.flush()
requirement_plan_db_session.add(
MountedDraftWorkspace(
tenant_id=3,
topic_workspace_id=int(topic.id),
state="collecting",
draft_version=0,
requirement_card={
"plan_profile": "it_project",
"title": "Claude 模型信息卡片",
"goal": "先把模型信息卡片需求收清楚。",
"project_nature": "轻量即用型工具",
"question_round": 1,
"last_round_options": {
"field": "project_nature",
"options": [
{"key": "A", "title": "轻量即用型工具"},
{"key": "B", "title": "业务协同平台"},
{"key": "C", "title": "AI能力底座模块"},
],
"recommendation": {"key": "A"},
},
"round_option_maps": {
"project_nature": {
"A": "轻量即用型工具",
"B": "业务协同平台",
"C": "AI能力底座模块",
}
},
},
required_confirmations=[],
)
)
await requirement_plan_db_session.flush()
executor = FakeExecutor()
intent = SimpleNamespace(
project_id=None,
metadata={
"source_chat_id": "chat-degraded-plan",
"source_sender_open_id": "ou-degraded-plan",
"semantic_planner_degraded": True,
},
)
result = await executor._execute_query_intent(
intent=intent,
message_text="A",
operator_user_id=999,
source_system="feishu",
source_message_id="msg-degraded-plan-1",
)
assert result["status"] == "succeeded"
assert str(result["workflow_route"]) == "requirement_plan.collect"
assert str(result["rag_result"]["query_type"]) == "plan"
assert isinstance(captured.get("direct_kwargs"), dict)
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_execute_query_intent_promotes_requirement_followup_from_structured_round_context_without_render_markers( async def test_execute_query_intent_promotes_requirement_followup_from_structured_round_context_without_render_markers(
monkeypatch, monkeypatch,
@ -11068,6 +11684,7 @@ async def test_should_route_to_draft_confirmation_session_does_not_route_free_te
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_external_freshness_followup_stays_in_query_answer_without_internal_target( async def test_external_freshness_followup_stays_in_query_answer_without_internal_target(
external_qa_db_session, external_qa_db_session,
monkeypatch,
) -> None: ) -> None:
conversation_scope_key = build_feishu_conversation_scope_key( conversation_scope_key = build_feishu_conversation_scope_key(
source_chat_id="chat-external-qa", source_chat_id="chat-external-qa",
@ -11113,13 +11730,41 @@ async def test_external_freshness_followup_stays_in_query_answer_without_interna
self.db = external_qa_db_session self.db = external_qa_db_session
self.tenant_id = 1 self.tenant_id = 1
async def _fake_aplan_message(cls, *, message_text: str, source_system: str, metadata=None):
_ = (cls, message_text, source_system, metadata)
return WorkflowSemanticPlanner.deserialize_bundle(
{
"bundle_id": "freshness-followup-query",
"semantic_version": "v1",
"source_system": "feishu",
"primary_workflow": "query_flow",
"workflows": [
{
"workflow_type": "query_flow",
"workflow_action": "answer",
"target_scope": "global",
"operation_codes": ["query.answer"],
"requires_confirmation": False,
"confidence": 0.91,
"clarification_needed": False,
"metadata": {},
}
],
"message_gate": "auto",
"reply_contract": {"mode": "text", "template_key": "workflow_bundle", "summary": "query_flow"},
"diagnostics": {"message_intent": "question"},
}
)
monkeypatch.setattr(WorkflowSemanticPlanner, "aplan_message", classmethod(_fake_aplan_message))
kernel = FakeKernel() kernel = FakeKernel()
result = await kernel._maybe_handle_external_freshness_followup( result = await kernel._maybe_handle_external_freshness_followup(
source_system="feishu", source_system="feishu",
source_message_id="om-external-followup-1", source_message_id="om-external-followup-1",
channel_ref="chat-external-qa", channel_ref="chat-external-qa",
source_sender_open_id="ou-external-qa", source_sender_open_id="ou-external-qa",
message_text="接入外部来源继续核实", message_text="重新联网核实一下",
thread_context={"reply_to_message_id": "om-external-question-1"}, thread_context={"reply_to_message_id": "om-external-question-1"},
) )
@ -11134,6 +11779,7 @@ async def test_external_freshness_followup_stays_in_query_answer_without_interna
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_external_freshness_followup_allows_explicit_internal_source_request( async def test_external_freshness_followup_allows_explicit_internal_source_request(
external_qa_db_session, external_qa_db_session,
monkeypatch,
) -> None: ) -> None:
conversation_scope_key = build_feishu_conversation_scope_key( conversation_scope_key = build_feishu_conversation_scope_key(
source_chat_id="chat-external-qa", source_chat_id="chat-external-qa",
@ -11179,6 +11825,34 @@ async def test_external_freshness_followup_allows_explicit_internal_source_reque
self.db = external_qa_db_session self.db = external_qa_db_session
self.tenant_id = 1 self.tenant_id = 1
async def _fake_aplan_message(cls, *, message_text: str, source_system: str, metadata=None):
_ = (cls, message_text, source_system, metadata)
return WorkflowSemanticPlanner.deserialize_bundle(
{
"bundle_id": "freshness-followup-signal-source",
"semantic_version": "v1",
"source_system": "feishu",
"primary_workflow": "signal_source_flow",
"workflows": [
{
"workflow_type": "signal_source_flow",
"workflow_action": "onboard",
"target_scope": "global",
"operation_codes": ["signal_source.onboard"],
"requires_confirmation": False,
"confidence": 0.95,
"clarification_needed": False,
"metadata": {},
}
],
"message_gate": "auto",
"reply_contract": {"mode": "text", "template_key": "workflow_bundle", "summary": "signal_source_flow"},
"diagnostics": {"message_intent": "question"},
}
)
monkeypatch.setattr(WorkflowSemanticPlanner, "aplan_message", classmethod(_fake_aplan_message))
kernel = FakeKernel() kernel = FakeKernel()
result = await kernel._maybe_handle_external_freshness_followup( result = await kernel._maybe_handle_external_freshness_followup(
source_system="feishu", source_system="feishu",
@ -11301,13 +11975,40 @@ async def test_process_ingress_short_circuits_external_freshness_followup_before
async def _prepare_mixed_sidecar_context(self, **kwargs): async def _prepare_mixed_sidecar_context(self, **kwargs):
raise AssertionError("external freshness followup should not enter mixed sidecar") raise AssertionError("external freshness followup should not enter mixed sidecar")
async def _fake_aplan_message(cls, *, message_text: str, source_system: str, metadata=None):
_ = (cls, message_text, source_system, metadata)
return WorkflowSemanticPlanner.deserialize_bundle(
{
"bundle_id": "freshness-followup-query-process-ingress",
"semantic_version": "v1",
"source_system": "feishu",
"primary_workflow": "query_flow",
"workflows": [
{
"workflow_type": "query_flow",
"workflow_action": "answer",
"target_scope": "global",
"operation_codes": ["query.answer"],
"requires_confirmation": False,
"confidence": 0.93,
"clarification_needed": False,
"metadata": {},
}
],
"message_gate": "auto",
"reply_contract": {"mode": "text", "template_key": "workflow_bundle", "summary": "query_flow"},
"diagnostics": {"message_intent": "question"},
}
)
monkeypatch.setattr(mixin_entry, "MountedAIRuntimeWriteService", FakeMountedAIRuntimeWriteService) monkeypatch.setattr(mixin_entry, "MountedAIRuntimeWriteService", FakeMountedAIRuntimeWriteService)
monkeypatch.setattr(WorkflowSemanticPlanner, "aplan_message", classmethod(_fake_aplan_message))
kernel = FakeKernel() kernel = FakeKernel()
result = await kernel.process_ingress( result = await kernel.process_ingress(
source_system="feishu", source_system="feishu",
source_message_id="om-external-followup-3", source_message_id="om-external-followup-3",
message_text="接入外部来源继续核实", message_text="继续联网查最新结果",
sender="林宇航", sender="林宇航",
channel_ref="chat-external-qa", channel_ref="chat-external-qa",
source_sender_open_id="ou-external-qa", source_sender_open_id="ou-external-qa",

View File

@ -636,6 +636,192 @@ async def test_openai_compatible_chat_accepts_structured_json_from_reasoning_con
assert json.loads(result.text)["title"] == "AI直播复盘助手" assert json.loads(result.text)["title"] == "AI直播复盘助手"
@pytest.mark.asyncio
async def test_openai_compatible_chat_ignores_invalid_structured_candidate_from_reasoning_content_and_retries(
monkeypatch,
) -> None:
calls: list[dict[str, object]] = []
class FakeResponse:
def __init__(self, payload: dict[str, object]) -> None:
self.status_code = 200
self._payload = payload
def json(self):
return self._payload
valid_payload = {
"topic_relation": "new_topic",
"mode_decision": "answer_only",
"followup_scope": "none",
"followup_action": "none",
"revision_note": "",
"draft_patch": {},
"workflow_plans": [
{
"workflow_type": "query_flow",
"workflow_action": "answer",
"target_scope": "global",
"operation_codes": ["query.answer"],
"requires_confirmation": False,
"confidence": 0.91,
"clarification_needed": False,
"metadata": {},
}
],
"scene_metadata": {},
"assistant_reply": "我会查询 Microsoft Learn 并返回最新官方链接。",
"requires_confirmation": False,
"requested_actions": ["workflow.bundle.compose"],
"confidence": 0.91,
"uncertainties": [],
}
class FakeClient:
def __init__(self, *args, **kwargs):
_ = (args, kwargs)
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc, tb):
return False
async def post(self, *args, **kwargs):
calls.append(dict(kwargs.get("json") or {}))
if len(calls) == 1:
return FakeResponse(
{
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": "",
"reasoning_content": (
"1. 分析输入\n"
"packet.thread.recent_turns: []\n"
"packet.ingress.allowed_context_scopes: []\n"
"接下来应进入 query.answer。"
),
},
"finish_reason": "length",
}
]
}
)
return FakeResponse(
{
"choices": [
{
"index": 0,
"message": {
"role": "assistant",
"content": json.dumps(valid_payload, ensure_ascii=False),
"reasoning_content": "",
},
"finish_reason": "stop",
}
]
}
)
async def fake_sleep(_seconds: float):
return None
monkeypatch.setattr(mixin_chat_openai.httpx, "AsyncClient", FakeClient)
monkeypatch.setattr(mixin_chat_openai.asyncio, "sleep", fake_sleep)
result = await llm_service._chat_completion_via_openai_compatible(
key="test-key",
base_url="https://api.edgefn.net/v1",
model="GLM-5",
temperature=0.0,
max_tokens=512,
messages=[{"role": "user", "content": "帮我判断这句消息应该走哪个 workflow"}],
request_timeout=1.0,
structured_output=True,
function_label="pmflow_semantic_context_decision",
provider_name="openai",
is_control_plane_gateway_request=False,
control_plane_agents_timeout_seconds=120,
request_started_at_perf=time.perf_counter(),
request_trace_id="trace-invalid-reasoning-structured",
control_plane_agent_timeout=None,
routing_meta={"response_contract": "structured"},
)
assert len(calls) == 2
assert json.loads(result.text)["topic_relation"] == "new_topic"
assert "最终 JSON 对象" in str(calls[1]["messages"][-1]["content"])
@pytest.mark.asyncio
async def test_chat_completion_raw_opens_provider_circuit_immediately_on_billing_block(monkeypatch) -> None:
circuit_key = "glm|https://dashscope.aliyuncs.com/compatible-mode/v1|qwen-plus"
original_state = dict(llm_service._provider_circuit_state)
original_signature = llm_service._provider_circuit_config_signature
class FakeResponse:
status_code = 400
headers: dict[str, str] = {}
text = json.dumps(
{
"error": {
"message": "Access denied, please make sure your account is in good standing.",
"type": "Arrearage",
"code": "Arrearage",
}
},
ensure_ascii=False,
)
def json(self):
return json.loads(self.text)
def raise_for_status(self):
request = httpx.Request("POST", "https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions")
response = httpx.Response(self.status_code, request=request, text=self.text)
raise httpx.HTTPStatusError("Client error '400 Bad Request'", request=request, response=response)
class FakeClient:
def __init__(self, *args, **kwargs):
_ = (args, kwargs)
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc, tb):
return False
async def post(self, *args, **kwargs):
_ = (args, kwargs)
return FakeResponse()
monkeypatch.setattr(mixin_chat_openai.httpx, "AsyncClient", FakeClient)
try:
with pytest.raises(ValueError, match="provider_billing_blocked:glm"):
await llm_service._chat_completion_raw(
provider="glm",
prompt="输出 JSON",
system_prompt="你是 JSON 生成器",
api_key="glm-key",
base_url="https://dashscope.aliyuncs.com/compatible-mode/v1",
model="qwen-plus",
temperature=0.0,
max_tokens=64,
function_name="pmflow_semantic_context_decision",
response_mode="fast",
routing_meta={"response_contract": "structured"},
)
assert llm_service._is_provider_circuit_open(circuit_key) is True
finally:
llm_service._provider_circuit_state.clear()
llm_service._provider_circuit_state.update(original_state)
llm_service._provider_circuit_config_signature = original_signature
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_openai_compatible_chat_retries_empty_reasoning_response_with_bigger_token_budget(monkeypatch) -> None: async def test_openai_compatible_chat_retries_empty_reasoning_response_with_bigger_token_budget(monkeypatch) -> None:
calls: list[dict[str, object]] = [] calls: list[dict[str, object]] = []

View File

@ -1568,6 +1568,112 @@ async def test_resolve_active_mounted_draft_keeps_retryable_pending_confirm_with
assert result == (draft_row, topic_row) assert result == (draft_row, topic_row)
@pytest.mark.asyncio
async def test_resolve_active_mounted_draft_does_not_fall_back_to_same_scope_row_when_explicit_anchor_misses(
) -> None:
class _FakeResult:
def __init__(self, rows) -> None:
self._rows = list(rows)
def all(self):
return list(self._rows)
draft_row = SimpleNamespace(
id=18,
topic_workspace_id=177,
active_work_item_id=None,
state=STATE_DRAFT_PENDING_CONFIRM,
latest_reply_message_id="om-stale-draft-reply-1",
latest_source_message_id="src-stale-draft-1",
)
topic_row = SimpleNamespace(
id=177,
tenant_id=1,
source_system="feishu",
mode="plan",
status="active",
channel_ref="chat-1",
conversation_scope_key="feishu:chat-1:ou_xxx",
plan_status=STATE_DRAFT_PENDING_CONFIRM,
root_message_id="msg-stale-draft-root",
thread_key="",
last_message_id="src-stale-draft-1",
updated_at=None,
)
class _FakeDB:
async def execute(self, query): # noqa: ARG002
return _FakeResult([(draft_row, topic_row)])
handler = DraftConfirmHandler(type("Kernel", (), {"db": _FakeDB(), "tenant_id": 1})())
result = await handler._resolve_active_mounted_draft(
sender="林宇航",
source_system="feishu",
channel_ref="chat-1",
source_sender_open_id="ou_xxx",
thread_context={
"reply_to_message_id": "om-current-plan-reply-1",
"root_message_id": "msg-current-plan-root",
},
)
assert result is None
@pytest.mark.asyncio
async def test_has_active_draft_session_does_not_lock_same_scope_when_explicit_anchor_targets_other_thread(
) -> None:
class _FakeResult:
def __init__(self, rows) -> None:
self._rows = list(rows)
def all(self):
return list(self._rows)
stale_draft = SimpleNamespace(
id=28,
topic_workspace_id=277,
active_work_item_id=None,
state=STATE_DRAFT_PENDING_CONFIRM,
latest_reply_message_id="om-stale-draft-reply-2",
latest_source_message_id="src-stale-draft-2",
)
stale_topic = SimpleNamespace(
id=277,
tenant_id=1,
source_system="feishu",
mode="plan",
status="active",
channel_ref="chat-1",
conversation_scope_key="feishu:chat-1:ou_xxx",
plan_status=STATE_DRAFT_PENDING_CONFIRM,
root_message_id="msg-stale-draft-root-2",
thread_key="",
last_message_id="src-stale-draft-2",
updated_at=None,
)
class _FakeDB:
async def execute(self, query): # noqa: ARG002
return _FakeResult([(stale_draft, stale_topic)])
handler = DraftConfirmHandler(type("Kernel", (), {"db": _FakeDB(), "tenant_id": 1})())
has_session = await handler.has_active_draft_session(
sender="林宇航",
source_system="feishu",
channel_ref="chat-1",
source_sender_open_id="ou_xxx",
thread_context={
"reply_to_message_id": "om-current-plan-reply-2",
"root_message_id": "msg-current-plan-root-2",
},
)
assert has_session is False
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_resolve_pending_draft_falls_back_to_anchor_matched_pending_work_item_when_active_mounted_draft_has_no_active_work_item_id( async def test_resolve_pending_draft_falls_back_to_anchor_matched_pending_work_item_when_active_mounted_draft_has_no_active_work_item_id(
monkeypatch, monkeypatch,

View File

@ -0,0 +1,51 @@
from app.services.external_query_service.planner import ExternalQueryPlanner
def test_fallback_plan_stays_generic_for_fresh_weather_semantics() -> None:
planner = ExternalQueryPlanner(
llm_service=object(),
db=object(),
tenant_id=1,
)
plan = planner._build_fallback_plan(
question="帮我看看未来5天的广州天气如何",
query_semantics={
"query_type": "status",
"object_mentions": ["广州"],
"query_profile": {
"needs_freshness": True,
"referenced_objects": ["weather"],
},
},
)
assert plan.query_kind == "generic_web"
assert plan.answer_contract == "summary_with_sources"
assert plan.location_hint is None
assert plan.result_limit == 3
def test_fallback_plan_uses_compare_contract_without_object_kind_mapping() -> None:
planner = ExternalQueryPlanner(
llm_service=object(),
db=object(),
tenant_id=1,
)
plan = planner._build_fallback_plan(
question="对比 OpenAI 和 Anthropic 最新模型能力差异",
query_semantics={
"query_type": "compare",
"object_mentions": ["OpenAI", "Anthropic"],
"query_profile": {
"needs_freshness": True,
"referenced_objects": ["public_model"],
},
},
)
assert plan.query_kind == "generic_web"
assert plan.answer_contract == "compare_with_sources"
assert plan.requires_multi_source is True
assert plan.result_limit == 3

View File

@ -277,7 +277,12 @@ async def test_handle_card_action_trigger_plan_choice_dispatches_synthetic_ingre
assert data.get("target_work_item_id") == 88 assert data.get("target_work_item_id") == 88
ingress_kwargs = called["ingress_kwargs"] ingress_kwargs = called["ingress_kwargs"]
assert ingress_kwargs["message_text"] == "C" assert ingress_kwargs["message_text"] == "C"
assert ingress_kwargs["source_system"] == "ai_pmo_ui" assert ingress_kwargs["source_system"] == "feishu"
assert str(ingress_kwargs["source_message_id"]).startswith("feishu-card-plan-88-plan_choice-")
ingress_options = ingress_kwargs.get("ingress_options") if isinstance(ingress_kwargs.get("ingress_options"), dict) else {}
thread_context = ingress_options.get("thread_context") if isinstance(ingress_options.get("thread_context"), dict) else {}
assert thread_context["reply_to_message_id"] == "om_plan_1"
assert thread_context["root_message_id"] == "om_plan_1"
assert called["reply_chat"] == "oc_plan_1" assert called["reply_chat"] == "oc_plan_1"
assert isinstance(called["reply_card"], dict) assert isinstance(called["reply_card"], dict)
assert called["reply_card"]["header"]["title"]["content"] == "AI PMO 需求收敛" assert called["reply_card"]["header"]["title"]["content"] == "AI PMO 需求收敛"

View File

@ -0,0 +1,269 @@
from __future__ import annotations
from types import SimpleNamespace
import pytest
import pytest_asyncio
from fastapi import FastAPI
from httpx import ASGITransport, AsyncClient
from sqlalchemy import JSON, select
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from app.api.mcp_api import routes as mcp_routes
from app.core.database import get_db
from app.dependencies.auth import get_current_user
from app.models.base import Base
from app.models.integration import Integration
from app.services.mcp_config import McpConfigService
def _use_sqlite_json_types(*tables) -> None:
for table in tables:
for column in table.columns:
if column.type.__class__.__name__ in {"JSONB", "ARRAY", "Vector"}:
column.type = JSON()
@pytest_asyncio.fixture
async def db_session():
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
tables = [Integration.__table__]
_use_sqlite_json_types(*tables)
async with engine.begin() as conn:
await conn.run_sync(lambda sync_conn: Base.metadata.create_all(sync_conn, tables=tables))
maker = async_sessionmaker(engine, expire_on_commit=False)
async with maker() as session:
yield session
await engine.dispose()
def _build_app(db_session, *, role: str, tenant_id: int = 1) -> FastAPI:
app = FastAPI()
app.include_router(mcp_routes.router)
async def _fake_get_db():
yield db_session
app.dependency_overrides[get_db] = _fake_get_db
app.dependency_overrides[get_current_user] = lambda: SimpleNamespace(
id=7,
tenant_id=tenant_id,
role=role,
username="tester",
email="tester@example.com",
name="Tester",
)
return app
@pytest.mark.asyncio
async def test_mcp_admin_api_requires_manager_role(db_session) -> None:
app = _build_app(db_session, role="AI")
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
response = await client.get("/mcp/admin/catalog")
assert response.status_code == 403
@pytest.mark.asyncio
async def test_mcp_admin_catalog_and_server_lifecycle(db_session, monkeypatch) -> None:
async def _fake_probe(self, *, template, params): # noqa: ANN001
_ = (self, template, params)
return {
"status": "ready",
"checked_at": "2026-04-15T12:01:00+08:00",
"message": "PMFlow backend can reach Microsoft Learn MCP.",
"tool_names": ["microsoft_docs_search", "microsoft_docs_fetch"],
}
monkeypatch.setattr(McpConfigService, "_probe_backend_runtime", _fake_probe)
app = _build_app(db_session, role="ADMIN")
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
catalog_response = await client.get("/mcp/admin/catalog")
create_response = await client.post(
"/mcp/admin/servers",
json={
"name": "microsoft-learn",
"display_name": "Microsoft Learn MCP",
"template_id": "microsoft_learn",
"params": {},
},
)
created = create_response.json()["data"]
list_response = await client.get("/mcp/admin/servers")
validate_response = await client.post(f"/mcp/admin/servers/{created['id']}/validate")
update_response = await client.put(
f"/mcp/admin/servers/{created['id']}",
json={
"display_name": "Microsoft Learn Docs",
"params": {},
},
)
confirm_response = await client.post(f"/mcp/admin/servers/{created['id']}/confirm-client-apply")
assert catalog_response.status_code == 200
assert catalog_response.json()["code"] == 0
assert catalog_response.json()["data"]["recommended_template_id"] == "microsoft_learn"
assert catalog_response.json()["data"]["items"][0]["execution_mode"] == "backend_direct"
assert create_response.status_code == 200
assert create_response.json()["code"] == 0
assert created["execution_mode"] == "backend_direct"
assert created["client_apply_state"] == "not_required"
assert created["runtime_probe"]["status"] == "unknown"
assert list_response.status_code == 200
assert list_response.json()["data"]["items"][0]["name"] == "microsoft-learn"
assert validate_response.status_code == 200
assert validate_response.json()["code"] == 0
assert validate_response.json()["data"]["is_valid"] is True
assert validate_response.json()["data"]["preview"]["add_command"] == "无需本机命令。保存后由 PMFlow 后端直接连接 Microsoft Learn MCP。"
assert validate_response.json()["data"]["preview"]["verify_command"] == "点击“重新校验”后,PMFlow 会从后端执行 MCP initialize 和 tools/list。"
assert validate_response.json()["data"]["runtime_probe"]["status"] == "ready"
assert validate_response.json()["data"]["runtime_probe"]["tool_names"] == ["microsoft_docs_search", "microsoft_docs_fetch"]
assert update_response.status_code == 200
assert update_response.json()["code"] == 0
assert update_response.json()["data"]["display_name"] == "Microsoft Learn Docs"
assert confirm_response.status_code == 400
assert confirm_response.json()["code"] == 400
assert "后端直连" in confirm_response.json()["message"]
@pytest.mark.asyncio
async def test_mcp_admin_servers_are_tenant_scoped(db_session) -> None:
tenant1_app = _build_app(db_session, role="ADMIN", tenant_id=1)
tenant2_app = _build_app(db_session, role="ADMIN", tenant_id=2)
async with AsyncClient(transport=ASGITransport(app=tenant1_app), base_url="http://test") as client:
create_response = await client.post(
"/mcp/admin/servers",
json={
"name": "playwright",
"display_name": "Playwright MCP",
"template_id": "playwright",
"params": {"output_dir": "~/.codex/mcp/playwright-output"},
},
)
async with AsyncClient(transport=ASGITransport(app=tenant2_app), base_url="http://test") as client:
list_response = await client.get("/mcp/admin/servers")
assert create_response.status_code == 200
assert create_response.json()["code"] == 0
assert list_response.status_code == 200
assert list_response.json()["code"] == 0
assert list_response.json()["data"]["items"] == []
@pytest.mark.asyncio
async def test_mcp_admin_github_template_rejects_raw_token_and_only_persists_env_var_name(db_session) -> None:
app = _build_app(db_session, role="ADMIN")
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
invalid_response = await client.post(
"/mcp/admin/servers",
json={
"name": "github",
"display_name": "GitHub MCP",
"template_id": "github",
"params": {
"token_env_var_name": "GITHUB_PAT",
"access_token": "ghp_secret_should_not_be_saved",
},
},
)
valid_response = await client.post(
"/mcp/admin/servers",
json={
"name": "github",
"display_name": "GitHub MCP",
"template_id": "github",
"params": {
"token_env_var_name": "GITHUB_PAT",
},
},
)
row = (
await db_session.execute(
select(Integration).where(
Integration.tenant_id == 1,
Integration.type == "mcp_template_center",
)
)
).scalar_one()
server = row.config["servers"][0]
assert invalid_response.status_code == 400
assert invalid_response.json()["code"] == 400
assert "token" in invalid_response.json()["message"].lower()
assert valid_response.status_code == 200
assert valid_response.json()["code"] == 0
assert valid_response.json()["data"]["execution_mode"] == "local_bridge"
assert server["params"]["token_env_var_name"] == "GITHUB_PAT"
assert "access_token" not in server["params"]
@pytest.mark.asyncio
async def test_mcp_admin_filesystem_requires_allowed_paths(db_session) -> None:
app = _build_app(db_session, role="ADMIN")
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
response = await client.post(
"/mcp/admin/servers",
json={
"name": "filesystem",
"display_name": "Filesystem MCP",
"template_id": "filesystem",
"params": {},
},
)
assert response.status_code == 400
assert response.json()["code"] == 400
assert "allowed_paths" in response.json()["message"]
@pytest.mark.asyncio
async def test_local_bridge_template_uses_manual_instructions_and_can_be_confirmed(db_session) -> None:
app = _build_app(db_session, role="ADMIN")
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
create_response = await client.post(
"/mcp/admin/servers",
json={
"name": "filesystem",
"display_name": "Filesystem MCP",
"template_id": "filesystem",
"params": {
"allowed_paths": ["/Users/yuuuu/Documents"],
},
},
)
created = create_response.json()["data"]
validate_response = await client.post(f"/mcp/admin/servers/{created['id']}/validate")
confirm_response = await client.post(f"/mcp/admin/servers/{created['id']}/confirm-client-apply")
assert create_response.status_code == 200
assert created["execution_mode"] == "local_bridge"
assert created["client_apply_state"] == "pending_local_apply"
assert "codex mcp add" not in created["preview"]["add_command"].lower()
assert "local bridge" in created["preview"]["add_command"].lower()
assert validate_response.status_code == 200
assert validate_response.json()["data"]["runtime_probe"]["status"] == "local_bridge_required"
assert confirm_response.status_code == 200
assert confirm_response.json()["data"]["client_apply_state"] == "user_confirmed"

View File

@ -0,0 +1,448 @@
from __future__ import annotations
from types import SimpleNamespace
import pytest
import pytest_asyncio
from sqlalchemy import JSON
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from app.models.base import Base
from app.models.integration import Integration
from app.services.adaptive_rag_service.mixin_answer_retrieval import AdaptiveRAGAnswerRetrievalMixin
from app.services.adaptive_rag_service.mixin_deep_guard_rules import AdaptiveRAGDeepGuardRulesMixin
from app.services.adaptive_rag_service.shared_part_02 import RetrievedItem
from app.services.external_query_service.adapters import MCPExternalAdapter
from app.services.external_query_service.gateway import ExternalQueryGateway
from app.services.external_query_service.registry import ExternalProviderRegistry
from app.services.external_query_service.schemas import ExternalAdapterResult, ExternalEvidenceItem, ExternalQueryPlan
from app.services.mcp_config import McpConfigService
from app.services.mcp_config.runtime import MicrosoftLearnMCPRuntime
def _use_sqlite_json_types(*tables) -> None:
for table in tables:
for column in table.columns:
if column.type.__class__.__name__ in {"JSONB", "ARRAY", "Vector"}:
column.type = JSON()
@pytest_asyncio.fixture
async def db_session():
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
tables = [Integration.__table__]
_use_sqlite_json_types(*tables)
async with engine.begin() as conn:
await conn.run_sync(lambda sync_conn: Base.metadata.create_all(sync_conn, tables=tables))
maker = async_sessionmaker(engine, expire_on_commit=False)
async with maker() as session:
yield session
await engine.dispose()
async def _seed_backend_direct_server(db_session) -> None:
service = McpConfigService(db_session, tenant_id=1)
await service.create_server(
name="microsoft-learn",
display_name="Microsoft Learn MCP",
template_id="microsoft_learn",
params={},
)
await db_session.commit()
@pytest.mark.asyncio
async def test_mcp_config_service_search_external_uses_backend_direct_server(db_session, monkeypatch) -> None:
await _seed_backend_direct_server(db_session)
async def _fake_search(self, *, query: str, result_limit: int = 3): # noqa: ANN001
_ = (self, query, result_limit)
return [
ExternalEvidenceItem(
source_type="mcp_search",
source_label="Microsoft Learn MCP",
title="Azure OpenAI quickstart",
target_url="https://learn.microsoft.com/azure/ai-foundry/openai/chatgpt-quickstart",
snippet="Official Microsoft quickstart.",
retrieved_at="2026-04-15T12:30:00+08:00",
freshness_score=0.9,
authority_score=0.98,
grounded_fields=["microsoft_docs_search"],
raw_metadata={"provider": "microsoft_learn_mcp"},
)
]
monkeypatch.setattr(MicrosoftLearnMCPRuntime, "search", _fake_search)
evidence = await McpConfigService(db_session, tenant_id=1).search_external(
query="azure openai quickstart",
result_limit=3,
)
assert len(evidence) == 1
assert evidence[0].source_type == "mcp_search"
assert evidence[0].target_url == "https://learn.microsoft.com/azure/ai-foundry/openai/chatgpt-quickstart"
@pytest.mark.asyncio
async def test_external_query_gateway_prefers_backend_direct_mcp_when_available(db_session, monkeypatch) -> None:
await _seed_backend_direct_server(db_session)
async def _fake_search_external(self, *, query: str, result_limit: int = 3): # noqa: ANN001
_ = (self, query, result_limit)
return [
ExternalEvidenceItem(
source_type="mcp_search",
source_label="Microsoft Learn MCP",
title="Azure OpenAI quickstart",
target_url="https://learn.microsoft.com/azure/ai-foundry/openai/chatgpt-quickstart",
snippet="Official Microsoft quickstart.",
retrieved_at="2026-04-15T12:30:00+08:00",
freshness_score=0.9,
authority_score=0.98,
grounded_fields=["microsoft_docs_search"],
raw_metadata={"provider": "microsoft_learn_mcp"},
)
]
monkeypatch.setattr(McpConfigService, "search_external", _fake_search_external)
class _FakeService(AdaptiveRAGAnswerRetrievalMixin, AdaptiveRAGDeepGuardRulesMixin):
def __init__(self, db):
self.db = db
self.tenant_id = 1
self.llm = None
service = _FakeService(db_session)
rows, meta = await service._execute_external_query_gateway(
question="Azure OpenAI quickstart",
query_semantics={},
result_limit=3,
max_pages=1,
)
assert len(rows) == 1
assert isinstance(rows[0], RetrievedItem)
assert rows[0].source == "mcp_search"
assert rows[0].metadata["external_adapter"] == "mcp"
assert meta["adapter"] == "mcp"
@pytest.mark.asyncio
async def test_external_query_gateway_falls_back_when_mcp_evidence_is_off_topic() -> None:
plan = ExternalQueryPlan(
plan_version="v1",
query_kind="weather",
answer_contract="summary_with_sources",
freshness_required=True,
topic="广州未来5天天气",
subject_entities=["广州"],
location_hint="广州",
time_range_hint="未来5天",
preferred_domains=[],
search_queries=["广州 未来5天天气"],
result_limit=3,
requires_multi_source=False,
clarify_on_ambiguity=False,
)
class _Planner:
async def build_plan(self, *, question: str, query_semantics: dict | None = None):
_ = (question, query_semantics)
return plan, {"planner": "test"}
class _MCPAdapter:
adapter_name = "mcp"
async def execute(self, current_plan: ExternalQueryPlan) -> ExternalAdapterResult:
_ = current_plan
return ExternalAdapterResult(
adapter_name="mcp",
status="success",
evidence=[
ExternalEvidenceItem(
source_type="mcp_search",
source_label="Microsoft Learn MCP",
title="Azure Maps Weather services",
target_url="https://learn.microsoft.com/azure/azure-maps/weather-services-concepts",
snippet="Learn how Azure Maps Weather APIs expose forecast data models and request options.",
retrieved_at="2026-04-15T12:30:00+08:00",
freshness_score=0.82,
authority_score=0.98,
grounded_fields=["microsoft_docs_search"],
raw_metadata={"provider": "microsoft_learn_mcp"},
)
],
meta={"provider": "mcp"},
)
class _LiveSearchAdapter:
adapter_name = "live_search"
async def execute(self, current_plan: ExternalQueryPlan) -> ExternalAdapterResult:
_ = current_plan
return ExternalAdapterResult(
adapter_name="live_search",
status="success",
evidence=[
ExternalEvidenceItem(
source_type="url",
source_label="Weather Example",
title="广州未来5天天气预报",
target_url="https://weather.example/guangzhou-5day",
snippet="广州未来5天有阵雨转多云,最高温31度,最低温24度。",
retrieved_at="2026-04-15T12:35:00+08:00",
freshness_score=0.91,
authority_score=0.76,
grounded_fields=["forecast"],
raw_metadata={"provider": "live_search"},
)
],
meta={"provider": "live_search"},
)
execution = await ExternalQueryGateway(
planner=_Planner(),
registry=ExternalProviderRegistry(adapters=[_MCPAdapter(), _LiveSearchAdapter()]),
).execute(
question="帮我看看未来5天的广州天气如何",
query_semantics={"query_profile": {"needs_freshness": True}},
)
assert execution.adapter_name == "live_search"
assert execution.meta["fallback_used"] is True
assert execution.verifier["ok"] is True
@pytest.mark.asyncio
async def test_mcp_adapter_fetches_target_page_content_after_search_result() -> None:
plan = ExternalQueryPlan(
plan_version="v1",
query_kind="weather",
answer_contract="summary_with_sources",
freshness_required=True,
topic="湛江市未来五天天气",
subject_entities=["湛江市"],
location_hint="湛江市",
time_range_hint="未来5天",
preferred_domains=["weather.example"],
search_queries=["湛江市未来五天天气预报"],
result_limit=3,
requires_multi_source=False,
clarify_on_ambiguity=False,
)
class _Planner:
async def build_plan(self, *, question: str, query_semantics: dict | None = None):
_ = (question, query_semantics)
return plan, {"planner": "test"}
async def _search_callable(current_plan: ExternalQueryPlan) -> ExternalAdapterResult:
_ = current_plan
return ExternalAdapterResult(
adapter_name="mcp",
status="success",
evidence=[
ExternalEvidenceItem(
source_type="mcp_search",
source_label="Weather MCP",
title="Forecast detail page",
target_url="https://weather.example/forecast/zhanjiang-five-day",
snippet="Landing page",
retrieved_at="2026-04-16T01:05:00+08:00",
freshness_score=0.6,
authority_score=0.86,
grounded_fields=["mcp_search"],
raw_metadata={"provider": "weather_mcp"},
)
],
meta={"provider": "weather_mcp"},
)
async def _fetch_text(url: str) -> str:
assert url == "https://weather.example/forecast/zhanjiang-five-day"
return "页面说明。湛江未来5天有阵雨转多云,最高温33度,最低温26度。末尾。"
def _chunk_text(text: str, *, chunk_size: int = 900, overlap: int = 120) -> list[str]:
_ = (text, chunk_size, overlap)
return ["页面说明。", "湛江未来5天有阵雨转多云,最高温33度,最低温26度。"]
def _score_chunk(*, question: str, title: str, content: str, url: str) -> float:
_ = (question, title, url)
return 0.94 if "湛江未来5天" in content else 0.08
execution = await ExternalQueryGateway(
planner=_Planner(),
registry=ExternalProviderRegistry(
adapters=[
MCPExternalAdapter(
search_callable=_search_callable,
fetch_text_callable=_fetch_text,
chunk_text_callable=_chunk_text,
score_chunk_callable=_score_chunk,
max_pages=1,
)
]
),
).execute(
question="湛江市未来五天的天气",
query_semantics={"query_profile": {"needs_freshness": True}},
)
assert execution.adapter_name == "mcp"
assert execution.verifier["ok"] is True
assert execution.evidence[0].snippet == "湛江未来5天有阵雨转多云,最高温33度,最低温26度。"
assert execution.evidence[0].raw_metadata["search_snippet"] == "Landing page"
assert "page_content_snippet" in execution.evidence[0].grounded_fields
@pytest.mark.asyncio
async def test_external_query_gateway_accepts_bundle_with_normalized_anchor_match() -> None:
plan = ExternalQueryPlan(
plan_version="v1",
query_kind="weather",
answer_contract="summary_with_sources",
freshness_required=True,
topic="湛江市未来五天天气",
subject_entities=["湛江市"],
location_hint="湛江市",
time_range_hint="未来5天",
preferred_domains=["weather.com.cn", "nmc.cn"],
search_queries=["湛江市未来五天天气预报"],
result_limit=3,
requires_multi_source=False,
clarify_on_ambiguity=False,
)
class _Planner:
async def build_plan(self, *, question: str, query_semantics: dict | None = None):
_ = (question, query_semantics)
return plan, {"planner": "test"}
class _LiveSearchAdapter:
adapter_name = "live_search"
async def execute(self, current_plan: ExternalQueryPlan) -> ExternalAdapterResult:
_ = current_plan
return ExternalAdapterResult(
adapter_name="live_search",
status="success",
evidence=[
ExternalEvidenceItem(
source_type="url",
source_label="中国天气网",
title="湛江天气预报,湛江7天天气预报,湛江15天天气预报,湛江天气查询",
target_url="https://www.weather.com.cn/weather/101281001.shtml",
snippet="湛江未来5天有阵雨转多云,最高温33度,最低温26度。",
retrieved_at="2026-04-16T00:30:00+08:00",
freshness_score=0.92,
authority_score=0.95,
grounded_fields=["forecast"],
raw_metadata={"provider": "live_search"},
)
],
meta={"provider": "live_search"},
)
execution = await ExternalQueryGateway(
planner=_Planner(),
registry=ExternalProviderRegistry(adapters=[_LiveSearchAdapter()]),
).execute(
question="湛江市未来五天的天气",
query_semantics={"query_profile": {"needs_freshness": True}},
)
assert execution.adapter_name == "live_search"
assert execution.verifier["ok"] is True
@pytest.mark.asyncio
async def test_external_query_gateway_prefers_weather_capable_adapter_before_doc_mcp() -> None:
plan = ExternalQueryPlan(
plan_version="v1",
query_kind="weather",
answer_contract="summary_with_sources",
freshness_required=True,
topic="湛江市未来五天天气",
subject_entities=["湛江市"],
location_hint="湛江市",
time_range_hint="未来5天",
preferred_domains=["weather.com.cn", "nmc.cn"],
search_queries=["湛江市未来五天天气预报"],
result_limit=3,
requires_multi_source=False,
clarify_on_ambiguity=False,
)
call_order: list[str] = []
class _Planner:
async def build_plan(self, *, question: str, query_semantics: dict | None = None):
_ = (question, query_semantics)
return plan, {"planner": "test"}
class _MCPAdapter:
adapter_name = "mcp"
supported_query_kinds = {"model", "company", "generic_web", "unknown"}
async def execute(self, current_plan: ExternalQueryPlan) -> ExternalAdapterResult:
_ = current_plan
call_order.append("mcp")
return ExternalAdapterResult(
adapter_name="mcp",
status="success",
evidence=[
ExternalEvidenceItem(
source_type="mcp_search",
source_label="Microsoft Learn MCP",
title="Azure Maps Weather service coverage",
target_url="https://learn.microsoft.com/azure/azure-maps/weather-coverage",
snippet="Weather service coverage.",
retrieved_at="2026-04-16T00:30:00+08:00",
freshness_score=0.82,
authority_score=0.98,
grounded_fields=["microsoft_docs_search"],
raw_metadata={"provider": "mcp"},
)
],
meta={"provider": "mcp"},
)
class _LiveSearchAdapter:
adapter_name = "live_search"
supported_query_kinds = {"weather", "news", "market", "generic_web", "unknown"}
async def execute(self, current_plan: ExternalQueryPlan) -> ExternalAdapterResult:
_ = current_plan
call_order.append("live_search")
return ExternalAdapterResult(
adapter_name="live_search",
status="success",
evidence=[
ExternalEvidenceItem(
source_type="url",
source_label="中国天气网",
title="湛江天气预报,湛江7天天气预报,湛江15天天气预报,湛江天气查询",
target_url="https://www.weather.com.cn/weather/101281001.shtml",
snippet="湛江未来5天有阵雨转多云,最高温33度,最低温26度。",
retrieved_at="2026-04-16T00:30:00+08:00",
freshness_score=0.92,
authority_score=0.95,
grounded_fields=["forecast"],
raw_metadata={"provider": "live_search"},
)
],
meta={"provider": "live_search"},
)
execution = await ExternalQueryGateway(
planner=_Planner(),
registry=ExternalProviderRegistry(adapters=[_MCPAdapter(), _LiveSearchAdapter()]),
).execute(
question="湛江市未来五天的天气",
query_semantics={"query_profile": {"needs_freshness": True}},
)
assert call_order == ["live_search"]
assert execution.adapter_name == "live_search"
assert execution.verifier["ok"] is True

View File

@ -0,0 +1,120 @@
from __future__ import annotations
import json
import httpx
import pytest
from app.services.mcp_config.runtime import MicrosoftLearnMCPRuntime
def _sse(payload: dict) -> httpx.Response:
body = f"event: message\ndata: {json.dumps(payload, ensure_ascii=False)}\n\n"
return httpx.Response(
200,
headers={"content-type": "text/event-stream", "mcp-session-id": "test-session"},
text=body,
)
@pytest.mark.asyncio
async def test_microsoft_learn_runtime_probe_initializes_and_lists_tools() -> None:
calls: list[dict] = []
def _handler(request: httpx.Request) -> httpx.Response:
payload = json.loads(request.content.decode("utf-8"))
calls.append(payload)
if payload["method"] == "initialize":
return _sse(
{
"jsonrpc": "2.0",
"id": payload["id"],
"result": {
"serverInfo": {"name": "Microsoft Learn MCP Server", "version": "1.0.0"},
},
}
)
if payload["method"] == "tools/list":
assert request.headers["mcp-session-id"] == "test-session"
return _sse(
{
"jsonrpc": "2.0",
"id": payload["id"],
"result": {
"tools": [
{"name": "microsoft_docs_search"},
{"name": "microsoft_docs_fetch"},
]
},
}
)
raise AssertionError(f"unexpected method: {payload['method']}")
runtime = MicrosoftLearnMCPRuntime(
endpoint_url="https://learn.microsoft.com/api/mcp",
client_factory=lambda: httpx.AsyncClient(transport=httpx.MockTransport(_handler)),
)
probe = await runtime.probe()
assert probe["status"] == "ready"
assert probe["tool_names"] == ["microsoft_docs_search", "microsoft_docs_fetch"]
assert [item["method"] for item in calls] == ["initialize", "tools/list"]
@pytest.mark.asyncio
async def test_microsoft_learn_runtime_search_returns_external_evidence() -> None:
def _handler(request: httpx.Request) -> httpx.Response:
payload = json.loads(request.content.decode("utf-8"))
if payload["method"] == "initialize":
return _sse(
{
"jsonrpc": "2.0",
"id": payload["id"],
"result": {
"serverInfo": {"name": "Microsoft Learn MCP Server", "version": "1.0.0"},
},
}
)
if payload["method"] == "tools/call":
assert payload["params"]["name"] == "microsoft_docs_search"
assert payload["params"]["arguments"]["query"] == "azure openai quickstart"
return _sse(
{
"jsonrpc": "2.0",
"id": payload["id"],
"result": {
"content": [
{
"type": "text",
"text": json.dumps(
{
"results": [
{
"title": "Quickstart: Azure OpenAI",
"content": "Use Azure OpenAI with official quickstart guidance.",
"contentUrl": "https://learn.microsoft.com/azure/ai-foundry/openai/chatgpt-quickstart",
}
]
},
ensure_ascii=False,
),
}
]
},
}
)
raise AssertionError(f"unexpected method: {payload['method']}")
runtime = MicrosoftLearnMCPRuntime(
endpoint_url="https://learn.microsoft.com/api/mcp",
client_factory=lambda: httpx.AsyncClient(transport=httpx.MockTransport(_handler)),
)
evidence = await runtime.search(query="azure openai quickstart", result_limit=3)
assert len(evidence) == 1
assert evidence[0].source_type == "mcp_search"
assert evidence[0].title == "Quickstart: Azure OpenAI"
assert evidence[0].target_url == "https://learn.microsoft.com/azure/ai-foundry/openai/chatgpt-quickstart"
assert evidence[0].authority_score >= 0.9

View File

@ -362,3 +362,107 @@ async def test_model_context_bundle_summary_uses_routing_names(db_env, monkeypat
first_row = summary["function_routing_template_matrix"][0] first_row = summary["function_routing_template_matrix"][0]
assert "routing_snapshot" in first_row assert "routing_snapshot" in first_row
assert "model_routing" not in first_row assert "model_routing" not in first_row
@pytest.mark.asyncio
async def test_get_embedding_async_falls_back_to_local_ollama_embed_when_active_engine_is_not_embedding_capable(
db_env,
monkeypatch,
) -> None:
db_session, _maker = db_env
captured: dict[str, object] = {}
class _FakeResponse:
def raise_for_status(self) -> None:
return None
def json(self) -> dict[str, object]:
return {"embeddings": [[0.11, 0.22, 0.33]]}
class _FakeAsyncClient:
def __init__(self, *args, **kwargs) -> None:
_ = args
captured["init_kwargs"] = dict(kwargs)
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc, tb) -> None:
_ = (exc_type, exc, tb)
return None
async def post(self, url: str, headers: dict[str, str], json: dict[str, object]):
captured["url"] = url
captured["headers"] = dict(headers)
captured["payload"] = dict(json)
return _FakeResponse()
async def _fake_resolve_active_engine(db, *, tenant_id: int):
_ = (db, tenant_id)
return {
"provider": "glm",
"model": "qwen-plus",
"base_url": "https://dashscope.aliyuncs.com/compatible-mode/v1",
"api_key": "glm-key",
}
monkeypatch.setattr(llm_service, "_resolve_active_engine", _fake_resolve_active_engine)
monkeypatch.setattr("app.services.llm_service.mixin_task_helpers.httpx.AsyncClient", _FakeAsyncClient)
monkeypatch.setenv("OLLAMA_HOST", "http://127.0.0.1:11434")
monkeypatch.setenv("OLLAMA_EMBED_MODEL", "nomic-embed-text:latest")
embedding = await llm_service.get_embedding_async(
"广州今天天气如何",
db=db_session,
tenant_id=1,
)
assert embedding == [0.11, 0.22, 0.33]
assert captured["url"] == "http://127.0.0.1:11434/api/embed"
assert captured["payload"] == {"model": "nomic-embed-text:latest", "input": "广州今天天气如何"}
assert captured["init_kwargs"]["trust_env"] is False
@pytest.mark.asyncio
async def test_get_embedding_async_returns_empty_when_all_real_backends_fail_instead_of_pseudo_vector(
db_env,
monkeypatch,
) -> None:
db_session, _maker = db_env
class _FailingAsyncClient:
def __init__(self, *args, **kwargs) -> None:
_ = (args, kwargs)
async def __aenter__(self):
return self
async def __aexit__(self, exc_type, exc, tb) -> None:
_ = (exc_type, exc, tb)
return None
async def post(self, url: str, headers: dict[str, str], json: dict[str, object]):
_ = (url, headers, json)
raise RuntimeError("embedding backend unavailable")
async def _fake_resolve_active_engine(db, *, tenant_id: int):
_ = (db, tenant_id)
return {
"provider": "glm",
"model": "qwen-plus",
"base_url": "https://dashscope.aliyuncs.com/compatible-mode/v1",
"api_key": "glm-key",
}
monkeypatch.setattr(llm_service, "_resolve_active_engine", _fake_resolve_active_engine)
monkeypatch.setattr("app.services.llm_service.mixin_task_helpers.httpx.AsyncClient", _FailingAsyncClient)
monkeypatch.setenv("OLLAMA_HOST", "http://127.0.0.1:11434")
monkeypatch.setenv("OLLAMA_EMBED_MODEL", "nomic-embed-text:latest")
embedding = await llm_service.get_embedding_async(
"今天荔湾区天气如何",
db=db_session,
tenant_id=1,
)
assert embedding == []

View File

@ -17,6 +17,8 @@ from app.models.kg_evidence import KGEvidence
from app.models.kg_relation import KGRelation from app.models.kg_relation import KGRelation
from app.models.kg_sync_run import KGSyncRun from app.models.kg_sync_run import KGSyncRun
from app.models.project import Project from app.models.project import Project
from app.models.semantic_fact import SemanticFact
from app.models.semantic_fact_extraction_run import SemanticFactExtractionRun
from app.models.signal import Signal from app.models.signal import Signal
from app.models.user import User from app.models.user import User
from app.models.work_item import WorkItem from app.models.work_item import WorkItem
@ -42,6 +44,8 @@ async def db_session():
IngestOffset.__table__, IngestOffset.__table__,
WorkItem.__table__, WorkItem.__table__,
AIOrchestrationRun.__table__, AIOrchestrationRun.__table__,
SemanticFact.__table__,
SemanticFactExtractionRun.__table__,
KGEntity.__table__, KGEntity.__table__,
KGRelation.__table__, KGRelation.__table__,
KGEvidence.__table__, KGEvidence.__table__,
@ -191,6 +195,77 @@ async def test_enqueue_work_item_delta_events_reads_feishu_work_items_into_kg(db
assert relation is not None assert relation is not None
@pytest.mark.asyncio
async def test_process_work_item_event_enqueues_semantic_fact_extract_for_project(db_session) -> None:
project = Project(
tenant_id=1,
name="alpha",
title="Alpha 项目",
status="active",
owner_id=None,
assigned_role="backend",
)
seed_ingest_event = IngestEvent(
tenant_id=1,
channel="seed",
event_key="seed-semantic-fact",
status="done",
payload={},
)
db_session.add_all([project, seed_ingest_event])
await db_session.flush()
target_work_item = WorkItem(
tenant_id=1,
ingest_event_id=int(seed_ingest_event.id),
source_system="feishu",
source_message_id="om_work_item_semantic_fact",
state="waiting_manual",
intent="project.create",
project_id=int(project.id),
execution_result={"status": "pending_manual"},
review_summary={"verified": False, "issues": ["pending_manual"]},
)
db_session.add(target_work_item)
await db_session.flush()
run = AIOrchestrationRun(
tenant_id=1,
trace_id="trace-semantic-fact-1",
run_type="ingress",
status="completed",
source_system="feishu",
source_message_id="om_work_item_semantic_fact",
sender="测试用户",
channel_ref="chat-alpha",
message_text="请创建项目 Alpha",
work_item_id=int(target_work_item.id),
project_id=int(project.id),
)
db_session.add(run)
await db_session.commit()
service = ProjectKnowledgeGraphService(db_session, tenant_id=1)
await service.enqueue_work_item_delta_events(limit=20)
process_result = await service.process_pending_events(max_batch=20)
semantic_event = (
await db_session.execute(
select(IngestEvent).where(
IngestEvent.tenant_id == 1,
IngestEvent.channel == "semantic_fact_extract",
IngestEvent.event_key == f"project:{int(project.id)}",
)
)
).scalar_one_or_none()
assert process_result["done"] >= 1
assert semantic_event is not None
assert semantic_event.payload["entity_type"] == "project"
assert semantic_event.payload["entity_id"] == int(project.id)
assert semantic_event.payload["trigger_type"] == "work_item_delta"
@pytest.mark.asyncio @pytest.mark.asyncio
async def test_enqueue_feishu_delta_events_is_compat_alias_for_work_item_delta(db_session) -> None: async def test_enqueue_feishu_delta_events_is_compat_alias_for_work_item_delta(db_session) -> None:
project = Project( project = Project(

View File

@ -0,0 +1,178 @@
from __future__ import annotations
from types import SimpleNamespace
import pytest
import pytest_asyncio
from fastapi import FastAPI
from httpx import ASGITransport, AsyncClient
from sqlalchemy import JSON, select
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from app.api.semantic_facts_api import routes as semantic_fact_routes
from app.dependencies.auth import get_current_user, get_tenant_id
from app.core.database import get_db
from app.models.base import Base
from app.models.ingest_event import IngestEvent
from app.models.knowledge_base_doc import KnowledgeBaseDoc
from app.models.project import Project
from app.models.semantic_fact import SemanticFact
from app.models.semantic_fact_extraction_run import SemanticFactExtractionRun
from app.models.signal import Signal
from app.models.task import Task
from app.models.truth_node import TruthNode
from app.models.user import User
def _use_sqlite_json_types(*tables) -> None:
for table in tables:
for column in table.columns:
if column.type.__class__.__name__ in {"JSONB", "ARRAY", "Vector"}:
column.type = JSON()
@pytest_asyncio.fixture
async def db_session():
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
tables = [
User.__table__,
Project.__table__,
Signal.__table__,
TruthNode.__table__,
Task.__table__,
KnowledgeBaseDoc.__table__,
SemanticFact.__table__,
SemanticFactExtractionRun.__table__,
IngestEvent.__table__,
]
_use_sqlite_json_types(*tables)
async with engine.begin() as conn:
await conn.run_sync(lambda sync_conn: Base.metadata.create_all(sync_conn, tables=tables))
maker = async_sessionmaker(engine, expire_on_commit=False)
async with maker() as session:
yield session
await engine.dispose()
def _build_app(db_session, *, role: str) -> FastAPI:
app = FastAPI()
app.include_router(semantic_fact_routes.router)
async def _fake_get_db():
yield db_session
app.dependency_overrides[get_db] = _fake_get_db
app.dependency_overrides[get_tenant_id] = lambda: 1
app.dependency_overrides[get_current_user] = lambda: SimpleNamespace(
id=7,
tenant_id=1,
role=role,
username="tester",
email="tester@example.com",
name="Tester",
)
return app
@pytest.mark.asyncio
async def test_semantic_fact_admin_api_requires_manager_role(db_session) -> None:
app = _build_app(db_session, role="AI")
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
response = await client.get("/semantic-facts/admin/overview")
assert response.status_code == 403
@pytest.mark.asyncio
async def test_semantic_fact_admin_api_exposes_overview_objects_detail_and_actions(db_session) -> None:
project = Project(
tenant_id=1,
name="invoice-helper",
title="开票助手",
description="自动生成发票",
owner_id=None,
status="active",
assigned_role="backend",
)
db_session.add(project)
await db_session.flush()
db_session.add(
SemanticFact(
tenant_id=1,
entity_type="project",
entity_id=int(project.id),
fact_namespace="alias",
fact_key="aliases",
fact_value_text="开票系统",
fact_value_norm="invoice_system",
value_type="keyword",
confidence=0.95,
evidence_source_type="project",
evidence_source_id=int(project.id),
evidence_excerpt="自动生成发票",
extract_version="semantic-facts-v1",
freshness_at=None,
is_active=True,
)
)
db_session.add(
SemanticFactExtractionRun(
tenant_id=1,
entity_type="project",
entity_id=int(project.id),
trigger_type="manual_retry",
status="completed",
input_fingerprint="fp-1",
facts_written=1,
failure_reason=None,
extract_version="semantic-facts-v1",
started_at="2026-04-15T10:00:00+00:00",
finished_at="2026-04-15T10:00:02+00:00",
)
)
await db_session.commit()
app = _build_app(db_session, role="ADMIN")
transport = ASGITransport(app=app)
async with AsyncClient(transport=transport, base_url="http://test") as client:
overview_response = await client.get("/semantic-facts/admin/overview")
objects_response = await client.get("/semantic-facts/admin/objects", params={"entity_type": "project", "q": "开票"})
detail_response = await client.get(f"/semantic-facts/admin/objects/project/{int(project.id)}")
retry_response = await client.post(f"/semantic-facts/admin/objects/project/{int(project.id)}/reextract")
retry_event = (
await db_session.execute(
select(IngestEvent).where(
IngestEvent.tenant_id == 1,
IngestEvent.channel == "semantic_fact_extract",
IngestEvent.event_key == f"project:{int(project.id)}",
)
)
).scalar_one()
retry_trigger_type = retry_event.payload["trigger_type"]
backfill_response = await client.post(
"/semantic-facts/admin/backfill",
json={"entity_types": ["project"], "only_missing": False},
)
assert overview_response.status_code == 200
assert overview_response.json()["code"] == 0
assert overview_response.json()["data"]["queue"]["pending"] >= 1
assert objects_response.status_code == 200
assert objects_response.json()["code"] == 0
assert objects_response.json()["data"]["items"][0]["display_name"] == "开票助手"
assert detail_response.status_code == 200
assert detail_response.json()["code"] == 0
assert detail_response.json()["data"]["facts"][0]["fact_value_text"] == "开票系统"
assert retry_response.status_code == 200
assert retry_response.json()["code"] == 0
assert retry_trigger_type == "manual_retry"
assert backfill_response.status_code == 200
assert backfill_response.json()["code"] == 0
assert backfill_response.json()["data"]["enqueued"] >= 1

View File

@ -0,0 +1,250 @@
from __future__ import annotations
import pytest
import pytest_asyncio
from sqlalchemy import JSON, select
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from app.models.base import Base
from app.models.ingest_event import IngestEvent
from app.models.knowledge_base_doc import KnowledgeBaseDoc
from app.models.project import Project
from app.models.signal import Signal
from app.models.task import Task
from app.models.truth_node import TruthNode
from app.models.user import User
from app.services.semantic_fact_service.hooks import register_semantic_fact_hooks
def _use_sqlite_json_types(*tables) -> None:
for table in tables:
for column in table.columns:
if column.type.__class__.__name__ in {"JSONB", "ARRAY", "Vector"}:
column.type = JSON()
@pytest_asyncio.fixture
async def db_session():
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
tables = [
User.__table__,
Project.__table__,
Signal.__table__,
TruthNode.__table__,
Task.__table__,
KnowledgeBaseDoc.__table__,
IngestEvent.__table__,
]
_use_sqlite_json_types(*tables)
async with engine.begin() as conn:
await conn.run_sync(lambda sync_conn: Base.metadata.create_all(sync_conn, tables=tables))
maker = async_sessionmaker(engine, expire_on_commit=False)
async with maker() as session:
yield session
await engine.dispose()
@pytest.mark.asyncio
async def test_semantic_fact_hooks_enqueue_extract_events_for_supported_entities(db_session) -> None:
register_semantic_fact_hooks()
user = User(
tenant_id=1,
username="semantic_admin",
email="semantic_admin@example.com",
password_hash="hash",
name="管理员",
role="ADMIN",
status="active",
is_active=True,
)
db_session.add(user)
await db_session.flush()
project = Project(
tenant_id=1,
name="alpha",
title="Alpha 项目",
description="项目描述",
owner_id=int(user.id),
status="active",
assigned_role="backend",
)
signal = Signal(
tenant_id=1,
title="信号标题",
content="信号内容",
status="待处理",
processed=False,
priority="high",
)
doc = KnowledgeBaseDoc(
tenant_id=1,
title="知识文档",
file_type="md",
source_type="file",
status="complete",
uploaded_by=int(user.id),
)
db_session.add_all([project, signal, doc])
await db_session.flush()
charter = TruthNode(
tenant_id=1,
node_type="charter",
parent_id=None,
project_id=int(project.id),
signal_id=None,
created_by=int(user.id),
title="开票系统需求",
content={"summary": "建设开票系统"},
version=1,
status="draft",
)
release = TruthNode(
tenant_id=1,
node_type="release",
parent_id=None,
project_id=int(project.id),
signal_id=None,
created_by=int(user.id),
title="发布记录",
content={"summary": "上线版本"},
version=1,
status="pending",
)
review = TruthNode(
tenant_id=1,
node_type="review",
parent_id=None,
project_id=int(project.id),
signal_id=None,
created_by=int(user.id),
title="阶段复盘",
content={"summary": "复盘内容"},
version=1,
status="draft",
)
db_session.add_all([charter, release, review])
await db_session.flush()
task = Task(
tenant_id=1,
truth_node_id=int(charter.id),
project_id=int(project.id),
assignee_id=int(user.id),
ai_recommended_assignees=None,
ai_breakdown_payload=None,
manual_overrides=None,
title="实现开票接口",
description="任务描述",
required_skill_tag_ids=[],
status="todo",
priority="high",
estimated_hours=None,
actual_hours=None,
quality_score=None,
due_date=None,
completed_at=None,
)
db_session.add(task)
await db_session.commit()
rows = (
await db_session.execute(
select(IngestEvent).where(
IngestEvent.tenant_id == 1,
IngestEvent.channel == "semantic_fact_extract",
).order_by(IngestEvent.event_key.asc())
)
).scalars().all()
payload_by_key = {str(row.event_key): row.payload for row in rows}
assert payload_by_key["document:%s" % int(doc.id)]["operation"] == "extract"
assert payload_by_key["project:%s" % int(project.id)]["operation"] == "extract"
assert payload_by_key["release:%s" % int(release.id)]["entity_type"] == "release"
assert payload_by_key["requirement:%s" % int(charter.id)]["entity_type"] == "requirement"
assert payload_by_key["review:%s" % int(review.id)]["entity_type"] == "review"
assert payload_by_key["signal:%s" % int(signal.id)]["entity_type"] == "signal"
assert payload_by_key["task:%s" % int(task.id)]["entity_type"] == "task"
assert payload_by_key["user:%s" % int(user.id)]["entity_type"] == "user"
@pytest.mark.asyncio
async def test_semantic_fact_hooks_switch_to_deactivate_for_archive_disable_and_delete(db_session) -> None:
register_semantic_fact_hooks()
user = User(
tenant_id=1,
username="semantic_admin_2",
email="semantic_admin_2@example.com",
password_hash="hash",
name="管理员2",
role="ADMIN",
status="active",
is_active=True,
)
project = Project(
tenant_id=1,
name="alpha",
title="Alpha 项目",
description="项目描述",
owner_id=None,
status="active",
assigned_role="backend",
)
signal = Signal(
tenant_id=1,
title="信号标题",
content="信号内容",
status="待处理",
processed=False,
priority="high",
)
doc = KnowledgeBaseDoc(
tenant_id=1,
title="知识文档",
file_type="md",
source_type="file",
status="complete",
uploaded_by=None,
)
db_session.add_all([user, project, signal, doc])
await db_session.flush()
review = TruthNode(
tenant_id=1,
node_type="review",
parent_id=None,
project_id=int(project.id),
signal_id=None,
created_by=int(user.id),
title="阶段复盘",
content={"summary": "复盘内容"},
version=1,
status="draft",
)
db_session.add(review)
await db_session.commit()
user.is_active = False
user.status = "offline"
signal.status = "已关闭"
review.status = "archived"
await db_session.delete(doc)
await db_session.commit()
rows = (
await db_session.execute(
select(IngestEvent).where(
IngestEvent.tenant_id == 1,
IngestEvent.channel == "semantic_fact_extract",
)
)
).scalars().all()
payload_by_key = {str(row.event_key): row.payload for row in rows}
assert payload_by_key["document:%s" % int(doc.id)]["operation"] == "deactivate"
assert payload_by_key["review:%s" % int(review.id)]["operation"] == "deactivate"
assert payload_by_key["signal:%s" % int(signal.id)]["operation"] == "deactivate"
assert payload_by_key["user:%s" % int(user.id)]["operation"] == "deactivate"

View File

@ -0,0 +1,575 @@
from __future__ import annotations
import pytest
import pytest_asyncio
from sqlalchemy import JSON, select
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
from app.models.base import Base
from app.models.knowledge_base_chunk import KnowledgeBaseChunk
from app.models.knowledge_base_doc import KnowledgeBaseDoc
from app.models.project import Project
from app.models.semantic_fact import SemanticFact
from app.models.semantic_fact_extraction_run import SemanticFactExtractionRun
from app.models.signal import Signal
from app.models.task import Task
from app.models.truth_node import TruthNode
from app.models.user import User
from app.services.adaptive_rag_service.mixin_answer_payload import AdaptiveRAGAnswerPayloadMixin
from app.services.adaptive_rag_service.mixin_fast_answer_project_main import AdaptiveRAGFastAnswerProjectMainMixin
from app.services.adaptive_rag_service.mixin_fast_answer_project_utils import AdaptiveRAGFastAnswerProjectUtilsMixin
from app.services.adaptive_rag_service.mixin_retrieval_router_db import AdaptiveRAGRetrievalRouterDBMixin
from app.services.llm_service import StructuredExtractionResult
from app.services.semantic_fact_service.service import SemanticFactService
def _use_sqlite_json_types(*tables) -> None:
for table in tables:
for column in table.columns:
if column.type.__class__.__name__ in {"JSONB", "ARRAY", "Vector"}:
column.type = JSON()
@pytest_asyncio.fixture
async def db_session():
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
tables = [
User.__table__,
Project.__table__,
Signal.__table__,
TruthNode.__table__,
Task.__table__,
KnowledgeBaseDoc.__table__,
KnowledgeBaseChunk.__table__,
SemanticFact.__table__,
SemanticFactExtractionRun.__table__,
]
_use_sqlite_json_types(*tables)
async with engine.begin() as conn:
await conn.run_sync(lambda sync_conn: Base.metadata.create_all(sync_conn, tables=tables))
maker = async_sessionmaker(engine, expire_on_commit=False)
async with maker() as session:
yield session
await engine.dispose()
@pytest.mark.asyncio
async def test_semantic_fact_service_upserts_and_queries_project_fact(db_session) -> None:
project = Project(
tenant_id=1,
name="a-share-analyzer",
title="A股趋势打分工具",
description="面向证券投研团队,聚焦 A 股公告解析。",
owner_id=None,
status="active",
assigned_role="backend",
)
db_session.add(project)
await db_session.flush()
service = SemanticFactService(db_session, tenant_id=1)
await service.upsert_fact(
entity_type="project",
entity_id=int(project.id),
fact_namespace="theme",
fact_key="market_theme",
fact_value_text="A股",
fact_value_norm="a_share",
value_type="keyword",
confidence=0.93,
evidence_source_type="project",
evidence_source_id=int(project.id),
evidence_excerpt="聚焦 A 股公告解析",
extract_version="semantic-facts-v1",
freshness_at=None,
)
await db_session.commit()
rows = await service.query_facts(
entity_type="project",
query_type="existence",
filters=[{"fact_key": "market_theme", "value": "A股", "normalized_value": "a_share"}],
)
assert len(rows) == 1
assert rows[0].fact_key == "market_theme"
assert rows[0].fact_value_norm == "a_share"
@pytest.mark.asyncio
async def test_semantic_fact_service_extracts_project_facts_from_llm_payload(monkeypatch, db_session) -> None:
project = Project(
tenant_id=1,
name="invoice-helper",
title="开票助手",
description="面向财务运营,自动生成增值税发票草稿。",
owner_id=None,
status="active",
assigned_role="backend",
)
db_session.add(project)
await db_session.commit()
async def _fake_generate_structured_object(prompt: str, **kwargs):
_ = (prompt, kwargs)
return StructuredExtractionResult(
payload={
"facts": [
{
"fact_namespace": "domain",
"fact_key": "business_domain",
"fact_value_text": "开票系统",
"fact_value_norm": "invoicing",
"value_type": "keyword",
"confidence": 0.94,
"evidence_excerpt": "自动生成增值税发票草稿",
},
{
"fact_namespace": "alias",
"fact_key": "aliases",
"fact_value_text": "开票系统",
"fact_value_norm": "invoice_system",
"value_type": "keyword",
"confidence": 0.9,
"evidence_excerpt": "开票助手",
},
]
},
source_field="payload",
provider="test",
model="test-model",
validation="valid",
)
monkeypatch.setattr(
"app.services.llm_service.llm_service.generate_structured_object",
_fake_generate_structured_object,
)
service = SemanticFactService(db_session, tenant_id=1)
run = await service.extract_project_facts(project_id=int(project.id), trigger_type="manual_retry")
await db_session.commit()
facts = (
await db_session.execute(
select(SemanticFact).where(
SemanticFact.tenant_id == 1,
SemanticFact.entity_type == "project",
SemanticFact.entity_id == int(project.id),
)
)
).scalars().all()
run_row = (
await db_session.execute(
select(SemanticFactExtractionRun).where(
SemanticFactExtractionRun.tenant_id == 1,
SemanticFactExtractionRun.id == int(run.id),
)
)
).scalar_one()
assert len(facts) == 2
assert {row.fact_key for row in facts} == {"business_domain", "aliases"}
assert str(run_row.status) == "completed"
assert int(run_row.facts_written or 0) == 2
@pytest.mark.asyncio
async def test_semantic_fact_service_extract_entity_facts_replaces_active_rows_for_requirement(monkeypatch, db_session) -> None:
user = User(
tenant_id=1,
username="semantic_owner",
email="semantic_owner@example.com",
password_hash="hash",
name="语义负责人",
role="ADMIN",
status="active",
is_active=True,
)
db_session.add(user)
await db_session.flush()
charter = TruthNode(
tenant_id=1,
node_type="charter",
parent_id=None,
project_id=None,
signal_id=None,
created_by=int(user.id),
title="开票系统需求",
content={"summary": "为财务团队建设开票平台"},
version=1,
status="draft",
)
db_session.add(charter)
await db_session.commit()
payloads = [
{
"facts": [
{
"fact_namespace": "alias",
"fact_key": "aliases",
"fact_value_text": "开票系统",
"fact_value_norm": "invoice_system",
"value_type": "keyword",
"confidence": 0.93,
"evidence_excerpt": "建设开票平台",
}
]
},
{
"facts": [
{
"fact_namespace": "alias",
"fact_key": "aliases",
"fact_value_text": "财务开票平台",
"fact_value_norm": "finance_invoice_platform",
"value_type": "keyword",
"confidence": 0.95,
"evidence_excerpt": "建设开票平台",
}
]
},
]
async def _fake_generate_structured_object(prompt: str, **kwargs):
_ = (prompt, kwargs)
return StructuredExtractionResult(
payload=payloads.pop(0),
source_field="payload",
provider="test",
model="test-model",
validation="valid",
)
monkeypatch.setattr(
"app.services.llm_service.llm_service.generate_structured_object",
_fake_generate_structured_object,
)
service = SemanticFactService(db_session, tenant_id=1)
first_run = await service.extract_entity_facts(
entity_type="requirement",
entity_id=int(charter.id),
trigger_type="manual_retry",
)
second_run = await service.extract_entity_facts(
entity_type="requirement",
entity_id=int(charter.id),
trigger_type="manual_retry",
)
await db_session.commit()
all_rows = (
await db_session.execute(
select(SemanticFact).where(
SemanticFact.tenant_id == 1,
SemanticFact.entity_type == "requirement",
SemanticFact.entity_id == int(charter.id),
).order_by(SemanticFact.id.asc())
)
).scalars().all()
active_rows = [row for row in all_rows if row.is_active]
assert str(first_run.status) == "completed"
assert str(second_run.status) == "completed"
assert len(all_rows) == 2
assert len(active_rows) == 1
assert active_rows[0].fact_value_text == "财务开票平台"
assert all_rows[0].is_active is False
@pytest.mark.asyncio
async def test_semantic_fact_service_deactivate_entity_facts_marks_rows_inactive(db_session) -> None:
project = Project(
tenant_id=1,
name="market-tool",
title="趋势分析工具",
description="这是一个证券行业分析工具。",
owner_id=None,
status="active",
assigned_role="backend",
)
db_session.add(project)
await db_session.flush()
service = SemanticFactService(db_session, tenant_id=1)
await service.upsert_fact(
entity_type="project",
entity_id=int(project.id),
fact_namespace="theme",
fact_key="market_theme",
fact_value_text="A股",
fact_value_norm="a_share",
value_type="keyword",
confidence=0.95,
evidence_source_type="project",
evidence_source_id=int(project.id),
evidence_excerpt="A股",
extract_version="semantic-facts-v1",
freshness_at=None,
)
await db_session.commit()
deactivated = await service.deactivate_entity_facts(entity_type="project", entity_id=int(project.id))
await db_session.commit()
rows = (
await db_session.execute(
select(SemanticFact).where(
SemanticFact.tenant_id == 1,
SemanticFact.entity_type == "project",
SemanticFact.entity_id == int(project.id),
)
)
).scalars().all()
assert deactivated == 1
assert rows
assert all(row.is_active is False for row in rows)
@pytest.mark.asyncio
async def test_project_existence_answer_prefers_semantic_facts_before_text_fallback(db_session) -> None:
project = Project(
tenant_id=1,
name="market-tool",
title="趋势分析工具",
description="这是一个证券行业分析工具。",
owner_id=None,
status="active",
assigned_role="backend",
)
db_session.add(project)
await db_session.flush()
fact_service = SemanticFactService(db_session, tenant_id=1)
await fact_service.upsert_fact(
entity_type="project",
entity_id=int(project.id),
fact_namespace="theme",
fact_key="market_theme",
fact_value_text="A股",
fact_value_norm="a_share",
value_type="keyword",
confidence=0.95,
evidence_source_type="project",
evidence_source_id=int(project.id),
evidence_excerpt="A股",
extract_version="semantic-facts-v1",
freshness_at=None,
)
await db_session.commit()
class _FakeService(
AdaptiveRAGFastAnswerProjectMainMixin,
AdaptiveRAGFastAnswerProjectUtilsMixin,
):
def __init__(self) -> None:
self.db = db_session
self.tenant_id = 1
@staticmethod
def _extract_project_lookup_keyword(question: str) -> str: # pragma: no cover - should not be called
raise AssertionError(f"lexical keyword fallback should be skipped: {question}")
@staticmethod
def _extract_query_terms(question: str, *, preferred: str = "") -> list[str]: # pragma: no cover - should not be called
raise AssertionError(f"lexical term fallback should be skipped: {question}, {preferred}")
service = _FakeService()
result = await service._try_fast_project_existence_answer(
clean_q="系统里有没有A股项目?",
project_id=None,
query_semantics={
"subject_focus": "project",
"scope_hint": "global",
"query_type": "existence",
"operation_contract": {
"query_type": "existence",
"object_mentions": ["A股项目"],
},
},
)
assert result is not None
assert "趋势分析工具" in str(result["answer"])
assert str(result["sources"][0]["metadata"]["table"]) == "semantic_facts"
@pytest.mark.asyncio
async def test_structured_db_retrieval_prefers_semantic_fact_context_for_project_queries(db_session) -> None:
project = Project(
tenant_id=1,
name="market-tool",
title="趋势分析工具",
description="这是一个证券行业分析工具。",
owner_id=None,
status="active",
assigned_role="backend",
)
db_session.add(project)
await db_session.flush()
fact_service = SemanticFactService(db_session, tenant_id=1)
await fact_service.upsert_fact(
entity_type="project",
entity_id=int(project.id),
fact_namespace="theme",
fact_key="market_theme",
fact_value_text="A股",
fact_value_norm="a_share",
value_type="keyword",
confidence=0.95,
evidence_source_type="project",
evidence_source_id=int(project.id),
evidence_excerpt="A股",
extract_version="semantic-facts-v1",
freshness_at=None,
)
await db_session.commit()
class _FakeService(
AdaptiveRAGRetrievalRouterDBMixin,
AdaptiveRAGFastAnswerProjectUtilsMixin,
AdaptiveRAGAnswerPayloadMixin,
):
def __init__(self) -> None:
self.db = db_session
self.tenant_id = 1
@staticmethod
def _extract_project_lookup_keyword(question: str) -> str: # pragma: no cover - should not be called
raise AssertionError(f"lexical keyword fallback should be skipped: {question}")
@staticmethod
def _extract_query_terms(question: str, *, preferred: str = "") -> list[str]: # pragma: no cover - should not be called
raise AssertionError(f"lexical term fallback should be skipped: {question}, {preferred}")
service = _FakeService()
rows = await service._retrieve_structured_db(
"系统里有没有A股项目?",
project_id=None,
query_semantics={
"subject_focus": "project",
"scope_hint": "global",
"query_type": "existence",
"operation_contract": {
"query_type": "existence",
"object_mentions": ["A股项目"],
},
},
)
assert rows
assert rows[0].metadata["table"] == "semantic_facts"
assert "趋势分析工具" in rows[0].content
@pytest.mark.asyncio
async def test_structured_db_retrieval_prefers_semantic_fact_context_for_task_queries(db_session) -> None:
user = User(
tenant_id=1,
username="task_owner",
email="task_owner@example.com",
password_hash="hash",
name="任务负责人",
role="ADMIN",
status="active",
is_active=True,
)
project = Project(
tenant_id=1,
name="invoice-project",
title="开票平台项目",
description="项目描述",
owner_id=None,
status="active",
assigned_role="backend",
)
db_session.add_all([user, project])
await db_session.flush()
charter = TruthNode(
tenant_id=1,
node_type="charter",
parent_id=None,
project_id=int(project.id),
signal_id=None,
created_by=int(user.id),
title="开票需求",
content={"summary": "建设开票系统"},
version=1,
status="draft",
)
db_session.add(charter)
await db_session.flush()
task = Task(
tenant_id=1,
truth_node_id=int(charter.id),
project_id=int(project.id),
assignee_id=int(user.id),
ai_recommended_assignees=None,
ai_breakdown_payload=None,
manual_overrides=None,
title="实现开票接口",
description="对接发票创建能力",
required_skill_tag_ids=[],
status="todo",
priority="high",
estimated_hours=None,
actual_hours=None,
quality_score=None,
due_date=None,
completed_at=None,
)
db_session.add(task)
await db_session.flush()
fact_service = SemanticFactService(db_session, tenant_id=1)
await fact_service.upsert_fact(
entity_type="task",
entity_id=int(task.id),
fact_namespace="alias",
fact_key="aliases",
fact_value_text="开票任务",
fact_value_norm="invoice_task",
value_type="keyword",
confidence=0.96,
evidence_source_type="task",
evidence_source_id=int(task.id),
evidence_excerpt="实现开票接口",
extract_version="semantic-facts-v1",
freshness_at=None,
)
await db_session.commit()
class _FakeService(
AdaptiveRAGRetrievalRouterDBMixin,
AdaptiveRAGFastAnswerProjectUtilsMixin,
AdaptiveRAGAnswerPayloadMixin,
):
def __init__(self) -> None:
self.db = db_session
self.tenant_id = 1
service = _FakeService()
rows = await service._retrieve_structured_db(
"有没有开票任务?",
project_id=None,
query_semantics={
"subject_focus": "task",
"scope_hint": "global",
"query_type": "existence",
"operation_contract": {
"query_type": "existence",
"object_mentions": ["开票任务"],
},
},
)
assert rows
assert rows[0].metadata["table"] == "semantic_facts"
assert "开票任务" in rows[0].content

View File

@ -48,6 +48,64 @@ def test_semantic_lint_flags_workflow_semantic_planner_directory(tmp_path) -> No
assert problems[0].rule_id == "semantic_hint_constant" assert problems[0].rule_id == "semantic_hint_constant"
def test_semantic_lint_flags_query_route_object_tables_in_adaptive_rag(tmp_path) -> None:
module = _load_semantic_lint_module()
target = tmp_path / "backend" / "app" / "services" / "adaptive_rag_service" / "route_gateway.py"
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text('_INTERNAL_ROUTE_OBJECTS = {"system_config"}\n', encoding="utf-8")
problems = module.collect_problems(
root=tmp_path,
scan_roots=(Path("backend/app/services/adaptive_rag_service"),),
baseline_path=Path("scripts/semantic_lint_baseline.txt"),
)
assert len(problems) == 1
assert problems[0].rule_id == "semantic_route_object_table"
def test_semantic_lint_flags_query_route_probe_helpers_in_adaptive_rag(tmp_path) -> None:
module = _load_semantic_lint_module()
target = tmp_path / "backend" / "app" / "services" / "adaptive_rag_service" / "route_gateway.py"
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text("def _external_probe_is_strong(snapshot):\n return bool(snapshot)\n", encoding="utf-8")
problems = module.collect_problems(
root=tmp_path,
scan_roots=(Path("backend/app/services/adaptive_rag_service"),),
baseline_path=Path("scripts/semantic_lint_baseline.txt"),
)
assert len(problems) == 1
assert problems[0].rule_id == "semantic_route_probe_helper"
def test_semantic_lint_flags_external_object_kind_mapping(tmp_path) -> None:
module = _load_semantic_lint_module()
target = tmp_path / "backend" / "app" / "services" / "external_query_service" / "planner.py"
target.parent.mkdir(parents=True, exist_ok=True)
target.write_text(
'if "weather" in referenced_objects:\n query_kind = "weather"\n',
encoding="utf-8",
)
problems = module.collect_problems(
root=tmp_path,
scan_roots=(Path("backend/app/services/external_query_service"),),
baseline_path=Path("scripts/semantic_lint_baseline.txt"),
)
assert len(problems) == 1
assert problems[0].rule_id == "semantic_object_kind_mapping"
def test_semantic_lint_default_roots_cover_adaptive_rag_and_external_query() -> None:
module = _load_semantic_lint_module()
assert Path("backend/app/services/adaptive_rag_service") in module.DEFAULT_SCAN_ROOTS
assert Path("backend/app/services/external_query_service") in module.DEFAULT_SCAN_ROOTS
def test_semantic_lint_repo_has_no_compat_baseline_exceptions() -> None: def test_semantic_lint_repo_has_no_compat_baseline_exceptions() -> None:
repo_root = Path(__file__).resolve().parents[2] repo_root = Path(__file__).resolve().parents[2]
baseline_path = repo_root / "scripts" / "semantic_lint_baseline.txt" baseline_path = repo_root / "scripts" / "semantic_lint_baseline.txt"

View File

@ -3,3 +3,5 @@
{"generated_at": "2026-03-28T08:52:16.466761", "case": "challenge", "source_message_id": "codex-feishu-challenge-1774659109", "question": "我要把整个系统一次性全部重构掉,这么做靠谱吗", "run_status": "missing", "processing": {"status": "success", "route": "ai_pmo.query.answer", "intent": "query.answer", "reply_source": "action_succeeded", "error_message": ""}, "outbox": [{"id": 655, "status": "sent", "attempts": 1, "sent_message_id": "om_x100b53462dfeb8a4b4a22d394e5186c", "text": "不靠谱。 \n依据:当前系统后端模块统计为全零(api/services/models/schemas均为0),但热点模块集中在`backend/app/api/sstc.py`等5个文件,说明核心逻辑实际集中、未模块化;前端仅6个路由,功能边界清晰;156个待处理信号中P0级3个、P1级46个,系统处于高负载运行态。 \n下一步:立即暂停重构决策,优先完成`sstc.py`模块拆解图谱与接口契约定义,并确认杨宇(唯一跨角色开发者,workload_ratio=796.67)是否参与重构设计。"}], "final_reply": "不靠谱。 \n依据:当前系统后端模块统计为全零(api/services/models/schemas均为0),但热点模块集中在`backend/app/api/sstc.py`等5个文件,说明核心逻辑实际集中、未模块化;前端仅6个路由,功能边界清晰;156个待处理信号中P0级3个、P1级46个,系统处于高负载运行态。 \n下一步:立即暂停重构决策,优先完成`sstc.py`模块拆解图谱与接口契约定义,并确认杨宇(唯一跨角色开发者,workload_ratio=796.67)是否参与重构设计。"} {"generated_at": "2026-03-28T08:52:16.466761", "case": "challenge", "source_message_id": "codex-feishu-challenge-1774659109", "question": "我要把整个系统一次性全部重构掉,这么做靠谱吗", "run_status": "missing", "processing": {"status": "success", "route": "ai_pmo.query.answer", "intent": "query.answer", "reply_source": "action_succeeded", "error_message": ""}, "outbox": [{"id": 655, "status": "sent", "attempts": 1, "sent_message_id": "om_x100b53462dfeb8a4b4a22d394e5186c", "text": "不靠谱。 \n依据:当前系统后端模块统计为全零(api/services/models/schemas均为0),但热点模块集中在`backend/app/api/sstc.py`等5个文件,说明核心逻辑实际集中、未模块化;前端仅6个路由,功能边界清晰;156个待处理信号中P0级3个、P1级46个,系统处于高负载运行态。 \n下一步:立即暂停重构决策,优先完成`sstc.py`模块拆解图谱与接口契约定义,并确认杨宇(唯一跨角色开发者,workload_ratio=796.67)是否参与重构设计。"}], "final_reply": "不靠谱。 \n依据:当前系统后端模块统计为全零(api/services/models/schemas均为0),但热点模块集中在`backend/app/api/sstc.py`等5个文件,说明核心逻辑实际集中、未模块化;前端仅6个路由,功能边界清晰;156个待处理信号中P0级3个、P1级46个,系统处于高负载运行态。 \n下一步:立即暂停重构决策,优先完成`sstc.py`模块拆解图谱与接口契约定义,并确认杨宇(唯一跨角色开发者,workload_ratio=796.67)是否参与重构设计。"}
{"generated_at": "2026-03-28T08:53:35.385412", "case": "brainstorm", "source_message_id": "codex-feishu-brainstorm-1774659145", "question": "我们一起想几个 AI 劝退助手的方向", "run_status": "missing", "processing": {"status": "success", "route": "ai_pmo.draft_pending_confirm", "intent": "project_signal.create", "reply_source": "draft_pending_confirm", "error_message": ""}, "outbox": [{"id": 656, "status": "sent", "attempts": 1, "sent_message_id": "om_x100b5346298338a4b2c4f90aaa6bc9f", "text": "📋 我先按产品经理视角起草一版评审建议,请确认:\n\n📌 需求草稿\n我们一起想几个 AI 劝退助手的方向\n\n🔎 同名 / 相似项目:\n· #51 AI劝退助手:现有项目已有一定描述,建议对比当前需求的目标、范围和交付边界。;若范围重叠,可能造成重复建设或维护分散。;名称或目标高度接近,优先确认是否复用或合并。\n\n🚀 MVP 建议:\n· 明确项目目标与边界\n· 完成同名/相似项目查重\n· 交付最小可验证闭环\n\n一、定位\n· 一句话描述:我们一起想几个 AI 劝退助手的方向\n· 目标客户:待补充\n\n二、核心痛点\n· 待补充核心痛点\n\n三、MVP 功能范围(8周交付)\nPhase 1:核心功能(Week 1-4)\n· 明确项目目标与边界\n· 完成同名/相似项目查重\n· 交付最小可验证闭环\nPhase 2:差异化功能(Week 5-8)\n· 待补充:确认差异化方向后再展开\n\n四、技术架构(精简版)\n· 待补充技术架构\n\n五、商业模式\n· 待补充商业模式\n\n六、验证指标\n验证指标:\n· 待补充\n\n七、快速启动建议\n快速启动建议:\n· 待补充\n\n八、团队配置\n· 待补充团队配置\n\n📈 前景判断:\n· 该项目具备落地价值,但建议先确认范围边界和现有项目复用情况,再决定是否推进\n\n🧩 需求清单:\n· 项目定位与目标\n· MVP 功能范围\n\n❓ 还缺的信息:\n· 暂无\n\n✅ 可行性:待讨论\n信息仍需补充,建议先确认边界。\n\n👤 推荐负责人:系统管理员 · 当前负载100%\n基于当前负载与任务匹配度给出的建议。\n\n📅 跟进计划:\n· 明天:确认触发时机与验收口径\n· 一周后:复核执行效果并决定是否扩展\n\n---\n回复 确认 → 入库\n回复 修改 [具体修改意见] → 重新起草\n回复 取消 → 放弃\n⏰ 24小时内未确认自动取消"}], "final_reply": "📋 我先按产品经理视角起草一版评审建议,请确认:\n\n📌 需求草稿\n我们一起想几个 AI 劝退助手的方向\n\n🔎 同名 / 相似项目:\n· #51 AI劝退助手:现有项目已有一定描述,建议对比当前需求的目标、范围和交付边界。;若范围重叠,可能造成重复建设或维护分散。;名称或目标高度接近,优先确认是否复用或合并。\n\n🚀 MVP 建议:\n· 明确项目目标与边界\n· 完成同名/相似项目查重\n· 交付最小可验证闭环\n\n一、定位\n· 一句话描述:我们一起想几个 AI 劝退助手的方向\n· 目标客户:待补充\n\n二、核心痛点\n· 待补充核心痛点\n\n三、MVP 功能范围(8周交付)\nPhase 1:核心功能(Week 1-4)\n· 明确项目目标与边界\n· 完成同名/相似项目查重\n· 交付最小可验证闭环\nPhase 2:差异化功能(Week 5-8)\n· 待补充:确认差异化方向后再展开\n\n四、技术架构(精简版)\n· 待补充技术架构\n\n五、商业模式\n· 待补充商业模式\n\n六、验证指标\n验证指标:\n· 待补充\n\n七、快速启动建议\n快速启动建议:\n· 待补充\n\n八、团队配置\n· 待补充团队配置\n\n📈 前景判断:\n· 该项目具备落地价值,但建议先确认范围边界和现有项目复用情况,再决定是否推进\n\n🧩 需求清单:\n· 项目定位与目标\n· MVP 功能范围\n\n❓ 还缺的信息:\n· 暂无\n\n✅ 可行性:待讨论\n信息仍需补充,建议先确认边界。\n\n👤 推荐负责人:系统管理员 · 当前负载100%\n基于当前负载与任务匹配度给出的建议。\n\n📅 跟进计划:\n· 明天:确认触发时机与验收口径\n· 一周后:复核执行效果并决定是否扩展\n\n---\n回复 确认 → 入库\n回复 修改 [具体修改意见] → 重新起草\n回复 取消 → 放弃\n⏰ 24小时内未确认自动取消"} {"generated_at": "2026-03-28T08:53:35.385412", "case": "brainstorm", "source_message_id": "codex-feishu-brainstorm-1774659145", "question": "我们一起想几个 AI 劝退助手的方向", "run_status": "missing", "processing": {"status": "success", "route": "ai_pmo.draft_pending_confirm", "intent": "project_signal.create", "reply_source": "draft_pending_confirm", "error_message": ""}, "outbox": [{"id": 656, "status": "sent", "attempts": 1, "sent_message_id": "om_x100b5346298338a4b2c4f90aaa6bc9f", "text": "📋 我先按产品经理视角起草一版评审建议,请确认:\n\n📌 需求草稿\n我们一起想几个 AI 劝退助手的方向\n\n🔎 同名 / 相似项目:\n· #51 AI劝退助手:现有项目已有一定描述,建议对比当前需求的目标、范围和交付边界。;若范围重叠,可能造成重复建设或维护分散。;名称或目标高度接近,优先确认是否复用或合并。\n\n🚀 MVP 建议:\n· 明确项目目标与边界\n· 完成同名/相似项目查重\n· 交付最小可验证闭环\n\n一、定位\n· 一句话描述:我们一起想几个 AI 劝退助手的方向\n· 目标客户:待补充\n\n二、核心痛点\n· 待补充核心痛点\n\n三、MVP 功能范围(8周交付)\nPhase 1:核心功能(Week 1-4)\n· 明确项目目标与边界\n· 完成同名/相似项目查重\n· 交付最小可验证闭环\nPhase 2:差异化功能(Week 5-8)\n· 待补充:确认差异化方向后再展开\n\n四、技术架构(精简版)\n· 待补充技术架构\n\n五、商业模式\n· 待补充商业模式\n\n六、验证指标\n验证指标:\n· 待补充\n\n七、快速启动建议\n快速启动建议:\n· 待补充\n\n八、团队配置\n· 待补充团队配置\n\n📈 前景判断:\n· 该项目具备落地价值,但建议先确认范围边界和现有项目复用情况,再决定是否推进\n\n🧩 需求清单:\n· 项目定位与目标\n· MVP 功能范围\n\n❓ 还缺的信息:\n· 暂无\n\n✅ 可行性:待讨论\n信息仍需补充,建议先确认边界。\n\n👤 推荐负责人:系统管理员 · 当前负载100%\n基于当前负载与任务匹配度给出的建议。\n\n📅 跟进计划:\n· 明天:确认触发时机与验收口径\n· 一周后:复核执行效果并决定是否扩展\n\n---\n回复 确认 → 入库\n回复 修改 [具体修改意见] → 重新起草\n回复 取消 → 放弃\n⏰ 24小时内未确认自动取消"}], "final_reply": "📋 我先按产品经理视角起草一版评审建议,请确认:\n\n📌 需求草稿\n我们一起想几个 AI 劝退助手的方向\n\n🔎 同名 / 相似项目:\n· #51 AI劝退助手:现有项目已有一定描述,建议对比当前需求的目标、范围和交付边界。;若范围重叠,可能造成重复建设或维护分散。;名称或目标高度接近,优先确认是否复用或合并。\n\n🚀 MVP 建议:\n· 明确项目目标与边界\n· 完成同名/相似项目查重\n· 交付最小可验证闭环\n\n一、定位\n· 一句话描述:我们一起想几个 AI 劝退助手的方向\n· 目标客户:待补充\n\n二、核心痛点\n· 待补充核心痛点\n\n三、MVP 功能范围(8周交付)\nPhase 1:核心功能(Week 1-4)\n· 明确项目目标与边界\n· 完成同名/相似项目查重\n· 交付最小可验证闭环\nPhase 2:差异化功能(Week 5-8)\n· 待补充:确认差异化方向后再展开\n\n四、技术架构(精简版)\n· 待补充技术架构\n\n五、商业模式\n· 待补充商业模式\n\n六、验证指标\n验证指标:\n· 待补充\n\n七、快速启动建议\n快速启动建议:\n· 待补充\n\n八、团队配置\n· 待补充团队配置\n\n📈 前景判断:\n· 该项目具备落地价值,但建议先确认范围边界和现有项目复用情况,再决定是否推进\n\n🧩 需求清单:\n· 项目定位与目标\n· MVP 功能范围\n\n❓ 还缺的信息:\n· 暂无\n\n✅ 可行性:待讨论\n信息仍需补充,建议先确认边界。\n\n👤 推荐负责人:系统管理员 · 当前负载100%\n基于当前负载与任务匹配度给出的建议。\n\n📅 跟进计划:\n· 明天:确认触发时机与验收口径\n· 一周后:复核执行效果并决定是否扩展\n\n---\n回复 确认 → 入库\n回复 修改 [具体修改意见] → 重新起草\n回复 取消 → 放弃\n⏰ 24小时内未确认自动取消"}
{"generated_at": "2026-04-07T00:07:35.291848", "idx": 9001, "source_message_id": "real-scene-p1-manual-9001-1775491344", "question": "跟你聊个新需求,关于换季系统自动推荐搭配衣服", "message": "跟你聊个新需求,关于换季系统自动推荐搭配衣服", "capture_status": "answered", "capture_direct_reply_text": "可以,我们先不创建项目,先把这件事收成一版需求。\n\n我先把结果主要给谁用定一下,这会直接影响后面的内容粒度和表达方式。\n\nA. 核心执行人 / 内部 PM\n更适合:先服务负责跟进、拆动作、推动落地的人\n\nB. 负责人 / 管理者\n更适合:先看结论、风险和待拍板项\n\nC. 整个团队 / 终端用户\n更适合:需要直接同步所有相关方,或结果就是给最终使用者看的\n\n你回 A / B / C,也可以直接说更准确的对象。", "consultant_dispatch": null, "run_status": "missing", "ack": "", "final_reply": "可以,我们先不创建项目,先把这件事收成一版需求。\n\n我先把结果主要给谁用定一下,这会直接影响后面的内容粒度和表达方式。\n\nA. 核心执行人 / 内部 PM\n更适合:先服务负责跟进、拆动作、推动落地的人\n\nB. 负责人 / 管理者\n更适合:先看结论、风险和待拍板项\n\nC. 整个团队 / 终端用户\n更适合:需要直接同步所有相关方,或结果就是给最终使用者看的\n\n你回 A / B / C,也可以直接说更准确的对象。", "prompt_scene": "unknown", "inferred_scene": "unknown", "actual_scene": "unknown", "thinking_leak": false, "has_capacity_warning": false, "processing": {"route": "ai_pmo.query.answer", "intent": "query.answer", "status": "success", "duration_ms": 9984, "error_message": "", "reply_source": "action_succeeded"}, "outbox": [{"id": 966, "status": "pending", "attempts": 0, "created_at": "2026-04-07 00:02:34.138267", "text": "可以,我们先不创建项目,先把这件事收成一版需求。\n\n我先把结果主要给谁用定一下,这会直接影响后面的内容粒度和表达方式。\n\nA. 核心执行人 / 内部 PM\n更适合:先服务负责跟进、拆动作、推动落地的人\n\nB. 负责人 / 管理者\n更适合:先看结论、风险和待拍板项\n\nC. 整个团队 / 终端用户\n更适合:需要直接同步所有相关方,或结果就是给最终使用者看的\n\n你回 A / B / C,也可以直接说更准确的对象。"}], "call_log_candidates": [], "pass": false, "pass_rule": "unknown"} {"generated_at": "2026-04-07T00:07:35.291848", "idx": 9001, "source_message_id": "real-scene-p1-manual-9001-1775491344", "question": "跟你聊个新需求,关于换季系统自动推荐搭配衣服", "message": "跟你聊个新需求,关于换季系统自动推荐搭配衣服", "capture_status": "answered", "capture_direct_reply_text": "可以,我们先不创建项目,先把这件事收成一版需求。\n\n我先把结果主要给谁用定一下,这会直接影响后面的内容粒度和表达方式。\n\nA. 核心执行人 / 内部 PM\n更适合:先服务负责跟进、拆动作、推动落地的人\n\nB. 负责人 / 管理者\n更适合:先看结论、风险和待拍板项\n\nC. 整个团队 / 终端用户\n更适合:需要直接同步所有相关方,或结果就是给最终使用者看的\n\n你回 A / B / C,也可以直接说更准确的对象。", "consultant_dispatch": null, "run_status": "missing", "ack": "", "final_reply": "可以,我们先不创建项目,先把这件事收成一版需求。\n\n我先把结果主要给谁用定一下,这会直接影响后面的内容粒度和表达方式。\n\nA. 核心执行人 / 内部 PM\n更适合:先服务负责跟进、拆动作、推动落地的人\n\nB. 负责人 / 管理者\n更适合:先看结论、风险和待拍板项\n\nC. 整个团队 / 终端用户\n更适合:需要直接同步所有相关方,或结果就是给最终使用者看的\n\n你回 A / B / C,也可以直接说更准确的对象。", "prompt_scene": "unknown", "inferred_scene": "unknown", "actual_scene": "unknown", "thinking_leak": false, "has_capacity_warning": false, "processing": {"route": "ai_pmo.query.answer", "intent": "query.answer", "status": "success", "duration_ms": 9984, "error_message": "", "reply_source": "action_succeeded"}, "outbox": [{"id": 966, "status": "pending", "attempts": 0, "created_at": "2026-04-07 00:02:34.138267", "text": "可以,我们先不创建项目,先把这件事收成一版需求。\n\n我先把结果主要给谁用定一下,这会直接影响后面的内容粒度和表达方式。\n\nA. 核心执行人 / 内部 PM\n更适合:先服务负责跟进、拆动作、推动落地的人\n\nB. 负责人 / 管理者\n更适合:先看结论、风险和待拍板项\n\nC. 整个团队 / 终端用户\n更适合:需要直接同步所有相关方,或结果就是给最终使用者看的\n\n你回 A / B / C,也可以直接说更准确的对象。"}], "call_log_candidates": [], "pass": false, "pass_rule": "unknown"}
{"generated_at": "2026-04-16T00:59:02.932070", "idx": 99, "source_message_id": "feishu-weather-e2e-99-1776271963", "question": "湛江市未来五天的天气", "message": "湛江市未来五天的天气", "capture_status": "answered", "capture_direct_reply_text": "当前正在需求模式,这条线程先只处理“当前这条需求”的收敛。你可以继续回 A / B / C / D、补一句、出草稿、重出这轮;如果要聊别的,请先回复“退出需求模式”。", "consultant_dispatch": null, "run_status": "missing", "ack": "", "final_reply": "当前正在需求模式,这条线程先只处理“当前这条需求”的收敛。你可以继续回 A / B / C / D、补一句、出草稿、重出这轮;如果要聊别的,请先回复“退出需求模式”。", "prompt_scene": "unknown", "inferred_scene": "unknown", "actual_scene": "unknown", "thinking_leak": false, "has_capacity_warning": false, "processing": {"route": "ai_pmo.query.answer", "intent": "query.answer", "status": "success", "duration_ms": 41383, "error_message": "", "reply_source": "action_succeeded"}, "outbox": [{"id": 1146, "status": "pending", "attempts": 0, "created_at": "2026-04-16 00:54:01.540858", "text": "当前正在需求模式,这条线程先只处理“当前这条需求”的收敛。你可以继续回 A / B / C / D、补一句、出草稿、重出这轮;如果要聊别的,请先回复“退出需求模式”。"}], "call_log_candidates": [], "pass": false, "pass_rule": "unknown"}
{"generated_at": "2026-04-16T10:04:28.903335", "idx": 901, "source_message_id": "codex-feishu-weather-901-1776304695", "question": "湛江市未来五天的天气", "message": "湛江市未来五天的天气", "capture_status": "answered", "capture_direct_reply_text": "当前正在需求模式,这条线程先只处理“当前这条需求”的收敛。你可以继续回 A / B / C / D、补一句、出草稿、重出这轮;如果要聊别的,请先回复“退出需求模式”。", "consultant_dispatch": null, "run_status": "missing", "ack": "", "final_reply": "当前正在需求模式,这条线程先只处理“当前这条需求”的收敛。你可以继续回 A / B / C / D、补一句、出草稿、重出这轮;如果要聊别的,请先回复“退出需求模式”。", "prompt_scene": "unknown", "inferred_scene": "unknown", "actual_scene": "unknown", "thinking_leak": false, "has_capacity_warning": false, "processing": {"route": "ai_pmo.query.answer", "intent": "query.answer", "status": "success", "duration_ms": 56505, "error_message": "", "reply_source": "action_succeeded"}, "outbox": [{"id": 1151, "status": "pending", "attempts": 0, "created_at": "2026-04-16 09:59:27.962651", "text": "当前正在需求模式,这条线程先只处理“当前这条需求”的收敛。你可以继续回 A / B / C / D、补一句、出草稿、重出这轮;如果要聊别的,请先回复“退出需求模式”。"}], "call_log_candidates": [{"id": 6075, "scene": "unknown", "function": "adaptive_rag_answer_normal", "created_at": "2026-04-16 10:02:19.531104"}], "pass": false, "pass_rule": "unknown"}

File diff suppressed because one or more lines are too long

View File

@ -25,6 +25,7 @@
- `docs/architecture/DATAFLOW.md`:主链与数据流说明 - `docs/architecture/DATAFLOW.md`:主链与数据流说明
- `docs/architecture/TASK_BRIEF_TEMPLATE.md`:任务包模板 - `docs/architecture/TASK_BRIEF_TEMPLATE.md`:任务包模板
- `docs/superpowers/plans/`:实现计划 - `docs/superpowers/plans/`:实现计划
- `docs/superpowers/specs/`:需求/方案规格
- `docs/task_briefs/`:任务包产物与验收截图 - `docs/task_briefs/`:任务包产物与验收截图
## 4. 后端库存 ## 4. 后端库存
@ -37,6 +38,7 @@
- `backend/app/api/sstc_api/` - `backend/app/api/sstc_api/`
- `backend/app/api/signals_api/` - `backend/app/api/signals_api/`
- `backend/app/api/signal_sources_api/` - `backend/app/api/signal_sources_api/`
- `backend/app/api/semantic_facts_api/`
- `backend/app/api/skill_tags_api/` - `backend/app/api/skill_tags_api/`
- `backend/app/api/ai_logs_api/` - `backend/app/api/ai_logs_api/`
@ -51,6 +53,9 @@
### 4.3 问答、模型与生成 ### 4.3 问答、模型与生成
- `backend/app/services/adaptive_rag_service/` - `backend/app/services/adaptive_rag_service/`
- `backend/app/services/adaptive_rag_service/mixin_query_route_gateway.py`
- `backend/app/services/semantic_fact_service/`
- `backend/app/services/semantic_fact_service/hooks.py`
- `backend/app/services/llm_service/` - `backend/app/services/llm_service/`
- `backend/app/services/ai_steward_service/` - `backend/app/services/ai_steward_service/`
- `backend/app/services/feedback_center/` - `backend/app/services/feedback_center/`
@ -80,6 +85,9 @@
- `backend/app/models/reply_iteration.py` - `backend/app/models/reply_iteration.py`
- `backend/app/models/training_pipeline.py` - `backend/app/models/training_pipeline.py`
- `backend/app/models/mounted_ai_runtime.py` - `backend/app/models/mounted_ai_runtime.py`
- `backend/app/models/query_route_memory.py`
- `backend/app/models/semantic_fact.py`
- `backend/app/models/semantic_fact_extraction_run.py`
- `backend/app/api/ai_steward_api/routes_reply_iteration.py` - `backend/app/api/ai_steward_api/routes_reply_iteration.py`
- `backend/app/api/ai_steward_api/routes_training_pipeline.py` - `backend/app/api/ai_steward_api/routes_training_pipeline.py`
- `backend/app/api/ai_steward_api/routes_mounted_ai.py` - `backend/app/api/ai_steward_api/routes_mounted_ai.py`
@ -88,7 +96,16 @@
- `backend/app/services/` 根目录已目录化收敛,新增服务优先进入对应子包。 - `backend/app/services/` 根目录已目录化收敛,新增服务优先进入对应子包。
- API canonical 入口优先使用 `*_api/` 子包,不再优先引用顶层兼容壳。 - API canonical 入口优先使用 `*_api/` 子包,不再优先引用顶层兼容壳。
- 运行时业务动作归因优先走“对象/上下文绑定 -> `SemanticContextPacket` -> `SemanticDecision` -> gatekeeper”。 - 运行时业务动作归因优先走“对象/上下文绑定 -> `SemanticContextPacket` -> `SemanticDecision` -> gatekeeper”。
- non-plan `query.answer` 现已在 `AdaptiveRAG` 内增加 `Query Route Gateway`:只消费上游单次结构化 `query_profile/source_policy/effective_source_policy` 与显式 hint,归一化产出 `route_mode / decision_origin / reply_contract / probe_snapshot(contract audit)`,不再在运行时执行 internal/external probe 或 route memory 强拍板,再进入检索与回答主链。
- internal `query.answer` 的下一阶段基线已经确认:`Query Route Gateway` 之后不再直接靠快答分支拍板,而是统一进入 `Operation Contract -> Target Resolver -> Query DSL Plan -> 白名单受控执行器 -> semantic facts 优先 -> verifier` 主链;相关设计规格见 `docs/superpowers/specs/2026-04-15-internal-query-dsl-plan-design.md` 与 `docs/superpowers/specs/2026-04-15-semantic-facts-read-model-design.md`。
- external `query.answer` 已完成 first-pass 收口:`Query Route Gateway` 之后统一进入 `backend/app/services/external_query_service/`,按 `External Query Planner -> External Query Gateway -> External Provider Registry -> MCP adapter 优先 -> bounded fallback live search -> external answer verifier` 执行;AdaptiveRAG external probe、freshness fetch 与 external route retrieval 已接到这条链上,并对 external sources 执行 fail-closed 过滤;相关设计规格见 `docs/superpowers/specs/2026-04-15-external-query-gateway-design.md`。
- `query.answer` 的客户可见回答展示也已确认独立基线:运行时证据必须拆成 `customer_safe` 与 `admin_debug` 双视图,客户侧统一消费中文化 `display_label`,禁止直接暴露 `raw_source_label`、机器来源或内部术语;相关设计规格见 `docs/superpowers/specs/2026-04-15-answer-presentation-policy-design.md`。
- `semantic_fact_service` 已升级为统一基线:`semantic_facts / semantic_fact_extraction_runs` 读模型、ORM 提交钩子自动收集对象变更并 upsert `ingest_events(channel=semantic_fact_extract)`、统一 worker 抽取/失活,以及 internal query 的 facts-first 主路径。
- semantic facts 自动抽取覆盖 `project / task / signal / document / user / requirement(charter) / release / review` 八类业务对象;manual retry 与 backfill 走同一队列、同一 worker,不再开旁路。
- MCP 模板接入中心已接入 `backend/app/api/mcp_api/` + `backend/app/services/mcp_config/`:租户内统一保存 MCP 模板 catalog、server 配置 bundle、命令预览、validate 结果、runtime probe 与 `pending_local_apply / user_confirmed / not_required` 状态;远程 `streamable_http` 模板可由 PMFlow 后端直连,本地 `stdio` 模板只输出 Local Bridge/兼容 host 接入说明,服务端不直写客户机器 `~/.codex/config.toml`。
- `query_route_memories` 作为独立路由记忆表,承接 `QARecord` 的 route audit、feedback 回灌和离线评估样本,不复用原 answer-quality 资产表。
- `backend/tests/test_runtime_structured_semantics_contract.py` 与 `backend/tests/test_feishu_pipeline_queue_contract.py` 共同守门 runtime 结构化语义收口,锁定飞书入口、signal capture、reply/runtime routing 不得回流旧词法拍板 helper。 - `backend/tests/test_runtime_structured_semantics_contract.py` 与 `backend/tests/test_feishu_pipeline_queue_contract.py` 共同守门 runtime 结构化语义收口,锁定飞书入口、signal capture、reply/runtime routing 不得回流旧词法拍板 helper。
- `backend/tests/test_external_query_planner.py` 现已补进 external fallback 回归,锁定 planner fallback 只能产出 generic external plan,不得再根据 `weather/news/public_model/...` 之类对象映射在运行时硬分 query_kind。
## 5. 前端库存 ## 5. 前端库存
@ -103,6 +120,7 @@
- `frontend/src/pages/feishu-config/` - `frontend/src/pages/feishu-config/`
- `frontend/src/pages/ai-config/` - `frontend/src/pages/ai-config/`
- `frontend/src/pages/model-routing-config/` - `frontend/src/pages/model-routing-config/`
- `frontend/src/pages/ai-config/mcp-config/`
- `frontend/src/pages/task-brief-center/` - `frontend/src/pages/task-brief-center/`
- `frontend/src/pages/task-brief-runs/` - `frontend/src/pages/task-brief-runs/`
- `frontend/src/pages/task-brief-template/` - `frontend/src/pages/task-brief-template/`
@ -110,6 +128,7 @@
- `frontend/src/pages/contract-sync-board/` - `frontend/src/pages/contract-sync-board/`
- `frontend/src/pages/dashboard/` - `frontend/src/pages/dashboard/`
- `frontend/src/pages/knowledge-graph-context-board/` - `frontend/src/pages/knowledge-graph-context-board/`
- `frontend/src/pages/semantic-facts-board/`
- `frontend/src/pages/login/` - `frontend/src/pages/login/`
- `frontend/src/pages/skill-tasks/` - `frontend/src/pages/skill-tasks/`
- `frontend/src/pages/skills/` - `frontend/src/pages/skills/`
@ -123,7 +142,9 @@
- `frontend/src/api/aiSteward.ts` - `frontend/src/api/aiSteward.ts`
- `frontend/src/api/taskBriefs.ts` - `frontend/src/api/taskBriefs.ts`
- `frontend/src/api/modelRouting.ts` - `frontend/src/api/modelRouting.ts`
- `frontend/src/api/mcp.ts`
- `frontend/src/api/knowledgeGraph.ts` - `frontend/src/api/knowledgeGraph.ts`
- `frontend/src/api/semanticFacts.ts`
- `frontend/src/api/mountedAi.ts` - `frontend/src/api/mountedAi.ts`
- `frontend/src/api/feedbackCenter.ts` - `frontend/src/api/feedbackCenter.ts`
- `frontend/src/api/sstc.ts` - `frontend/src/api/sstc.ts`
@ -137,6 +158,7 @@
- 页面级编排逻辑优先收敛到 `pages/*/` 子目录。 - 页面级编排逻辑优先收敛到 `pages/*/` 子目录。
- API 调用统一走 `frontend/src/api/`,不在页面或组件内直接 `fetch`。 - API 调用统一走 `frontend/src/api/`,不在页面或组件内直接 `fetch`。
- `frontend/src/components/sstc/` 继续作为 SSTC 共享组件目录维护。 - `frontend/src/components/sstc/` 继续作为 SSTC 共享组件目录维护。
- `AI Config -> Governance Tools` 现新增 `/ai-config/mcp-config` 与 `/ai-config/semantic-facts-board` 入口;两类治理页都只对 `BOSS/ADMIN` 开放。MCP 页负责模板化配置、后端直连/Local Bridge 状态展示、runtime probe、命令预览与本机确认;semantic facts 页负责 KPI、队列、对象表与详情抽屉。
## 6. 工具与运行库存 ## 6. 工具与运行库存
- `backend/scripts/repair_requirement_plan_history.py` - `backend/scripts/repair_requirement_plan_history.py`
@ -145,6 +167,7 @@
- `scripts/gen_task_brief.sh` - `scripts/gen_task_brief.sh`
- `scripts/check_contract_sync.sh` - `scripts/check_contract_sync.sh`
- `frontend/playwright.config.ts` - `frontend/playwright.config.ts`
- `frontend/vitest.setup.ts`
- `frontend/e2e/feishu-project-full-flow.spec.ts` - `frontend/e2e/feishu-project-full-flow.spec.ts`
## 7. 维护建议 ## 7. 维护建议

View File

@ -30,8 +30,19 @@
### 2.3 RAG / AI PMO / Knowledge Graph ### 2.3 RAG / AI PMO / Knowledge Graph
- RAG 问答主入口:参考 `backend/app/api/rag_api/` 与 `backend/app/api/rag_ask_contract.py` - RAG 问答主入口:参考 `backend/app/api/rag_api/` 与 `backend/app/api/rag_ask_contract.py`
- Query 路由网关与显式来源回复合同:参考 `backend/app/services/adaptive_rag_service/mixin_query_route_gateway.py`
- External Query Service 主链:参考 `backend/app/services/external_query_service/`(`planner.py`、`gateway.py`、`registry.py`、`adapters/`)
- 运行时语义守门与回归:参考 `scripts/semantic_lint.py`、`backend/tests/test_semantic_lint.py`、`backend/tests/test_external_query_planner.py`
- MCP backend-direct runtime / Microsoft Learn bridge:参考 `backend/app/services/mcp_config/runtime.py` 与 `backend/app/services/mcp_config/service.py`
- 路由记忆样本模型:参考 `backend/app/models/query_route_memory.py`
- Internal Query `Operation Contract / Target Resolver / DSL / verifier` 主链:参考 `backend/app/services/adaptive_rag_service/mixin_target_resolution.py`、`mixin_answer_mainline.py`、`mixin_deterministic_query_core.py`、`mixin_answer_payload.py`
- Semantic facts 读模型与抽取服务:参考 `backend/app/models/semantic_fact.py`、`backend/app/models/semantic_fact_extraction_run.py`、`backend/app/services/semantic_fact_service/`
- Semantic facts 自动入队钩子:参考 `backend/app/services/semantic_fact_service/hooks.py`
- Semantic facts 异步事件消费:参考 `backend/app/services/project_knowledge_graph_service/mixin_event_queue.py`
- Semantic facts 治理后台接口:参考 `backend/app/api/semantic_facts_api/routes.py`
- KG 上下文闭环接口:参考 `backend/app/api/knowledge_graph.py` - KG 上下文闭环接口:参考 `backend/app/api/knowledge_graph.py`
- 任务包中心接口:参考 `backend/app/api/task_briefs.py` - 任务包中心接口:参考 `backend/app/api/task_briefs.py`
- MCP 模板接入中心接口:参考 `backend/app/api/mcp_api/routes.py`
- AI PMO 恢复接口:参考 `POST /ai-pmo/work-items/{work_item_id}/resume` - AI PMO 恢复接口:参考 `POST /ai-pmo/work-items/{work_item_id}/resume`
- AI PMO 草稿决策接口:参考 `POST /ai-pmo/work-items/{work_item_id}/draft-decision` - AI PMO 草稿决策接口:参考 `POST /ai-pmo/work-items/{work_item_id}/draft-decision`
- AI PMO 审批接口:参考 `POST /ai-pmo/work-items/{work_item_id}/decision` - AI PMO 审批接口:参考 `POST /ai-pmo/work-items/{work_item_id}/decision`
@ -60,6 +71,11 @@
- Scheduler Center / Mounted AI Console:参考 `frontend/src/pages/scheduler-center/` - Scheduler Center / Mounted AI Console:参考 `frontend/src/pages/scheduler-center/`
- SSTC 工作流组件:参考 `frontend/src/components/sstc/` - SSTC 工作流组件:参考 `frontend/src/components/sstc/`
- Model Routing 页面:参考 `frontend/src/pages/model-routing-config/` - Model Routing 页面:参考 `frontend/src/pages/model-routing-config/`
- MCP 模板接入页:参考 `frontend/src/pages/ai-config/mcp-config/`
- Semantic facts 治理页:参考 `frontend/src/pages/semantic-facts-board/`
- MCP 前端 API:参考 `frontend/src/api/mcp.ts`
- Semantic facts 前端 API:参考 `frontend/src/api/semanticFacts.ts`
- AI Config 治理工具入口:参考 `frontend/src/pages/ai-config/constants.ts` 与 `frontend/src/router/index.tsx`
## 4. 模型与配置真源 ## 4. 模型与配置真源
- 模型配置唯一来源:参考 `backend/app/services/model/model_config.py` - 模型配置唯一来源:参考 `backend/app/services/model/model_config.py`
@ -67,6 +83,7 @@
- Prompt 聚合:`backend/app/config/ai_prompts.py` - Prompt 聚合:`backend/app/config/ai_prompts.py`
- Prompt 场景化分片:`backend/app/config/ai_prompts_catalog/` - Prompt 场景化分片:`backend/app/config/ai_prompts_catalog/`
- Prompt registry:`backend/app/config/ai_prompt_registry.py` - Prompt registry:`backend/app/config/ai_prompt_registry.py`
- External Query Planner Prompt:`backend/app/config/ai_prompts.py` 中的 `PROMPT_EXTERNAL_QUERY_PLAN`
## 5. 维护建议 ## 5. 维护建议
- 新增接口时:先更新本文档,再同步 `API_CONTRACTS.md` - 新增接口时:先更新本文档,再同步 `API_CONTRACTS.md`

Some files were not shown because too many files have changed in this diff Show More