feat: add external query and semantic facts runtime
This commit is contained in:
parent
e42652b619
commit
4720c8117d
|
|
@ -0,0 +1,2 @@
|
||||||
|
{"type":"click","text":"C. Tab 分对象类型\n \n 概览 | 项目 | 任务 | 信号 | 文档 | 人员 | 宪章 | 发布 | 复盘\n \n 当前 tab facts 数\n 失败数\n 最后抽取\n \n 当前对象类型的列表和详情\n \n \n \n \n 方案 C\n 对象边界最清楚,但你要横向看整体积压和失败会来回切 tab,运营效率最低。","choice":"c","id":null,"timestamp":1776240598958}
|
||||||
|
{"type":"click","text":"C. Tab 分对象类型\n \n 概览 | 项目 | 任务 | 信号 | 文档 | 人员 | 宪章 | 发布 | 复盘\n \n 当前 tab facts 数\n 失败数\n 最后抽取\n \n 当前对象类型的列表和详情\n \n \n \n \n 方案 C\n 对象边界最清楚,但你要横向看整体积压和失败会来回切 tab,运营效率最低。","choice":"c","id":null,"timestamp":1776241280184}
|
||||||
|
|
@ -0,0 +1 @@
|
||||||
|
{"reason":"idle timeout","timestamp":1776243098908}
|
||||||
|
|
@ -0,0 +1 @@
|
||||||
|
95325
|
||||||
|
|
@ -0,0 +1,96 @@
|
||||||
|
<h2>Semantic Facts 状态页设计稿</h2>
|
||||||
|
<p class="subtitle">先确认页面信息架构和操作区布局。后端统一按“提交后自动入队 + 失活旧 facts + 手动重抽入口”设计。</p>
|
||||||
|
|
||||||
|
<div class="cards">
|
||||||
|
<div class="card" data-choice="a" onclick="toggleSelect(this)">
|
||||||
|
<div class="card-image">
|
||||||
|
<div class="mockup">
|
||||||
|
<div class="mockup-header">A. 总览台 + 队列栏 + 对象表 + 详情抽屉</div>
|
||||||
|
<div class="mockup-body">
|
||||||
|
<div style="display:grid;grid-template-columns:repeat(4,1fr);gap:10px;margin-bottom:10px;">
|
||||||
|
<div class="placeholder" style="height:68px;">对象总数</div>
|
||||||
|
<div class="placeholder" style="height:68px;">active facts</div>
|
||||||
|
<div class="placeholder" style="height:68px;">失败队列</div>
|
||||||
|
<div class="placeholder" style="height:68px;">积压时长</div>
|
||||||
|
</div>
|
||||||
|
<div style="display:grid;grid-template-columns:1.2fr 0.8fr;gap:10px;margin-bottom:10px;">
|
||||||
|
<div class="placeholder" style="height:92px;">按对象类型统计:项目 / 任务 / 信号 / 文档 / 人员 / 宪章 / 发布 / 复盘</div>
|
||||||
|
<div class="placeholder" style="height:92px;">抽取队列状态:pending / running / retry / dead_letter</div>
|
||||||
|
</div>
|
||||||
|
<div style="display:grid;grid-template-columns:1fr 0.55fr;gap:10px;">
|
||||||
|
<div class="placeholder" style="height:170px;">可搜索对象表:对象类型 / 标题 / facts 条数 / 最后抽取 / 队列状态 / 操作</div>
|
||||||
|
<div class="placeholder" style="height:170px;">右侧详情抽屉:facts 列表 / 证据 / 最近 run / 重抽按钮</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="card-body">
|
||||||
|
<h3>方案 A</h3>
|
||||||
|
<p>一页看全:先看风险和积压,再搜对象,再在右侧展开 facts 详情。我推荐这个,最适合排查“哪个对象没抽上、为什么失败、要不要重跑”。</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="card" data-choice="b" onclick="toggleSelect(this)">
|
||||||
|
<div class="card-image">
|
||||||
|
<div class="mockup">
|
||||||
|
<div class="mockup-header">B. 双栏工作台</div>
|
||||||
|
<div class="mockup-body">
|
||||||
|
<div style="display:grid;grid-template-columns:0.9fr 1.1fr;gap:10px;">
|
||||||
|
<div>
|
||||||
|
<div class="placeholder" style="height:78px;margin-bottom:10px;">队列状态总览</div>
|
||||||
|
<div class="placeholder" style="height:194px;">对象列表 + 类型筛选 + 搜索</div>
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<div class="placeholder" style="height:88px;margin-bottom:10px;">当前对象摘要:facts 数 / 最后抽取 / 最近失败</div>
|
||||||
|
<div class="placeholder" style="height:184px;">facts 明细、run 时间线、手动重抽</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="card-body">
|
||||||
|
<h3>方案 B</h3>
|
||||||
|
<p>偏“工作台”操作感,适合长时间盯一个对象排障;但全局统计和横向比较会弱一点。</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="card" data-choice="c" onclick="toggleSelect(this)">
|
||||||
|
<div class="card-image">
|
||||||
|
<div class="mockup">
|
||||||
|
<div class="mockup-header">C. Tab 分对象类型</div>
|
||||||
|
<div class="mockup-body">
|
||||||
|
<div class="mock-nav" style="margin-bottom:10px;">概览 | 项目 | 任务 | 信号 | 文档 | 人员 | 宪章 | 发布 | 复盘</div>
|
||||||
|
<div style="display:grid;grid-template-columns:repeat(3,1fr);gap:10px;margin-bottom:10px;">
|
||||||
|
<div class="placeholder" style="height:76px;">当前 tab facts 数</div>
|
||||||
|
<div class="placeholder" style="height:76px;">失败数</div>
|
||||||
|
<div class="placeholder" style="height:76px;">最后抽取</div>
|
||||||
|
</div>
|
||||||
|
<div class="placeholder" style="height:190px;">当前对象类型的列表和详情</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div class="card-body">
|
||||||
|
<h3>方案 C</h3>
|
||||||
|
<p>对象边界最清楚,但你要横向看整体积压和失败会来回切 tab,运营效率最低。</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
<div class="pros-cons" style="margin-top:18px;">
|
||||||
|
<div class="pros">
|
||||||
|
<h4>我推荐 A 的原因</h4>
|
||||||
|
<ul>
|
||||||
|
<li>既能看全局积压,也能快速钻到单对象 facts。</li>
|
||||||
|
<li>和现有治理子页风格更一致,适合放到智能配置中心。</li>
|
||||||
|
<li>后面要加 bulk backfill、失败重放、对象覆盖率都比较自然。</li>
|
||||||
|
</ul>
|
||||||
|
</div>
|
||||||
|
<div class="cons">
|
||||||
|
<h4>设计约束</h4>
|
||||||
|
<ul>
|
||||||
|
<li>这轮“需求”按宪章(charter)展示,不含运行中 draft。</li>
|
||||||
|
<li>队列状态会直接暴露 pending / retry / dead_letter,不做粉饰。</li>
|
||||||
|
<li>对象详情默认展示 active facts,历史失活 facts 放二级查看。</li>
|
||||||
|
</ul>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
10
AGENTS.md
10
AGENTS.md
|
|
@ -95,14 +95,24 @@ frontend/ # React 前端
|
||||||
- 新增接口必须遵循响应包络:`{code, message, data}`,不得裸返回
|
- 新增接口必须遵循响应包络:`{code, message, data}`,不得裸返回
|
||||||
- 禁止在 API 层写业务逻辑,业务逻辑归属 `services/`
|
- 禁止在 API 层写业务逻辑,业务逻辑归属 `services/`
|
||||||
- 禁止在前端组件内直接调用 `fetch`,必须通过 `frontend/src/api/` 封装层
|
- 禁止在前端组件内直接调用 `fetch`,必须通过 `frontend/src/api/` 封装层
|
||||||
|
- MCP 模板接入中心只允许保存模板参数、命令预览、runtime probe 与“本机已执行”确认状态;远程 `streamable_http` 模板允许由 PMFlow 后端直连校验/调用,本地 `stdio` 模板只能走 PMFlow Local Bridge 或兼容 MCP host;禁止从服务端直接写宿主机 `~/.codex/config.toml`、禁止替客户执行本机命令,GitHub 类模板只允许保存 env var 名称或占位符,不得落明文 token。
|
||||||
- 前端状态管理:服务端数据用 React Query,本地 UI 状态用 `useState`
|
- 前端状态管理:服务端数据用 React Query,本地 UI 状态用 `useState`
|
||||||
- 用户可见核心业务内容零兜底:AI 失败、证据不足、结构化失败时必须显式失败或标记待确认,禁止用默认句、模板句、stub 冒充成功。
|
- 用户可见核心业务内容零兜底:AI 失败、证据不足、结构化失败时必须显式失败或标记待确认,禁止用默认句、模板句、stub 冒充成功。
|
||||||
- 业务语义、问答/执行归因与前台动作拍板必须优先走“对象/上下文绑定 -> `SemanticContextPacket` -> `SemanticDecision` -> gatekeeper”主链,禁止在 `backend/app` 运行时代码里恢复 `looks_like_*`、关键词仓或散落正则拍板。
|
- 业务语义、问答/执行归因与前台动作拍板必须优先走“对象/上下文绑定 -> `SemanticContextPacket` -> `SemanticDecision` -> gatekeeper”主链,禁止在 `backend/app` 运行时代码里恢复 `looks_like_*`、关键词仓或散落正则拍板。
|
||||||
|
- `query.answer` 路由与外部查询规划不得在运行时代码里维护 `_INTERNAL_ROUTE_OBJECTS / _EXTERNAL_ROUTE_OBJECTS / _SEMANTIC_QUERY_OBJECT_TYPES` 一类对象词表,也不得新增 `*_probe_is_strong / *_route_probe / query_kind(weather/news/company/market/model)` 这类二次语义拍板。`query_profile / source_policy_hint / SemanticDecision` 只能作为上游结构化输入;若证据不足,只能显式返回 `uncertain/clarify`,禁止在执行层靠对象集合、`subject_focus`、`needs_freshness`、`referenced_objects` 再次裁决 internal vs external。
|
||||||
- `llm_service` 运行时必须显式传入 `routing_meta`;禁止再通过函数名片段、hint 常量或模板词在底层推断 tier、结构化输出或 fallback 策略。
|
- `llm_service` 运行时必须显式传入 `routing_meta`;禁止再通过函数名片段、hint 常量或模板词在底层推断 tier、结构化输出或 fallback 策略。
|
||||||
- 显式 `reply/thread`、卡片动作和 `work_item_id` 属于更强锚点,必须优先于旧 session 和记忆续写。
|
- 显式 `reply/thread`、卡片动作和 `work_item_id` 属于更强锚点,必须优先于旧 session 和记忆续写。
|
||||||
- 进入 `requirement_plan` / draft confirm 的线程必须留在对应主链,禁止掉回 `query.answer`;其中 draft confirm 只允许显式 `reply/thread`、卡片按钮或 `work_item_id` 进入,禁止无锚点按“同群/同人”回捞旧 pending draft。若同 scope 旧 plan/draft 已被判定污染,运行时必须先终止该 scope 内的 active topic/work item,再回退到 `query.answer`。选项协议回复必须复用单点结构化解析器,不得散落关键词/正则。
|
- 进入 `requirement_plan` / draft confirm 的线程必须留在对应主链,禁止掉回 `query.answer`;其中 draft confirm 只允许显式 `reply/thread`、卡片按钮或 `work_item_id` 进入,禁止无锚点按“同群/同人”回捞旧 pending draft。若同 scope 旧 plan/draft 已被判定污染,运行时必须先终止该 scope 内的 active topic/work item,再回退到 `query.answer`。选项协议回复必须复用单点结构化解析器,不得散落关键词/正则。
|
||||||
|
- 已绑定 active requirement context 的线程里,显式 `退出需求模式` 必须先挂起当前 collecting/ready draft 对应的 requirement topic,再允许回到普通 `query.answer`;同一 `conversation_scope` 下的 topic 选择必须优先命中仍带 active draft 的 live requirement workspace,禁止被后续 idle query topic 抢走。
|
||||||
|
- draft confirm 若命中显式 `reply/thread` 锚点,候选 mounted draft / pending draft 必须先通过该锚点精确命中;同群、同 scope、同人但锚点不匹配的历史草稿不得参与打分或回捞,避免 requirement-plan A/B/C 线程被旧 draft 劫持。
|
||||||
|
- Feishu `requirement_plan` 首轮/追问回复若命中 `reply_template_key=requirement_plan_followup`,必须保留模板键并优先渲染交互卡片;卡片按钮回流必须复用 canonical Feishu `reply/thread` 上下文,禁止因 query renderer 吞模板键或按钮回流丢锚点而退回纯文本旧模板。
|
||||||
|
- 冷启动 `WorkflowSemanticPlanner` / `pmflow_semantic_context_decision` 失败时,运行时只能降级到带 `semantic_planner_degraded=true` 审计标记的中性 `query.answer` 场景;后续优先重试 semantic backfill,显式 active requirement context / reply anchor 仍留在 requirement-plan 主链,无锚点事实问句禁止硬降级成 requirement consultation。
|
||||||
- Feishu 显式 `reply/thread` 到机器人消息时,必须先通过 `feishu_reply_outbox.sent_message_id -> source_message_id` 归一化 canonical thread context,再进入后续主链。
|
- Feishu 显式 `reply/thread` 到机器人消息时,必须先通过 `feishu_reply_outbox.sent_message_id -> source_message_id` 归一化 canonical thread context,再进入后续主链。
|
||||||
- 外部 freshness 问答的 follow-up 若锚定上一轮“未接入实时外部数据/无法确认最新结论”的 QA 回复,默认继续留在 `query.answer` 聊天链并先尝试实时外部检索;只有用户显式点明系统/项目/配置等内部对象并提出接入动作时,才允许落入内部接入/入库流程。
|
- 外部 freshness 问答的 follow-up 若锚定上一轮“未接入实时外部数据/无法确认最新结论”的 QA 回复,默认继续留在 `query.answer` 聊天链并先尝试实时外部检索;只有用户显式点明系统/项目/配置等内部对象并提出接入动作时,才允许落入内部接入/入库流程。
|
||||||
|
- non-plan `query.answer` 必须先经过 `Query Route Gateway` 产出 `route_mode / decision_origin / reply_contract / probe_snapshot`,再进入 AdaptiveRAG;回复正文必须遵守显式来源合同(`内部知识库查询:` / `外部实时查询:`),`uncertain` 只允许 `external-first` 或澄清,禁止默认混合答案。
|
||||||
|
- `route_mode=external` 后不得再直接把 DuckDuckGo HTML 抓取当主执行层;必须统一进入 `External Query Planner -> External Query Gateway -> External Provider Registry -> MCP adapter 优先 -> bounded fallback live search -> external answer verifier` 主链。external 来源过滤必须 fail-closed,禁止把 `Feishu history / doc / pmflow` 这类内部来源混回 `外部实时查询` 的来源摘要。
|
||||||
|
- `query.answer` 的客户可见回复必须再经过 `Answer Presentation Policy`:客户视图只允许看到中文化 `display_label/display_summary`,禁止直接暴露 `raw_source_label`、`Project database / Semantic facts / Knowledge Graph / Feishu history` 等机器来源或内部术语。prompt context 也只能喂展示字段,不能再把原始英文来源标签直接喂给模型。
|
||||||
|
- internal `query.answer` 一旦命中 `route_mode=internal`,必须再走“`Operation Contract -> Target Resolver -> Query DSL Plan -> 白名单受控执行器 -> semantic facts 优先(可用时)-> bounded fallback retrieval -> answer verifier`”链路;禁止让 LLM 自由写 SQL、自由决定 join/表名,或在没有证据时直接给内部结论。
|
||||||
- `MountedDraftWorkspace.requirement_card` 是 requirement-plan 唯一业务真源;assistant prose、topic facts、run 文案只可审计,不可反推业务字段。
|
- `MountedDraftWorkspace.requirement_card` 是 requirement-plan 唯一业务真源;assistant prose、topic facts、run 文案只可审计,不可反推业务字段。
|
||||||
- `project_create.create / followup.confirm` 的任务拆解是从属子步骤:运行时必须先 low-first 结构化生成,必要时升级 high;仍失败则显式记为 `pending`,禁止冒充成功。
|
- `project_create.create / followup.confirm` 的任务拆解是从属子步骤:运行时必须先 low-first 结构化生成,必要时升级 high;仍失败则显式记为 `pending`,禁止冒充成功。
|
||||||
- 运行时长清单、Feishu / requirement-plan / AdaptiveRAG 细则、已知边界与假设,统一以 `docs/architecture/RUNTIME_GUARDRAILS.md` 为准。
|
- 运行时长清单、Feishu / requirement-plan / AdaptiveRAG 细则、已知边界与假设,统一以 `docs/architecture/RUNTIME_GUARDRAILS.md` 为准。
|
||||||
|
|
|
||||||
18
CLAUDE.md
18
CLAUDE.md
|
|
@ -32,7 +32,7 @@ AIProjectManger/
|
||||||
```
|
```
|
||||||
- 详细架构库存已迁到 [`docs/architecture/ARCHITECTURE_INVENTORY.md`](/Users/yuuuu/Documents/AIProjectManger/docs/architecture/ARCHITECTURE_INVENTORY.md)。
|
- 详细架构库存已迁到 [`docs/architecture/ARCHITECTURE_INVENTORY.md`](/Users/yuuuu/Documents/AIProjectManger/docs/architecture/ARCHITECTURE_INVENTORY.md)。
|
||||||
- `docs/architecture/` 当前包含:`MASTER_CONTRACT.md`、`RUNTIME_GUARDRAILS.md`、`CODE_REFERENCE_INDEX.md`、`API_CONTRACTS.md`、`DATAFLOW.md`、`TASK_BRIEF_TEMPLATE.md`、`ARCHITECTURE_INVENTORY.md`
|
- `docs/architecture/` 当前包含:`MASTER_CONTRACT.md`、`RUNTIME_GUARDRAILS.md`、`CODE_REFERENCE_INDEX.md`、`API_CONTRACTS.md`、`DATAFLOW.md`、`TASK_BRIEF_TEMPLATE.md`、`ARCHITECTURE_INVENTORY.md`
|
||||||
- `docs/superpowers/plans/` 保存实现计划,`docs/task_briefs/` 保存任务包产物与验收截图。
|
- `docs/superpowers/plans/` 保存实现计划,`docs/superpowers/specs/` 保存需求/方案规格,`docs/task_briefs/` 保存任务包产物与验收截图。
|
||||||
|
|
||||||
## 4. 后端结构(`backend/`)
|
## 4. 后端结构(`backend/`)
|
||||||
```text
|
```text
|
||||||
|
|
@ -56,10 +56,18 @@ backend/
|
||||||
- 当前后端重点入口优先看:
|
- 当前后端重点入口优先看:
|
||||||
- `backend/app/api/` 下的 `*_api/` 子包与聚合 router
|
- `backend/app/api/` 下的 `*_api/` 子包与聚合 router
|
||||||
- `backend/app/services/ai_pmo/`(AI PMO 主链、Mounted Runtime、semantic context)
|
- `backend/app/services/ai_pmo/`(AI PMO 主链、Mounted Runtime、semantic context)
|
||||||
- `backend/app/services/adaptive_rag_service/`(只读问答主链)
|
- `backend/app/services/adaptive_rag_service/`(只读问答主链;`mixin_query_route_gateway.py` 负责 non-plan `query.answer` 的 internal/external/uncertain 来源合同归一化、contract 审计与显式来源回复合同;`mixin_answer_retrieval.py` 现已把 `route_mode=external` 接到独立外部查询主链;内部查询基线已升级为 `Operation Contract -> Target Resolver -> Query DSL Plan -> 受控执行器 -> semantic facts 优先 -> verifier`)
|
||||||
|
- `backend/app/services/external_query_service/`(external `query.answer` 主链:`External Query Planner -> External Query Gateway -> External Provider Registry -> MCP 优先 -> bounded fallback live search -> verifier`)
|
||||||
|
- `backend/app/services/mcp_config/`(MCP 模板接入中心:catalog、bundle 持久化、后端直连 runtime probe / Microsoft Learn 检索桥接、Local Bridge 接入说明与本机确认状态)
|
||||||
|
- `backend/app/services/semantic_fact_service/`(semantic facts 读模型、八类业务对象统一抽取/失活、ORM 提交钩子入队、治理看板查询)
|
||||||
|
- `backend/app/api/semantic_facts_api/`(semantic facts 治理后台接口:overview / objects / detail / reextract / backfill)
|
||||||
|
- `backend/app/api/mcp_api/`(MCP 模板接入中心后台接口:catalog / servers / validate / confirm-client-apply)
|
||||||
- `backend/app/services/llm_service/`(显式 `routing_meta` 协议)
|
- `backend/app/services/llm_service/`(显式 `routing_meta` 协议)
|
||||||
- `backend/app/services/integrations/feishu_integration_service/`(飞书入站与卡片回调)
|
- `backend/app/services/integrations/feishu_integration_service/`(飞书入站与卡片回调)
|
||||||
- `backend/app/services/signal/`(信号接入、编排、planner、repair)
|
- `backend/app/services/signal/`(信号接入、编排、planner、repair)
|
||||||
|
- 本轮新增的问答路由审计存储为 `backend/app/models/query_route_memory.py`,通过 `backend/alembic/versions/20260415_add_query_route_memories.py` 建表;只用于 route audit / feedback 回灌 / 离线评估,不做在线自动调参。
|
||||||
|
- 本轮新增的语义事实读模型为 `backend/app/models/semantic_fact.py` 与 `backend/app/models/semantic_fact_extraction_run.py`,通过 `backend/alembic/versions/20260415_add_semantic_facts.py` 建表;用于内部查询的 facts-first 命中与异步抽取审计。
|
||||||
|
- 本轮新增的外部查询 fallback 回归测试位于 `backend/tests/test_external_query_planner.py`;配合 `backend/tests/test_adaptive_rag_mainline_contract.py`、`backend/tests/test_ai_pmo_ingress_mainline.py` 与 `backend/tests/test_semantic_lint.py` 一起锁定“Query Route Gateway 不再跑 runtime probes、External Query Planner fallback 不再按对象重分类、external freshness follow-up 不再靠硬编码词表”。
|
||||||
- 详细目录迁移、兼容壳清理与新增模块清单见库存文档。
|
- 详细目录迁移、兼容壳清理与新增模块清单见库存文档。
|
||||||
|
|
||||||
## 5. 前端结构(`frontend/`)
|
## 5. 前端结构(`frontend/`)
|
||||||
|
|
@ -83,6 +91,8 @@ frontend/
|
||||||
- `frontend/src/api/`(统一 API 封装)
|
- `frontend/src/api/`(统一 API 封装)
|
||||||
- `frontend/src/pages/inbox/`(消息台)
|
- `frontend/src/pages/inbox/`(消息台)
|
||||||
- `frontend/src/pages/ai-steward/`(AI 管家)
|
- `frontend/src/pages/ai-steward/`(AI 管家)
|
||||||
|
- `frontend/src/pages/ai-config/mcp-config/`(AI Config -> Governance Tools 下的 MCP 模板接入中心)
|
||||||
|
- `frontend/src/pages/semantic-facts-board/`(AI Config -> Governance Tools 下的 semantic facts 治理页)
|
||||||
- `frontend/src/pages/scheduler-center/`(Mounted AI Console)
|
- `frontend/src/pages/scheduler-center/`(Mounted AI Console)
|
||||||
- `frontend/src/components/sstc/`(SSTC 工作流组件)
|
- `frontend/src/components/sstc/`(SSTC 工作流组件)
|
||||||
- 页面目录化、Mounted AI Console、Inbox、AI Steward 与 Recipes/Projects/Users 等模块化拆分的详细库存见库存文档。
|
- 页面目录化、Mounted AI Console、Inbox、AI Steward 与 Recipes/Projects/Users 等模块化拆分的详细库存见库存文档。
|
||||||
|
|
@ -143,6 +153,7 @@ frontend/
|
||||||
cd frontend
|
cd frontend
|
||||||
npm run dev
|
npm run dev
|
||||||
npm run build
|
npm run build
|
||||||
|
npm test
|
||||||
npm run e2e:feishu
|
npm run e2e:feishu
|
||||||
```
|
```
|
||||||
|
|
||||||
|
|
@ -165,9 +176,12 @@ uvicorn app.main:app --reload --host 0.0.0.0 --port 8000
|
||||||
- 当前最重要的摘要只保留:
|
- 当前最重要的摘要只保留:
|
||||||
- 写操作必须经 Alembic / 受控执行器 / outbox / 审计日志,禁止业务路径直写或同步直发。
|
- 写操作必须经 Alembic / 受控执行器 / outbox / 审计日志,禁止业务路径直写或同步直发。
|
||||||
- 用户可见核心业务内容禁止 fallback、stub、模板句冒充成功;失败必须显式失败或标记待确认。
|
- 用户可见核心业务内容禁止 fallback、stub、模板句冒充成功;失败必须显式失败或标记待确认。
|
||||||
|
- MCP 模板接入中心只保存模板参数、命令预览、runtime probe 和手动确认状态;远程 `streamable_http` 模板可由 PMFlow 后端直连,`stdio` 模板只能走 Local Bridge / 兼容 MCP host;禁止从服务端直写宿主机 `~/.codex/config.toml` 或替客户执行本机命令。
|
||||||
- 运行时业务动作归因统一走“对象/上下文绑定 -> `SemanticContextPacket` -> `SemanticDecision` -> gatekeeper”。
|
- 运行时业务动作归因统一走“对象/上下文绑定 -> `SemanticContextPacket` -> `SemanticDecision` -> gatekeeper”。
|
||||||
- `backend/app` 运行时代码禁止恢复散落的中文关键词、正则、`looks_like_*`、`*_HINTS/*_KEYWORDS` 业务拍板逻辑。
|
- `backend/app` 运行时代码禁止恢复散落的中文关键词、正则、`looks_like_*`、`*_HINTS/*_KEYWORDS` 业务拍板逻辑。
|
||||||
- `llm_service` 只认显式 `routing_meta` 协议,不再靠函数名或 hint 推断 tier / contract / fallback。
|
- `llm_service` 只认显式 `routing_meta` 协议,不再靠函数名或 hint 推断 tier / contract / fallback。
|
||||||
|
- 内部只读查询必须走 `Operation Contract -> Target Resolver -> Query DSL Plan -> 白名单受控执行器 -> semantic facts 优先 -> verifier`,禁止自由 SQL 和无证据脑补。
|
||||||
|
- `semantic_facts` 自动抽取统一走 ORM 提交钩子 -> `ingest_events(channel=semantic_fact_extract)` -> 统一 worker,支持 `project/task/signal/document/user/requirement/release/review` 八类对象;禁止在各路由散落手工 enqueue。
|
||||||
- AI Steward、Requirement Plan、Signal Source、AdaptiveRAG 等核心链路的详细 guardrails 统一以 `RUNTIME_GUARDRAILS.md` 为准。
|
- AI Steward、Requirement Plan、Signal Source、AdaptiveRAG 等核心链路的详细 guardrails 统一以 `RUNTIME_GUARDRAILS.md` 为准。
|
||||||
|
|
||||||
## 9. 新增代码的参考模式
|
## 9. 新增代码的参考模式
|
||||||
|
|
|
||||||
10
HEARTBEAT.md
10
HEARTBEAT.md
|
|
@ -3,3 +3,13 @@
|
||||||
# Keep this file empty (or with only comments) to skip heartbeat API calls.
|
# Keep this file empty (or with only comments) to skip heartbeat API calls.
|
||||||
|
|
||||||
# Add tasks below when you want the agent to check something periodically.
|
# Add tasks below when you want the agent to check something periodically.
|
||||||
|
|
||||||
|
## 每日工单推送
|
||||||
|
|
||||||
|
- 触发时间:每天 08:30
|
||||||
|
- 任务:查询当天工单并推送到飞书
|
||||||
|
- 执行命令:
|
||||||
|
```bash
|
||||||
|
docker exec pmflow-db psql "postgresql://readonly:readonly%40%232025@173.1.1.32:5432/middle_platform" -c "SELECT task_code, task_name, excutor_name, status, start_time FROM plat_task_center.task_instance WHERE start_time >= CURRENT_DATE AND start_time < CURRENT_DATE + INTERVAL '1 day' AND ext3 IN ('90101', '901') ORDER BY start_time DESC LIMIT 50;" -x
|
||||||
|
```
|
||||||
|
- 目标用户:ou_9e1fefbcf4cc68a84f4a4ff7d1749048
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,98 @@
|
||||||
|
"""add query route memories
|
||||||
|
|
||||||
|
Revision ID: 20260415_add_query_route_memories
|
||||||
|
Revises: 20260413_rename_control_plane_status
|
||||||
|
Create Date: 2026-04-15
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
try:
|
||||||
|
from sqlalchemy.dialects.postgresql import JSONB
|
||||||
|
except Exception: # pragma: no cover
|
||||||
|
JSONB = sa.JSON
|
||||||
|
|
||||||
|
|
||||||
|
revision = "20260415_add_query_route_memories"
|
||||||
|
down_revision = "20260413_rename_control_plane_status"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def _json_type(bind):
|
||||||
|
return JSONB() if bind.dialect.name == "postgresql" else sa.JSON()
|
||||||
|
|
||||||
|
|
||||||
|
def _table_exists(bind, table_name: str) -> bool:
|
||||||
|
return sa.inspect(bind).has_table(table_name)
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
bind = op.get_bind()
|
||||||
|
if _table_exists(bind, "query_route_memories"):
|
||||||
|
return
|
||||||
|
|
||||||
|
op.create_table(
|
||||||
|
"query_route_memories",
|
||||||
|
sa.Column("id", sa.Integer(), primary_key=True, autoincrement=True),
|
||||||
|
sa.Column("tenant_id", sa.BigInteger(), nullable=False),
|
||||||
|
sa.Column("created_at", sa.TIMESTAMP(), server_default=sa.func.now(), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.TIMESTAMP(), server_default=sa.func.now(), nullable=False),
|
||||||
|
sa.Column("qa_record_id", sa.Integer(), sa.ForeignKey("qa_records.id", ondelete="CASCADE"), nullable=False),
|
||||||
|
sa.Column("project_id", sa.BigInteger(), nullable=True),
|
||||||
|
sa.Column("question_text", sa.Text(), nullable=False),
|
||||||
|
sa.Column("route_mode_predicted", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("route_mode_effective", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("decision_origin", sa.String(length=64), nullable=True),
|
||||||
|
sa.Column("reply_contract", _json_type(bind), nullable=True),
|
||||||
|
sa.Column("probe_snapshot", _json_type(bind), nullable=True),
|
||||||
|
sa.Column("query_profile", _json_type(bind), nullable=True),
|
||||||
|
sa.Column("feedback_outcome", _json_type(bind), nullable=True),
|
||||||
|
sa.Column("question_embedding", _json_type(bind), nullable=True),
|
||||||
|
sa.UniqueConstraint(
|
||||||
|
"tenant_id",
|
||||||
|
"qa_record_id",
|
||||||
|
name="uq_query_route_memories_tenant_qa_record",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
op.create_index("ix_query_route_memories_tenant_id", "query_route_memories", ["tenant_id"], unique=False)
|
||||||
|
op.create_index("ix_query_route_memories_qa_record_id", "query_route_memories", ["qa_record_id"], unique=False)
|
||||||
|
op.create_index("ix_query_route_memories_project_id", "query_route_memories", ["project_id"], unique=False)
|
||||||
|
op.create_index(
|
||||||
|
"ix_query_route_memories_route_mode_predicted",
|
||||||
|
"query_route_memories",
|
||||||
|
["route_mode_predicted"],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
"ix_query_route_memories_route_mode_effective",
|
||||||
|
"query_route_memories",
|
||||||
|
["route_mode_effective"],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
"ix_query_route_memories_decision_origin",
|
||||||
|
"query_route_memories",
|
||||||
|
["decision_origin"],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
bind = op.get_bind()
|
||||||
|
if not _table_exists(bind, "query_route_memories"):
|
||||||
|
return
|
||||||
|
|
||||||
|
for index_name in (
|
||||||
|
"ix_query_route_memories_decision_origin",
|
||||||
|
"ix_query_route_memories_route_mode_effective",
|
||||||
|
"ix_query_route_memories_route_mode_predicted",
|
||||||
|
"ix_query_route_memories_project_id",
|
||||||
|
"ix_query_route_memories_qa_record_id",
|
||||||
|
"ix_query_route_memories_tenant_id",
|
||||||
|
):
|
||||||
|
op.drop_index(index_name, table_name="query_route_memories")
|
||||||
|
op.drop_table("query_route_memories")
|
||||||
|
|
@ -0,0 +1,138 @@
|
||||||
|
"""add semantic facts
|
||||||
|
|
||||||
|
Revision ID: 20260415_add_semantic_facts
|
||||||
|
Revises: 20260415_add_query_route_memories
|
||||||
|
Create Date: 2026-04-15
|
||||||
|
"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from alembic import op
|
||||||
|
import sqlalchemy as sa
|
||||||
|
|
||||||
|
|
||||||
|
revision = "20260415_add_semantic_facts"
|
||||||
|
down_revision = "20260415_add_query_route_memories"
|
||||||
|
branch_labels = None
|
||||||
|
depends_on = None
|
||||||
|
|
||||||
|
|
||||||
|
def _table_exists(bind, table_name: str) -> bool:
|
||||||
|
return sa.inspect(bind).has_table(table_name)
|
||||||
|
|
||||||
|
|
||||||
|
def upgrade() -> None:
|
||||||
|
bind = op.get_bind()
|
||||||
|
if not _table_exists(bind, "semantic_facts"):
|
||||||
|
op.create_table(
|
||||||
|
"semantic_facts",
|
||||||
|
sa.Column("id", sa.Integer(), primary_key=True, autoincrement=True),
|
||||||
|
sa.Column("tenant_id", sa.BigInteger(), nullable=False),
|
||||||
|
sa.Column("created_at", sa.TIMESTAMP(), server_default=sa.func.now(), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.TIMESTAMP(), server_default=sa.func.now(), nullable=False),
|
||||||
|
sa.Column("entity_type", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("entity_id", sa.BigInteger(), nullable=False),
|
||||||
|
sa.Column("fact_namespace", sa.String(length=64), nullable=False),
|
||||||
|
sa.Column("fact_key", sa.String(length=64), nullable=False),
|
||||||
|
sa.Column("fact_value_text", sa.Text(), nullable=False),
|
||||||
|
sa.Column("fact_value_norm", sa.String(length=255), nullable=False),
|
||||||
|
sa.Column("value_type", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("confidence", sa.Float(), nullable=False, server_default="0"),
|
||||||
|
sa.Column("evidence_source_type", sa.String(length=32), nullable=True),
|
||||||
|
sa.Column("evidence_source_id", sa.BigInteger(), nullable=True),
|
||||||
|
sa.Column("evidence_excerpt", sa.Text(), nullable=True),
|
||||||
|
sa.Column("extract_version", sa.String(length=64), nullable=True),
|
||||||
|
sa.Column("freshness_at", sa.String(length=64), nullable=True),
|
||||||
|
sa.Column("is_active", sa.Boolean(), nullable=False, server_default=sa.true()),
|
||||||
|
sa.UniqueConstraint(
|
||||||
|
"tenant_id",
|
||||||
|
"entity_type",
|
||||||
|
"entity_id",
|
||||||
|
"fact_key",
|
||||||
|
"fact_value_norm",
|
||||||
|
name="uq_semantic_facts_tenant_entity_key_norm",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
op.create_index("ix_semantic_facts_tenant_id", "semantic_facts", ["tenant_id"], unique=False)
|
||||||
|
op.create_index("ix_semantic_facts_entity_type", "semantic_facts", ["entity_type"], unique=False)
|
||||||
|
op.create_index("ix_semantic_facts_entity_id", "semantic_facts", ["entity_id"], unique=False)
|
||||||
|
op.create_index("ix_semantic_facts_fact_namespace", "semantic_facts", ["fact_namespace"], unique=False)
|
||||||
|
op.create_index("ix_semantic_facts_fact_key", "semantic_facts", ["fact_key"], unique=False)
|
||||||
|
op.create_index("ix_semantic_facts_fact_value_norm", "semantic_facts", ["fact_value_norm"], unique=False)
|
||||||
|
op.create_index("ix_semantic_facts_is_active", "semantic_facts", ["is_active"], unique=False)
|
||||||
|
|
||||||
|
if not _table_exists(bind, "semantic_fact_extraction_runs"):
|
||||||
|
op.create_table(
|
||||||
|
"semantic_fact_extraction_runs",
|
||||||
|
sa.Column("id", sa.Integer(), primary_key=True, autoincrement=True),
|
||||||
|
sa.Column("tenant_id", sa.BigInteger(), nullable=False),
|
||||||
|
sa.Column("created_at", sa.TIMESTAMP(), server_default=sa.func.now(), nullable=False),
|
||||||
|
sa.Column("updated_at", sa.TIMESTAMP(), server_default=sa.func.now(), nullable=False),
|
||||||
|
sa.Column("entity_type", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("entity_id", sa.BigInteger(), nullable=False),
|
||||||
|
sa.Column("trigger_type", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("status", sa.String(length=32), nullable=False),
|
||||||
|
sa.Column("input_fingerprint", sa.String(length=64), nullable=True),
|
||||||
|
sa.Column("facts_written", sa.Integer(), nullable=False, server_default="0"),
|
||||||
|
sa.Column("failure_reason", sa.Text(), nullable=True),
|
||||||
|
sa.Column("extract_version", sa.String(length=64), nullable=True),
|
||||||
|
sa.Column("started_at", sa.String(length=64), nullable=True),
|
||||||
|
sa.Column("finished_at", sa.String(length=64), nullable=True),
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
"ix_semantic_fact_extraction_runs_tenant_id",
|
||||||
|
"semantic_fact_extraction_runs",
|
||||||
|
["tenant_id"],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
"ix_semantic_fact_extraction_runs_entity_type",
|
||||||
|
"semantic_fact_extraction_runs",
|
||||||
|
["entity_type"],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
"ix_semantic_fact_extraction_runs_entity_id",
|
||||||
|
"semantic_fact_extraction_runs",
|
||||||
|
["entity_id"],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
"ix_semantic_fact_extraction_runs_status",
|
||||||
|
"semantic_fact_extraction_runs",
|
||||||
|
["status"],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
op.create_index(
|
||||||
|
"ix_semantic_fact_extraction_runs_input_fingerprint",
|
||||||
|
"semantic_fact_extraction_runs",
|
||||||
|
["input_fingerprint"],
|
||||||
|
unique=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def downgrade() -> None:
|
||||||
|
bind = op.get_bind()
|
||||||
|
if _table_exists(bind, "semantic_fact_extraction_runs"):
|
||||||
|
for index_name in (
|
||||||
|
"ix_semantic_fact_extraction_runs_input_fingerprint",
|
||||||
|
"ix_semantic_fact_extraction_runs_status",
|
||||||
|
"ix_semantic_fact_extraction_runs_entity_id",
|
||||||
|
"ix_semantic_fact_extraction_runs_entity_type",
|
||||||
|
"ix_semantic_fact_extraction_runs_tenant_id",
|
||||||
|
):
|
||||||
|
op.drop_index(index_name, table_name="semantic_fact_extraction_runs")
|
||||||
|
op.drop_table("semantic_fact_extraction_runs")
|
||||||
|
|
||||||
|
if _table_exists(bind, "semantic_facts"):
|
||||||
|
for index_name in (
|
||||||
|
"ix_semantic_facts_is_active",
|
||||||
|
"ix_semantic_facts_fact_value_norm",
|
||||||
|
"ix_semantic_facts_fact_key",
|
||||||
|
"ix_semantic_facts_fact_namespace",
|
||||||
|
"ix_semantic_facts_entity_id",
|
||||||
|
"ix_semantic_facts_entity_type",
|
||||||
|
"ix_semantic_facts_tenant_id",
|
||||||
|
):
|
||||||
|
op.drop_index(index_name, table_name="semantic_facts")
|
||||||
|
op.drop_table("semantic_facts")
|
||||||
|
|
@ -0,0 +1,4 @@
|
||||||
|
from . import routes
|
||||||
|
from .routes import router
|
||||||
|
|
||||||
|
__all__ = ["router", "routes"]
|
||||||
|
|
@ -0,0 +1,136 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends
|
||||||
|
from fastapi.responses import JSONResponse
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
|
||||||
|
from ...core.database import get_db
|
||||||
|
from ...core.response import error_response, success_response
|
||||||
|
from ...dependencies import require_manager_user
|
||||||
|
from ...models import User
|
||||||
|
from ...services.mcp_config import McpConfigService
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/mcp", tags=["mcp_config"])
|
||||||
|
|
||||||
|
|
||||||
|
class McpServerCreateRequest(BaseModel):
|
||||||
|
name: str
|
||||||
|
display_name: str | None = None
|
||||||
|
template_id: str
|
||||||
|
params: dict[str, Any] = Field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
|
class McpServerUpdateRequest(BaseModel):
|
||||||
|
name: str | None = None
|
||||||
|
display_name: str | None = None
|
||||||
|
template_id: str | None = None
|
||||||
|
params: dict[str, Any] | None = None
|
||||||
|
|
||||||
|
|
||||||
|
def _service(db: AsyncSession, current_user: User) -> McpConfigService:
|
||||||
|
return McpConfigService(db, tenant_id=int(current_user.tenant_id))
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/admin/catalog")
|
||||||
|
async def get_mcp_catalog(
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
current_user: User = Depends(require_manager_user),
|
||||||
|
):
|
||||||
|
return success_response(await _service(db, current_user).get_catalog())
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/admin/servers")
|
||||||
|
async def list_mcp_servers(
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
current_user: User = Depends(require_manager_user),
|
||||||
|
):
|
||||||
|
items = await _service(db, current_user).list_servers()
|
||||||
|
return success_response({"items": items})
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/admin/servers")
|
||||||
|
async def create_mcp_server(
|
||||||
|
body: McpServerCreateRequest,
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
current_user: User = Depends(require_manager_user),
|
||||||
|
):
|
||||||
|
service = _service(db, current_user)
|
||||||
|
try:
|
||||||
|
item = await service.create_server(
|
||||||
|
name=body.name,
|
||||||
|
display_name=body.display_name,
|
||||||
|
template_id=body.template_id,
|
||||||
|
params=body.params,
|
||||||
|
)
|
||||||
|
except ValueError as exc:
|
||||||
|
await db.rollback()
|
||||||
|
return JSONResponse(status_code=400, content=error_response(400, str(exc)))
|
||||||
|
await db.commit()
|
||||||
|
return success_response(item)
|
||||||
|
|
||||||
|
|
||||||
|
@router.put("/admin/servers/{server_id}")
|
||||||
|
async def update_mcp_server(
|
||||||
|
server_id: str,
|
||||||
|
body: McpServerUpdateRequest,
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
current_user: User = Depends(require_manager_user),
|
||||||
|
):
|
||||||
|
service = _service(db, current_user)
|
||||||
|
try:
|
||||||
|
item = await service.update_server(
|
||||||
|
server_id=server_id,
|
||||||
|
name=body.name,
|
||||||
|
display_name=body.display_name,
|
||||||
|
template_id=body.template_id,
|
||||||
|
params=body.params,
|
||||||
|
)
|
||||||
|
except ValueError as exc:
|
||||||
|
await db.rollback()
|
||||||
|
return JSONResponse(status_code=400, content=error_response(400, str(exc)))
|
||||||
|
except LookupError as exc:
|
||||||
|
await db.rollback()
|
||||||
|
return JSONResponse(status_code=404, content=error_response(404, str(exc)))
|
||||||
|
await db.commit()
|
||||||
|
return success_response(item)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/admin/servers/{server_id}/validate")
|
||||||
|
async def validate_mcp_server(
|
||||||
|
server_id: str,
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
current_user: User = Depends(require_manager_user),
|
||||||
|
):
|
||||||
|
service = _service(db, current_user)
|
||||||
|
try:
|
||||||
|
payload = await service.validate_server(server_id=server_id)
|
||||||
|
except ValueError as exc:
|
||||||
|
await db.rollback()
|
||||||
|
return JSONResponse(status_code=400, content=error_response(400, str(exc)))
|
||||||
|
except LookupError as exc:
|
||||||
|
await db.rollback()
|
||||||
|
return JSONResponse(status_code=404, content=error_response(404, str(exc)))
|
||||||
|
await db.commit()
|
||||||
|
return success_response(payload)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/admin/servers/{server_id}/confirm-client-apply")
|
||||||
|
async def confirm_mcp_server_client_apply(
|
||||||
|
server_id: str,
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
current_user: User = Depends(require_manager_user),
|
||||||
|
):
|
||||||
|
service = _service(db, current_user)
|
||||||
|
try:
|
||||||
|
payload = await service.confirm_client_apply(server_id=server_id)
|
||||||
|
except ValueError as exc:
|
||||||
|
await db.rollback()
|
||||||
|
return JSONResponse(status_code=400, content=error_response(400, str(exc)))
|
||||||
|
except LookupError as exc:
|
||||||
|
await db.rollback()
|
||||||
|
return JSONResponse(status_code=404, content=error_response(404, str(exc)))
|
||||||
|
await db.commit()
|
||||||
|
return success_response(payload)
|
||||||
|
|
@ -36,7 +36,9 @@ from .operation_catalog import router as operation_catalog_router
|
||||||
from .ai_pmo_api import router as ai_pmo_router
|
from .ai_pmo_api import router as ai_pmo_router
|
||||||
from .audit import router as audit_router
|
from .audit import router as audit_router
|
||||||
from .model_routing import router as model_routing_router
|
from .model_routing import router as model_routing_router
|
||||||
|
from .semantic_facts_api import router as semantic_facts_router
|
||||||
from .task_briefs import router as task_briefs_router
|
from .task_briefs import router as task_briefs_router
|
||||||
|
from .mcp_api import router as mcp_router
|
||||||
|
|
||||||
# 创建主 API 路由器
|
# 创建主 API 路由器
|
||||||
api_router = APIRouter()
|
api_router = APIRouter()
|
||||||
|
|
@ -72,4 +74,6 @@ api_router.include_router(operation_catalog_router)
|
||||||
api_router.include_router(ai_pmo_router)
|
api_router.include_router(ai_pmo_router)
|
||||||
api_router.include_router(audit_router)
|
api_router.include_router(audit_router)
|
||||||
api_router.include_router(model_routing_router)
|
api_router.include_router(model_routing_router)
|
||||||
|
api_router.include_router(semantic_facts_router)
|
||||||
api_router.include_router(task_briefs_router)
|
api_router.include_router(task_briefs_router)
|
||||||
|
api_router.include_router(mcp_router)
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,4 @@
|
||||||
|
from . import routes
|
||||||
|
from .routes import router
|
||||||
|
|
||||||
|
__all__ = ["router", "routes"]
|
||||||
|
|
@ -0,0 +1,104 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
from fastapi import APIRouter, Depends, Query
|
||||||
|
from fastapi.responses import JSONResponse
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
|
||||||
|
from ...core.database import get_db
|
||||||
|
from ...core.response import error_response, success_response
|
||||||
|
from ...dependencies import require_manager_user
|
||||||
|
from ...models import User
|
||||||
|
from ...services.semantic_fact_service import SemanticFactService
|
||||||
|
|
||||||
|
router = APIRouter(prefix="/semantic-facts", tags=["semantic_facts"])
|
||||||
|
|
||||||
|
|
||||||
|
class SemanticFactsBackfillRequest(BaseModel):
|
||||||
|
entity_types: Optional[list[str]] = Field(default=None)
|
||||||
|
only_missing: bool = Field(default=True)
|
||||||
|
limit_per_type: int = Field(default=200, ge=1, le=1000)
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/admin/overview")
|
||||||
|
async def get_semantic_facts_overview(
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
current_user: User = Depends(require_manager_user),
|
||||||
|
):
|
||||||
|
service = SemanticFactService(db, tenant_id=int(current_user.tenant_id))
|
||||||
|
return success_response(await service.get_admin_overview())
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/admin/objects")
|
||||||
|
async def list_semantic_fact_objects(
|
||||||
|
entity_type: str | None = Query(default=None),
|
||||||
|
q: str | None = Query(default=None),
|
||||||
|
status: str | None = Query(default=None),
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
current_user: User = Depends(require_manager_user),
|
||||||
|
):
|
||||||
|
service = SemanticFactService(db, tenant_id=int(current_user.tenant_id))
|
||||||
|
items = await service.list_admin_objects(
|
||||||
|
entity_types=[entity_type] if entity_type else None,
|
||||||
|
q=q,
|
||||||
|
status=status,
|
||||||
|
)
|
||||||
|
return success_response({"items": items})
|
||||||
|
|
||||||
|
|
||||||
|
@router.get("/admin/objects/{entity_type}/{entity_id}")
|
||||||
|
async def get_semantic_fact_object_detail(
|
||||||
|
entity_type: str,
|
||||||
|
entity_id: int,
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
current_user: User = Depends(require_manager_user),
|
||||||
|
):
|
||||||
|
service = SemanticFactService(db, tenant_id=int(current_user.tenant_id))
|
||||||
|
try:
|
||||||
|
detail = await service.get_admin_detail(entity_type=entity_type, entity_id=int(entity_id))
|
||||||
|
except ValueError as exc:
|
||||||
|
return JSONResponse(status_code=404, content=error_response(404, str(exc)))
|
||||||
|
return success_response(detail)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/admin/objects/{entity_type}/{entity_id}/reextract")
|
||||||
|
async def reextract_semantic_fact_object(
|
||||||
|
entity_type: str,
|
||||||
|
entity_id: int,
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
current_user: User = Depends(require_manager_user),
|
||||||
|
):
|
||||||
|
service = SemanticFactService(db, tenant_id=int(current_user.tenant_id))
|
||||||
|
row = await service.enqueue_entity_event(
|
||||||
|
entity_type=entity_type,
|
||||||
|
entity_id=int(entity_id),
|
||||||
|
operation="extract",
|
||||||
|
trigger_type="manual_retry",
|
||||||
|
)
|
||||||
|
await db.commit()
|
||||||
|
return success_response(
|
||||||
|
{
|
||||||
|
"queued": True,
|
||||||
|
"event_id": int(row.id),
|
||||||
|
"entity_type": str(entity_type),
|
||||||
|
"entity_id": int(entity_id),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@router.post("/admin/backfill")
|
||||||
|
async def enqueue_semantic_fact_backfill(
|
||||||
|
body: SemanticFactsBackfillRequest,
|
||||||
|
db: AsyncSession = Depends(get_db),
|
||||||
|
current_user: User = Depends(require_manager_user),
|
||||||
|
):
|
||||||
|
service = SemanticFactService(db, tenant_id=int(current_user.tenant_id))
|
||||||
|
result = await service.enqueue_backfill(
|
||||||
|
entity_types=body.entity_types,
|
||||||
|
only_missing=bool(body.only_missing),
|
||||||
|
limit_per_type=int(body.limit_per_type),
|
||||||
|
)
|
||||||
|
await db.commit()
|
||||||
|
return success_response(result)
|
||||||
|
|
@ -165,23 +165,31 @@ PROMPT_RAG_QUERY_SEMANTICS = """
|
||||||
|
|
||||||
你的职责不是靠关键词仓库猜,而是基于当前消息表达的目的、对象、作用域和 bundle 上下文,输出:
|
你的职责不是靠关键词仓库猜,而是基于当前消息表达的目的、对象、作用域和 bundle 上下文,输出:
|
||||||
- conversation intent(question / statement / mixed)
|
- conversation intent(question / statement / mixed)
|
||||||
- query_type(count / list / status / explain / unknown)
|
- query_type(existence / count / list / status / explain / diagnose / compare / unknown)
|
||||||
|
- object_mentions(用户当前想解析的对象提及,保留原始业务名词短语)
|
||||||
- retrieval_domains(db / graph / doc / feishu / pmflow / url)
|
- retrieval_domains(db / graph / doc / feishu / pmflow / url)
|
||||||
- subject_focus(people / project / task / signal / charter / workload / feature / unknown)
|
- subject_focus(people / project / task / signal / charter / workload / feature / unknown)
|
||||||
- scope_hint(none / global / project)
|
- scope_hint(none / global / project)
|
||||||
- feature_focus(none / purpose / entry / catalog)
|
- feature_focus(none / purpose / entry / catalog)
|
||||||
|
- query_profile(needs_private_context / needs_freshness / referenced_objects)
|
||||||
|
|
||||||
硬约束:
|
硬约束:
|
||||||
- 不要复述 workflow bundle 的内部字段名给用户。
|
- 不要复述 workflow bundle 的内部字段名给用户。
|
||||||
- 如果 bundle 已经明确显示这是 query_flow,优先按 query 处理;只有消息本身同时包含明确执行动作时才输出 mixed。
|
- 如果 bundle 已经明确显示这是 query_flow,优先按 query 处理;只有消息本身同时包含明确执行动作时才输出 mixed。
|
||||||
- query_type 只用于 Adaptive RAG 的检索与快路径规划,不代表最终回复场景。
|
- query_type 只用于 Adaptive RAG 的检索与快路径规划,不代表最终回复场景。
|
||||||
- query_type 取值只能是:count、list、status、explain、unknown。
|
- query_type 取值只能是:existence、count、list、status、explain、diagnose、compare、unknown。
|
||||||
- retrieval_domains 只能从:db、graph、doc、feishu、pmflow、url 中选择。
|
- retrieval_domains 只能从:db、graph、doc、feishu、pmflow、url 中选择。
|
||||||
- subject_focus 只能是:people、project、task、signal、charter、workload、feature、unknown。
|
- subject_focus 只能是:people、project、task、signal、charter、workload、feature、unknown。
|
||||||
- scope_hint 只能是:none、global、project。
|
- scope_hint 只能是:none、global、project。
|
||||||
- feature_focus 只能是:none、purpose、entry、catalog。
|
- feature_focus 只能是:none、purpose、entry、catalog。
|
||||||
|
- query_profile.referenced_objects 只能从:unknown、system_config、model_routing、project_manifest、feature_catalog、project_object、internal_document、person_object、public_model、public_company、weather、market_data、sports_event、news_topic 中选择。
|
||||||
- 如果问题是在问人员、成员、负责人清单或人数,subject_focus 应为 people。
|
- 如果问题是在问人员、成员、负责人清单或人数,subject_focus 应为 people。
|
||||||
- 如果问题是在问功能做什么、入口在哪、模块如何进入,subject_focus 应为 feature,并结合 feature_focus 标明是 purpose / entry / catalog。
|
- 如果问题是在问功能做什么、入口在哪、模块如何进入,subject_focus 应为 feature,并结合 feature_focus 标明是 purpose / entry / catalog。
|
||||||
|
- 如果问题是在问“有没有 / 是否存在 / 系统里有无”这类内部对象存在性,query_type 应为 existence;不要因为对象名里出现“系统 / 平台 / 助手”就改判成 explain。像 `帮我看看系统里面有没有开票系统。`,应输出 `query_type=existence`,并把 `object_mentions=["开票系统"]`。
|
||||||
|
- 外部公开事实示例:`当前 codex 最新的模型用的是什么?`
|
||||||
|
query_profile 应体现 `needs_freshness=true`、`referenced_objects=["public_model"]`,retrieval_domains 应优先为 ["url"],不要因为出现“模型”二字就改判成系统内部配置问题。
|
||||||
|
- 内部运行时配置示例:`我们系统当前接的是哪个 codex 模型配置?`
|
||||||
|
query_profile 应体现 `needs_private_context=true`、`referenced_objects=["system_config","model_routing"]`,retrieval_domains 应优先为 ["db","pmflow"],这是在问系统配置而不是公开外部事实。
|
||||||
- 若证据不足,query_type 输出 unknown,retrieval_domains 选择保守但完整的查询范围。
|
- 若证据不足,query_type 输出 unknown,retrieval_domains 选择保守但完整的查询范围。
|
||||||
- 输出必须是一个 JSON 对象,不要输出 Markdown。
|
- 输出必须是一个 JSON 对象,不要输出 Markdown。
|
||||||
|
|
||||||
|
|
@ -190,11 +198,18 @@ JSON schema:
|
||||||
"intent": "question | statement | mixed",
|
"intent": "question | statement | mixed",
|
||||||
"question_part": "string",
|
"question_part": "string",
|
||||||
"statement_part": "string",
|
"statement_part": "string",
|
||||||
"query_type": "count | list | status | explain | unknown",
|
"query_type": "existence | count | list | status | explain | diagnose | compare | unknown",
|
||||||
|
"object_mentions": ["string"],
|
||||||
"retrieval_domains": ["db | graph | doc | feishu | pmflow | url"],
|
"retrieval_domains": ["db | graph | doc | feishu | pmflow | url"],
|
||||||
"subject_focus": "people | project | task | signal | charter | workload | feature | unknown",
|
"subject_focus": "people | project | task | signal | charter | workload | feature | unknown",
|
||||||
"scope_hint": "none | global | project",
|
"scope_hint": "none | global | project",
|
||||||
"feature_focus": "none | purpose | entry | catalog",
|
"feature_focus": "none | purpose | entry | catalog",
|
||||||
|
"query_profile": {{
|
||||||
|
"needs_private_context": false,
|
||||||
|
"needs_freshness": false,
|
||||||
|
"referenced_objects": ["unknown"],
|
||||||
|
"confidence": 0.0
|
||||||
|
}},
|
||||||
"confidence": 0.0
|
"confidence": 0.0
|
||||||
}}
|
}}
|
||||||
|
|
||||||
|
|
@ -248,6 +263,241 @@ message:
|
||||||
{message}
|
{message}
|
||||||
""".strip()
|
""".strip()
|
||||||
|
|
||||||
|
PROMPT_INTERNAL_TARGET_RESOLUTION = """
|
||||||
|
你是 PMFlow 内部查询链路的 Target Resolver rerank 层。
|
||||||
|
|
||||||
|
你的输入已经包含:
|
||||||
|
1. 用户问题
|
||||||
|
2. 已经确定的 operation_contract
|
||||||
|
3. 受控候选对象列表
|
||||||
|
|
||||||
|
你的职责只有一个:
|
||||||
|
- 在给定候选里选出最符合当前问题语义的目标对象
|
||||||
|
- 如果候选之间仍然无法稳定区分,就要求澄清
|
||||||
|
|
||||||
|
硬约束:
|
||||||
|
- 只能在候选列表里选择,禁止生成新的对象、ID、类型或 target_ref。
|
||||||
|
- 如果是存在性问题,优先选择“被问到的业务对象”,不要把对象名里带“系统/平台/助手”的词面自动改判成 feature explain。
|
||||||
|
- 若前两名候选分差很小、证据不足或候选语义冲突,输出 `clarify=true`。
|
||||||
|
- 输出必须是 JSON 对象,不要输出 Markdown。
|
||||||
|
|
||||||
|
JSON schema:
|
||||||
|
{{
|
||||||
|
"selected_index": 0,
|
||||||
|
"clarify": false,
|
||||||
|
"confidence": 0.0,
|
||||||
|
"reason": "string"
|
||||||
|
}}
|
||||||
|
|
||||||
|
question:
|
||||||
|
{question}
|
||||||
|
|
||||||
|
operation_contract:
|
||||||
|
{operation_contract}
|
||||||
|
|
||||||
|
candidates:
|
||||||
|
{candidates}
|
||||||
|
""".strip()
|
||||||
|
|
||||||
|
PROMPT_EXTERNAL_QUERY_PLAN = """
|
||||||
|
你是 PMFlow External Query Planner。
|
||||||
|
|
||||||
|
你的职责不是直接回答外部事实,而是把当前问题整理成一个受控的外部查询计划。
|
||||||
|
|
||||||
|
硬约束:
|
||||||
|
- 只输出 JSON 对象,不要输出 Markdown。
|
||||||
|
- 不要直接给外部事实结论。
|
||||||
|
- `query_kind` 只能是:weather、news、company、model、market、generic_web、unknown。
|
||||||
|
- `answer_contract` 只能是:direct_fact、summary_with_sources、compare_with_sources、clarify。
|
||||||
|
- `search_queries` 必须是简短、可直接执行的查询短语。
|
||||||
|
- `preferred_domains` 只放域名,不要带协议头。
|
||||||
|
- 若用户问题明显要求最新、当前、今天、最近等外部信息,`freshness_required=true`。
|
||||||
|
- 若问题更像对比/比较,`requires_multi_source=true` 且 `answer_contract=compare_with_sources`。
|
||||||
|
- 结构化失败时优先保守,不要编造实体。
|
||||||
|
|
||||||
|
JSON schema:
|
||||||
|
{{
|
||||||
|
"plan_version": "v1",
|
||||||
|
"query_kind": "weather | news | company | model | market | generic_web | unknown",
|
||||||
|
"answer_contract": "direct_fact | summary_with_sources | compare_with_sources | clarify",
|
||||||
|
"freshness_required": false,
|
||||||
|
"topic": "string",
|
||||||
|
"subject_entities": ["string"],
|
||||||
|
"location_hint": "string | null",
|
||||||
|
"time_range_hint": "string | null",
|
||||||
|
"preferred_domains": ["string"],
|
||||||
|
"search_queries": ["string"],
|
||||||
|
"result_limit": 3,
|
||||||
|
"requires_multi_source": false,
|
||||||
|
"clarify_on_ambiguity": false
|
||||||
|
}}
|
||||||
|
|
||||||
|
question:
|
||||||
|
{question}
|
||||||
|
|
||||||
|
query_semantics:
|
||||||
|
{query_semantics}
|
||||||
|
|
||||||
|
fallback_queries:
|
||||||
|
{fallback_queries}
|
||||||
|
""".strip()
|
||||||
|
|
||||||
|
def _build_semantic_fact_extract_prompt(
|
||||||
|
*,
|
||||||
|
entity_label: str,
|
||||||
|
goal_text: str,
|
||||||
|
allowed_examples: list[str],
|
||||||
|
payload_key: str,
|
||||||
|
) -> str:
|
||||||
|
examples_text = "\n".join(f"- `{item}`" for item in allowed_examples)
|
||||||
|
payload_marker = f"<<{payload_key}>>"
|
||||||
|
return f"""
|
||||||
|
你是 PMFlow 的{entity_label}语义事实抽取器。
|
||||||
|
|
||||||
|
目标:
|
||||||
|
- {goal_text}
|
||||||
|
- 只输出有证据支撑的事实,不要脑补
|
||||||
|
- 如果没有明确事实,返回空数组
|
||||||
|
|
||||||
|
允许抽取的事实示例:
|
||||||
|
{examples_text}
|
||||||
|
|
||||||
|
输出要求:
|
||||||
|
- 只输出 JSON 对象
|
||||||
|
- `facts` 必须是数组
|
||||||
|
- 每个 fact 必须包含:
|
||||||
|
- `fact_namespace`
|
||||||
|
- `fact_key`
|
||||||
|
- `fact_value_text`
|
||||||
|
- `fact_value_norm`
|
||||||
|
- `value_type`
|
||||||
|
- `confidence`
|
||||||
|
- `evidence_excerpt`
|
||||||
|
|
||||||
|
JSON schema:
|
||||||
|
{{
|
||||||
|
"facts": [
|
||||||
|
{{
|
||||||
|
"fact_namespace": "domain | alias | purpose | audience | status | product | role",
|
||||||
|
"fact_key": "string",
|
||||||
|
"fact_value_text": "string",
|
||||||
|
"fact_value_norm": "string",
|
||||||
|
"value_type": "keyword | text",
|
||||||
|
"confidence": 0.0,
|
||||||
|
"evidence_excerpt": "string"
|
||||||
|
}}
|
||||||
|
]
|
||||||
|
}}
|
||||||
|
|
||||||
|
{payload_key}:
|
||||||
|
{payload_marker}
|
||||||
|
""".strip()
|
||||||
|
|
||||||
|
|
||||||
|
PROMPT_SEMANTIC_FACT_EXTRACT_PROJECT = _build_semantic_fact_extract_prompt(
|
||||||
|
entity_label="项目",
|
||||||
|
goal_text="从项目名称、标题、描述里抽取稳定、可复用、适合内部查询的语义事实",
|
||||||
|
allowed_examples=[
|
||||||
|
"market_theme:A股、港股、美股",
|
||||||
|
"business_domain:开票、财务、采购、投研",
|
||||||
|
"aliases:开票系统、A股项目、库存报表工具",
|
||||||
|
"audience:投顾团队、研究团队、财务运营",
|
||||||
|
"product_form:助手、平台、报表工具",
|
||||||
|
],
|
||||||
|
payload_key="project_payload",
|
||||||
|
)
|
||||||
|
|
||||||
|
PROMPT_SEMANTIC_FACT_EXTRACT_TASK = _build_semantic_fact_extract_prompt(
|
||||||
|
entity_label="任务",
|
||||||
|
goal_text="从任务标题、描述、状态、优先级里抽取任务主题、对象别名、交付目标和责任线索",
|
||||||
|
allowed_examples=[
|
||||||
|
"deliverable:开票接口、财务报表页、信号看板",
|
||||||
|
"business_domain:开票、财务、采购、风控",
|
||||||
|
"aliases:开票任务、A股需求任务",
|
||||||
|
"owner_role:后端、前端、AI",
|
||||||
|
"status_hint:待办、进行中、已完成",
|
||||||
|
],
|
||||||
|
payload_key="entity_payload",
|
||||||
|
)
|
||||||
|
|
||||||
|
PROMPT_SEMANTIC_FACT_EXTRACT_SIGNAL = _build_semantic_fact_extract_prompt(
|
||||||
|
entity_label="信号",
|
||||||
|
goal_text="从信号标题、摘要、正文、状态里抽取业务主题、问题对象、影响域和别名",
|
||||||
|
allowed_examples=[
|
||||||
|
"business_domain:开票、结算、投研、采购",
|
||||||
|
"problem_object:开票系统、报表平台、A股项目",
|
||||||
|
"aliases:开票需求、发票流程问题",
|
||||||
|
"signal_type:bug、需求、机会",
|
||||||
|
"status_hint:待处理、已关闭",
|
||||||
|
],
|
||||||
|
payload_key="entity_payload",
|
||||||
|
)
|
||||||
|
|
||||||
|
PROMPT_SEMANTIC_FACT_EXTRACT_DOCUMENT = _build_semantic_fact_extract_prompt(
|
||||||
|
entity_label="文档",
|
||||||
|
goal_text="从文档标题、来源、内容预览里抽取文档主题、业务对象、别名和适用团队",
|
||||||
|
allowed_examples=[
|
||||||
|
"business_domain:开票、财务、采购、投研",
|
||||||
|
"document_topic:实施方案、接口文档、复盘总结",
|
||||||
|
"aliases:开票系统文档、A股项目说明",
|
||||||
|
"audience:财务团队、研发团队、运营团队",
|
||||||
|
"source_form:文件、网页、知识库条目",
|
||||||
|
],
|
||||||
|
payload_key="entity_payload",
|
||||||
|
)
|
||||||
|
|
||||||
|
PROMPT_SEMANTIC_FACT_EXTRACT_USER = _build_semantic_fact_extract_prompt(
|
||||||
|
entity_label="人员",
|
||||||
|
goal_text="从成员姓名、角色、领域、简介里抽取稳定的职责、领域、别名和角色事实",
|
||||||
|
allowed_examples=[
|
||||||
|
"role_scope:后端、前端、AI、管理员",
|
||||||
|
"business_domain:财务、采购、投研",
|
||||||
|
"aliases:开票负责人、A股项目 owner",
|
||||||
|
"audience:财务团队、研发团队",
|
||||||
|
"status_hint:在岗、离线",
|
||||||
|
],
|
||||||
|
payload_key="entity_payload",
|
||||||
|
)
|
||||||
|
|
||||||
|
PROMPT_SEMANTIC_FACT_EXTRACT_REQUIREMENT = _build_semantic_fact_extract_prompt(
|
||||||
|
entity_label="需求宪章",
|
||||||
|
goal_text="从需求宪章标题、状态和内容里抽取需求主题、业务对象、别名和目标团队",
|
||||||
|
allowed_examples=[
|
||||||
|
"business_domain:开票、财务、采购、投研",
|
||||||
|
"target_object:开票系统、报表平台、A股项目",
|
||||||
|
"aliases:开票需求、开票平台需求",
|
||||||
|
"audience:财务团队、研究团队",
|
||||||
|
"status_hint:草稿、对齐中、已对齐",
|
||||||
|
],
|
||||||
|
payload_key="entity_payload",
|
||||||
|
)
|
||||||
|
|
||||||
|
PROMPT_SEMANTIC_FACT_EXTRACT_RELEASE = _build_semantic_fact_extract_prompt(
|
||||||
|
entity_label="发布",
|
||||||
|
goal_text="从发布标题、状态和内容里抽取发布对象、业务域、别名和阶段状态",
|
||||||
|
allowed_examples=[
|
||||||
|
"business_domain:开票、财务、采购、投研",
|
||||||
|
"release_object:开票系统、结算平台、A股工具",
|
||||||
|
"aliases:开票版本、财务发布",
|
||||||
|
"status_hint:pending、verified、online",
|
||||||
|
"audience:财务团队、运营团队",
|
||||||
|
],
|
||||||
|
payload_key="entity_payload",
|
||||||
|
)
|
||||||
|
|
||||||
|
PROMPT_SEMANTIC_FACT_EXTRACT_REVIEW = _build_semantic_fact_extract_prompt(
|
||||||
|
entity_label="复盘",
|
||||||
|
goal_text="从复盘标题、状态和内容里抽取复盘对象、业务主题、别名和适用团队",
|
||||||
|
allowed_examples=[
|
||||||
|
"business_domain:开票、财务、采购、投研",
|
||||||
|
"review_object:开票系统、A股项目、财务流程",
|
||||||
|
"aliases:开票复盘、阶段复盘",
|
||||||
|
"status_hint:draft、archived",
|
||||||
|
"audience:财务团队、研发团队",
|
||||||
|
],
|
||||||
|
payload_key="entity_payload",
|
||||||
|
)
|
||||||
|
|
||||||
PROMPT_RAG_ANSWER_SCENE_SEMANTICS = """
|
PROMPT_RAG_ANSWER_SCENE_SEMANTICS = """
|
||||||
你是 PMFlow Adaptive RAG 的 answer scene 规划层。
|
你是 PMFlow Adaptive RAG 的 answer scene 规划层。
|
||||||
|
|
||||||
|
|
@ -584,6 +834,8 @@ __all__ = [
|
||||||
"PMFLOW_SEMANTIC_CONTEXT_DECISION_PROMPT",
|
"PMFLOW_SEMANTIC_CONTEXT_DECISION_PROMPT",
|
||||||
"PROMPT_RAG_QUERY_SEMANTICS",
|
"PROMPT_RAG_QUERY_SEMANTICS",
|
||||||
"PROMPT_RAG_FOLLOWUP_ANCHOR_SEMANTICS",
|
"PROMPT_RAG_FOLLOWUP_ANCHOR_SEMANTICS",
|
||||||
|
"PROMPT_INTERNAL_TARGET_RESOLUTION",
|
||||||
|
"PROMPT_EXTERNAL_QUERY_PLAN",
|
||||||
"PROMPT_RAG_ANSWER_SCENE_SEMANTICS",
|
"PROMPT_RAG_ANSWER_SCENE_SEMANTICS",
|
||||||
"PMFLOW_REQUIREMENT_PLAN_ROUND_PROMPT",
|
"PMFLOW_REQUIREMENT_PLAN_ROUND_PROMPT",
|
||||||
"PMFLOW_REQUIREMENT_PLAN_DRAFT_PROMPT",
|
"PMFLOW_REQUIREMENT_PLAN_DRAFT_PROMPT",
|
||||||
|
|
|
||||||
|
|
@ -31,10 +31,12 @@ from .services.scheduler_center_core import (
|
||||||
WorkloadRecalcScheduler,
|
WorkloadRecalcScheduler,
|
||||||
)
|
)
|
||||||
from .services.skill import init_system_skill_tags
|
from .services.skill import init_system_skill_tags
|
||||||
|
from .services.semantic_fact_service import register_semantic_fact_hooks
|
||||||
|
|
||||||
configure_logging(settings.debug)
|
configure_logging(settings.debug)
|
||||||
logger = get_logger(__name__)
|
logger = get_logger(__name__)
|
||||||
register_context_cache_invalidation_hooks()
|
register_context_cache_invalidation_hooks()
|
||||||
|
register_semantic_fact_hooks()
|
||||||
|
|
||||||
|
|
||||||
@asynccontextmanager
|
@asynccontextmanager
|
||||||
|
|
|
||||||
|
|
@ -45,6 +45,9 @@ from .knowledge_base_doc import KnowledgeBaseDoc
|
||||||
from .knowledge_base_chunk import KnowledgeBaseChunk
|
from .knowledge_base_chunk import KnowledgeBaseChunk
|
||||||
from .qa_record import QARecord
|
from .qa_record import QARecord
|
||||||
from .qa_feedback import QAFeedback
|
from .qa_feedback import QAFeedback
|
||||||
|
from .query_route_memory import QueryRouteMemory
|
||||||
|
from .semantic_fact import SemanticFact
|
||||||
|
from .semantic_fact_extraction_run import SemanticFactExtractionRun
|
||||||
from .feedback_center import FeedbackCenterRecord, FeedbackCenterRecordRevision
|
from .feedback_center import FeedbackCenterRecord, FeedbackCenterRecordRevision
|
||||||
from .reply_iteration import QualityAsset, QualityAssetPair, JudgeRun, OptimizerCandidate, PromptRelease
|
from .reply_iteration import QualityAsset, QualityAssetPair, JudgeRun, OptimizerCandidate, PromptRelease
|
||||||
from .training_pipeline import (
|
from .training_pipeline import (
|
||||||
|
|
@ -130,6 +133,9 @@ __all__ = [
|
||||||
"KnowledgeBaseChunk",
|
"KnowledgeBaseChunk",
|
||||||
"QARecord",
|
"QARecord",
|
||||||
"QAFeedback",
|
"QAFeedback",
|
||||||
|
"QueryRouteMemory",
|
||||||
|
"SemanticFact",
|
||||||
|
"SemanticFactExtractionRun",
|
||||||
"FeedbackCenterRecord",
|
"FeedbackCenterRecord",
|
||||||
"FeedbackCenterRecordRevision",
|
"FeedbackCenterRecordRevision",
|
||||||
"QualityAsset",
|
"QualityAsset",
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,44 @@
|
||||||
|
"""问答路由记忆样本。"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any, Optional
|
||||||
|
|
||||||
|
from sqlalchemy import BigInteger, ForeignKey, Integer, Text, UniqueConstraint, VARCHAR
|
||||||
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
|
from .base import Base
|
||||||
|
from .types import JSONB
|
||||||
|
|
||||||
|
|
||||||
|
class QueryRouteMemory(Base):
|
||||||
|
__tablename__ = "query_route_memories"
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
qa_record_id: Mapped[int] = mapped_column(
|
||||||
|
Integer,
|
||||||
|
ForeignKey("qa_records.id", ondelete="CASCADE"),
|
||||||
|
nullable=False,
|
||||||
|
index=True,
|
||||||
|
)
|
||||||
|
project_id: Mapped[Optional[int]] = mapped_column(BigInteger, nullable=True, index=True)
|
||||||
|
question_text: Mapped[str] = mapped_column(Text, nullable=False)
|
||||||
|
route_mode_predicted: Mapped[str] = mapped_column(VARCHAR(32), nullable=False, index=True)
|
||||||
|
route_mode_effective: Mapped[str] = mapped_column(VARCHAR(32), nullable=False, index=True)
|
||||||
|
decision_origin: Mapped[Optional[str]] = mapped_column(VARCHAR(64), nullable=True, index=True)
|
||||||
|
reply_contract: Mapped[Optional[Any]] = mapped_column(JSONB, nullable=True)
|
||||||
|
probe_snapshot: Mapped[Optional[Any]] = mapped_column(JSONB, nullable=True)
|
||||||
|
query_profile: Mapped[Optional[Any]] = mapped_column(JSONB, nullable=True)
|
||||||
|
feedback_outcome: Mapped[Optional[Any]] = mapped_column(JSONB, nullable=True)
|
||||||
|
question_embedding: Mapped[Optional[Any]] = mapped_column(JSONB, nullable=True)
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint(
|
||||||
|
"tenant_id",
|
||||||
|
"qa_record_id",
|
||||||
|
name="uq_query_route_memories_tenant_qa_record",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["QueryRouteMemory"]
|
||||||
|
|
@ -0,0 +1,44 @@
|
||||||
|
"""语义事实读模型。"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
from sqlalchemy import BigInteger, Boolean, Float, Integer, Text, UniqueConstraint, VARCHAR
|
||||||
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
|
from .base import Base
|
||||||
|
|
||||||
|
|
||||||
|
class SemanticFact(Base):
|
||||||
|
__tablename__ = "semantic_facts"
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
entity_type: Mapped[str] = mapped_column(VARCHAR(32), nullable=False, index=True)
|
||||||
|
entity_id: Mapped[int] = mapped_column(BigInteger, nullable=False, index=True)
|
||||||
|
fact_namespace: Mapped[str] = mapped_column(VARCHAR(64), nullable=False, index=True)
|
||||||
|
fact_key: Mapped[str] = mapped_column(VARCHAR(64), nullable=False, index=True)
|
||||||
|
fact_value_text: Mapped[str] = mapped_column(Text, nullable=False)
|
||||||
|
fact_value_norm: Mapped[str] = mapped_column(VARCHAR(255), nullable=False, index=True)
|
||||||
|
value_type: Mapped[str] = mapped_column(VARCHAR(32), nullable=False, default="keyword")
|
||||||
|
confidence: Mapped[float] = mapped_column(Float, nullable=False, default=0.0)
|
||||||
|
evidence_source_type: Mapped[Optional[str]] = mapped_column(VARCHAR(32), nullable=True)
|
||||||
|
evidence_source_id: Mapped[Optional[int]] = mapped_column(BigInteger, nullable=True)
|
||||||
|
evidence_excerpt: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
|
||||||
|
extract_version: Mapped[Optional[str]] = mapped_column(VARCHAR(64), nullable=True)
|
||||||
|
freshness_at: Mapped[Optional[str]] = mapped_column(VARCHAR(64), nullable=True)
|
||||||
|
is_active: Mapped[bool] = mapped_column(Boolean, nullable=False, default=True, index=True)
|
||||||
|
|
||||||
|
__table_args__ = (
|
||||||
|
UniqueConstraint(
|
||||||
|
"tenant_id",
|
||||||
|
"entity_type",
|
||||||
|
"entity_id",
|
||||||
|
"fact_key",
|
||||||
|
"fact_value_norm",
|
||||||
|
name="uq_semantic_facts_tenant_entity_key_norm",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["SemanticFact"]
|
||||||
|
|
@ -0,0 +1,29 @@
|
||||||
|
"""语义事实抽取运行记录。"""
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
from sqlalchemy import BigInteger, Integer, Text, VARCHAR
|
||||||
|
from sqlalchemy.orm import Mapped, mapped_column
|
||||||
|
|
||||||
|
from .base import Base
|
||||||
|
|
||||||
|
|
||||||
|
class SemanticFactExtractionRun(Base):
|
||||||
|
__tablename__ = "semantic_fact_extraction_runs"
|
||||||
|
|
||||||
|
id: Mapped[int] = mapped_column(Integer, primary_key=True, autoincrement=True)
|
||||||
|
entity_type: Mapped[str] = mapped_column(VARCHAR(32), nullable=False, index=True)
|
||||||
|
entity_id: Mapped[int] = mapped_column(BigInteger, nullable=False, index=True)
|
||||||
|
trigger_type: Mapped[str] = mapped_column(VARCHAR(32), nullable=False, default="manual_retry")
|
||||||
|
status: Mapped[str] = mapped_column(VARCHAR(32), nullable=False, default="queued", index=True)
|
||||||
|
input_fingerprint: Mapped[Optional[str]] = mapped_column(VARCHAR(64), nullable=True, index=True)
|
||||||
|
facts_written: Mapped[int] = mapped_column(Integer, nullable=False, default=0)
|
||||||
|
failure_reason: Mapped[Optional[str]] = mapped_column(Text, nullable=True)
|
||||||
|
extract_version: Mapped[Optional[str]] = mapped_column(VARCHAR(64), nullable=True)
|
||||||
|
started_at: Mapped[Optional[str]] = mapped_column(VARCHAR(64), nullable=True)
|
||||||
|
finished_at: Mapped[Optional[str]] = mapped_column(VARCHAR(64), nullable=True)
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["SemanticFactExtractionRun"]
|
||||||
|
|
@ -68,6 +68,9 @@ class AdaptiveRAGAnswerEntryMixin:
|
||||||
message_intent: Optional[str] = None,
|
message_intent: Optional[str] = None,
|
||||||
mixed_signal_part: Optional[str] = None,
|
mixed_signal_part: Optional[str] = None,
|
||||||
consultant_mode_preference: Optional[str] = None,
|
consultant_mode_preference: Optional[str] = None,
|
||||||
|
source_policy: Optional[str] = None,
|
||||||
|
precomputed_query_semantics: Optional[dict[str, Any]] = None,
|
||||||
|
route_decision: Optional[dict[str, Any]] = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
started = time.perf_counter()
|
started = time.perf_counter()
|
||||||
clean_q = str(question or "").strip()
|
clean_q = str(question or "").strip()
|
||||||
|
|
@ -108,6 +111,9 @@ class AdaptiveRAGAnswerEntryMixin:
|
||||||
action_mode=action_mode,
|
action_mode=action_mode,
|
||||||
source_sender_open_id=source_sender_open_id,
|
source_sender_open_id=source_sender_open_id,
|
||||||
thread_context=thread_context,
|
thread_context=thread_context,
|
||||||
|
source_policy=source_policy,
|
||||||
|
precomputed_query_semantics=precomputed_query_semantics,
|
||||||
|
route_decision=route_decision,
|
||||||
)
|
)
|
||||||
clean_q = str(runtime.get("resolved_question") or clean_q).strip() or clean_q
|
clean_q = str(runtime.get("resolved_question") or clean_q).strip() or clean_q
|
||||||
session_id = runtime["session_id"]
|
session_id = runtime["session_id"]
|
||||||
|
|
@ -121,6 +127,7 @@ class AdaptiveRAGAnswerEntryMixin:
|
||||||
query_semantics = runtime["query_semantics"] if isinstance(runtime.get("query_semantics"), dict) else {}
|
query_semantics = runtime["query_semantics"] if isinstance(runtime.get("query_semantics"), dict) else {}
|
||||||
retrieval_domains = list(runtime.get("retrieval_domains") or [])
|
retrieval_domains = list(runtime.get("retrieval_domains") or [])
|
||||||
scene_semantics = runtime["scene_semantics"] if isinstance(runtime.get("scene_semantics"), dict) else {}
|
scene_semantics = runtime["scene_semantics"] if isinstance(runtime.get("scene_semantics"), dict) else {}
|
||||||
|
route_decision = runtime["route_decision"] if isinstance(runtime.get("route_decision"), dict) else {}
|
||||||
runtime_message_intent = str(runtime.get("message_intent") or "").strip().lower() or None
|
runtime_message_intent = str(runtime.get("message_intent") or "").strip().lower() or None
|
||||||
effective_message_intent = str(message_intent or runtime_message_intent or "").strip().lower() or None
|
effective_message_intent = str(message_intent or runtime_message_intent or "").strip().lower() or None
|
||||||
enforce_scene_prompt_only = self._should_enforce_scene_prompt_only(
|
enforce_scene_prompt_only = self._should_enforce_scene_prompt_only(
|
||||||
|
|
@ -181,4 +188,5 @@ class AdaptiveRAGAnswerEntryMixin:
|
||||||
mixed_signal_part=mixed_signal_part,
|
mixed_signal_part=mixed_signal_part,
|
||||||
anchor_resolution=anchor_resolution,
|
anchor_resolution=anchor_resolution,
|
||||||
consultant_mode_preference=consultant_mode_preference,
|
consultant_mode_preference=consultant_mode_preference,
|
||||||
|
route_decision=route_decision,
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -5,80 +5,150 @@ from .shared import *
|
||||||
|
|
||||||
|
|
||||||
class AdaptiveRAGAnswerMainlineMixin:
|
class AdaptiveRAGAnswerMainlineMixin:
|
||||||
_EXTERNAL_FACT_FRESHNESS_MARKERS = (
|
@staticmethod
|
||||||
"最新",
|
def _internal_answer_contract_for_query_type(query_type: str) -> str:
|
||||||
"目前",
|
normalized = str(query_type or QUERY_TYPE_UNKNOWN).strip().lower()
|
||||||
"当前",
|
if normalized == QUERY_TYPE_EXISTENCE:
|
||||||
"截至现在",
|
return "yes_no_with_evidence"
|
||||||
"截至当前",
|
if normalized == QUERY_TYPE_COUNT:
|
||||||
"截止当前",
|
return "count_scalar"
|
||||||
"今天",
|
if normalized == QUERY_TYPE_LIST:
|
||||||
"现在",
|
return "entity_list"
|
||||||
"最近",
|
if normalized == QUERY_TYPE_STATUS:
|
||||||
"出到什么",
|
return "entity_snapshot"
|
||||||
"发布了吗",
|
if normalized == QUERY_TYPE_EXPLAIN:
|
||||||
"发布到",
|
return "explanation_with_citations"
|
||||||
"版本",
|
if normalized == QUERY_TYPE_DIAGNOSE:
|
||||||
"最高模型",
|
return "diagnosis_with_evidence"
|
||||||
)
|
if normalized == QUERY_TYPE_COMPARE:
|
||||||
_MODEL_ENTITY_MARKERS = (
|
return "comparison_with_basis"
|
||||||
"codex",
|
return "clarify_or_fail"
|
||||||
"gpt",
|
|
||||||
"chatgpt",
|
@staticmethod
|
||||||
"claude",
|
def _internal_entity_type_for_subject_focus(subject_focus: str) -> str:
|
||||||
"deepseek",
|
normalized = str(subject_focus or "").strip().lower()
|
||||||
"qwen",
|
if normalized == "people":
|
||||||
"gemini",
|
return "user"
|
||||||
"copilot",
|
if normalized in {"project", "task", "signal", "feature"}:
|
||||||
"grok",
|
return normalized
|
||||||
"kimi",
|
return "unknown"
|
||||||
"llama",
|
|
||||||
"openai",
|
def _build_internal_query_plan(
|
||||||
"anthropic",
|
self,
|
||||||
)
|
*,
|
||||||
_INTERNAL_MODEL_CONFIG_MARKERS = (
|
question: str,
|
||||||
"ai-config",
|
project_id: Optional[int],
|
||||||
"/ai-config",
|
query_semantics: Optional[dict[str, Any]],
|
||||||
"模型配置",
|
operation_contract: Optional[dict[str, Any]] = None,
|
||||||
"配置项",
|
target_resolution: Optional[dict[str, Any]] = None,
|
||||||
"配置值",
|
) -> Optional[dict[str, Any]]:
|
||||||
"路由规则",
|
semantics = query_semantics if isinstance(query_semantics, dict) else {}
|
||||||
"初始化参数",
|
operation = operation_contract if isinstance(operation_contract, dict) else (
|
||||||
"llm服务",
|
semantics.get("operation_contract") if isinstance(semantics.get("operation_contract"), dict) else {}
|
||||||
"llm 服务",
|
)
|
||||||
"运行时",
|
target = target_resolution if isinstance(target_resolution, dict) else (
|
||||||
"当前系统",
|
semantics.get("target_resolution") if isinstance(semantics.get("target_resolution"), dict) else {}
|
||||||
"我们系统",
|
)
|
||||||
"pmflow",
|
source_policy = str(
|
||||||
"本系统",
|
semantics.get("effective_source_policy") or semantics.get("source_policy") or ""
|
||||||
"接入的模型",
|
).strip().lower()
|
||||||
"接的是",
|
if source_policy != "internal":
|
||||||
"用的是哪个模型配置",
|
return None
|
||||||
)
|
|
||||||
_PROJECT_CONTEXT_POSITIVE_MARKERS = (
|
query_type = str(
|
||||||
"这个项目",
|
operation.get("query_type")
|
||||||
"当前项目",
|
or semantics.get("query_type")
|
||||||
"本项目",
|
or QUERY_TYPE_UNKNOWN
|
||||||
"我们项目",
|
).strip().lower() or QUERY_TYPE_UNKNOWN
|
||||||
"项目所在行业",
|
subject_focus = self._query_subject_focus(semantics) if hasattr(self, "_query_subject_focus") else "unknown"
|
||||||
"对项目的影响",
|
entity_type = str(
|
||||||
"对这个项目的影响",
|
target.get("target_entity_type")
|
||||||
"对当前项目的影响",
|
or self._internal_entity_type_for_subject_focus(subject_focus)
|
||||||
"对我们项目的影响",
|
).strip().lower() or "unknown"
|
||||||
"结合项目",
|
scope_hint = self._query_scope_hint(semantics) if hasattr(self, "_query_scope_hint") else "none"
|
||||||
"结合当前项目",
|
target_ref = str(target.get("target_ref") or "").strip() or None
|
||||||
"落到项目",
|
|
||||||
)
|
filters: list[dict[str, Any]] = []
|
||||||
_PROJECT_CONTEXT_NEGATIVE_MARKERS = (
|
if entity_type == "project" and target_ref and target_ref.startswith("project:"):
|
||||||
"不要提项目",
|
try:
|
||||||
"别提项目",
|
resolved_project_id = int(target_ref.split(":", 1)[1])
|
||||||
"不要结合项目",
|
except Exception:
|
||||||
"不用结合项目",
|
resolved_project_id = None
|
||||||
"不需要结合项目",
|
if resolved_project_id is not None:
|
||||||
"先别结合项目",
|
filters.append(
|
||||||
"不要绑定项目",
|
{
|
||||||
"别绑定项目",
|
"field": "project_id",
|
||||||
)
|
"operator": "equals",
|
||||||
|
"value": resolved_project_id,
|
||||||
|
"value_source": "target_resolution",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
elif entity_type == "project":
|
||||||
|
mention_collector = getattr(self, "_collect_lookup_terms", None)
|
||||||
|
if callable(mention_collector):
|
||||||
|
operation_mentions = mention_collector(
|
||||||
|
question=question,
|
||||||
|
query_semantics=semantics,
|
||||||
|
operation_contract=operation,
|
||||||
|
target_resolution=target,
|
||||||
|
limit=3,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
operation_mentions = [
|
||||||
|
str(item or "").strip()
|
||||||
|
for item in list(operation.get("object_mentions") or [])
|
||||||
|
if str(item or "").strip()
|
||||||
|
]
|
||||||
|
if operation_mentions:
|
||||||
|
for raw in operation_mentions[:3]:
|
||||||
|
filters.append(
|
||||||
|
{
|
||||||
|
"field": "semantic_label",
|
||||||
|
"operator": "equals_or_alias",
|
||||||
|
"value": raw,
|
||||||
|
"value_source": "operation_contract",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
elif entity_type == "feature" and target_ref:
|
||||||
|
filters.append(
|
||||||
|
{
|
||||||
|
"field": "feature_ref",
|
||||||
|
"operator": "equals",
|
||||||
|
"value": target_ref,
|
||||||
|
"value_source": "target_resolution",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
elif entity_type == "config" and target_ref:
|
||||||
|
filters.append(
|
||||||
|
{
|
||||||
|
"field": "config_ref",
|
||||||
|
"operator": "equals",
|
||||||
|
"value": target_ref,
|
||||||
|
"value_source": "target_resolution",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
return {
|
||||||
|
"plan_version": "v2" if operation or target else "v1",
|
||||||
|
"query_type": query_type,
|
||||||
|
"entity_type": entity_type,
|
||||||
|
"answer_contract": self._internal_answer_contract_for_query_type(query_type),
|
||||||
|
"target_ref": target_ref,
|
||||||
|
"filters": filters,
|
||||||
|
"scope": {
|
||||||
|
"tenant_bound": True,
|
||||||
|
"project_id": int(project_id) if project_id is not None else None,
|
||||||
|
"scope_hint": scope_hint,
|
||||||
|
},
|
||||||
|
"retrieval_policy": {
|
||||||
|
"facts_first": True,
|
||||||
|
"allow_fallback_text": True,
|
||||||
|
},
|
||||||
|
"verifier_checks": [
|
||||||
|
"contract_matches_query_type",
|
||||||
|
"evidence_backed",
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def _extract_prompt_variant_from_policy_version(cls, policy_version: Optional[str]) -> Optional[str]:
|
def _extract_prompt_variant_from_policy_version(cls, policy_version: Optional[str]) -> Optional[str]:
|
||||||
|
|
@ -93,46 +163,6 @@ class AdaptiveRAGAnswerMainlineMixin:
|
||||||
return value or None
|
return value or None
|
||||||
return None
|
return None
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def _question_explicitly_requests_project_context(
|
|
||||||
cls,
|
|
||||||
*,
|
|
||||||
question: str,
|
|
||||||
project_id: Optional[int],
|
|
||||||
) -> bool:
|
|
||||||
if project_id is None:
|
|
||||||
return False
|
|
||||||
normalized = re.sub(r"\s+", "", str(question or "")).strip().lower()
|
|
||||||
if not normalized:
|
|
||||||
return False
|
|
||||||
if any(marker in normalized for marker in cls._PROJECT_CONTEXT_NEGATIVE_MARKERS):
|
|
||||||
return False
|
|
||||||
return any(marker in normalized for marker in cls._PROJECT_CONTEXT_POSITIVE_MARKERS)
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def _looks_like_external_freshness_question(cls, question: str) -> bool:
|
|
||||||
normalized = re.sub(r"\s+", "", str(question or "")).strip().lower()
|
|
||||||
if not normalized:
|
|
||||||
return False
|
|
||||||
return any(marker in normalized for marker in cls._EXTERNAL_FACT_FRESHNESS_MARKERS)
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def _looks_like_internal_model_config_question(cls, question: str) -> bool:
|
|
||||||
normalized = re.sub(r"\s+", "", str(question or "")).strip().lower()
|
|
||||||
if not normalized:
|
|
||||||
return False
|
|
||||||
return any(marker in normalized for marker in cls._INTERNAL_MODEL_CONFIG_MARKERS)
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def _looks_like_model_fact_question(cls, question: str) -> bool:
|
|
||||||
normalized = re.sub(r"\s+", "", str(question or "")).strip().lower()
|
|
||||||
if not normalized:
|
|
||||||
return False
|
|
||||||
has_entity = any(marker in normalized for marker in cls._MODEL_ENTITY_MARKERS)
|
|
||||||
if not has_entity:
|
|
||||||
return False
|
|
||||||
return "模型" in normalized or cls._looks_like_external_freshness_question(question)
|
|
||||||
|
|
||||||
async def _resolve_anchor_prompt_variant(
|
async def _resolve_anchor_prompt_variant(
|
||||||
self,
|
self,
|
||||||
*,
|
*,
|
||||||
|
|
@ -161,40 +191,18 @@ class AdaptiveRAGAnswerMainlineMixin:
|
||||||
scene_semantics: Optional[dict[str, Any]],
|
scene_semantics: Optional[dict[str, Any]],
|
||||||
anchor_prompt_variant: Optional[str] = None,
|
anchor_prompt_variant: Optional[str] = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
|
_ = question
|
||||||
normalized = dict(scene_semantics or {})
|
normalized = dict(scene_semantics or {})
|
||||||
scene_key = str(normalized.get("scene_key") or "default").strip().lower() or "default"
|
scene_key = str(normalized.get("scene_key") or "default").strip().lower() or "default"
|
||||||
anchor_variant = str(anchor_prompt_variant or "").strip().lower()
|
anchor_variant = str(anchor_prompt_variant or "").strip().lower()
|
||||||
explicit_project_context = self._question_explicitly_requests_project_context(
|
if scene_key in {"ai_pmo", "ai_pmo_correction"} and anchor_variant in {"external_trend", "industry_trend"}:
|
||||||
question=question,
|
scene_key = anchor_variant
|
||||||
project_id=project_id,
|
|
||||||
)
|
|
||||||
requires_freshness = self._looks_like_external_freshness_question(question)
|
|
||||||
internal_model_config_question = self._looks_like_internal_model_config_question(question)
|
|
||||||
model_fact_question = self._looks_like_model_fact_question(question)
|
|
||||||
anchored_trend_followup = anchor_variant in {"external_trend", "industry_trend"}
|
|
||||||
|
|
||||||
if scene_key == "ai_pmo" and model_fact_question and requires_freshness and not internal_model_config_question:
|
|
||||||
scene_key = "external_trend" if explicit_project_context else "industry_trend"
|
|
||||||
normalized["snapshot_focus"] = "none"
|
normalized["snapshot_focus"] = "none"
|
||||||
normalized["deep_intent"] = "none"
|
normalized["deep_intent"] = "none"
|
||||||
normalized["external_fact_chat"] = True
|
if scene_key == "external_trend" and project_id is None:
|
||||||
normalized["requires_freshness"] = True
|
|
||||||
|
|
||||||
if scene_key == "ai_pmo_correction" and anchored_trend_followup:
|
|
||||||
scene_key = "external_trend" if explicit_project_context else "industry_trend"
|
|
||||||
normalized["snapshot_focus"] = "none"
|
|
||||||
normalized["deep_intent"] = "none"
|
|
||||||
normalized["external_fact_chat"] = True
|
|
||||||
|
|
||||||
if scene_key == "external_trend" and not explicit_project_context:
|
|
||||||
scene_key = "industry_trend"
|
scene_key = "industry_trend"
|
||||||
|
|
||||||
if scene_key in {"external_trend", "industry_trend"}:
|
if scene_key in {"external_trend", "industry_trend"}:
|
||||||
normalized["external_fact_chat"] = True
|
normalized["external_fact_chat"] = True
|
||||||
normalized["requires_freshness"] = bool(requires_freshness)
|
|
||||||
if requires_freshness:
|
|
||||||
normalized["deep_intent"] = "none"
|
|
||||||
|
|
||||||
normalized["scene_key"] = scene_key
|
normalized["scene_key"] = scene_key
|
||||||
return normalized
|
return normalized
|
||||||
|
|
||||||
|
|
@ -217,7 +225,12 @@ class AdaptiveRAGAnswerMainlineMixin:
|
||||||
normalized_thinking_mode = normalize_thinking_mode(requested_thinking_mode)
|
normalized_thinking_mode = normalize_thinking_mode(requested_thinking_mode)
|
||||||
normalized_query_type = str(query_type or QUERY_TYPE_UNKNOWN).strip().lower()
|
normalized_query_type = str(query_type or QUERY_TYPE_UNKNOWN).strip().lower()
|
||||||
normalized_consultant_mode = str(consultant_mode_preference or "").strip().lower()
|
normalized_consultant_mode = str(consultant_mode_preference or "").strip().lower()
|
||||||
fast_path_candidate = normalized_query_type in {QUERY_TYPE_COUNT, QUERY_TYPE_LIST, QUERY_TYPE_STATUS}
|
fast_path_candidate = normalized_query_type in {
|
||||||
|
QUERY_TYPE_EXISTENCE,
|
||||||
|
QUERY_TYPE_COUNT,
|
||||||
|
QUERY_TYPE_LIST,
|
||||||
|
QUERY_TYPE_STATUS,
|
||||||
|
}
|
||||||
normalized_question = str(question or "").strip()
|
normalized_question = str(question or "").strip()
|
||||||
normalized_effective = str(effective_question or "").strip()
|
normalized_effective = str(effective_question or "").strip()
|
||||||
|
|
||||||
|
|
@ -290,6 +303,9 @@ class AdaptiveRAGAnswerMainlineMixin:
|
||||||
source_message_id: Optional[str] = None,
|
source_message_id: Optional[str] = None,
|
||||||
source_sender_open_id: Optional[str] = None,
|
source_sender_open_id: Optional[str] = None,
|
||||||
thread_context: Optional[dict[str, Any]] = None,
|
thread_context: Optional[dict[str, Any]] = None,
|
||||||
|
source_policy: Optional[str] = None,
|
||||||
|
precomputed_query_semantics: Optional[dict[str, Any]] = None,
|
||||||
|
route_decision: Optional[dict[str, Any]] = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
external_conversation_key = self._build_external_conversation_key(
|
external_conversation_key = self._build_external_conversation_key(
|
||||||
asked_via=asked_via,
|
asked_via=asked_via,
|
||||||
|
|
@ -358,20 +374,57 @@ class AdaptiveRAGAnswerMainlineMixin:
|
||||||
session_state=state_row,
|
session_state=state_row,
|
||||||
anchor_resolution=anchor_resolution,
|
anchor_resolution=anchor_resolution,
|
||||||
)
|
)
|
||||||
query_semantics = await self._resolve_query_semantics(
|
query_semantics = dict(precomputed_query_semantics or {}) if isinstance(precomputed_query_semantics, dict) else {}
|
||||||
message=effective_q,
|
route_preview = await self.preview_query_route_decision(
|
||||||
source_system=asked_via,
|
question=effective_q,
|
||||||
metadata={
|
asked_via=asked_via,
|
||||||
"project_id": project_id,
|
project_id=project_id,
|
||||||
"source_chat_id": source_chat_id,
|
source_chat_id=source_chat_id,
|
||||||
"source_message_id": source_message_id,
|
source_message_id=source_message_id,
|
||||||
"conversation_scope_key": external_conversation_key,
|
conversation_scope_key=external_conversation_key,
|
||||||
"reply_to_message_id": str((thread_context or {}).get("reply_to_message_id") or "").strip() or None,
|
reply_to_message_id=str((thread_context or {}).get("reply_to_message_id") or "").strip() or None,
|
||||||
"root_message_id": str((thread_context or {}).get("root_message_id") or "").strip() or None,
|
root_message_id=str((thread_context or {}).get("root_message_id") or "").strip() or None,
|
||||||
},
|
source_policy_hint=source_policy,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
) if not isinstance(route_decision, dict) or not query_semantics else {
|
||||||
|
"query_semantics": self._apply_route_decision_to_query_semantics(
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
route_decision=route_decision,
|
||||||
|
),
|
||||||
|
"route_decision": self._normalize_query_route_decision(route_decision),
|
||||||
|
}
|
||||||
|
query_semantics = (
|
||||||
|
dict(route_preview.get("query_semantics") or {})
|
||||||
|
if isinstance(route_preview.get("query_semantics"), dict)
|
||||||
|
else {}
|
||||||
)
|
)
|
||||||
query_semantics = dict(query_semantics) if isinstance(query_semantics, dict) else {}
|
operation_contract = None
|
||||||
query_type = str(query_semantics.get("query_type") or QUERY_TYPE_UNKNOWN).strip().lower() or QUERY_TYPE_UNKNOWN
|
operation_contract_builder = getattr(self, "_build_operation_contract", None)
|
||||||
|
if callable(operation_contract_builder):
|
||||||
|
operation_contract = operation_contract_builder(
|
||||||
|
question=effective_q,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
)
|
||||||
|
if isinstance(operation_contract, dict):
|
||||||
|
query_semantics["operation_contract"] = operation_contract
|
||||||
|
internal_query_plan = self._build_internal_query_plan(
|
||||||
|
question=effective_q,
|
||||||
|
project_id=project_id,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
operation_contract=operation_contract,
|
||||||
|
)
|
||||||
|
if internal_query_plan:
|
||||||
|
query_semantics["query_plan"] = internal_query_plan
|
||||||
|
route_decision = (
|
||||||
|
dict(route_preview.get("route_decision") or {})
|
||||||
|
if isinstance(route_preview.get("route_decision"), dict)
|
||||||
|
else {}
|
||||||
|
)
|
||||||
|
query_type = str(
|
||||||
|
(operation_contract or {}).get("query_type")
|
||||||
|
or query_semantics.get("query_type")
|
||||||
|
or QUERY_TYPE_UNKNOWN
|
||||||
|
).strip().lower() or QUERY_TYPE_UNKNOWN
|
||||||
conversation_text = "\n".join(
|
conversation_text = "\n".join(
|
||||||
[
|
[
|
||||||
f"Q: {str(row.get('question') or '').strip()}\nA: {str(row.get('answer') or '').strip()}"
|
f"Q: {str(row.get('question') or '').strip()}\nA: {str(row.get('answer') or '').strip()}"
|
||||||
|
|
@ -386,6 +439,16 @@ class AdaptiveRAGAnswerMainlineMixin:
|
||||||
message_intent=str(query_semantics.get("message_intent") or query_semantics.get("intent") or "").strip().lower() or None,
|
message_intent=str(query_semantics.get("message_intent") or query_semantics.get("intent") or "").strip().lower() or None,
|
||||||
query_type=query_type,
|
query_type=query_type,
|
||||||
conversation_text=conversation_text,
|
conversation_text=conversation_text,
|
||||||
|
source_policy=str(
|
||||||
|
route_decision.get("effective_source_policy")
|
||||||
|
or query_semantics.get("effective_source_policy")
|
||||||
|
or query_semantics.get("source_policy")
|
||||||
|
or source_policy
|
||||||
|
or ""
|
||||||
|
).strip().lower()
|
||||||
|
or None,
|
||||||
|
project_id=project_id,
|
||||||
|
query_semantics=query_semantics,
|
||||||
)
|
)
|
||||||
anchor_prompt_variant = await self._resolve_anchor_prompt_variant(
|
anchor_prompt_variant = await self._resolve_anchor_prompt_variant(
|
||||||
anchor_resolution=anchor_resolution,
|
anchor_resolution=anchor_resolution,
|
||||||
|
|
@ -400,6 +463,24 @@ class AdaptiveRAGAnswerMainlineMixin:
|
||||||
if bool(scene_semantics.get("requires_freshness")):
|
if bool(scene_semantics.get("requires_freshness")):
|
||||||
query_semantics["external_freshness_only"] = True
|
query_semantics["external_freshness_only"] = True
|
||||||
query_semantics["retrieval_domains"] = ["url"]
|
query_semantics["retrieval_domains"] = ["url"]
|
||||||
|
query_semantics["source_policy"] = "external"
|
||||||
|
query_semantics["effective_source_policy"] = "external"
|
||||||
|
route_decision = self._normalize_query_route_decision(
|
||||||
|
{
|
||||||
|
**route_decision,
|
||||||
|
"route_mode": str(route_decision.get("route_mode") or "external"),
|
||||||
|
"effective_source_policy": "external",
|
||||||
|
"decision_origin": str(route_decision.get("decision_origin") or "freshness_guard"),
|
||||||
|
"reply_contract": {
|
||||||
|
"mode": "explicit_source",
|
||||||
|
"head": "外部实时查询:",
|
||||||
|
"tail_hint": None,
|
||||||
|
},
|
||||||
|
"query_profile": query_semantics.get("query_profile"),
|
||||||
|
"probe_snapshot_summary": route_decision.get("probe_snapshot_summary"),
|
||||||
|
"probe_snapshot": route_decision.get("probe_snapshot"),
|
||||||
|
}
|
||||||
|
)
|
||||||
requested_action_mode = self._normalize_action_mode(action_mode)
|
requested_action_mode = self._normalize_action_mode(action_mode)
|
||||||
effective_action_mode = await self._resolve_action_mode_async(
|
effective_action_mode = await self._resolve_action_mode_async(
|
||||||
requested_mode=requested_action_mode,
|
requested_mode=requested_action_mode,
|
||||||
|
|
@ -423,6 +504,7 @@ class AdaptiveRAGAnswerMainlineMixin:
|
||||||
"scene_semantics": scene_semantics if isinstance(scene_semantics, dict) else {},
|
"scene_semantics": scene_semantics if isinstance(scene_semantics, dict) else {},
|
||||||
"effective_action_mode": effective_action_mode,
|
"effective_action_mode": effective_action_mode,
|
||||||
"anchor_resolution": anchor_resolution,
|
"anchor_resolution": anchor_resolution,
|
||||||
|
"route_decision": route_decision,
|
||||||
}
|
}
|
||||||
|
|
||||||
async def _execute_base_answer_mainline(
|
async def _execute_base_answer_mainline(
|
||||||
|
|
@ -455,7 +537,26 @@ class AdaptiveRAGAnswerMainlineMixin:
|
||||||
mixed_signal_part: Optional[str],
|
mixed_signal_part: Optional[str],
|
||||||
anchor_resolution: Optional[FollowupAnchorResolution] = None,
|
anchor_resolution: Optional[FollowupAnchorResolution] = None,
|
||||||
consultant_mode_preference: Optional[str] = None,
|
consultant_mode_preference: Optional[str] = None,
|
||||||
|
route_decision: Optional[dict[str, Any]] = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
|
operation_contract = (
|
||||||
|
dict(query_semantics.get("operation_contract") or {})
|
||||||
|
if isinstance(query_semantics.get("operation_contract"), dict)
|
||||||
|
else {}
|
||||||
|
)
|
||||||
|
operation_contract_builder = getattr(self, "_build_operation_contract", None)
|
||||||
|
if not operation_contract and callable(operation_contract_builder):
|
||||||
|
operation_contract = operation_contract_builder(
|
||||||
|
question=effective_q,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
)
|
||||||
|
if isinstance(operation_contract, dict):
|
||||||
|
query_semantics["operation_contract"] = operation_contract
|
||||||
|
query_type = str(
|
||||||
|
operation_contract.get("query_type")
|
||||||
|
or query_type
|
||||||
|
or QUERY_TYPE_UNKNOWN
|
||||||
|
).strip().lower() or QUERY_TYPE_UNKNOWN
|
||||||
planner = self._plan_query_runtime(
|
planner = self._plan_query_runtime(
|
||||||
question=clean_q,
|
question=clean_q,
|
||||||
effective_question=effective_q,
|
effective_question=effective_q,
|
||||||
|
|
@ -483,11 +584,31 @@ class AdaptiveRAGAnswerMainlineMixin:
|
||||||
anchor_resolution=anchor_resolution,
|
anchor_resolution=anchor_resolution,
|
||||||
session_state=state_row,
|
session_state=state_row,
|
||||||
)
|
)
|
||||||
|
route_clarification_payload = self._build_route_clarification_payload(route_decision=route_decision)
|
||||||
meta_followup_payload = await self._build_meta_followup_payload(
|
meta_followup_payload = await self._build_meta_followup_payload(
|
||||||
question=clean_q,
|
question=clean_q,
|
||||||
anchor_resolution=anchor_resolution,
|
anchor_resolution=anchor_resolution,
|
||||||
conversation_scope_key=self._get_active_external_qa_context().get("conversation_scope_key"),
|
conversation_scope_key=self._get_active_external_qa_context().get("conversation_scope_key"),
|
||||||
)
|
)
|
||||||
|
clarification_result = await self._maybe_return_clarification_response(
|
||||||
|
clarification_payload=clarification_payload or route_clarification_payload,
|
||||||
|
started=started,
|
||||||
|
clean_q=clean_q,
|
||||||
|
effective_q=effective_q,
|
||||||
|
query_type=query_type,
|
||||||
|
asked_by=asked_by,
|
||||||
|
asked_via=asked_via,
|
||||||
|
project_id=project_id,
|
||||||
|
session_id=session_id,
|
||||||
|
turn_index=turn_index,
|
||||||
|
state_row=state_row,
|
||||||
|
rewrite_meta=rewrite_meta,
|
||||||
|
context_session_id=context_session_id,
|
||||||
|
active_thinking_mode=active_thinking_mode,
|
||||||
|
route_decision=route_decision,
|
||||||
|
)
|
||||||
|
if clarification_result is not None:
|
||||||
|
return clarification_result
|
||||||
meta_followup_result = await self._maybe_return_meta_followup_response(
|
meta_followup_result = await self._maybe_return_meta_followup_response(
|
||||||
meta_followup_payload=meta_followup_payload,
|
meta_followup_payload=meta_followup_payload,
|
||||||
started=started,
|
started=started,
|
||||||
|
|
@ -503,27 +624,10 @@ class AdaptiveRAGAnswerMainlineMixin:
|
||||||
rewrite_meta=rewrite_meta,
|
rewrite_meta=rewrite_meta,
|
||||||
context_session_id=context_session_id,
|
context_session_id=context_session_id,
|
||||||
active_thinking_mode=active_thinking_mode,
|
active_thinking_mode=active_thinking_mode,
|
||||||
|
route_decision=route_decision,
|
||||||
)
|
)
|
||||||
if meta_followup_result is not None:
|
if meta_followup_result is not None:
|
||||||
return meta_followup_result
|
return meta_followup_result
|
||||||
clarification_result = await self._maybe_return_clarification_response(
|
|
||||||
clarification_payload=clarification_payload,
|
|
||||||
started=started,
|
|
||||||
clean_q=clean_q,
|
|
||||||
effective_q=effective_q,
|
|
||||||
query_type=query_type,
|
|
||||||
asked_by=asked_by,
|
|
||||||
asked_via=asked_via,
|
|
||||||
project_id=project_id,
|
|
||||||
session_id=session_id,
|
|
||||||
turn_index=turn_index,
|
|
||||||
state_row=state_row,
|
|
||||||
rewrite_meta=rewrite_meta,
|
|
||||||
context_session_id=context_session_id,
|
|
||||||
active_thinking_mode=active_thinking_mode,
|
|
||||||
)
|
|
||||||
if clarification_result is not None:
|
|
||||||
return clarification_result
|
|
||||||
|
|
||||||
system_snapshot = await self.context_cache.get_system_snapshot(
|
system_snapshot = await self.context_cache.get_system_snapshot(
|
||||||
db=self.db,
|
db=self.db,
|
||||||
|
|
@ -534,8 +638,44 @@ class AdaptiveRAGAnswerMainlineMixin:
|
||||||
tenant_id=self.tenant_id,
|
tenant_id=self.tenant_id,
|
||||||
user_id=asked_by,
|
user_id=asked_by,
|
||||||
)
|
)
|
||||||
|
prefers_external_retrieval = (
|
||||||
|
str(query_semantics.get("effective_source_policy") or query_semantics.get("source_policy") or "").strip().lower()
|
||||||
|
== "external"
|
||||||
|
)
|
||||||
|
target_resolution = (
|
||||||
|
dict(query_semantics.get("target_resolution") or {})
|
||||||
|
if isinstance(query_semantics.get("target_resolution"), dict)
|
||||||
|
else {}
|
||||||
|
)
|
||||||
|
target_resolver = getattr(self, "_resolve_internal_target", None)
|
||||||
|
if not prefers_external_retrieval and callable(target_resolver):
|
||||||
|
target_resolution = await target_resolver(
|
||||||
|
question=effective_q,
|
||||||
|
project_id=project_id,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
system_snapshot=system_snapshot,
|
||||||
|
operation_contract=operation_contract,
|
||||||
|
)
|
||||||
|
if isinstance(target_resolution, dict):
|
||||||
|
query_semantics["target_resolution"] = target_resolution
|
||||||
|
patch_scene_semantics = getattr(self, "_patch_scene_semantics_with_target_resolution", None)
|
||||||
|
if callable(patch_scene_semantics):
|
||||||
|
scene_semantics = patch_scene_semantics(
|
||||||
|
scene_semantics=scene_semantics,
|
||||||
|
operation_contract=operation_contract,
|
||||||
|
target_resolution=target_resolution,
|
||||||
|
)
|
||||||
|
internal_query_plan = self._build_internal_query_plan(
|
||||||
|
question=effective_q,
|
||||||
|
project_id=project_id,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
operation_contract=operation_contract,
|
||||||
|
target_resolution=target_resolution,
|
||||||
|
)
|
||||||
|
if internal_query_plan:
|
||||||
|
query_semantics["query_plan"] = internal_query_plan
|
||||||
|
|
||||||
if planner.allow_fast_path:
|
if planner.allow_fast_path and not prefers_external_retrieval:
|
||||||
deterministic_result = await self._run_deterministic_query(
|
deterministic_result = await self._run_deterministic_query(
|
||||||
question=effective_q,
|
question=effective_q,
|
||||||
project_id=project_id,
|
project_id=project_id,
|
||||||
|
|
@ -559,37 +699,78 @@ class AdaptiveRAGAnswerMainlineMixin:
|
||||||
rewrite_meta=rewrite_meta,
|
rewrite_meta=rewrite_meta,
|
||||||
thinking_mode=active_thinking_mode,
|
thinking_mode=active_thinking_mode,
|
||||||
answer_mode=ANSWER_MODE_FAST_PATH,
|
answer_mode=ANSWER_MODE_FAST_PATH,
|
||||||
|
route_decision=route_decision,
|
||||||
|
query_semantics=query_semantics,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
internal_model_config_fast_answer = None
|
||||||
|
internal_model_config_builder = getattr(self, "_build_internal_model_config_fast_answer", None)
|
||||||
|
if not prefers_external_retrieval and callable(internal_model_config_builder):
|
||||||
|
internal_model_config_fast_answer = await internal_model_config_builder(
|
||||||
|
question=effective_q,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
)
|
||||||
|
if internal_model_config_fast_answer is not None:
|
||||||
|
return await self._persist_fast_answer(
|
||||||
|
question=clean_q,
|
||||||
|
asked_by=asked_by,
|
||||||
|
asked_via=asked_via,
|
||||||
|
project_id=project_id,
|
||||||
|
fast_result=internal_model_config_fast_answer,
|
||||||
|
session_id=session_id,
|
||||||
|
external_session_id=context_session_id,
|
||||||
|
turn_index=turn_index,
|
||||||
|
query_type=QUERY_TYPE_STATUS,
|
||||||
|
missing_fields=[],
|
||||||
|
state_row=state_row,
|
||||||
|
effective_question=effective_q,
|
||||||
|
rewrite_meta=rewrite_meta,
|
||||||
|
thinking_mode=active_thinking_mode,
|
||||||
|
answer_mode=ANSWER_MODE_FAST_PATH,
|
||||||
|
route_decision=route_decision,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
)
|
||||||
|
|
||||||
direct_answer = None
|
direct_answer = None
|
||||||
if planner.allow_fast_path:
|
allow_snapshot_direct_answer = planner.allow_fast_path and not prefers_external_retrieval
|
||||||
|
snapshot_direct_guard = getattr(self, "_should_allow_snapshot_direct_answer", None)
|
||||||
|
if callable(snapshot_direct_guard):
|
||||||
|
allow_snapshot_direct_answer = allow_snapshot_direct_answer and snapshot_direct_guard(
|
||||||
|
scene_semantics=scene_semantics,
|
||||||
|
operation_contract=operation_contract,
|
||||||
|
target_resolution=target_resolution,
|
||||||
|
)
|
||||||
|
if allow_snapshot_direct_answer:
|
||||||
direct_answer = self._build_snapshot_direct_answer(
|
direct_answer = self._build_snapshot_direct_answer(
|
||||||
effective_q,
|
effective_q,
|
||||||
system_snapshot=system_snapshot,
|
system_snapshot=system_snapshot,
|
||||||
user_snapshot=user_snapshot,
|
user_snapshot=user_snapshot,
|
||||||
snapshot_focus=str((scene_semantics or {}).get("snapshot_focus") or "none"),
|
snapshot_focus=str((scene_semantics or {}).get("snapshot_focus") or "none"),
|
||||||
)
|
)
|
||||||
direct_result = await self._maybe_return_snapshot_direct_response(
|
direct_result = None
|
||||||
direct_answer=direct_answer,
|
if allow_snapshot_direct_answer:
|
||||||
started=started,
|
direct_result = await self._maybe_return_snapshot_direct_response(
|
||||||
clean_q=clean_q,
|
direct_answer=direct_answer,
|
||||||
effective_q=effective_q,
|
started=started,
|
||||||
asked_by=asked_by,
|
clean_q=clean_q,
|
||||||
asked_via=asked_via,
|
effective_q=effective_q,
|
||||||
project_id=project_id,
|
asked_by=asked_by,
|
||||||
session_id=session_id,
|
asked_via=asked_via,
|
||||||
turn_index=turn_index,
|
project_id=project_id,
|
||||||
state_row=state_row,
|
session_id=session_id,
|
||||||
rewrite_meta=rewrite_meta,
|
turn_index=turn_index,
|
||||||
context_session_id=context_session_id,
|
state_row=state_row,
|
||||||
active_thinking_mode=active_thinking_mode,
|
rewrite_meta=rewrite_meta,
|
||||||
user_snapshot=user_snapshot,
|
context_session_id=context_session_id,
|
||||||
)
|
active_thinking_mode=active_thinking_mode,
|
||||||
|
user_snapshot=user_snapshot,
|
||||||
|
route_decision=route_decision,
|
||||||
|
)
|
||||||
if direct_result is not None:
|
if direct_result is not None:
|
||||||
return direct_result
|
return direct_result
|
||||||
|
|
||||||
is_snapshot_direct_question = str((scene_semantics or {}).get("snapshot_focus") or "none").strip().lower() != "none"
|
is_snapshot_direct_question = str((scene_semantics or {}).get("snapshot_focus") or "none").strip().lower() != "none"
|
||||||
if planner.allow_fast_path and not is_snapshot_direct_question:
|
if planner.allow_fast_path and not prefers_external_retrieval and not is_snapshot_direct_question:
|
||||||
glm_semantic_result = await self._try_glm_project_semantic_answer(
|
glm_semantic_result = await self._try_glm_project_semantic_answer(
|
||||||
clean_q=effective_q,
|
clean_q=effective_q,
|
||||||
project_id=project_id,
|
project_id=project_id,
|
||||||
|
|
@ -612,6 +793,8 @@ class AdaptiveRAGAnswerMainlineMixin:
|
||||||
rewrite_meta=rewrite_meta,
|
rewrite_meta=rewrite_meta,
|
||||||
thinking_mode=active_thinking_mode,
|
thinking_mode=active_thinking_mode,
|
||||||
answer_mode=ANSWER_MODE_FAST_PATH,
|
answer_mode=ANSWER_MODE_FAST_PATH,
|
||||||
|
route_decision=route_decision,
|
||||||
|
query_semantics=query_semantics,
|
||||||
)
|
)
|
||||||
|
|
||||||
return await self._build_retrieval_response(
|
return await self._build_retrieval_response(
|
||||||
|
|
@ -644,4 +827,5 @@ class AdaptiveRAGAnswerMainlineMixin:
|
||||||
mixed_signal_part=mixed_signal_part,
|
mixed_signal_part=mixed_signal_part,
|
||||||
answer_mode=planner.retrieval_answer_mode,
|
answer_mode=planner.retrieval_answer_mode,
|
||||||
consultant_mode_preference=consultant_mode_preference,
|
consultant_mode_preference=consultant_mode_preference,
|
||||||
|
route_decision=route_decision,
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -10,10 +10,12 @@ class AdaptiveRAGAnswerPayloadMixin:
|
||||||
"db": 90,
|
"db": 90,
|
||||||
"graph": 80,
|
"graph": 80,
|
||||||
"doc": 70,
|
"doc": 70,
|
||||||
|
"mcp_search": 68,
|
||||||
"url": 65,
|
"url": 65,
|
||||||
"feishu": 60,
|
"feishu": 60,
|
||||||
"system": 10,
|
"system": 10,
|
||||||
}
|
}
|
||||||
|
_INTERNAL_SOURCE_TYPES = {"db", "graph", "doc", "pmflow", "feishu"}
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _safe_source_metadata(raw: Any) -> dict[str, Any]:
|
def _safe_source_metadata(raw: Any) -> dict[str, Any]:
|
||||||
|
|
@ -69,7 +71,7 @@ class AdaptiveRAGAnswerPayloadMixin:
|
||||||
if record_type and record_id is not None:
|
if record_type and record_id is not None:
|
||||||
return f"{record_type}:{record_id}"
|
return f"{record_type}:{record_id}"
|
||||||
return record_id
|
return record_id
|
||||||
if source_type in {"doc", "url"}:
|
if source_type in {"doc", "url", "mcp_search"}:
|
||||||
return self._source_ref_value(
|
return self._source_ref_value(
|
||||||
metadata.get("doc_id")
|
metadata.get("doc_id")
|
||||||
or metadata.get("knowledge_doc_id")
|
or metadata.get("knowledge_doc_id")
|
||||||
|
|
@ -366,6 +368,191 @@ class AdaptiveRAGAnswerPayloadMixin:
|
||||||
"tool_trace": tool_trace if isinstance(tool_trace, list) else [],
|
"tool_trace": tool_trace if isinstance(tool_trace, list) else [],
|
||||||
}
|
}
|
||||||
|
|
||||||
|
def _is_internal_answer_contract_route(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
route_decision: Optional[dict[str, Any]] = None,
|
||||||
|
sources: Optional[list[dict[str, Any]]] = None,
|
||||||
|
) -> bool:
|
||||||
|
normalized_route = (
|
||||||
|
self._normalize_query_route_decision(route_decision)
|
||||||
|
if hasattr(self, "_normalize_query_route_decision")
|
||||||
|
else {}
|
||||||
|
)
|
||||||
|
if str(normalized_route.get("route_mode") or "").strip().lower() == "internal":
|
||||||
|
return True
|
||||||
|
if str(normalized_route.get("effective_source_policy") or "").strip().lower() == "internal":
|
||||||
|
return True
|
||||||
|
for item in sources or []:
|
||||||
|
if not isinstance(item, dict):
|
||||||
|
continue
|
||||||
|
if str(item.get("source") or "").strip().lower() in self._INTERNAL_SOURCE_TYPES:
|
||||||
|
return True
|
||||||
|
return False
|
||||||
|
|
||||||
|
def _verify_external_answer_contract(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
answer: str,
|
||||||
|
sources: Optional[list[dict[str, Any]]] = None,
|
||||||
|
route_decision: Optional[dict[str, Any]] = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
normalized_route = (
|
||||||
|
self._normalize_query_route_decision(route_decision)
|
||||||
|
if hasattr(self, "_normalize_query_route_decision")
|
||||||
|
else {}
|
||||||
|
)
|
||||||
|
if str(normalized_route.get("effective_source_policy") or "").strip().lower() != "external":
|
||||||
|
return {"ok": True, "reason_code": None, "missing_fields": []}
|
||||||
|
|
||||||
|
source_rows = sources if isinstance(sources, list) else []
|
||||||
|
if not source_rows:
|
||||||
|
return {
|
||||||
|
"ok": False,
|
||||||
|
"reason_code": "missing_external_evidence",
|
||||||
|
"missing_fields": ["external_evidence"],
|
||||||
|
}
|
||||||
|
|
||||||
|
external_source_seen = False
|
||||||
|
for item in source_rows:
|
||||||
|
if not isinstance(item, dict):
|
||||||
|
continue
|
||||||
|
source = str(item.get("source") or "").strip().lower()
|
||||||
|
metadata = item.get("metadata") if isinstance(item.get("metadata"), dict) else {}
|
||||||
|
if source == "system":
|
||||||
|
continue
|
||||||
|
if source not in {"url", "mcp_search"}:
|
||||||
|
return {
|
||||||
|
"ok": False,
|
||||||
|
"reason_code": "external_source_contract_violation",
|
||||||
|
"missing_fields": ["external_source_type"],
|
||||||
|
}
|
||||||
|
target_url = str(
|
||||||
|
metadata.get("target_url")
|
||||||
|
or metadata.get("file_url")
|
||||||
|
or metadata.get("url")
|
||||||
|
or ""
|
||||||
|
).strip().lower()
|
||||||
|
if not target_url.startswith("http://") and not target_url.startswith("https://"):
|
||||||
|
return {
|
||||||
|
"ok": False,
|
||||||
|
"reason_code": "external_target_url_missing",
|
||||||
|
"missing_fields": ["target_url"],
|
||||||
|
}
|
||||||
|
external_source_seen = True
|
||||||
|
|
||||||
|
if not external_source_seen:
|
||||||
|
return {
|
||||||
|
"ok": False,
|
||||||
|
"reason_code": "missing_external_grounding",
|
||||||
|
"missing_fields": ["external_evidence"],
|
||||||
|
}
|
||||||
|
_ = answer
|
||||||
|
return {"ok": True, "reason_code": None, "missing_fields": []}
|
||||||
|
|
||||||
|
def _build_external_answer_contract_guard_payload(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
reason_code: Optional[str],
|
||||||
|
missing_fields: Optional[list[str]] = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
normalized_reason = str(reason_code or "").strip().lower() or "external_answer_contract_violation"
|
||||||
|
normalized_missing = [str(item).strip() for item in (missing_fields or []) if str(item).strip()]
|
||||||
|
return {
|
||||||
|
"answer": "当前没有拿到可核验的外部实时证据,已阻止直接返回外部结论。请稍后重试,或把要核验的公开对象说得更具体一些。",
|
||||||
|
"sources": [
|
||||||
|
{
|
||||||
|
"source": "system",
|
||||||
|
"label": "External Query Contract Guard",
|
||||||
|
"score": 0.99,
|
||||||
|
"metadata": {
|
||||||
|
"mode": "guard",
|
||||||
|
"reason": normalized_reason,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"missing_fields": normalized_missing or ["external_answer_contract"],
|
||||||
|
"reason_code": normalized_reason,
|
||||||
|
"answer_mode": ANSWER_MODE_CLARIFY,
|
||||||
|
"grounding_status": GROUNDING_STATUS_INSUFFICIENT_EVIDENCE,
|
||||||
|
}
|
||||||
|
|
||||||
|
def _verify_internal_answer_contract(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
query_type: str,
|
||||||
|
answer: str,
|
||||||
|
sources: Optional[list[dict[str, Any]]] = None,
|
||||||
|
route_decision: Optional[dict[str, Any]] = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
source_rows = sources if isinstance(sources, list) else []
|
||||||
|
if not self._is_internal_answer_contract_route(route_decision=route_decision, sources=source_rows):
|
||||||
|
return {"ok": True, "reason_code": None, "missing_fields": []}
|
||||||
|
|
||||||
|
normalized_query_type = str(query_type or QUERY_TYPE_UNKNOWN).strip().lower()
|
||||||
|
normalized_answer = str(answer or "").strip()
|
||||||
|
compact_answer = re.sub(r"\s+", "", normalized_answer.lower())
|
||||||
|
|
||||||
|
if normalized_query_type == QUERY_TYPE_EXISTENCE:
|
||||||
|
if "项目总数" in normalized_answer or "总数为" in normalized_answer:
|
||||||
|
return {
|
||||||
|
"ok": False,
|
||||||
|
"reason_code": "no_total_count_substitution",
|
||||||
|
"missing_fields": ["matched_evidence", "existence_contract"],
|
||||||
|
}
|
||||||
|
if "总数" in compact_answer and "匹配到" not in normalized_answer and "未检索到" not in normalized_answer:
|
||||||
|
return {
|
||||||
|
"ok": False,
|
||||||
|
"reason_code": "no_total_count_substitution",
|
||||||
|
"missing_fields": ["matched_evidence", "existence_contract"],
|
||||||
|
}
|
||||||
|
|
||||||
|
if not source_rows:
|
||||||
|
return {
|
||||||
|
"ok": False,
|
||||||
|
"reason_code": "missing_internal_evidence",
|
||||||
|
"missing_fields": ["internal_evidence"],
|
||||||
|
}
|
||||||
|
|
||||||
|
return {"ok": True, "reason_code": None, "missing_fields": []}
|
||||||
|
|
||||||
|
def _build_internal_answer_contract_guard_payload(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
query_type: str,
|
||||||
|
reason_code: Optional[str],
|
||||||
|
missing_fields: Optional[list[str]] = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
normalized_query_type = str(query_type or QUERY_TYPE_UNKNOWN).strip().lower()
|
||||||
|
normalized_reason = str(reason_code or "").strip().lower() or "internal_answer_contract_violation"
|
||||||
|
normalized_missing = [str(item).strip() for item in (missing_fields or []) if str(item).strip()]
|
||||||
|
if normalized_reason == "no_total_count_substitution" and normalized_query_type == QUERY_TYPE_EXISTENCE:
|
||||||
|
answer = "当前内部查询结果与问题类型不一致,已阻止直接返回。你问的是“有没有”,请给我更明确的对象条件后我按存在性重新查询。"
|
||||||
|
elif normalized_reason == "missing_internal_evidence":
|
||||||
|
answer = "当前内部查询证据不足,已阻止直接返回结论。请补充更明确的对象或范围后我再查。"
|
||||||
|
else:
|
||||||
|
answer = "当前内部查询结果未通过回答合同校验,已阻止直接返回。请补充更明确的对象或范围后我再查。"
|
||||||
|
|
||||||
|
return {
|
||||||
|
"answer": answer,
|
||||||
|
"sources": [
|
||||||
|
{
|
||||||
|
"source": "system",
|
||||||
|
"label": "Internal Query Contract Guard",
|
||||||
|
"score": 0.99,
|
||||||
|
"metadata": {
|
||||||
|
"mode": "guard",
|
||||||
|
"reason": normalized_reason,
|
||||||
|
"query_type": normalized_query_type,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"missing_fields": normalized_missing or ["internal_answer_contract"],
|
||||||
|
"reason_code": normalized_reason,
|
||||||
|
"answer_mode": ANSWER_MODE_BLOCKED,
|
||||||
|
"grounding_status": GROUNDING_STATUS_BLOCKED,
|
||||||
|
}
|
||||||
|
|
||||||
def _build_response_payload(
|
def _build_response_payload(
|
||||||
self,
|
self,
|
||||||
*,
|
*,
|
||||||
|
|
@ -408,6 +595,7 @@ class AdaptiveRAGAnswerPayloadMixin:
|
||||||
sources_used: Optional[list[dict[str, Any]]] = None,
|
sources_used: Optional[list[dict[str, Any]]] = None,
|
||||||
evidence_pack: Optional[dict[str, Any]] = None,
|
evidence_pack: Optional[dict[str, Any]] = None,
|
||||||
prompt_variant_id: Optional[str] = None,
|
prompt_variant_id: Optional[str] = None,
|
||||||
|
route_decision: Optional[dict[str, Any]] = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
evidence: list[dict[str, Any]] = []
|
evidence: list[dict[str, Any]] = []
|
||||||
for item in sources[:8]:
|
for item in sources[:8]:
|
||||||
|
|
@ -452,6 +640,11 @@ class AdaptiveRAGAnswerPayloadMixin:
|
||||||
reason_code=reason_code,
|
reason_code=reason_code,
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
normalized_route_decision = (
|
||||||
|
self._normalize_query_route_decision(route_decision)
|
||||||
|
if hasattr(self, "_normalize_query_route_decision")
|
||||||
|
else {}
|
||||||
|
)
|
||||||
return {
|
return {
|
||||||
"answer": str(answer or "").strip(),
|
"answer": str(answer or "").strip(),
|
||||||
"evidence": evidence,
|
"evidence": evidence,
|
||||||
|
|
@ -498,6 +691,7 @@ class AdaptiveRAGAnswerPayloadMixin:
|
||||||
"citations": normalized_citations,
|
"citations": normalized_citations,
|
||||||
"sources_used": normalized_sources_used,
|
"sources_used": normalized_sources_used,
|
||||||
"evidence_pack": normalized_evidence_pack,
|
"evidence_pack": normalized_evidence_pack,
|
||||||
|
"route_decision": normalized_route_decision,
|
||||||
}
|
}
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
|
|
|
||||||
|
|
@ -22,6 +22,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
|
||||||
rewrite_meta: dict[str, Any],
|
rewrite_meta: dict[str, Any],
|
||||||
context_session_id: Optional[int],
|
context_session_id: Optional[int],
|
||||||
active_thinking_mode: str,
|
active_thinking_mode: str,
|
||||||
|
route_decision: Optional[dict[str, Any]] = None,
|
||||||
) -> Optional[dict[str, Any]]:
|
) -> Optional[dict[str, Any]]:
|
||||||
if not isinstance(meta_followup_payload, dict):
|
if not isinstance(meta_followup_payload, dict):
|
||||||
return None
|
return None
|
||||||
|
|
@ -52,8 +53,16 @@ class AdaptiveRAGAnswerPrecheckMixin:
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
|
render_with_contract = getattr(self, "_render_answer_with_route_contract", None)
|
||||||
|
rendered_answer = answer
|
||||||
|
if callable(render_with_contract):
|
||||||
|
rendered_answer = render_with_contract(
|
||||||
|
answer=answer,
|
||||||
|
sources=direct_sources,
|
||||||
|
route_decision=route_decision,
|
||||||
|
)
|
||||||
runtime_fields = self._build_qa_record_runtime_fields(
|
runtime_fields = self._build_qa_record_runtime_fields(
|
||||||
answer=answer,
|
answer=rendered_answer,
|
||||||
sources=direct_sources,
|
sources=direct_sources,
|
||||||
missing_fields=[],
|
missing_fields=[],
|
||||||
answer_mode=ANSWER_MODE_FAST_PATH,
|
answer_mode=ANSWER_MODE_FAST_PATH,
|
||||||
|
|
@ -76,7 +85,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
|
||||||
qa = QARecord(
|
qa = QARecord(
|
||||||
tenant_id=self.tenant_id,
|
tenant_id=self.tenant_id,
|
||||||
question=clean_q,
|
question=clean_q,
|
||||||
answer=answer,
|
answer=rendered_answer,
|
||||||
sources=direct_sources,
|
sources=direct_sources,
|
||||||
intent="question",
|
intent="question",
|
||||||
domains_used=["system_context"],
|
domains_used=["system_context"],
|
||||||
|
|
@ -90,6 +99,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
|
||||||
anchor_kind=anchor_kind,
|
anchor_kind=anchor_kind,
|
||||||
anchor_ref=str(meta_followup_payload.get("anchor_ref") or "").strip() or None,
|
anchor_ref=str(meta_followup_payload.get("anchor_ref") or "").strip() or None,
|
||||||
routing_snapshot=routing_snapshot,
|
routing_snapshot=routing_snapshot,
|
||||||
|
route_decision=route_decision,
|
||||||
),
|
),
|
||||||
**runtime_fields,
|
**runtime_fields,
|
||||||
)
|
)
|
||||||
|
|
@ -98,22 +108,31 @@ class AdaptiveRAGAnswerPrecheckMixin:
|
||||||
session_state=state_row,
|
session_state=state_row,
|
||||||
question=clean_q,
|
question=clean_q,
|
||||||
effective_question=effective_q,
|
effective_question=effective_q,
|
||||||
answer=answer,
|
answer=rendered_answer,
|
||||||
query_type=query_type,
|
query_type=query_type,
|
||||||
ambiguity="clear",
|
ambiguity="clear",
|
||||||
rewrite_meta=rewrite_meta,
|
rewrite_meta=rewrite_meta,
|
||||||
sources=direct_sources,
|
sources=direct_sources,
|
||||||
)
|
)
|
||||||
|
await self.db.flush()
|
||||||
|
persist_route_memory = getattr(self, "_persist_query_route_memory", None)
|
||||||
|
if callable(persist_route_memory):
|
||||||
|
await persist_route_memory(
|
||||||
|
qa_record_id=int(qa.id) if getattr(qa, "id", None) is not None else None,
|
||||||
|
question_text=clean_q,
|
||||||
|
project_id=project_id,
|
||||||
|
route_decision=route_decision,
|
||||||
|
)
|
||||||
await self.db.commit()
|
await self.db.commit()
|
||||||
await self.db.refresh(qa)
|
await self.db.refresh(qa)
|
||||||
await self.context_cache.append_session_messages(
|
await self.context_cache.append_session_messages(
|
||||||
tenant_id=self.tenant_id,
|
tenant_id=self.tenant_id,
|
||||||
session_id=context_session_id,
|
session_id=context_session_id,
|
||||||
question=clean_q,
|
question=clean_q,
|
||||||
answer=answer,
|
answer=rendered_answer,
|
||||||
)
|
)
|
||||||
return self._build_response_payload(
|
return self._build_response_payload(
|
||||||
answer=answer,
|
answer=rendered_answer,
|
||||||
sources=direct_sources,
|
sources=direct_sources,
|
||||||
domains_used=["system_context"],
|
domains_used=["system_context"],
|
||||||
response_time_ms=elapsed,
|
response_time_ms=elapsed,
|
||||||
|
|
@ -141,6 +160,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
|
||||||
evidence_pack=runtime_fields["evidence_pack"],
|
evidence_pack=runtime_fields["evidence_pack"],
|
||||||
tool_trace=runtime_fields["tool_trace"],
|
tool_trace=runtime_fields["tool_trace"],
|
||||||
prompt_variant_id="meta_followup",
|
prompt_variant_id="meta_followup",
|
||||||
|
route_decision=route_decision,
|
||||||
)
|
)
|
||||||
|
|
||||||
async def _maybe_return_clarification_response(
|
async def _maybe_return_clarification_response(
|
||||||
|
|
@ -160,6 +180,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
|
||||||
rewrite_meta: dict[str, Any],
|
rewrite_meta: dict[str, Any],
|
||||||
context_session_id: Optional[int],
|
context_session_id: Optional[int],
|
||||||
active_thinking_mode: str,
|
active_thinking_mode: str,
|
||||||
|
route_decision: Optional[dict[str, Any]] = None,
|
||||||
) -> Optional[dict[str, Any]]:
|
) -> Optional[dict[str, Any]]:
|
||||||
if not isinstance(clarification_payload, dict):
|
if not isinstance(clarification_payload, dict):
|
||||||
return None
|
return None
|
||||||
|
|
@ -191,8 +212,16 @@ class AdaptiveRAGAnswerPrecheckMixin:
|
||||||
"metadata": {"mode": "clarification", "reason": clarification_code},
|
"metadata": {"mode": "clarification", "reason": clarification_code},
|
||||||
}
|
}
|
||||||
]
|
]
|
||||||
|
render_with_contract = getattr(self, "_render_answer_with_route_contract", None)
|
||||||
|
rendered_answer = clarification_answer
|
||||||
|
if callable(render_with_contract):
|
||||||
|
rendered_answer = render_with_contract(
|
||||||
|
answer=clarification_answer,
|
||||||
|
sources=clarify_sources,
|
||||||
|
route_decision=route_decision,
|
||||||
|
)
|
||||||
runtime_fields = self._build_qa_record_runtime_fields(
|
runtime_fields = self._build_qa_record_runtime_fields(
|
||||||
answer=clarification_answer,
|
answer=rendered_answer,
|
||||||
sources=clarify_sources,
|
sources=clarify_sources,
|
||||||
missing_fields=missing_fields,
|
missing_fields=missing_fields,
|
||||||
answer_mode=ANSWER_MODE_CLARIFY,
|
answer_mode=ANSWER_MODE_CLARIFY,
|
||||||
|
|
@ -214,7 +243,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
|
||||||
qa = QARecord(
|
qa = QARecord(
|
||||||
tenant_id=self.tenant_id,
|
tenant_id=self.tenant_id,
|
||||||
question=clean_q,
|
question=clean_q,
|
||||||
answer=clarification_answer,
|
answer=rendered_answer,
|
||||||
sources=clarify_sources,
|
sources=clarify_sources,
|
||||||
intent="question",
|
intent="question",
|
||||||
domains_used=["system_context"],
|
domains_used=["system_context"],
|
||||||
|
|
@ -224,7 +253,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
|
||||||
project_id=project_id,
|
project_id=project_id,
|
||||||
session_id=active_session_id,
|
session_id=active_session_id,
|
||||||
turn_index=turn_index,
|
turn_index=turn_index,
|
||||||
**self._build_qa_record_external_fields(),
|
**self._build_qa_record_external_fields(route_decision=route_decision),
|
||||||
**runtime_fields,
|
**runtime_fields,
|
||||||
)
|
)
|
||||||
self.db.add(qa)
|
self.db.add(qa)
|
||||||
|
|
@ -244,22 +273,31 @@ class AdaptiveRAGAnswerPrecheckMixin:
|
||||||
session_state=state_row,
|
session_state=state_row,
|
||||||
question=clean_q,
|
question=clean_q,
|
||||||
effective_question=effective_q,
|
effective_question=effective_q,
|
||||||
answer=clarification_answer,
|
answer=rendered_answer,
|
||||||
query_type=query_type,
|
query_type=query_type,
|
||||||
ambiguity="needs_clarification",
|
ambiguity="needs_clarification",
|
||||||
rewrite_meta=rewrite_meta,
|
rewrite_meta=rewrite_meta,
|
||||||
sources=clarify_sources,
|
sources=clarify_sources,
|
||||||
)
|
)
|
||||||
|
await self.db.flush()
|
||||||
|
persist_route_memory = getattr(self, "_persist_query_route_memory", None)
|
||||||
|
if callable(persist_route_memory):
|
||||||
|
await persist_route_memory(
|
||||||
|
qa_record_id=int(qa.id) if getattr(qa, "id", None) is not None else None,
|
||||||
|
question_text=clean_q,
|
||||||
|
project_id=project_id,
|
||||||
|
route_decision=route_decision,
|
||||||
|
)
|
||||||
await self.db.commit()
|
await self.db.commit()
|
||||||
await self.db.refresh(qa)
|
await self.db.refresh(qa)
|
||||||
await self.context_cache.append_session_messages(
|
await self.context_cache.append_session_messages(
|
||||||
tenant_id=self.tenant_id,
|
tenant_id=self.tenant_id,
|
||||||
session_id=context_session_id,
|
session_id=context_session_id,
|
||||||
question=clean_q,
|
question=clean_q,
|
||||||
answer=clarification_answer,
|
answer=rendered_answer,
|
||||||
)
|
)
|
||||||
return self._build_response_payload(
|
return self._build_response_payload(
|
||||||
answer=clarification_answer,
|
answer=rendered_answer,
|
||||||
sources=clarify_sources,
|
sources=clarify_sources,
|
||||||
domains_used=["system_context"],
|
domains_used=["system_context"],
|
||||||
response_time_ms=elapsed,
|
response_time_ms=elapsed,
|
||||||
|
|
@ -286,6 +324,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
|
||||||
evidence_pack=runtime_fields["evidence_pack"],
|
evidence_pack=runtime_fields["evidence_pack"],
|
||||||
tool_trace=runtime_fields["tool_trace"],
|
tool_trace=runtime_fields["tool_trace"],
|
||||||
prompt_variant_id="clarification",
|
prompt_variant_id="clarification",
|
||||||
|
route_decision=route_decision,
|
||||||
)
|
)
|
||||||
|
|
||||||
async def _maybe_return_snapshot_direct_response(
|
async def _maybe_return_snapshot_direct_response(
|
||||||
|
|
@ -305,6 +344,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
|
||||||
context_session_id: Optional[int],
|
context_session_id: Optional[int],
|
||||||
active_thinking_mode: str,
|
active_thinking_mode: str,
|
||||||
user_snapshot: Optional[dict[str, Any]],
|
user_snapshot: Optional[dict[str, Any]],
|
||||||
|
route_decision: Optional[dict[str, Any]] = None,
|
||||||
) -> Optional[dict[str, Any]]:
|
) -> Optional[dict[str, Any]]:
|
||||||
if not direct_answer:
|
if not direct_answer:
|
||||||
return None
|
return None
|
||||||
|
|
@ -327,8 +367,16 @@ class AdaptiveRAGAnswerPrecheckMixin:
|
||||||
"metadata": {"mode": "snapshot_direct", "topic": "current_user"},
|
"metadata": {"mode": "snapshot_direct", "topic": "current_user"},
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
|
render_with_contract = getattr(self, "_render_answer_with_route_contract", None)
|
||||||
|
rendered_answer = direct_answer
|
||||||
|
if callable(render_with_contract):
|
||||||
|
rendered_answer = render_with_contract(
|
||||||
|
answer=direct_answer,
|
||||||
|
sources=direct_sources,
|
||||||
|
route_decision=route_decision,
|
||||||
|
)
|
||||||
runtime_fields = self._build_qa_record_runtime_fields(
|
runtime_fields = self._build_qa_record_runtime_fields(
|
||||||
answer=direct_answer,
|
answer=rendered_answer,
|
||||||
sources=direct_sources,
|
sources=direct_sources,
|
||||||
missing_fields=[],
|
missing_fields=[],
|
||||||
answer_mode=ANSWER_MODE_FAST_PATH,
|
answer_mode=ANSWER_MODE_FAST_PATH,
|
||||||
|
|
@ -347,7 +395,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
|
||||||
qa = QARecord(
|
qa = QARecord(
|
||||||
tenant_id=self.tenant_id,
|
tenant_id=self.tenant_id,
|
||||||
question=clean_q,
|
question=clean_q,
|
||||||
answer=direct_answer,
|
answer=rendered_answer,
|
||||||
sources=direct_sources,
|
sources=direct_sources,
|
||||||
intent="question",
|
intent="question",
|
||||||
domains_used=["system_context"],
|
domains_used=["system_context"],
|
||||||
|
|
@ -357,7 +405,7 @@ class AdaptiveRAGAnswerPrecheckMixin:
|
||||||
project_id=project_id,
|
project_id=project_id,
|
||||||
session_id=int(session_id) if session_id is not None else (int(context_session_id) if context_session_id is not None else None),
|
session_id=int(session_id) if session_id is not None else (int(context_session_id) if context_session_id is not None else None),
|
||||||
turn_index=turn_index,
|
turn_index=turn_index,
|
||||||
**self._build_qa_record_external_fields(),
|
**self._build_qa_record_external_fields(route_decision=route_decision),
|
||||||
**runtime_fields,
|
**runtime_fields,
|
||||||
)
|
)
|
||||||
self.db.add(qa)
|
self.db.add(qa)
|
||||||
|
|
@ -365,22 +413,31 @@ class AdaptiveRAGAnswerPrecheckMixin:
|
||||||
session_state=state_row,
|
session_state=state_row,
|
||||||
question=clean_q,
|
question=clean_q,
|
||||||
effective_question=effective_q,
|
effective_question=effective_q,
|
||||||
answer=direct_answer,
|
answer=rendered_answer,
|
||||||
query_type=QUERY_TYPE_EXPLAIN,
|
query_type=QUERY_TYPE_EXPLAIN,
|
||||||
ambiguity="clear",
|
ambiguity="clear",
|
||||||
rewrite_meta=rewrite_meta,
|
rewrite_meta=rewrite_meta,
|
||||||
sources=direct_sources,
|
sources=direct_sources,
|
||||||
)
|
)
|
||||||
|
await self.db.flush()
|
||||||
|
persist_route_memory = getattr(self, "_persist_query_route_memory", None)
|
||||||
|
if callable(persist_route_memory):
|
||||||
|
await persist_route_memory(
|
||||||
|
qa_record_id=int(qa.id) if getattr(qa, "id", None) is not None else None,
|
||||||
|
question_text=clean_q,
|
||||||
|
project_id=project_id,
|
||||||
|
route_decision=route_decision,
|
||||||
|
)
|
||||||
await self.db.commit()
|
await self.db.commit()
|
||||||
await self.db.refresh(qa)
|
await self.db.refresh(qa)
|
||||||
await self.context_cache.append_session_messages(
|
await self.context_cache.append_session_messages(
|
||||||
tenant_id=self.tenant_id,
|
tenant_id=self.tenant_id,
|
||||||
session_id=context_session_id,
|
session_id=context_session_id,
|
||||||
question=clean_q,
|
question=clean_q,
|
||||||
answer=direct_answer,
|
answer=rendered_answer,
|
||||||
)
|
)
|
||||||
return self._build_response_payload(
|
return self._build_response_payload(
|
||||||
answer=direct_answer,
|
answer=rendered_answer,
|
||||||
sources=direct_sources,
|
sources=direct_sources,
|
||||||
domains_used=["system_context"],
|
domains_used=["system_context"],
|
||||||
response_time_ms=elapsed,
|
response_time_ms=elapsed,
|
||||||
|
|
@ -404,4 +461,5 @@ class AdaptiveRAGAnswerPrecheckMixin:
|
||||||
evidence_pack=runtime_fields["evidence_pack"],
|
evidence_pack=runtime_fields["evidence_pack"],
|
||||||
tool_trace=runtime_fields["tool_trace"],
|
tool_trace=runtime_fields["tool_trace"],
|
||||||
prompt_variant_id="snapshot_direct",
|
prompt_variant_id="snapshot_direct",
|
||||||
|
route_decision=route_decision,
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -5,6 +5,237 @@ from .shared import *
|
||||||
|
|
||||||
|
|
||||||
class AdaptiveRAGAnswerRetrievalMixin:
|
class AdaptiveRAGAnswerRetrievalMixin:
|
||||||
|
async def _execute_external_mcp_search(self, plan) -> dict[str, Any]:
|
||||||
|
from ..external_query_service.schemas import ExternalAdapterResult, ExternalQueryPlan
|
||||||
|
from ..mcp_config import McpConfigService
|
||||||
|
|
||||||
|
try:
|
||||||
|
normalized_plan = plan if isinstance(plan, ExternalQueryPlan) else ExternalQueryPlan.model_validate(plan)
|
||||||
|
except Exception as exc:
|
||||||
|
return ExternalAdapterResult(
|
||||||
|
adapter_name="mcp",
|
||||||
|
status="error",
|
||||||
|
evidence=[],
|
||||||
|
meta={"reason": "plan_invalid", "error": str(exc)},
|
||||||
|
).model_dump(mode="json")
|
||||||
|
|
||||||
|
query = str((normalized_plan.search_queries[:1] or [normalized_plan.topic])[0] or "").strip()
|
||||||
|
if not query:
|
||||||
|
return ExternalAdapterResult(
|
||||||
|
adapter_name="mcp",
|
||||||
|
status="empty",
|
||||||
|
evidence=[],
|
||||||
|
meta={"reason": "empty_query"},
|
||||||
|
).model_dump(mode="json")
|
||||||
|
|
||||||
|
db = getattr(self, "db", None)
|
||||||
|
if db is None:
|
||||||
|
return ExternalAdapterResult(
|
||||||
|
adapter_name="mcp",
|
||||||
|
status="unavailable",
|
||||||
|
evidence=[],
|
||||||
|
meta={"reason": "db_unavailable"},
|
||||||
|
).model_dump(mode="json")
|
||||||
|
|
||||||
|
try:
|
||||||
|
service = McpConfigService(db, tenant_id=int(getattr(self, "tenant_id", 1) or 1))
|
||||||
|
evidence = await service.search_external(query=query, result_limit=normalized_plan.result_limit)
|
||||||
|
except Exception as exc:
|
||||||
|
return ExternalAdapterResult(
|
||||||
|
adapter_name="mcp",
|
||||||
|
status="error",
|
||||||
|
evidence=[],
|
||||||
|
meta={"reason": "backend_direct_runtime_error", "error": str(exc), "query": query},
|
||||||
|
).model_dump(mode="json")
|
||||||
|
|
||||||
|
if not evidence:
|
||||||
|
return ExternalAdapterResult(
|
||||||
|
adapter_name="mcp",
|
||||||
|
status="empty",
|
||||||
|
evidence=[],
|
||||||
|
meta={"reason": "no_backend_direct_result", "query": query},
|
||||||
|
).model_dump(mode="json")
|
||||||
|
|
||||||
|
return ExternalAdapterResult(
|
||||||
|
adapter_name="mcp",
|
||||||
|
status="success",
|
||||||
|
evidence=evidence,
|
||||||
|
meta={"query": query, "provider": "mcp_backend_direct"},
|
||||||
|
).model_dump(mode="json")
|
||||||
|
|
||||||
|
async def _execute_external_query_gateway(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
question: str,
|
||||||
|
query_semantics: Optional[dict[str, Any]] = None,
|
||||||
|
result_limit: int = 3,
|
||||||
|
max_pages: int = 3,
|
||||||
|
) -> tuple[list[RetrievedItem], dict[str, Any]]:
|
||||||
|
live_search_ready = all(
|
||||||
|
callable(item)
|
||||||
|
for item in (
|
||||||
|
getattr(self, "_search_live_external_results", None),
|
||||||
|
getattr(self, "_fetch_url_text", None),
|
||||||
|
getattr(self, "_chunk_text", None),
|
||||||
|
getattr(self, "_score_live_external_search_result", None),
|
||||||
|
getattr(self, "_score_live_external_chunk", None),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
mcp_ready = callable(getattr(self, "_execute_external_mcp_search", None)) and getattr(self, "db", None) is not None
|
||||||
|
if not live_search_ready and not mcp_ready:
|
||||||
|
legacy_retrieval = getattr(self, "_retrieve_live_external_context", None)
|
||||||
|
if callable(legacy_retrieval):
|
||||||
|
rows, meta = await legacy_retrieval(
|
||||||
|
question=question,
|
||||||
|
max_results=result_limit,
|
||||||
|
max_pages=max_pages,
|
||||||
|
)
|
||||||
|
normalized_meta = dict(meta or {})
|
||||||
|
normalized_meta.setdefault("mode", "live_external_context" if rows else "live_external_context_empty")
|
||||||
|
normalized_meta.setdefault("domains_used", ["url"])
|
||||||
|
normalized_meta.setdefault("db_query_ms", 0)
|
||||||
|
normalized_meta.setdefault("vector_search_ms", 0)
|
||||||
|
normalized_meta.setdefault("graph_query_ms", 0)
|
||||||
|
normalized_meta.setdefault("db_timeout", False)
|
||||||
|
normalized_meta.setdefault("vector_timeout", False)
|
||||||
|
normalized_meta.setdefault("graph_timeout", False)
|
||||||
|
normalized_meta.setdefault("tool_trace", [])
|
||||||
|
return rows, normalized_meta
|
||||||
|
|
||||||
|
from ..external_query_service import ExternalQueryGateway, ExternalQueryPlanner, ExternalProviderRegistry
|
||||||
|
from ..external_query_service.adapters import LiveSearchFallbackAdapter, MCPExternalAdapter
|
||||||
|
|
||||||
|
planner = ExternalQueryPlanner(
|
||||||
|
llm_service=getattr(self, "llm", None),
|
||||||
|
db=getattr(self, "db", None),
|
||||||
|
tenant_id=getattr(self, "tenant_id", 1),
|
||||||
|
query_builder=getattr(self, "_build_live_external_search_queries", None),
|
||||||
|
)
|
||||||
|
registry = ExternalProviderRegistry(
|
||||||
|
adapters=[
|
||||||
|
MCPExternalAdapter(
|
||||||
|
search_callable=getattr(self, "_execute_external_mcp_search", None),
|
||||||
|
fetch_text_callable=getattr(self, "_fetch_url_text", None),
|
||||||
|
chunk_text_callable=getattr(self, "_chunk_text", None),
|
||||||
|
score_chunk_callable=getattr(self, "_score_live_external_chunk", None),
|
||||||
|
max_pages=max_pages,
|
||||||
|
),
|
||||||
|
LiveSearchFallbackAdapter(
|
||||||
|
build_queries_callable=getattr(self, "_build_live_external_search_queries", None),
|
||||||
|
search_results_callable=getattr(self, "_search_live_external_results", None),
|
||||||
|
fetch_text_callable=getattr(self, "_fetch_url_text", None),
|
||||||
|
chunk_text_callable=getattr(self, "_chunk_text", None),
|
||||||
|
score_search_result_callable=getattr(self, "_score_live_external_search_result", None),
|
||||||
|
score_chunk_callable=getattr(self, "_score_live_external_chunk", None),
|
||||||
|
max_pages=max_pages,
|
||||||
|
),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
execution = await ExternalQueryGateway(
|
||||||
|
planner=planner,
|
||||||
|
registry=registry,
|
||||||
|
).execute(
|
||||||
|
question=question,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
)
|
||||||
|
verifier = dict(execution.verifier or {})
|
||||||
|
if not bool(verifier.get("ok")):
|
||||||
|
return [], {
|
||||||
|
"mode": "external_query_gateway_verifier_blocked",
|
||||||
|
"adapter": execution.adapter_name,
|
||||||
|
"verifier": verifier,
|
||||||
|
"planner": dict(execution.planner_meta or {}),
|
||||||
|
"fallback_used": bool((execution.meta or {}).get("fallback_used")),
|
||||||
|
"adapter_attempts": list((execution.meta or {}).get("adapter_attempts") or []),
|
||||||
|
"query": str(((execution.meta or {}).get("query") or (execution.plan.search_queries[:1] or [""])[0])).strip(),
|
||||||
|
"search_result_count": int((execution.meta or {}).get("search_result_count") or 0),
|
||||||
|
"fetched_pages": int((execution.meta or {}).get("fetched_pages") or 0),
|
||||||
|
"tool_trace": [
|
||||||
|
{
|
||||||
|
"step": 1,
|
||||||
|
"tool": "external_query_gateway",
|
||||||
|
"adapter": execution.adapter_name,
|
||||||
|
"planner": str((execution.planner_meta or {}).get("planner") or "unknown"),
|
||||||
|
"verifier_ok": False,
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"domains_used": ["url"],
|
||||||
|
"db_query_ms": 0,
|
||||||
|
"vector_search_ms": 0,
|
||||||
|
"graph_query_ms": 0,
|
||||||
|
"db_timeout": False,
|
||||||
|
"vector_timeout": False,
|
||||||
|
"graph_timeout": False,
|
||||||
|
}
|
||||||
|
|
||||||
|
rows: list[RetrievedItem] = []
|
||||||
|
for item in list(execution.evidence or [])[: max(1, int(result_limit))]:
|
||||||
|
source_type = str(getattr(item, "source_type", "url") or "url").strip().lower() or "url"
|
||||||
|
normalized_source = source_type if source_type in {"url", "mcp_search"} else "url"
|
||||||
|
title = str(getattr(item, "title", "") or "").strip()
|
||||||
|
source_label = str(getattr(item, "source_label", "") or "").strip() or title
|
||||||
|
target_url = str(getattr(item, "target_url", "") or "").strip()
|
||||||
|
snippet = str(getattr(item, "snippet", "") or "").strip()
|
||||||
|
freshness_score = float(getattr(item, "freshness_score", 0.0) or 0.0)
|
||||||
|
authority_score = float(getattr(item, "authority_score", 0.0) or 0.0)
|
||||||
|
rows.append(
|
||||||
|
RetrievedItem(
|
||||||
|
source=normalized_source,
|
||||||
|
content=(snippet or title or target_url)[:1800],
|
||||||
|
score=round(max(freshness_score, authority_score, 0.55), 4),
|
||||||
|
source_label=(
|
||||||
|
f"MCP[{source_label[:120]}]"
|
||||||
|
if normalized_source == "mcp_search"
|
||||||
|
else f"URL[{source_label[:120]}]"
|
||||||
|
),
|
||||||
|
metadata={
|
||||||
|
"title": title[:255],
|
||||||
|
"url": target_url,
|
||||||
|
"file_url": target_url,
|
||||||
|
"target_url": target_url,
|
||||||
|
"retrieved_via": "external_query_gateway",
|
||||||
|
"external_source_type": source_type,
|
||||||
|
"external_adapter": execution.adapter_name,
|
||||||
|
"freshness_score": round(freshness_score, 4),
|
||||||
|
"authority_score": round(authority_score, 4),
|
||||||
|
"grounded_fields": list(getattr(item, "grounded_fields", []) or []),
|
||||||
|
**(dict(getattr(item, "raw_metadata", {}) or {})),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
meta = dict(execution.meta or {})
|
||||||
|
meta.update(
|
||||||
|
{
|
||||||
|
"mode": "external_query_gateway" if rows else "external_query_gateway_empty",
|
||||||
|
"adapter": execution.adapter_name,
|
||||||
|
"verifier": verifier,
|
||||||
|
"planner": dict(execution.planner_meta or {}),
|
||||||
|
"query": str(meta.get("query") or (execution.plan.search_queries[:1] or [""])[0]).strip(),
|
||||||
|
"search_result_count": int(meta.get("search_result_count") or 0),
|
||||||
|
"fetched_pages": int(meta.get("fetched_pages") or 0),
|
||||||
|
"domains_used": ["url"],
|
||||||
|
"db_query_ms": 0,
|
||||||
|
"vector_search_ms": 0,
|
||||||
|
"graph_query_ms": 0,
|
||||||
|
"db_timeout": False,
|
||||||
|
"vector_timeout": False,
|
||||||
|
"graph_timeout": False,
|
||||||
|
"tool_trace": [
|
||||||
|
{
|
||||||
|
"step": 1,
|
||||||
|
"tool": "external_query_gateway",
|
||||||
|
"adapter": execution.adapter_name,
|
||||||
|
"planner": str((execution.planner_meta or {}).get("planner") or "unknown"),
|
||||||
|
"verifier_ok": True,
|
||||||
|
"result_count": len(rows),
|
||||||
|
"fallback_used": bool(meta.get("fallback_used")),
|
||||||
|
}
|
||||||
|
],
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return rows, meta
|
||||||
|
|
||||||
async def _maybe_fetch_live_external_freshness_context(
|
async def _maybe_fetch_live_external_freshness_context(
|
||||||
self,
|
self,
|
||||||
*,
|
*,
|
||||||
|
|
@ -19,16 +250,30 @@ class AdaptiveRAGAnswerRetrievalMixin:
|
||||||
search_question = str(effective_question or question or "").strip()
|
search_question = str(effective_question or question or "").strip()
|
||||||
if not search_question:
|
if not search_question:
|
||||||
return [], {"mode": "skip_empty_question"}
|
return [], {"mode": "skip_empty_question"}
|
||||||
|
execute_gateway = getattr(self, "_execute_external_query_gateway", None)
|
||||||
try:
|
try:
|
||||||
rows, live_meta = await self._retrieve_live_external_context(
|
if callable(execute_gateway):
|
||||||
question=search_question,
|
rows, live_meta = await execute_gateway(
|
||||||
max_results=3,
|
question=search_question,
|
||||||
max_pages=3,
|
query_semantics=query_semantics,
|
||||||
)
|
result_limit=3,
|
||||||
|
max_pages=3,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
rows, live_meta = await self._retrieve_live_external_context(
|
||||||
|
question=search_question,
|
||||||
|
max_results=3,
|
||||||
|
max_pages=3,
|
||||||
|
)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
return [], {"mode": "live_external_context_error", "error": str(exc)}
|
mode = "external_query_gateway_error" if callable(execute_gateway) else "live_external_context_error"
|
||||||
|
return [], {"mode": mode, "error": str(exc)}
|
||||||
meta = dict(live_meta or {})
|
meta = dict(live_meta or {})
|
||||||
meta["mode"] = "live_external_context" if rows else "live_external_context_empty"
|
if not str(meta.get("mode") or "").strip():
|
||||||
|
if callable(execute_gateway):
|
||||||
|
meta["mode"] = "external_query_gateway" if rows else "external_query_gateway_empty"
|
||||||
|
else:
|
||||||
|
meta["mode"] = "live_external_context" if rows else "live_external_context_empty"
|
||||||
meta["external_freshness_only"] = bool(
|
meta["external_freshness_only"] = bool(
|
||||||
isinstance(query_semantics, dict) and query_semantics.get("external_freshness_only")
|
isinstance(query_semantics, dict) and query_semantics.get("external_freshness_only")
|
||||||
)
|
)
|
||||||
|
|
@ -66,22 +311,43 @@ class AdaptiveRAGAnswerRetrievalMixin:
|
||||||
mixed_signal_part: Optional[str] = None,
|
mixed_signal_part: Optional[str] = None,
|
||||||
answer_mode: str = ANSWER_MODE_RETRIEVAL_NORMAL,
|
answer_mode: str = ANSWER_MODE_RETRIEVAL_NORMAL,
|
||||||
consultant_mode_preference: Optional[str] = None,
|
consultant_mode_preference: Optional[str] = None,
|
||||||
|
route_decision: Optional[dict[str, Any]] = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
domains = [str(item).strip().lower() for item in retrieval_domains if str(item or "").strip()]
|
domains = [str(item).strip().lower() for item in retrieval_domains if str(item or "").strip()]
|
||||||
if not domains:
|
if not domains:
|
||||||
domains = ["db", "graph", "doc", "feishu", "pmflow", "url"]
|
domains = ["db", "graph", "doc", "feishu", "pmflow", "url"]
|
||||||
if str(asked_via or "").strip().lower() == "feishu" and source_chat_id and "feishu" not in domains:
|
if str(asked_via or "").strip().lower() == "feishu" and source_chat_id and "feishu" not in domains:
|
||||||
domains.append("feishu")
|
domains.append("feishu")
|
||||||
retrieved, search_meta = await self._multi_step_search(
|
normalize_route_decision = getattr(self, "_normalize_query_route_decision", None)
|
||||||
question=effective_q,
|
normalized_route_decision = (
|
||||||
domains=domains,
|
normalize_route_decision(route_decision)
|
||||||
project_id=project_id,
|
if callable(normalize_route_decision)
|
||||||
source_chat_id=source_chat_id,
|
else (dict(route_decision or {}) if isinstance(route_decision, dict) else {})
|
||||||
query_semantics=query_semantics,
|
|
||||||
session_state=state_row,
|
|
||||||
max_steps_override=deep_retrieval_steps if deep_thinking else None,
|
|
||||||
vector_top_k=deep_vector_top_k if deep_thinking else None,
|
|
||||||
)
|
)
|
||||||
|
external_route_only = str(
|
||||||
|
normalized_route_decision.get("effective_source_policy")
|
||||||
|
or query_semantics.get("effective_source_policy")
|
||||||
|
or query_semantics.get("source_policy")
|
||||||
|
or ""
|
||||||
|
).strip().lower() == "external"
|
||||||
|
if external_route_only:
|
||||||
|
retrieved, search_meta = await self._execute_external_query_gateway(
|
||||||
|
question=effective_q,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
result_limit=3,
|
||||||
|
max_pages=3,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
retrieved, search_meta = await self._multi_step_search(
|
||||||
|
question=effective_q,
|
||||||
|
domains=domains,
|
||||||
|
project_id=project_id,
|
||||||
|
source_chat_id=source_chat_id,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
session_state=state_row,
|
||||||
|
max_steps_override=deep_retrieval_steps if deep_thinking else None,
|
||||||
|
vector_top_k=deep_vector_top_k if deep_thinking else None,
|
||||||
|
)
|
||||||
db_query_ms = int(search_meta.get("db_query_ms") or 0)
|
db_query_ms = int(search_meta.get("db_query_ms") or 0)
|
||||||
vector_search_ms = int(search_meta.get("vector_search_ms") or 0)
|
vector_search_ms = int(search_meta.get("vector_search_ms") or 0)
|
||||||
graph_query_ms = int(search_meta.get("graph_query_ms") or 0)
|
graph_query_ms = int(search_meta.get("graph_query_ms") or 0)
|
||||||
|
|
@ -101,6 +367,16 @@ class AdaptiveRAGAnswerRetrievalMixin:
|
||||||
),
|
),
|
||||||
},
|
},
|
||||||
)
|
)
|
||||||
|
tool_trace.insert(
|
||||||
|
0,
|
||||||
|
{
|
||||||
|
"step": 0,
|
||||||
|
"tool": "query_route_gateway",
|
||||||
|
"route_mode": normalized_route_decision.get("route_mode"),
|
||||||
|
"effective_source_policy": normalized_route_decision.get("effective_source_policy"),
|
||||||
|
"decision_origin": normalized_route_decision.get("decision_origin"),
|
||||||
|
},
|
||||||
|
)
|
||||||
final_domains = [str(x) for x in (search_meta.get("domains_used") or domains)]
|
final_domains = [str(x) for x in (search_meta.get("domains_used") or domains)]
|
||||||
prompt_variant_id = self._peek_answer_prompt_variant()
|
prompt_variant_id = self._peek_answer_prompt_variant()
|
||||||
|
|
||||||
|
|
@ -155,7 +431,7 @@ class AdaptiveRAGAnswerRetrievalMixin:
|
||||||
auto_retry_reason = ""
|
auto_retry_reason = ""
|
||||||
if not deep_thinking:
|
if not deep_thinking:
|
||||||
auto_retry_reason = str(pre_retry_guard_semantics.get("auto_retry_reason") or "").strip()
|
auto_retry_reason = str(pre_retry_guard_semantics.get("auto_retry_reason") or "").strip()
|
||||||
if auto_retry_reason:
|
if auto_retry_reason and not external_route_only:
|
||||||
deep_retrieved, deep_meta = await self._multi_step_search(
|
deep_retrieved, deep_meta = await self._multi_step_search(
|
||||||
question=effective_q,
|
question=effective_q,
|
||||||
domains=domains,
|
domains=domains,
|
||||||
|
|
@ -426,6 +702,92 @@ class AdaptiveRAGAnswerRetrievalMixin:
|
||||||
"vector_timeout": bool(vector_timeout),
|
"vector_timeout": bool(vector_timeout),
|
||||||
"graph_timeout": bool(graph_timeout),
|
"graph_timeout": bool(graph_timeout),
|
||||||
}
|
}
|
||||||
|
filter_sources_for_route = getattr(self, "_filter_sources_for_route", None)
|
||||||
|
if callable(filter_sources_for_route):
|
||||||
|
used_sources = filter_sources_for_route(
|
||||||
|
sources=used_sources,
|
||||||
|
route_decision=normalized_route_decision,
|
||||||
|
)
|
||||||
|
render_with_contract = getattr(self, "_render_answer_with_route_contract", None)
|
||||||
|
if callable(render_with_contract):
|
||||||
|
answer_text = render_with_contract(
|
||||||
|
answer=answer_text,
|
||||||
|
sources=used_sources,
|
||||||
|
route_decision=normalized_route_decision,
|
||||||
|
)
|
||||||
|
effective_source_policy = str(normalized_route_decision.get("effective_source_policy") or "").strip().lower()
|
||||||
|
contract_verdict = {"ok": True, "reason_code": None, "missing_fields": []}
|
||||||
|
guard_payload: Optional[dict[str, Any]] = None
|
||||||
|
if effective_source_policy == "external" and answer_mode != ANSWER_MODE_CLARIFY:
|
||||||
|
contract_verdict = self._verify_external_answer_contract(
|
||||||
|
answer=answer_text,
|
||||||
|
sources=used_sources,
|
||||||
|
route_decision=normalized_route_decision,
|
||||||
|
)
|
||||||
|
if not bool(contract_verdict.get("ok")):
|
||||||
|
guard_payload = self._build_external_answer_contract_guard_payload(
|
||||||
|
reason_code=str(contract_verdict.get("reason_code") or ""),
|
||||||
|
missing_fields=list(contract_verdict.get("missing_fields") or []),
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
contract_verdict = self._verify_internal_answer_contract(
|
||||||
|
query_type=query_type,
|
||||||
|
answer=answer_text,
|
||||||
|
sources=used_sources,
|
||||||
|
route_decision=normalized_route_decision,
|
||||||
|
)
|
||||||
|
if not bool(contract_verdict.get("ok")):
|
||||||
|
guard_payload = self._build_internal_answer_contract_guard_payload(
|
||||||
|
query_type=query_type,
|
||||||
|
reason_code=str(contract_verdict.get("reason_code") or ""),
|
||||||
|
missing_fields=list(contract_verdict.get("missing_fields") or []),
|
||||||
|
)
|
||||||
|
if guard_payload:
|
||||||
|
answer_text = str(guard_payload.get("answer") or answer_text)
|
||||||
|
used_sources = list(guard_payload.get("sources") or [])
|
||||||
|
answer_mode = str(guard_payload.get("answer_mode") or answer_mode or ANSWER_MODE_BLOCKED)
|
||||||
|
max_score = min(float(max_score or 0.0), 0.35)
|
||||||
|
if "system_context" not in final_domains:
|
||||||
|
final_domains.append("system_context")
|
||||||
|
tool_trace.append(
|
||||||
|
{
|
||||||
|
"step": 100,
|
||||||
|
"tool": (
|
||||||
|
"external_answer_contract_guard"
|
||||||
|
if effective_source_policy == "external"
|
||||||
|
else "internal_answer_contract_guard"
|
||||||
|
),
|
||||||
|
"reason": str(
|
||||||
|
guard_payload.get("reason_code")
|
||||||
|
or (
|
||||||
|
"external_answer_contract_violation"
|
||||||
|
if effective_source_policy == "external"
|
||||||
|
else "internal_answer_contract_violation"
|
||||||
|
)
|
||||||
|
),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
grounding_verdict = GroundingVerdict(
|
||||||
|
status=str(
|
||||||
|
guard_payload.get("grounding_status")
|
||||||
|
or (GROUNDING_STATUS_INSUFFICIENT_EVIDENCE if effective_source_policy == "external" else GROUNDING_STATUS_BLOCKED)
|
||||||
|
),
|
||||||
|
missing_fields=list(guard_payload.get("missing_fields") or []),
|
||||||
|
reason_code=str(
|
||||||
|
guard_payload.get("reason_code")
|
||||||
|
or (
|
||||||
|
"external_answer_contract_violation"
|
||||||
|
if effective_source_policy == "external"
|
||||||
|
else "internal_answer_contract_violation"
|
||||||
|
)
|
||||||
|
),
|
||||||
|
)
|
||||||
|
inject_runtime_summaries = getattr(self, "_inject_internal_query_runtime_summaries", None)
|
||||||
|
if callable(inject_runtime_summaries):
|
||||||
|
retrieval_metrics = inject_runtime_summaries(
|
||||||
|
retrieval_metrics=retrieval_metrics,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
)
|
||||||
runtime_fields = self._build_qa_record_runtime_fields(
|
runtime_fields = self._build_qa_record_runtime_fields(
|
||||||
answer=answer_text,
|
answer=answer_text,
|
||||||
sources=used_sources,
|
sources=used_sources,
|
||||||
|
|
@ -455,6 +817,7 @@ class AdaptiveRAGAnswerRetrievalMixin:
|
||||||
anchor_kind=None,
|
anchor_kind=None,
|
||||||
anchor_ref=None,
|
anchor_ref=None,
|
||||||
routing_snapshot=routing_snapshot,
|
routing_snapshot=routing_snapshot,
|
||||||
|
route_decision=normalized_route_decision,
|
||||||
),
|
),
|
||||||
**runtime_fields,
|
**runtime_fields,
|
||||||
)
|
)
|
||||||
|
|
@ -473,6 +836,15 @@ class AdaptiveRAGAnswerRetrievalMixin:
|
||||||
rewrite_meta=rewrite_meta,
|
rewrite_meta=rewrite_meta,
|
||||||
sources=used_sources,
|
sources=used_sources,
|
||||||
)
|
)
|
||||||
|
await self.db.flush()
|
||||||
|
persist_route_memory = getattr(self, "_persist_query_route_memory", None)
|
||||||
|
if callable(persist_route_memory):
|
||||||
|
await persist_route_memory(
|
||||||
|
qa_record_id=int(qa.id) if getattr(qa, "id", None) is not None else None,
|
||||||
|
question_text=clean_q,
|
||||||
|
project_id=project_id,
|
||||||
|
route_decision=normalized_route_decision,
|
||||||
|
)
|
||||||
await self.db.commit()
|
await self.db.commit()
|
||||||
await self.db.refresh(qa)
|
await self.db.refresh(qa)
|
||||||
await self.context_cache.append_session_messages(
|
await self.context_cache.append_session_messages(
|
||||||
|
|
@ -510,4 +882,5 @@ class AdaptiveRAGAnswerRetrievalMixin:
|
||||||
sources_used=runtime_fields["sources_used"],
|
sources_used=runtime_fields["sources_used"],
|
||||||
evidence_pack=runtime_fields["evidence_pack"],
|
evidence_pack=runtime_fields["evidence_pack"],
|
||||||
prompt_variant_id=prompt_variant_id,
|
prompt_variant_id=prompt_variant_id,
|
||||||
|
route_decision=normalized_route_decision,
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -8,16 +8,22 @@ from ..ai_pmo.workflow_semantic_planner import WorkflowSemanticPlanner
|
||||||
class AdaptiveRAGCoreIntentMixin:
|
class AdaptiveRAGCoreIntentMixin:
|
||||||
_SEMANTIC_INTENTS = {"question", "statement", "mixed"}
|
_SEMANTIC_INTENTS = {"question", "statement", "mixed"}
|
||||||
_SEMANTIC_QUERY_TYPES = {
|
_SEMANTIC_QUERY_TYPES = {
|
||||||
|
QUERY_TYPE_EXISTENCE,
|
||||||
QUERY_TYPE_COUNT,
|
QUERY_TYPE_COUNT,
|
||||||
QUERY_TYPE_LIST,
|
QUERY_TYPE_LIST,
|
||||||
QUERY_TYPE_STATUS,
|
QUERY_TYPE_STATUS,
|
||||||
QUERY_TYPE_EXPLAIN,
|
QUERY_TYPE_EXPLAIN,
|
||||||
|
QUERY_TYPE_DIAGNOSE,
|
||||||
|
QUERY_TYPE_COMPARE,
|
||||||
QUERY_TYPE_UNKNOWN,
|
QUERY_TYPE_UNKNOWN,
|
||||||
}
|
}
|
||||||
_SEMANTIC_RETRIEVAL_DOMAINS = ("db", "graph", "doc", "feishu", "pmflow", "url")
|
_SEMANTIC_RETRIEVAL_DOMAINS = ("db", "graph", "doc", "feishu", "pmflow", "url")
|
||||||
|
_INTERNAL_RETRIEVAL_DOMAINS = ("db", "graph", "doc", "feishu", "pmflow")
|
||||||
|
_EXTERNAL_RETRIEVAL_DOMAINS = ("url",)
|
||||||
_SEMANTIC_SUBJECT_FOCI = {"people", "project", "task", "signal", "charter", "workload", "feature", "unknown"}
|
_SEMANTIC_SUBJECT_FOCI = {"people", "project", "task", "signal", "charter", "workload", "feature", "unknown"}
|
||||||
_ALLOWED_SCOPE_VALUES = {"none", "global", "project"}
|
_ALLOWED_SCOPE_VALUES = {"none", "global", "project"}
|
||||||
_SEMANTIC_FEATURE_FOCI = {"none", "purpose", "entry", "catalog"}
|
_SEMANTIC_FEATURE_FOCI = {"none", "purpose", "entry", "catalog"}
|
||||||
|
_SOURCE_POLICIES = {"internal", "external", "uncertain"}
|
||||||
|
|
||||||
def __init__(self, db: AsyncSession, *, tenant_id: int) -> None:
|
def __init__(self, db: AsyncSession, *, tenant_id: int) -> None:
|
||||||
self.db = db
|
self.db = db
|
||||||
|
|
@ -118,8 +124,43 @@ class AdaptiveRAGCoreIntentMixin:
|
||||||
"routing_decision": routing_decision or None,
|
"routing_decision": routing_decision or None,
|
||||||
"routing_config_version": routing_config_version or None,
|
"routing_config_version": routing_config_version or None,
|
||||||
}
|
}
|
||||||
|
query_route = payload.get("query_route") if isinstance(payload.get("query_route"), dict) else {}
|
||||||
|
if query_route:
|
||||||
|
normalized["query_route"] = {
|
||||||
|
"route_mode": str(query_route.get("route_mode") or "").strip().lower() or None,
|
||||||
|
"effective_source_policy": str(query_route.get("effective_source_policy") or "").strip().lower() or None,
|
||||||
|
"decision_origin": str(query_route.get("decision_origin") or "").strip() or None,
|
||||||
|
"reply_contract": (
|
||||||
|
dict(query_route.get("reply_contract") or {})
|
||||||
|
if isinstance(query_route.get("reply_contract"), dict)
|
||||||
|
else {}
|
||||||
|
),
|
||||||
|
"query_profile": (
|
||||||
|
dict(query_route.get("query_profile") or {})
|
||||||
|
if isinstance(query_route.get("query_profile"), dict)
|
||||||
|
else {}
|
||||||
|
),
|
||||||
|
"probe_snapshot_summary": (
|
||||||
|
dict(query_route.get("probe_snapshot_summary") or {})
|
||||||
|
if isinstance(query_route.get("probe_snapshot_summary"), dict)
|
||||||
|
else {}
|
||||||
|
),
|
||||||
|
}
|
||||||
return {key: value for key, value in normalized.items() if value is not None}
|
return {key: value for key, value in normalized.items() if value is not None}
|
||||||
|
|
||||||
|
def _merge_query_route_audit_into_routing_snapshot(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
routing_snapshot: Optional[dict[str, Any]],
|
||||||
|
route_decision: Optional[dict[str, Any]],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
normalized_snapshot = self._normalize_routing_snapshot(routing_snapshot)
|
||||||
|
if hasattr(self, "_build_query_route_audit_snapshot"):
|
||||||
|
query_route = self._build_query_route_audit_snapshot(route_decision=route_decision)
|
||||||
|
if query_route:
|
||||||
|
normalized_snapshot["query_route"] = query_route
|
||||||
|
return normalized_snapshot
|
||||||
|
|
||||||
def _peek_last_answer_routing_snapshot(self) -> dict[str, Any]:
|
def _peek_last_answer_routing_snapshot(self) -> dict[str, Any]:
|
||||||
payload = getattr(self.llm, "_last_ai_call_summary", None)
|
payload = getattr(self.llm, "_last_ai_call_summary", None)
|
||||||
if not isinstance(payload, dict):
|
if not isinstance(payload, dict):
|
||||||
|
|
@ -137,9 +178,13 @@ class AdaptiveRAGCoreIntentMixin:
|
||||||
anchor_kind: Optional[str] = None,
|
anchor_kind: Optional[str] = None,
|
||||||
anchor_ref: Optional[str] = None,
|
anchor_ref: Optional[str] = None,
|
||||||
routing_snapshot: Optional[dict[str, Any]] = None,
|
routing_snapshot: Optional[dict[str, Any]] = None,
|
||||||
|
route_decision: Optional[dict[str, Any]] = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
context = self._get_active_external_qa_context()
|
context = self._get_active_external_qa_context()
|
||||||
normalized_snapshot = self._normalize_routing_snapshot(routing_snapshot) or None
|
normalized_snapshot = self._merge_query_route_audit_into_routing_snapshot(
|
||||||
|
routing_snapshot=routing_snapshot,
|
||||||
|
route_decision=route_decision,
|
||||||
|
) or None
|
||||||
return {
|
return {
|
||||||
"source_message_id": context.get("source_message_id"),
|
"source_message_id": context.get("source_message_id"),
|
||||||
"source_chat_id": context.get("source_chat_id"),
|
"source_chat_id": context.get("source_chat_id"),
|
||||||
|
|
@ -270,6 +315,15 @@ class AdaptiveRAGCoreIntentMixin:
|
||||||
except Exception:
|
except Exception:
|
||||||
return float(default)
|
return float(default)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _safe_int_value(value: Any) -> Optional[int]:
|
||||||
|
try:
|
||||||
|
if value is None:
|
||||||
|
return None
|
||||||
|
return int(value)
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _parse_json_object_safe(raw: Any) -> dict[str, Any]:
|
def _parse_json_object_safe(raw: Any) -> dict[str, Any]:
|
||||||
if isinstance(raw, dict):
|
if isinstance(raw, dict):
|
||||||
|
|
@ -323,12 +377,94 @@ class AdaptiveRAGCoreIntentMixin:
|
||||||
return normalized
|
return normalized
|
||||||
return "none"
|
return "none"
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _normalize_source_policy(cls, value: Any) -> str:
|
||||||
|
normalized = str(value or "").strip().lower()
|
||||||
|
if normalized in cls._SOURCE_POLICIES:
|
||||||
|
return normalized
|
||||||
|
return "uncertain"
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _normalize_query_profile_object(cls, value: Any) -> str:
|
||||||
|
normalized = re.sub(r"[^a-z0-9:_\-]", "", str(value or "").strip().lower())
|
||||||
|
return normalized[:48]
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _normalize_query_profile(cls, payload: Any) -> dict[str, Any]:
|
||||||
|
raw = payload if isinstance(payload, dict) else {}
|
||||||
|
referenced_objects: list[str] = []
|
||||||
|
for item in list(raw.get("referenced_objects") or []):
|
||||||
|
normalized = cls._normalize_query_profile_object(item)
|
||||||
|
if not normalized:
|
||||||
|
continue
|
||||||
|
if normalized not in referenced_objects:
|
||||||
|
referenced_objects.append(normalized)
|
||||||
|
if not referenced_objects:
|
||||||
|
referenced_objects = ["unknown"]
|
||||||
|
try:
|
||||||
|
confidence = max(0.0, min(1.0, float(raw.get("confidence") or 0.0)))
|
||||||
|
except Exception:
|
||||||
|
confidence = 0.0
|
||||||
|
return {
|
||||||
|
"needs_private_context": bool(raw.get("needs_private_context")),
|
||||||
|
"needs_freshness": bool(raw.get("needs_freshness")),
|
||||||
|
"referenced_objects": referenced_objects,
|
||||||
|
"confidence": confidence,
|
||||||
|
}
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _normalize_semantic_object_mentions(payload: Any) -> list[str]:
|
||||||
|
rows = payload if isinstance(payload, list) else []
|
||||||
|
mentions: list[str] = []
|
||||||
|
for item in rows:
|
||||||
|
text = str(item or "").strip(" ,。,.!??::;;“”\"'()()")
|
||||||
|
if len(text) >= 2 and text not in mentions:
|
||||||
|
mentions.append(text[:32])
|
||||||
|
return mentions[:4]
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _apply_source_policy_to_domains(
|
||||||
|
cls,
|
||||||
|
*,
|
||||||
|
domains: list[str],
|
||||||
|
effective_source_policy: str,
|
||||||
|
) -> list[str]:
|
||||||
|
normalized_policy = cls._normalize_source_policy(effective_source_policy)
|
||||||
|
if normalized_policy == "internal":
|
||||||
|
internal_domains = [item for item in domains if item in cls._INTERNAL_RETRIEVAL_DOMAINS]
|
||||||
|
if internal_domains:
|
||||||
|
return internal_domains
|
||||||
|
return list(cls._INTERNAL_RETRIEVAL_DOMAINS)
|
||||||
|
external_domains = [item for item in domains if item in cls._EXTERNAL_RETRIEVAL_DOMAINS]
|
||||||
|
if external_domains:
|
||||||
|
return external_domains
|
||||||
|
return list(cls._EXTERNAL_RETRIEVAL_DOMAINS)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _should_normalize_to_external_contract(
|
||||||
|
cls,
|
||||||
|
*,
|
||||||
|
source_policy: str,
|
||||||
|
effective_source_policy: str,
|
||||||
|
parsed_domains: list[str],
|
||||||
|
query_profile: dict[str, Any],
|
||||||
|
) -> bool:
|
||||||
|
if source_policy != "uncertain" or effective_source_policy != "uncertain":
|
||||||
|
return False
|
||||||
|
normalized_domains = [str(item or "").strip().lower() for item in list(parsed_domains or []) if str(item or "").strip()]
|
||||||
|
if not normalized_domains:
|
||||||
|
return False
|
||||||
|
if any(item not in cls._EXTERNAL_RETRIEVAL_DOMAINS for item in normalized_domains):
|
||||||
|
return False
|
||||||
|
return bool(query_profile.get("needs_freshness")) and not bool(query_profile.get("needs_private_context"))
|
||||||
|
|
||||||
async def _resolve_query_semantics(
|
async def _resolve_query_semantics(
|
||||||
self,
|
self,
|
||||||
*,
|
*,
|
||||||
message: str,
|
message: str,
|
||||||
source_system: str,
|
source_system: str,
|
||||||
metadata: Optional[dict[str, Any]] = None,
|
metadata: Optional[dict[str, Any]] = None,
|
||||||
|
source_policy_hint: Optional[str] = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
text = str(message or "").strip()
|
text = str(message or "").strip()
|
||||||
default_domains = list(self._SEMANTIC_RETRIEVAL_DOMAINS)
|
default_domains = list(self._SEMANTIC_RETRIEVAL_DOMAINS)
|
||||||
|
|
@ -339,7 +475,14 @@ class AdaptiveRAGCoreIntentMixin:
|
||||||
"question_part": "",
|
"question_part": "",
|
||||||
"statement_part": "",
|
"statement_part": "",
|
||||||
"query_type": QUERY_TYPE_UNKNOWN,
|
"query_type": QUERY_TYPE_UNKNOWN,
|
||||||
|
"object_mentions": [],
|
||||||
"retrieval_domains": default_domains,
|
"retrieval_domains": default_domains,
|
||||||
|
"source_policy": "uncertain",
|
||||||
|
"effective_source_policy": "uncertain",
|
||||||
|
"source_reason": "empty_message",
|
||||||
|
"decision_origin": "empty_message",
|
||||||
|
"fallback_taken": False,
|
||||||
|
"query_profile": self._normalize_query_profile(None),
|
||||||
"confidence": 1.0,
|
"confidence": 1.0,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
@ -402,6 +545,50 @@ class AdaptiveRAGCoreIntentMixin:
|
||||||
query_type = str(parsed.get("query_type") or QUERY_TYPE_UNKNOWN).strip().lower()
|
query_type = str(parsed.get("query_type") or QUERY_TYPE_UNKNOWN).strip().lower()
|
||||||
if query_type not in self._SEMANTIC_QUERY_TYPES:
|
if query_type not in self._SEMANTIC_QUERY_TYPES:
|
||||||
query_type = QUERY_TYPE_UNKNOWN
|
query_type = QUERY_TYPE_UNKNOWN
|
||||||
|
object_mentions = self._normalize_semantic_object_mentions(parsed.get("object_mentions"))
|
||||||
|
|
||||||
|
subject_focus = self._normalize_semantic_subject_focus(parsed.get("subject_focus"))
|
||||||
|
scope_hint = self._normalize_semantic_scope_hint(parsed.get("scope_hint"))
|
||||||
|
feature_focus = self._normalize_semantic_feature_focus(parsed.get("feature_focus"))
|
||||||
|
parsed_domains = self._normalize_semantic_domains(parsed.get("retrieval_domains"))
|
||||||
|
raw_source_policy = self._normalize_source_policy(parsed.get("source_policy"))
|
||||||
|
raw_effective_source_policy = self._normalize_source_policy(parsed.get("effective_source_policy"))
|
||||||
|
query_profile = self._normalize_query_profile(parsed.get("query_profile"))
|
||||||
|
normalized_hint = self._normalize_source_policy(source_policy_hint)
|
||||||
|
fallback_taken = False
|
||||||
|
if normalized_hint in {"internal", "external"}:
|
||||||
|
source_policy = normalized_hint
|
||||||
|
effective_source_policy = normalized_hint
|
||||||
|
source_reason = "upstream_source_policy_hint"
|
||||||
|
decision_origin = "upstream_hint"
|
||||||
|
fallback_taken = raw_source_policy not in {"uncertain", normalized_hint}
|
||||||
|
else:
|
||||||
|
source_policy = raw_source_policy
|
||||||
|
if raw_effective_source_policy in {"internal", "external"}:
|
||||||
|
effective_source_policy = raw_effective_source_policy
|
||||||
|
elif source_policy in {"internal", "external"}:
|
||||||
|
effective_source_policy = source_policy
|
||||||
|
else:
|
||||||
|
effective_source_policy = "uncertain"
|
||||||
|
source_reason = "query_semantics_contract"
|
||||||
|
decision_origin = "query_semantics_contract"
|
||||||
|
if self._should_normalize_to_external_contract(
|
||||||
|
source_policy=source_policy,
|
||||||
|
effective_source_policy=effective_source_policy,
|
||||||
|
parsed_domains=parsed_domains,
|
||||||
|
query_profile=query_profile,
|
||||||
|
):
|
||||||
|
source_policy = "external"
|
||||||
|
effective_source_policy = "external"
|
||||||
|
source_reason = "query_semantics_url_only_external_contract"
|
||||||
|
decision_origin = "query_semantics_contract_normalized"
|
||||||
|
if effective_source_policy == "uncertain":
|
||||||
|
retrieval_domains = list(parsed_domains)
|
||||||
|
else:
|
||||||
|
retrieval_domains = self._apply_source_policy_to_domains(
|
||||||
|
domains=parsed_domains,
|
||||||
|
effective_source_policy=effective_source_policy,
|
||||||
|
)
|
||||||
|
|
||||||
confidence = self._safe_float(parsed.get("confidence"), planner_confidence)
|
confidence = self._safe_float(parsed.get("confidence"), planner_confidence)
|
||||||
normalized_message_intent = planner_message_intent or intent
|
normalized_message_intent = planner_message_intent or intent
|
||||||
|
|
@ -411,10 +598,17 @@ class AdaptiveRAGCoreIntentMixin:
|
||||||
"question_part": question_part,
|
"question_part": question_part,
|
||||||
"statement_part": statement_part,
|
"statement_part": statement_part,
|
||||||
"query_type": query_type,
|
"query_type": query_type,
|
||||||
"retrieval_domains": self._normalize_semantic_domains(parsed.get("retrieval_domains")),
|
"object_mentions": object_mentions,
|
||||||
"subject_focus": self._normalize_semantic_subject_focus(parsed.get("subject_focus")),
|
"retrieval_domains": retrieval_domains,
|
||||||
"scope_hint": self._normalize_semantic_scope_hint(parsed.get("scope_hint")),
|
"subject_focus": subject_focus,
|
||||||
"feature_focus": self._normalize_semantic_feature_focus(parsed.get("feature_focus")),
|
"scope_hint": scope_hint,
|
||||||
|
"feature_focus": feature_focus,
|
||||||
|
"query_profile": query_profile,
|
||||||
|
"source_policy": source_policy,
|
||||||
|
"effective_source_policy": effective_source_policy,
|
||||||
|
"source_reason": source_reason,
|
||||||
|
"decision_origin": decision_origin,
|
||||||
|
"fallback_taken": fallback_taken,
|
||||||
"confidence": max(0.0, min(1.0, confidence)),
|
"confidence": max(0.0, min(1.0, confidence)),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -29,81 +29,7 @@ class AdaptiveRAGDeepGuardAnswerMixin:
|
||||||
"engine_process_query",
|
"engine_process_query",
|
||||||
"runtime_state_query",
|
"runtime_state_query",
|
||||||
}
|
}
|
||||||
_PROJECT_CONTEXT_POSITIVE_MARKERS = (
|
_MODEL_TOKEN_RE = re.compile(
|
||||||
"这个项目",
|
|
||||||
"当前项目",
|
|
||||||
"本项目",
|
|
||||||
"我们项目",
|
|
||||||
"项目所在行业",
|
|
||||||
"对项目的影响",
|
|
||||||
"对这个项目的影响",
|
|
||||||
"对当前项目的影响",
|
|
||||||
"对我们项目的影响",
|
|
||||||
"结合项目",
|
|
||||||
"结合当前项目",
|
|
||||||
"落到项目",
|
|
||||||
)
|
|
||||||
_PROJECT_CONTEXT_NEGATIVE_MARKERS = (
|
|
||||||
"不要提项目",
|
|
||||||
"别提项目",
|
|
||||||
"不要结合项目",
|
|
||||||
"不用结合项目",
|
|
||||||
"不需要结合项目",
|
|
||||||
"先别结合项目",
|
|
||||||
"不要绑定项目",
|
|
||||||
"别绑定项目",
|
|
||||||
)
|
|
||||||
_EXTERNAL_FACT_FRESHNESS_MARKERS = (
|
|
||||||
"最新",
|
|
||||||
"目前",
|
|
||||||
"当前",
|
|
||||||
"截至现在",
|
|
||||||
"截至当前",
|
|
||||||
"截止当前",
|
|
||||||
"今天",
|
|
||||||
"现在",
|
|
||||||
"最近",
|
|
||||||
"出到什么",
|
|
||||||
"发布了吗",
|
|
||||||
"发布到",
|
|
||||||
"版本",
|
|
||||||
"最高模型",
|
|
||||||
)
|
|
||||||
_MODEL_ENTITY_MARKERS = (
|
|
||||||
"codex",
|
|
||||||
"gpt",
|
|
||||||
"chatgpt",
|
|
||||||
"claude",
|
|
||||||
"deepseek",
|
|
||||||
"qwen",
|
|
||||||
"gemini",
|
|
||||||
"copilot",
|
|
||||||
"grok",
|
|
||||||
"kimi",
|
|
||||||
"llama",
|
|
||||||
"openai",
|
|
||||||
"anthropic",
|
|
||||||
)
|
|
||||||
_INTERNAL_MODEL_CONFIG_MARKERS = (
|
|
||||||
"ai-config",
|
|
||||||
"/ai-config",
|
|
||||||
"模型配置",
|
|
||||||
"配置项",
|
|
||||||
"配置值",
|
|
||||||
"路由规则",
|
|
||||||
"初始化参数",
|
|
||||||
"llm服务",
|
|
||||||
"llm 服务",
|
|
||||||
"运行时",
|
|
||||||
"当前系统",
|
|
||||||
"我们系统",
|
|
||||||
"pmflow",
|
|
||||||
"本系统",
|
|
||||||
"接入的模型",
|
|
||||||
"接的是",
|
|
||||||
"用的是哪个模型配置",
|
|
||||||
)
|
|
||||||
_EXTERNAL_MODEL_TOKEN_RE = re.compile(
|
|
||||||
r"\b(?:GPT|gpt)[- ]?\d(?:\.\d)?(?:[- ]?(?:codex|turbo|mini|nano))?\b",
|
r"\b(?:GPT|gpt)[- ]?\d(?:\.\d)?(?:[- ]?(?:codex|turbo|mini|nano))?\b",
|
||||||
flags=re.IGNORECASE,
|
flags=re.IGNORECASE,
|
||||||
)
|
)
|
||||||
|
|
@ -115,46 +41,6 @@ class AdaptiveRAGDeepGuardAnswerMixin:
|
||||||
return text
|
return text
|
||||||
return None
|
return None
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def _question_explicitly_requests_project_context(
|
|
||||||
cls,
|
|
||||||
*,
|
|
||||||
question: str,
|
|
||||||
project_id: Optional[int],
|
|
||||||
) -> bool:
|
|
||||||
if project_id is None:
|
|
||||||
return False
|
|
||||||
normalized = re.sub(r"\s+", "", str(question or "")).strip().lower()
|
|
||||||
if not normalized:
|
|
||||||
return False
|
|
||||||
if any(marker in normalized for marker in cls._PROJECT_CONTEXT_NEGATIVE_MARKERS):
|
|
||||||
return False
|
|
||||||
return any(marker in normalized for marker in cls._PROJECT_CONTEXT_POSITIVE_MARKERS)
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def _looks_like_external_freshness_question(cls, question: str) -> bool:
|
|
||||||
normalized = re.sub(r"\s+", "", str(question or "")).strip().lower()
|
|
||||||
if not normalized:
|
|
||||||
return False
|
|
||||||
return any(marker in normalized for marker in cls._EXTERNAL_FACT_FRESHNESS_MARKERS)
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def _looks_like_internal_model_config_question(cls, question: str) -> bool:
|
|
||||||
normalized = re.sub(r"\s+", "", str(question or "")).strip().lower()
|
|
||||||
if not normalized:
|
|
||||||
return False
|
|
||||||
return any(marker in normalized for marker in cls._INTERNAL_MODEL_CONFIG_MARKERS)
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def _looks_like_model_fact_question(cls, question: str) -> bool:
|
|
||||||
normalized = re.sub(r"\s+", "", str(question or "")).strip().lower()
|
|
||||||
if not normalized:
|
|
||||||
return False
|
|
||||||
has_entity = any(marker in normalized for marker in cls._MODEL_ENTITY_MARKERS)
|
|
||||||
if not has_entity:
|
|
||||||
return False
|
|
||||||
return "模型" in normalized or cls._looks_like_external_freshness_question(question)
|
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def _normalize_external_model_evidence_text(cls, text: str) -> str:
|
def _normalize_external_model_evidence_text(cls, text: str) -> str:
|
||||||
normalized = str(text or "")
|
normalized = str(text or "")
|
||||||
|
|
@ -175,7 +61,7 @@ class AdaptiveRAGDeepGuardAnswerMixin:
|
||||||
search_windows = [latest_match.group(1)] if latest_match else []
|
search_windows = [latest_match.group(1)] if latest_match else []
|
||||||
search_windows.append(normalized)
|
search_windows.append(normalized)
|
||||||
for window in search_windows:
|
for window in search_windows:
|
||||||
match = cls._EXTERNAL_MODEL_TOKEN_RE.search(str(window or ""))
|
match = cls._MODEL_TOKEN_RE.search(str(window or ""))
|
||||||
if not match:
|
if not match:
|
||||||
continue
|
continue
|
||||||
token = re.sub(r"\s+", "-", str(match.group(0) or "").strip())
|
token = re.sub(r"\s+", "-", str(match.group(0) or "").strip())
|
||||||
|
|
@ -191,8 +77,7 @@ class AdaptiveRAGDeepGuardAnswerMixin:
|
||||||
retrieved: list[RetrievedItem],
|
retrieved: list[RetrievedItem],
|
||||||
scene_semantics: Optional[dict[str, Any]] = None,
|
scene_semantics: Optional[dict[str, Any]] = None,
|
||||||
) -> Optional[str]:
|
) -> Optional[str]:
|
||||||
if not cls._looks_like_model_fact_question(question) or not cls._looks_like_external_freshness_question(question):
|
_ = question
|
||||||
return None
|
|
||||||
normalized_scene = scene_semantics if isinstance(scene_semantics, dict) else {}
|
normalized_scene = scene_semantics if isinstance(scene_semantics, dict) else {}
|
||||||
scene_key = str(normalized_scene.get("scene_key") or "").strip().lower()
|
scene_key = str(normalized_scene.get("scene_key") or "").strip().lower()
|
||||||
if scene_key and scene_key not in {"industry_trend", "external_trend"}:
|
if scene_key and scene_key not in {"industry_trend", "external_trend"}:
|
||||||
|
|
@ -230,6 +115,141 @@ class AdaptiveRAGDeepGuardAnswerMixin:
|
||||||
)
|
)
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
async def _build_internal_model_config_fast_answer(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
question: str,
|
||||||
|
query_semantics: Optional[dict[str, Any]] = None,
|
||||||
|
) -> Optional[dict[str, Any]]:
|
||||||
|
_ = question
|
||||||
|
semantics = query_semantics if isinstance(query_semantics, dict) else {}
|
||||||
|
source_policy = str(
|
||||||
|
semantics.get("effective_source_policy") or semantics.get("source_policy") or ""
|
||||||
|
).strip().lower()
|
||||||
|
if source_policy and source_policy != "internal":
|
||||||
|
return None
|
||||||
|
operation_contract = semantics.get("operation_contract") if isinstance(semantics.get("operation_contract"), dict) else {}
|
||||||
|
query_type = str(
|
||||||
|
operation_contract.get("query_type")
|
||||||
|
or semantics.get("query_type")
|
||||||
|
or QUERY_TYPE_UNKNOWN
|
||||||
|
).strip().lower()
|
||||||
|
if query_type not in {QUERY_TYPE_STATUS, QUERY_TYPE_EXPLAIN}:
|
||||||
|
return None
|
||||||
|
query_profile = semantics.get("query_profile") if isinstance(semantics.get("query_profile"), dict) else {}
|
||||||
|
referenced_objects = {
|
||||||
|
str(item).strip().lower()
|
||||||
|
for item in list(query_profile.get("referenced_objects") or [])
|
||||||
|
if str(item).strip()
|
||||||
|
}
|
||||||
|
if not referenced_objects.intersection({"system_config", "model_routing"}):
|
||||||
|
return None
|
||||||
|
target_resolution = (
|
||||||
|
semantics.get("target_resolution")
|
||||||
|
if isinstance(semantics.get("target_resolution"), dict)
|
||||||
|
else {}
|
||||||
|
)
|
||||||
|
target_entity_type = str(target_resolution.get("target_entity_type") or "").strip().lower()
|
||||||
|
if target_entity_type and target_entity_type != "config":
|
||||||
|
return None
|
||||||
|
|
||||||
|
get_bundle = getattr(self.llm, "_get_or_init_bundle", None)
|
||||||
|
resolve_active_engine = getattr(self.llm, "_resolve_active_engine_from_bundle", None)
|
||||||
|
resolve_engine_for_call = getattr(self.llm, "_resolve_engine_for_call", None)
|
||||||
|
if not callable(get_bundle) or not callable(resolve_active_engine) or not callable(resolve_engine_for_call):
|
||||||
|
return None
|
||||||
|
|
||||||
|
try:
|
||||||
|
_row, bundle = await get_bundle(self.db, tenant_id=self.tenant_id)
|
||||||
|
bundle = dict(bundle) if isinstance(bundle, dict) else {}
|
||||||
|
active_engine = dict(resolve_active_engine(bundle) or {})
|
||||||
|
normal_engine = dict(
|
||||||
|
await resolve_engine_for_call(
|
||||||
|
self.db,
|
||||||
|
tenant_id=self.tenant_id,
|
||||||
|
function_name="adaptive_rag_answer_normal",
|
||||||
|
)
|
||||||
|
or {}
|
||||||
|
)
|
||||||
|
deep_engine = dict(
|
||||||
|
await resolve_engine_for_call(
|
||||||
|
self.db,
|
||||||
|
tenant_id=self.tenant_id,
|
||||||
|
function_name="adaptive_rag_answer_deep",
|
||||||
|
)
|
||||||
|
or {}
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
|
||||||
|
settings = bundle.get("settings") if isinstance(bundle.get("settings"), dict) else {}
|
||||||
|
local_kb_enabled = bool(settings.get("enable_local_kb_qa"))
|
||||||
|
|
||||||
|
def _engine_summary(engine: dict[str, Any]) -> str:
|
||||||
|
name = str(engine.get("name") or "unknown").strip() or "unknown"
|
||||||
|
provider = str(engine.get("provider") or "unknown").strip() or "unknown"
|
||||||
|
model = str(engine.get("model") or "unknown").strip() or "unknown"
|
||||||
|
routing_meta = engine.get("_routing_meta") if isinstance(engine.get("_routing_meta"), dict) else {}
|
||||||
|
tier = (
|
||||||
|
str(routing_meta.get("executed_tier") or routing_meta.get("planned_tier") or "")
|
||||||
|
.strip()
|
||||||
|
.lower()
|
||||||
|
)
|
||||||
|
if tier:
|
||||||
|
return f"{name}(provider={provider}, model={model}, tier={tier})"
|
||||||
|
return f"{name}(provider={provider}, model={model})"
|
||||||
|
|
||||||
|
active_summary = _engine_summary(active_engine)
|
||||||
|
normal_summary = _engine_summary(normal_engine)
|
||||||
|
deep_summary = _engine_summary(deep_engine)
|
||||||
|
answer_lines = [
|
||||||
|
f"当前活动引擎是 {active_summary}。",
|
||||||
|
f"`adaptive_rag_answer_normal` 当前走 {normal_summary}。",
|
||||||
|
f"`adaptive_rag_answer_deep` 当前走 {deep_summary}。",
|
||||||
|
f"`enable_local_kb_qa={'true' if local_kb_enabled else 'false'}`。",
|
||||||
|
]
|
||||||
|
active_engine_id = str(bundle.get("active_engine_id") or "").strip() or None
|
||||||
|
return {
|
||||||
|
"answer": "\n".join(answer_lines),
|
||||||
|
"sources": [
|
||||||
|
{
|
||||||
|
"source": "db",
|
||||||
|
"label": "Integration[llm_engine]",
|
||||||
|
"score": 0.99,
|
||||||
|
"metadata": {
|
||||||
|
"integration_type": "llm_engine",
|
||||||
|
"active_engine_id": active_engine_id,
|
||||||
|
"provider": str(active_engine.get("provider") or ""),
|
||||||
|
"model": str(active_engine.get("model") or ""),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"source": "db",
|
||||||
|
"label": "Integration[model_routing:adaptive_rag_answer_normal]",
|
||||||
|
"score": 0.98,
|
||||||
|
"metadata": {
|
||||||
|
"function_name": "adaptive_rag_answer_normal",
|
||||||
|
"provider": str(normal_engine.get("provider") or ""),
|
||||||
|
"model": str(normal_engine.get("model") or ""),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"source": "db",
|
||||||
|
"label": "Integration[model_routing:adaptive_rag_answer_deep]",
|
||||||
|
"score": 0.98,
|
||||||
|
"metadata": {
|
||||||
|
"function_name": "adaptive_rag_answer_deep",
|
||||||
|
"provider": str(deep_engine.get("provider") or ""),
|
||||||
|
"model": str(deep_engine.get("model") or ""),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
],
|
||||||
|
"query_type": QUERY_TYPE_STATUS,
|
||||||
|
"confidence": 0.99,
|
||||||
|
"reason_code": "internal_model_config_direct",
|
||||||
|
"prompt_variant_id": "internal_model_config_direct",
|
||||||
|
}
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def _apply_consultant_mode_preference(
|
def _apply_consultant_mode_preference(
|
||||||
cls,
|
cls,
|
||||||
|
|
@ -529,56 +549,54 @@ class AdaptiveRAGDeepGuardAnswerMixin:
|
||||||
message_intent: Optional[str],
|
message_intent: Optional[str],
|
||||||
query_type: Optional[str],
|
query_type: Optional[str],
|
||||||
conversation_text: str,
|
conversation_text: str,
|
||||||
|
source_policy: Optional[str] = None,
|
||||||
|
project_id: Optional[int] = None,
|
||||||
|
query_semantics: Optional[dict[str, Any]] = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
|
_ = (effective_question, conversation_text)
|
||||||
fallback = self._default_answer_scene_semantics(
|
fallback = self._default_answer_scene_semantics(
|
||||||
asked_via=asked_via,
|
asked_via=asked_via,
|
||||||
message_intent=message_intent,
|
message_intent=message_intent,
|
||||||
)
|
)
|
||||||
prompt = PROMPT_RAG_ANSWER_SCENE_SEMANTICS.format(
|
semantics = query_semantics if isinstance(query_semantics, dict) else {}
|
||||||
asked_via=str(asked_via or "").strip().lower() or "pmflow",
|
normalized_source_policy = str(source_policy or "").strip().lower()
|
||||||
message_intent=str(message_intent or "").strip().lower() or "unknown",
|
if normalized_source_policy == "external":
|
||||||
query_type=str(query_type or QUERY_TYPE_UNKNOWN).strip().lower() or QUERY_TYPE_UNKNOWN,
|
has_project_scope = (
|
||||||
conversation_tail=self._truncate_compact_text(conversation_text, limit=1200),
|
project_id is not None
|
||||||
question=str(question or "").strip()[:800],
|
and str(semantics.get("scope_hint") or "none").strip().lower() == "project"
|
||||||
effective_question=str(effective_question or question or "").strip()[:1000],
|
|
||||||
).strip()
|
|
||||||
parsed: dict[str, Any] = {}
|
|
||||||
try:
|
|
||||||
raw = await self.llm.generate(
|
|
||||||
prompt,
|
|
||||||
db=self.db,
|
|
||||||
tenant_id=self.tenant_id,
|
|
||||||
function_name="rag_answer_scene_semantics",
|
|
||||||
temperature=0.0,
|
|
||||||
max_tokens=260,
|
|
||||||
use_prompt_profile=False,
|
|
||||||
)
|
)
|
||||||
parser = getattr(self, "_parse_json_object_safe", None)
|
scene_key = "external_trend" if has_project_scope else "industry_trend"
|
||||||
parsed = parser(raw) if callable(parser) else {}
|
return {
|
||||||
parsed = dict(parsed) if isinstance(parsed, dict) else {}
|
"scene_key": scene_key,
|
||||||
except Exception:
|
"snapshot_focus": "none",
|
||||||
parsed = {}
|
"deep_intent": "none",
|
||||||
|
"confidence": 1.0,
|
||||||
|
"reason": "source_policy:external",
|
||||||
|
}
|
||||||
|
|
||||||
scene_key = str(parsed.get("scene_key") or fallback["scene_key"]).strip().lower()
|
subject_focus = str(semantics.get("subject_focus") or "unknown").strip().lower()
|
||||||
|
feature_focus = str(semantics.get("feature_focus") or "none").strip().lower()
|
||||||
|
scope_hint = str(semantics.get("scope_hint") or "none").strip().lower()
|
||||||
|
normalized_query_type = str(query_type or QUERY_TYPE_UNKNOWN).strip().lower()
|
||||||
|
|
||||||
|
scene_key = str(fallback["scene_key"])
|
||||||
|
if subject_focus == "workload":
|
||||||
|
scene_key = "member_workload"
|
||||||
|
elif subject_focus == "project" and normalized_query_type == QUERY_TYPE_STATUS:
|
||||||
|
scene_key = "project_status"
|
||||||
if scene_key not in self._ANSWER_SCENE_KEYS:
|
if scene_key not in self._ANSWER_SCENE_KEYS:
|
||||||
scene_key = str(fallback["scene_key"])
|
scene_key = str(fallback["scene_key"])
|
||||||
|
|
||||||
snapshot_focus = str(parsed.get("snapshot_focus") or fallback["snapshot_focus"]).strip().lower()
|
snapshot_focus = "feature" if (subject_focus == "feature" or feature_focus != "none") else str(fallback["snapshot_focus"])
|
||||||
if snapshot_focus not in self._SNAPSHOT_FOCI:
|
if snapshot_focus not in self._SNAPSHOT_FOCI:
|
||||||
snapshot_focus = "none"
|
snapshot_focus = str(fallback["snapshot_focus"])
|
||||||
|
confidence = 0.82 if scene_key != str(fallback["scene_key"]) or snapshot_focus != str(fallback["snapshot_focus"]) or scope_hint == "project" else 0.6
|
||||||
deep_intent = str(parsed.get("deep_intent") or fallback["deep_intent"]).strip().lower()
|
|
||||||
if deep_intent not in self._DEEP_INTENTS:
|
|
||||||
deep_intent = "none"
|
|
||||||
|
|
||||||
safe_float = getattr(self, "_safe_float", None)
|
|
||||||
confidence = safe_float(parsed.get("confidence"), 0.0) if callable(safe_float) else 0.0
|
|
||||||
return {
|
return {
|
||||||
"scene_key": scene_key,
|
"scene_key": scene_key,
|
||||||
"snapshot_focus": snapshot_focus,
|
"snapshot_focus": snapshot_focus,
|
||||||
"deep_intent": deep_intent,
|
"deep_intent": str(fallback["deep_intent"]),
|
||||||
"confidence": confidence,
|
"confidence": confidence,
|
||||||
"reason": str(parsed.get("reason") or "").strip(),
|
"reason": "query_semantics_mapping",
|
||||||
}
|
}
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
|
|
@ -655,18 +673,7 @@ class AdaptiveRAGDeepGuardAnswerMixin:
|
||||||
)
|
)
|
||||||
normalized_scene_key = str(scene_semantics.get("scene_key") or "default").strip().lower()
|
normalized_scene_key = str(scene_semantics.get("scene_key") or "default").strip().lower()
|
||||||
normalized_message_intent = str(message_intent or "").strip().lower()
|
normalized_message_intent = str(message_intent or "").strip().lower()
|
||||||
explicit_project_context = self._question_explicitly_requests_project_context(
|
if normalized_scene_key == "external_trend" and project_id is None:
|
||||||
question=question,
|
|
||||||
project_id=project_id,
|
|
||||||
)
|
|
||||||
internal_model_config_question = self._looks_like_internal_model_config_question(question)
|
|
||||||
model_fact_question = self._looks_like_model_fact_question(question)
|
|
||||||
freshness_question = self._looks_like_external_freshness_question(question)
|
|
||||||
|
|
||||||
if normalized_scene_key == "ai_pmo" and model_fact_question and freshness_question and not internal_model_config_question:
|
|
||||||
normalized_scene_key = "external_trend" if explicit_project_context else "industry_trend"
|
|
||||||
|
|
||||||
if normalized_scene_key == "external_trend" and not explicit_project_context:
|
|
||||||
normalized_scene_key = "industry_trend"
|
normalized_scene_key = "industry_trend"
|
||||||
|
|
||||||
if normalized_scene_key == "member_workload":
|
if normalized_scene_key == "member_workload":
|
||||||
|
|
@ -1170,6 +1177,14 @@ class AdaptiveRAGDeepGuardAnswerMixin:
|
||||||
message_intent=message_intent,
|
message_intent=message_intent,
|
||||||
query_type=query_type,
|
query_type=query_type,
|
||||||
conversation_text=conversation_text,
|
conversation_text=conversation_text,
|
||||||
|
source_policy=str(
|
||||||
|
(query_semantics or {}).get("effective_source_policy")
|
||||||
|
or (query_semantics or {}).get("source_policy")
|
||||||
|
or ""
|
||||||
|
).strip().lower()
|
||||||
|
or None,
|
||||||
|
project_id=project_id,
|
||||||
|
query_semantics=query_semantics,
|
||||||
)
|
)
|
||||||
member_workload_direct_answer = await self._build_member_workload_direct_answer(
|
member_workload_direct_answer = await self._build_member_workload_direct_answer(
|
||||||
question,
|
question,
|
||||||
|
|
|
||||||
|
|
@ -167,7 +167,7 @@ class AdaptiveRAGDeepGuardRulesMixin:
|
||||||
continue
|
continue
|
||||||
source = str(row.get("source") or "").strip().lower()
|
source = str(row.get("source") or "").strip().lower()
|
||||||
metadata = row.get("metadata") if isinstance(row.get("metadata"), dict) else {}
|
metadata = row.get("metadata") if isinstance(row.get("metadata"), dict) else {}
|
||||||
if source == "url":
|
if source in {"url", "mcp_search"}:
|
||||||
return True
|
return True
|
||||||
for key in ("target_url", "file_url", "doc_url", "url"):
|
for key in ("target_url", "file_url", "doc_url", "url"):
|
||||||
value = str(metadata.get(key) or "").strip().lower()
|
value = str(metadata.get(key) or "").strip().lower()
|
||||||
|
|
|
||||||
|
|
@ -26,14 +26,33 @@ class AdaptiveRAGDeterministicQueryCoreMixin:
|
||||||
project_id=project_id,
|
project_id=project_id,
|
||||||
query_semantics=query_semantics,
|
query_semantics=query_semantics,
|
||||||
)
|
)
|
||||||
|
if q_type == QUERY_TYPE_EXISTENCE:
|
||||||
|
existence = await self._try_fast_project_existence_answer(
|
||||||
|
clean_q=question,
|
||||||
|
project_id=project_id,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
)
|
||||||
|
if existence:
|
||||||
|
existence["query_type"] = QUERY_TYPE_EXISTENCE
|
||||||
|
existence.setdefault("confidence", 0.96)
|
||||||
|
existence.setdefault("missing_fields", [])
|
||||||
|
return existence
|
||||||
if q_type == QUERY_TYPE_STATUS:
|
if q_type == QUERY_TYPE_STATUS:
|
||||||
created_at = await self._try_fast_project_created_at_answer(clean_q=question, project_id=project_id)
|
created_at = await self._try_fast_project_created_at_answer(
|
||||||
|
clean_q=question,
|
||||||
|
project_id=project_id,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
)
|
||||||
if created_at:
|
if created_at:
|
||||||
created_at["query_type"] = QUERY_TYPE_STATUS
|
created_at["query_type"] = QUERY_TYPE_STATUS
|
||||||
created_at.setdefault("confidence", 0.97)
|
created_at.setdefault("confidence", 0.97)
|
||||||
created_at.setdefault("missing_fields", [])
|
created_at.setdefault("missing_fields", [])
|
||||||
return created_at
|
return created_at
|
||||||
progress = await self._try_fast_project_progress_answer(clean_q=question, project_id=project_id)
|
progress = await self._try_fast_project_progress_answer(
|
||||||
|
clean_q=question,
|
||||||
|
project_id=project_id,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
)
|
||||||
if progress:
|
if progress:
|
||||||
progress["query_type"] = QUERY_TYPE_STATUS
|
progress["query_type"] = QUERY_TYPE_STATUS
|
||||||
progress.setdefault("confidence", 0.98)
|
progress.setdefault("confidence", 0.98)
|
||||||
|
|
@ -56,6 +75,7 @@ class AdaptiveRAGDeterministicQueryCoreMixin:
|
||||||
*,
|
*,
|
||||||
question: str,
|
question: str,
|
||||||
project_id: Optional[int],
|
project_id: Optional[int],
|
||||||
|
query_semantics: Optional[dict[str, Any]] = None,
|
||||||
) -> tuple[Optional[int], Optional[str], str]:
|
) -> tuple[Optional[int], Optional[str], str]:
|
||||||
if project_id is not None:
|
if project_id is not None:
|
||||||
row = (
|
row = (
|
||||||
|
|
@ -72,18 +92,50 @@ class AdaptiveRAGDeterministicQueryCoreMixin:
|
||||||
return (None, None, "")
|
return (None, None, "")
|
||||||
return (int(row.id), str(row.title or row.name or f"项目#{int(row.id)}"), "")
|
return (int(row.id), str(row.title or row.name or f"项目#{int(row.id)}"), "")
|
||||||
|
|
||||||
|
semantics = query_semantics if isinstance(query_semantics, dict) else {}
|
||||||
|
target_resolution = semantics.get("target_resolution") if isinstance(semantics.get("target_resolution"), dict) else {}
|
||||||
|
target_ref = str(target_resolution.get("target_ref") or "").strip()
|
||||||
|
if target_ref.startswith("project:"):
|
||||||
|
try:
|
||||||
|
resolved_project_id = int(target_ref.split(":", 1)[1])
|
||||||
|
except Exception:
|
||||||
|
resolved_project_id = None
|
||||||
|
if resolved_project_id is not None:
|
||||||
|
row = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(Project).where(
|
||||||
|
and_(
|
||||||
|
Project.tenant_id == self.tenant_id,
|
||||||
|
Project.id == resolved_project_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if row is not None:
|
||||||
|
return (
|
||||||
|
int(row.id),
|
||||||
|
str(row.title or row.name or f"项目#{int(row.id)}"),
|
||||||
|
str(target_resolution.get("resolved_label") or row.title or row.name or "")[:32],
|
||||||
|
)
|
||||||
|
|
||||||
direct_project_id, direct_project_name = await self._match_project_name_in_question(question=question)
|
direct_project_id, direct_project_name = await self._match_project_name_in_question(question=question)
|
||||||
if direct_project_id is not None and direct_project_name:
|
if direct_project_id is not None and direct_project_name:
|
||||||
return (int(direct_project_id), str(direct_project_name), str(direct_project_name)[:24])
|
return (int(direct_project_id), str(direct_project_name), str(direct_project_name)[:24])
|
||||||
|
|
||||||
keyword = self._extract_project_lookup_keyword(question)
|
term_collector = getattr(self, "_collect_lookup_terms", None)
|
||||||
terms = self._extract_query_terms(question, preferred=keyword)
|
lookup_terms = term_collector(
|
||||||
for term in terms:
|
question=question,
|
||||||
matched = await self._search_projects_by_keyword(keyword=term, project_id=None, limit=5)
|
query_semantics=semantics,
|
||||||
|
target_resolution=target_resolution,
|
||||||
|
limit=4,
|
||||||
|
) if callable(term_collector) else []
|
||||||
|
searcher = getattr(self, "_search_projects_by_lookup_terms", None)
|
||||||
|
if callable(searcher) and lookup_terms:
|
||||||
|
matched, used_term = await searcher(query_terms=lookup_terms, project_id=None, limit=5)
|
||||||
if matched:
|
if matched:
|
||||||
target = matched[0]
|
target = matched[0]
|
||||||
return (int(target.id), str(target.title or target.name or f"项目#{int(target.id)}"), term)
|
return (int(target.id), str(target.title or target.name or f"项目#{int(target.id)}"), used_term[:24])
|
||||||
return (None, None, keyword[:24] if keyword else "")
|
return (None, None, str(lookup_terms[0] if lookup_terms else "")[:24])
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _normalize_project_subject_text(text: str) -> str:
|
def _normalize_project_subject_text(text: str) -> str:
|
||||||
|
|
@ -92,11 +144,6 @@ class AdaptiveRAGDeterministicQueryCoreMixin:
|
||||||
return ""
|
return ""
|
||||||
clean = clean.replace("—", "-").replace("_", " ")
|
clean = clean.replace("—", "-").replace("_", " ")
|
||||||
clean = re.sub(r"[^\w\u4e00-\u9fff\- ]+", " ", clean)
|
clean = re.sub(r"[^\w\u4e00-\u9fff\- ]+", " ", clean)
|
||||||
clean = re.sub(
|
|
||||||
r"(我说的是|我说的|请问|请|帮我|一下|这个里面|这里面|里面|这个|那个|一下子|目前|当前|现在|有多少个|多少个|信号源|信号|项目|系统)",
|
|
||||||
" ",
|
|
||||||
clean,
|
|
||||||
)
|
|
||||||
clean = re.sub(r"[-]+", " ", clean)
|
clean = re.sub(r"[-]+", " ", clean)
|
||||||
clean = re.sub(r"\s+", " ", clean).strip()
|
clean = re.sub(r"\s+", " ", clean).strip()
|
||||||
return clean
|
return clean
|
||||||
|
|
|
||||||
|
|
@ -16,6 +16,7 @@ class AdaptiveRAGFastAnswerProjectMainMixin:
|
||||||
*,
|
*,
|
||||||
clean_q: str,
|
clean_q: str,
|
||||||
project_id: Optional[int],
|
project_id: Optional[int],
|
||||||
|
query_semantics: Optional[dict[str, Any]] = None,
|
||||||
) -> Optional[dict[str, Any]]:
|
) -> Optional[dict[str, Any]]:
|
||||||
question = str(clean_q or "").strip()
|
question = str(clean_q or "").strip()
|
||||||
if not question:
|
if not question:
|
||||||
|
|
@ -26,24 +27,16 @@ class AdaptiveRAGFastAnswerProjectMainMixin:
|
||||||
if "项目" not in question and "系统" not in question:
|
if "项目" not in question and "系统" not in question:
|
||||||
return None
|
return None
|
||||||
|
|
||||||
primary_keyword = self._extract_project_lookup_keyword(question)
|
|
||||||
query_terms = self._extract_query_terms(question, preferred=primary_keyword)
|
|
||||||
if not query_terms:
|
|
||||||
return None
|
|
||||||
|
|
||||||
started = time.perf_counter()
|
started = time.perf_counter()
|
||||||
matched: list[Project] = []
|
target_project_id, target_name, used_keyword = await self._resolve_project_scope_for_question(
|
||||||
used_keyword = query_terms[0]
|
question=question,
|
||||||
for term in query_terms:
|
project_id=project_id,
|
||||||
rows = await self._search_projects_by_keyword(keyword=term, project_id=project_id, limit=3)
|
query_semantics=query_semantics,
|
||||||
if rows:
|
)
|
||||||
matched = rows
|
if target_project_id is None:
|
||||||
used_keyword = term
|
|
||||||
break
|
|
||||||
if not matched:
|
|
||||||
db_query_ms = int((time.perf_counter() - started) * 1000)
|
db_query_ms = int((time.perf_counter() - started) * 1000)
|
||||||
return {
|
return {
|
||||||
"answer": f"未检索到名称或描述包含“{used_keyword}”的项目,无法判断进度。",
|
"answer": f"未检索到可确认进度的项目,无法判断“{question}”对应对象。",
|
||||||
"sources": [
|
"sources": [
|
||||||
{
|
{
|
||||||
"source": "db",
|
"source": "db",
|
||||||
|
|
@ -56,7 +49,36 @@ class AdaptiveRAGFastAnswerProjectMainMixin:
|
||||||
"response_time_ms": db_query_ms,
|
"response_time_ms": db_query_ms,
|
||||||
}
|
}
|
||||||
|
|
||||||
target = matched[0]
|
target = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(Project).where(
|
||||||
|
and_(
|
||||||
|
Project.tenant_id == self.tenant_id,
|
||||||
|
Project.id == int(target_project_id),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if target is None:
|
||||||
|
db_query_ms = int((time.perf_counter() - started) * 1000)
|
||||||
|
return {
|
||||||
|
"answer": f"未检索到项目#{int(target_project_id)},无法判断进度。",
|
||||||
|
"sources": [
|
||||||
|
{
|
||||||
|
"source": "db",
|
||||||
|
"label": "Project database",
|
||||||
|
"score": 0.99,
|
||||||
|
"metadata": {
|
||||||
|
"table": "projects",
|
||||||
|
"project_id": int(target_project_id),
|
||||||
|
"query_keyword": used_keyword,
|
||||||
|
"match_count": 0,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"db_query_ms": db_query_ms,
|
||||||
|
"response_time_ms": db_query_ms,
|
||||||
|
}
|
||||||
task_rows = (
|
task_rows = (
|
||||||
await self.db.execute(
|
await self.db.execute(
|
||||||
select(Task).where(
|
select(Task).where(
|
||||||
|
|
@ -124,10 +146,6 @@ class AdaptiveRAGFastAnswerProjectMainMixin:
|
||||||
if asks_risk_or_blockers and latest_update is not None:
|
if asks_risk_or_blockers and latest_update is not None:
|
||||||
answer += f" 最近任务更新时间:{latest_update.strftime('%Y-%m-%d %H:%M')}。"
|
answer += f" 最近任务更新时间:{latest_update.strftime('%Y-%m-%d %H:%M')}。"
|
||||||
|
|
||||||
if len(matched) > 1:
|
|
||||||
others = "、".join([str(x.title or x.name or f'#{x.id}') for x in matched[1:]])
|
|
||||||
answer += f" 另有 {len(matched) - 1} 个同名候选项目:{others}。"
|
|
||||||
|
|
||||||
return {
|
return {
|
||||||
"answer": answer,
|
"answer": answer,
|
||||||
"sources": [
|
"sources": [
|
||||||
|
|
@ -165,6 +183,7 @@ class AdaptiveRAGFastAnswerProjectMainMixin:
|
||||||
*,
|
*,
|
||||||
clean_q: str,
|
clean_q: str,
|
||||||
project_id: Optional[int],
|
project_id: Optional[int],
|
||||||
|
query_semantics: Optional[dict[str, Any]] = None,
|
||||||
) -> Optional[dict[str, Any]]:
|
) -> Optional[dict[str, Any]]:
|
||||||
question = str(clean_q or "").strip()
|
question = str(clean_q or "").strip()
|
||||||
if not question or not self.PROJECT_CREATED_AT_RE.search(question):
|
if not question or not self.PROJECT_CREATED_AT_RE.search(question):
|
||||||
|
|
@ -172,22 +191,13 @@ class AdaptiveRAGFastAnswerProjectMainMixin:
|
||||||
if "项目" not in question and "系统" not in question:
|
if "项目" not in question and "系统" not in question:
|
||||||
return None
|
return None
|
||||||
|
|
||||||
primary_keyword = self._extract_project_lookup_keyword(question)
|
|
||||||
query_terms = self._extract_query_terms(question, preferred=primary_keyword)
|
|
||||||
started = time.perf_counter()
|
started = time.perf_counter()
|
||||||
|
|
||||||
target_project_id, target_name, used_keyword = await self._resolve_project_scope_for_question(
|
target_project_id, target_name, used_keyword = await self._resolve_project_scope_for_question(
|
||||||
question=question,
|
question=question,
|
||||||
project_id=project_id,
|
project_id=project_id,
|
||||||
|
query_semantics=query_semantics,
|
||||||
)
|
)
|
||||||
if target_project_id is None and query_terms:
|
|
||||||
for term in query_terms:
|
|
||||||
rows = await self._search_projects_by_keyword(keyword=term, project_id=project_id, limit=3)
|
|
||||||
if rows:
|
|
||||||
target_project_id = int(rows[0].id)
|
|
||||||
target_name = str(rows[0].title or rows[0].name or f"项目#{int(rows[0].id)}")
|
|
||||||
used_keyword = term
|
|
||||||
break
|
|
||||||
|
|
||||||
if target_project_id is None:
|
if target_project_id is None:
|
||||||
db_query_ms = int((time.perf_counter() - started) * 1000)
|
db_query_ms = int((time.perf_counter() - started) * 1000)
|
||||||
|
|
@ -284,6 +294,46 @@ class AdaptiveRAGFastAnswerProjectMainMixin:
|
||||||
subject_focus = self._query_subject_focus(semantics)
|
subject_focus = self._query_subject_focus(semantics)
|
||||||
scope_hint = self._query_scope_hint(semantics)
|
scope_hint = self._query_scope_hint(semantics)
|
||||||
message_intent = str(semantics.get("message_intent") or semantics.get("intent") or "").strip().lower()
|
message_intent = str(semantics.get("message_intent") or semantics.get("intent") or "").strip().lower()
|
||||||
|
target_resolution = semantics.get("target_resolution") if isinstance(semantics.get("target_resolution"), dict) else {}
|
||||||
|
target_entity_type = str(target_resolution.get("target_entity_type") or "").strip().lower()
|
||||||
|
target_ref = str(target_resolution.get("target_ref") or "").strip()
|
||||||
|
if target_entity_type == "project" and target_ref.startswith("project:"):
|
||||||
|
try:
|
||||||
|
resolved_project_id = int(target_ref.split(":", 1)[1])
|
||||||
|
except Exception:
|
||||||
|
resolved_project_id = None
|
||||||
|
if resolved_project_id is not None:
|
||||||
|
matched_project = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(Project).where(
|
||||||
|
and_(
|
||||||
|
Project.tenant_id == self.tenant_id,
|
||||||
|
Project.id == resolved_project_id,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if matched_project is not None:
|
||||||
|
title = str(matched_project.title or matched_project.name or f"项目#{int(matched_project.id)}")
|
||||||
|
status = str(matched_project.status or "unknown")
|
||||||
|
answer = f"有,当前匹配到 1 个相关项目:\n#{int(matched_project.id)} {title}(状态:{status})"
|
||||||
|
return {
|
||||||
|
"answer": answer,
|
||||||
|
"sources": [
|
||||||
|
{
|
||||||
|
"source": "db",
|
||||||
|
"label": "Project database",
|
||||||
|
"score": 0.99,
|
||||||
|
"metadata": {
|
||||||
|
"table": "projects",
|
||||||
|
"query_keyword": str(target_resolution.get("resolved_label") or title),
|
||||||
|
"match_count": 1,
|
||||||
|
"project_id": int(matched_project.id),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"db_query_ms": 0,
|
||||||
|
}
|
||||||
if (
|
if (
|
||||||
project_id is None
|
project_id is None
|
||||||
and message_intent != "project_lookup_question"
|
and message_intent != "project_lookup_question"
|
||||||
|
|
@ -292,20 +342,38 @@ class AdaptiveRAGFastAnswerProjectMainMixin:
|
||||||
):
|
):
|
||||||
return None
|
return None
|
||||||
|
|
||||||
primary_keyword = self._extract_project_lookup_keyword(question)
|
query_terms = self._collect_lookup_terms(
|
||||||
query_terms = self._extract_query_terms(question, preferred=primary_keyword)
|
question=question,
|
||||||
|
query_semantics=semantics,
|
||||||
|
operation_contract=semantics.get("operation_contract") if isinstance(semantics.get("operation_contract"), dict) else None,
|
||||||
|
target_resolution=target_resolution,
|
||||||
|
limit=4,
|
||||||
|
)
|
||||||
|
if not query_terms:
|
||||||
|
matched_project_id, matched_name = await self._match_project_name_in_question(question=question)
|
||||||
|
if matched_project_id is not None and matched_name:
|
||||||
|
query_terms = [str(matched_name)]
|
||||||
if not query_terms:
|
if not query_terms:
|
||||||
return None
|
return None
|
||||||
|
|
||||||
started = time.perf_counter()
|
started = time.perf_counter()
|
||||||
|
semantic_match = await self._try_fast_project_existence_answer_from_semantic_facts(
|
||||||
|
query_terms=query_terms,
|
||||||
|
project_id=project_id,
|
||||||
|
)
|
||||||
|
if semantic_match is not None:
|
||||||
|
semantic_match["db_query_ms"] = int((time.perf_counter() - started) * 1000)
|
||||||
|
semantic_match.setdefault("response_time_ms", semantic_match["db_query_ms"])
|
||||||
|
return semantic_match
|
||||||
|
|
||||||
matched: list[Project] = []
|
matched: list[Project] = []
|
||||||
used_keyword = query_terms[0]
|
used_keyword = query_terms[0]
|
||||||
for term in query_terms:
|
ranked_rows, used_keyword = await self._search_projects_by_lookup_terms(
|
||||||
rows = await self._search_projects_by_keyword(keyword=term, project_id=project_id, limit=5)
|
query_terms=query_terms,
|
||||||
if rows:
|
project_id=project_id,
|
||||||
matched = rows
|
limit=5,
|
||||||
used_keyword = term
|
)
|
||||||
break
|
matched = list(ranked_rows or [])
|
||||||
db_query_ms = int((time.perf_counter() - started) * 1000)
|
db_query_ms = int((time.perf_counter() - started) * 1000)
|
||||||
|
|
||||||
if matched:
|
if matched:
|
||||||
|
|
@ -334,6 +402,71 @@ class AdaptiveRAGFastAnswerProjectMainMixin:
|
||||||
"db_query_ms": db_query_ms,
|
"db_query_ms": db_query_ms,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async def _try_fast_project_existence_answer_from_semantic_facts(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
query_terms: list[str],
|
||||||
|
project_id: Optional[int],
|
||||||
|
) -> Optional[dict[str, Any]]:
|
||||||
|
fact_query = getattr(self, "_query_project_semantic_facts_by_lookup_terms", None)
|
||||||
|
if not callable(fact_query):
|
||||||
|
return None
|
||||||
|
matched_facts, used_keyword = await fact_query(
|
||||||
|
query_terms=query_terms,
|
||||||
|
project_id=project_id,
|
||||||
|
limit=10,
|
||||||
|
)
|
||||||
|
|
||||||
|
if not matched_facts:
|
||||||
|
return None
|
||||||
|
|
||||||
|
matched_project_ids = []
|
||||||
|
for row in matched_facts:
|
||||||
|
try:
|
||||||
|
entity_id = int(row.entity_id)
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
if entity_id not in matched_project_ids:
|
||||||
|
matched_project_ids.append(entity_id)
|
||||||
|
if not matched_project_ids:
|
||||||
|
return None
|
||||||
|
|
||||||
|
project_rows = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(Project).where(
|
||||||
|
and_(
|
||||||
|
Project.tenant_id == self.tenant_id,
|
||||||
|
Project.id.in_(matched_project_ids),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
if not project_rows:
|
||||||
|
return None
|
||||||
|
|
||||||
|
lines = [
|
||||||
|
f"#{int(row.id)} {str(row.title or row.name or '未命名项目')}(状态:{str(row.status or 'unknown')})"
|
||||||
|
for row in project_rows
|
||||||
|
]
|
||||||
|
answer = f"有,当前匹配到 {len(project_rows)} 个相关项目:\n" + "\n".join(lines)
|
||||||
|
return {
|
||||||
|
"answer": answer,
|
||||||
|
"sources": [
|
||||||
|
{
|
||||||
|
"source": "db",
|
||||||
|
"label": "Semantic facts",
|
||||||
|
"score": 0.99,
|
||||||
|
"metadata": {
|
||||||
|
"table": "semantic_facts",
|
||||||
|
"query_keyword": used_keyword,
|
||||||
|
"match_count": len(project_rows),
|
||||||
|
"matched_project_ids": matched_project_ids,
|
||||||
|
"fact_keys": sorted({str(row.fact_key or "") for row in matched_facts if str(row.fact_key or "").strip()}),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
],
|
||||||
|
}
|
||||||
|
|
||||||
async def _search_projects_by_keyword(
|
async def _search_projects_by_keyword(
|
||||||
self,
|
self,
|
||||||
*,
|
*,
|
||||||
|
|
@ -356,3 +489,67 @@ class AdaptiveRAGFastAnswerProjectMainMixin:
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
return (await self.db.execute(stmt.limit(max(1, int(limit))))).scalars().all()
|
return (await self.db.execute(stmt.limit(max(1, int(limit))))).scalars().all()
|
||||||
|
|
||||||
|
async def _search_projects_by_lookup_terms(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
query_terms: list[str],
|
||||||
|
project_id: Optional[int],
|
||||||
|
limit: int,
|
||||||
|
) -> tuple[list[Project], str]:
|
||||||
|
terms = self._dedupe_lookup_terms([str(item or "").strip() for item in query_terms], limit=6)
|
||||||
|
if not terms:
|
||||||
|
return ([], "")
|
||||||
|
|
||||||
|
for term in terms:
|
||||||
|
rows = await self._search_projects_by_keyword(keyword=term, project_id=project_id, limit=limit)
|
||||||
|
if rows:
|
||||||
|
return (rows, term)
|
||||||
|
|
||||||
|
stmt = select(Project).where(Project.tenant_id == self.tenant_id)
|
||||||
|
if project_id is not None:
|
||||||
|
stmt = stmt.where(Project.id == int(project_id))
|
||||||
|
candidates = (
|
||||||
|
await self.db.execute(
|
||||||
|
stmt.order_by(Project.updated_at.desc(), Project.id.desc()).limit(max(40, int(limit) * 20))
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
if not candidates:
|
||||||
|
return ([], terms[0])
|
||||||
|
|
||||||
|
ranked: list[tuple[float, Project, str]] = []
|
||||||
|
for row in candidates:
|
||||||
|
title = str(row.title or "").strip()
|
||||||
|
name = str(row.name or "").strip()
|
||||||
|
description = str(row.description or "").strip()
|
||||||
|
best_term = ""
|
||||||
|
best_score = 0.0
|
||||||
|
for term in terms:
|
||||||
|
score = max(
|
||||||
|
self._lookup_similarity_score(term, title),
|
||||||
|
self._lookup_similarity_score(term, name),
|
||||||
|
self._lookup_similarity_score(term, description) * 0.82,
|
||||||
|
)
|
||||||
|
if score > best_score:
|
||||||
|
best_score = score
|
||||||
|
best_term = term
|
||||||
|
if best_score >= 0.62 and best_term:
|
||||||
|
ranked.append((best_score, row, best_term))
|
||||||
|
if not ranked:
|
||||||
|
return ([], terms[0])
|
||||||
|
|
||||||
|
ranked.sort(key=lambda item: item[0], reverse=True)
|
||||||
|
deduped: list[Project] = []
|
||||||
|
used_keyword = ranked[0][2]
|
||||||
|
seen_ids: set[int] = set()
|
||||||
|
for _score, row, term in ranked:
|
||||||
|
row_id = int(row.id)
|
||||||
|
if row_id in seen_ids:
|
||||||
|
continue
|
||||||
|
seen_ids.add(row_id)
|
||||||
|
deduped.append(row)
|
||||||
|
if not used_keyword:
|
||||||
|
used_keyword = term
|
||||||
|
if len(deduped) >= max(1, int(limit)):
|
||||||
|
break
|
||||||
|
return (deduped, used_keyword)
|
||||||
|
|
|
||||||
|
|
@ -8,83 +8,188 @@ class AdaptiveRAGFastAnswerProjectUtilsMixin:
|
||||||
_SEMANTIC_SUBJECT_FOCI = {"people", "project", "task", "signal", "charter", "workload", "feature", "unknown"}
|
_SEMANTIC_SUBJECT_FOCI = {"people", "project", "task", "signal", "charter", "workload", "feature", "unknown"}
|
||||||
_ALLOWED_SCOPE_VALUES = {"none", "global", "project"}
|
_ALLOWED_SCOPE_VALUES = {"none", "global", "project"}
|
||||||
_SEMANTIC_FEATURE_FOCI = {"none", "purpose", "entry", "catalog"}
|
_SEMANTIC_FEATURE_FOCI = {"none", "purpose", "entry", "catalog"}
|
||||||
|
_LOOKUP_TEXT_TRANSLATION = str.maketrans(
|
||||||
|
{
|
||||||
|
",": " ",
|
||||||
|
"。": " ",
|
||||||
|
"!": " ",
|
||||||
|
"?": " ",
|
||||||
|
":": " ",
|
||||||
|
";": " ",
|
||||||
|
"(": " ",
|
||||||
|
")": " ",
|
||||||
|
"(": " ",
|
||||||
|
")": " ",
|
||||||
|
"\"": " ",
|
||||||
|
"'": " ",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _extract_project_lookup_keyword(question: str) -> str:
|
def _normalize_lookup_text(text: str) -> str:
|
||||||
text = str(question or "").strip()
|
normalized = str(text or "").strip().lower()
|
||||||
if not text:
|
if not normalized:
|
||||||
return ""
|
return ""
|
||||||
text = text.replace("?", "?")
|
normalized = normalized.translate(AdaptiveRAGFastAnswerProjectUtilsMixin._LOOKUP_TEXT_TRANSLATION)
|
||||||
|
normalized = normalized.replace("—", "-").replace("_", " ")
|
||||||
|
normalized = re.sub(r"[^\w\u4e00-\u9fff\- ]+", " ", normalized)
|
||||||
|
normalized = re.sub(r"\s+", " ", normalized).strip()
|
||||||
|
return normalized[:96]
|
||||||
|
|
||||||
direct_patterns = [
|
@classmethod
|
||||||
r"(?:是否有|有没有|有无)([\u4e00-\u9fffA-Za-z0-9_-]{1,24})(?:项目|系统)?",
|
def _collect_structured_object_mentions(
|
||||||
r"(?:是否存在|存在)([\u4e00-\u9fffA-Za-z0-9_-]{1,24})(?:项目|系统)?",
|
cls,
|
||||||
]
|
*,
|
||||||
for pattern in direct_patterns:
|
query_semantics: Optional[dict[str, Any]] = None,
|
||||||
m = re.search(pattern, text)
|
operation_contract: Optional[dict[str, Any]] = None,
|
||||||
if m:
|
target_resolution: Optional[dict[str, Any]] = None,
|
||||||
value = str(m.group(1) or "").strip(" ,。,.!??::;;“”\"'()()")
|
limit: int = 6,
|
||||||
value = QUESTION_NOISE_RE.sub("", value).strip()
|
) -> list[str]:
|
||||||
if len(value) >= 2:
|
semantics = query_semantics if isinstance(query_semantics, dict) else {}
|
||||||
return value[:24]
|
operation = operation_contract if isinstance(operation_contract, dict) else (
|
||||||
|
semantics.get("operation_contract") if isinstance(semantics.get("operation_contract"), dict) else {}
|
||||||
anchored_matches = list(re.finditer(r"([\u4e00-\u9fffA-Za-z0-9_-]{1,24})(?:项目|系统)", text))
|
|
||||||
for m in reversed(anchored_matches):
|
|
||||||
value = str(m.group(1) or "").strip(" ,。,.!??::;;“”\"'()()")
|
|
||||||
value = QUESTION_NOISE_RE.sub("", value).strip()
|
|
||||||
if len(value) >= 2:
|
|
||||||
return value[:24]
|
|
||||||
|
|
||||||
patterns = [
|
|
||||||
r"有(?:没有)?(.{1,24}?)(?:项目|系统).*\?",
|
|
||||||
r"项目.*?有(?:没有)?(.{1,24}?)(?:吗|\?)",
|
|
||||||
r"是否有(.{1,24}?)(?:项目|系统)",
|
|
||||||
r"(.{1,24}?)(?:项目|系统)(?:有吗|有没有|是否有)",
|
|
||||||
]
|
|
||||||
for pattern in patterns:
|
|
||||||
m = re.search(pattern, text)
|
|
||||||
if not m:
|
|
||||||
continue
|
|
||||||
kw = str(m.group(1) or "").strip(" ,。,.!??::;;“”\"'()()")
|
|
||||||
kw = QUESTION_NOISE_RE.sub("", kw).strip()
|
|
||||||
if kw:
|
|
||||||
return kw[:24]
|
|
||||||
|
|
||||||
fallback = re.sub(r"[??]", " ", text)
|
|
||||||
fallback = re.sub(
|
|
||||||
r"(项目|系统|目前|当前|现在|我们|有没有|是否有|有吗|有|吗|什么|进度|层面|阶段|是在|在|是|请问|一下|多少)",
|
|
||||||
" ",
|
|
||||||
fallback,
|
|
||||||
)
|
)
|
||||||
fallback = re.sub(r"\s+", " ", fallback).strip(" ,。,.!??::;;")
|
target = target_resolution if isinstance(target_resolution, dict) else (
|
||||||
return fallback[:24]
|
semantics.get("target_resolution") if isinstance(semantics.get("target_resolution"), dict) else {}
|
||||||
|
)
|
||||||
|
raw_rows: list[Any] = []
|
||||||
|
raw_rows.extend(list(operation.get("object_mentions") or []))
|
||||||
|
raw_rows.extend(list(semantics.get("object_mentions") or []))
|
||||||
|
resolved_label = str(target.get("resolved_label") or "").strip()
|
||||||
|
if resolved_label:
|
||||||
|
raw_rows.append(resolved_label)
|
||||||
|
|
||||||
@staticmethod
|
mentions: list[str] = []
|
||||||
def _extract_query_terms(question: str, *, preferred: str = "") -> list[str]:
|
for item in raw_rows:
|
||||||
base = str(question or "").strip()
|
text = str(item or "").strip(" ,。,.!??::;;“”\"'()()")
|
||||||
|
if len(text) < 2 or text in mentions:
|
||||||
|
continue
|
||||||
|
mentions.append(text[:32])
|
||||||
|
if len(mentions) >= max(1, int(limit)):
|
||||||
|
break
|
||||||
|
return mentions
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _lookup_similarity_score(cls, left: str, right: str) -> float:
|
||||||
|
lhs = cls._normalize_lookup_text(left)
|
||||||
|
rhs = cls._normalize_lookup_text(right)
|
||||||
|
if not lhs or not rhs:
|
||||||
|
return 0.0
|
||||||
|
if lhs == rhs:
|
||||||
|
return 1.0
|
||||||
|
if lhs in rhs:
|
||||||
|
return min(0.98, 0.82 + min(0.16, len(lhs) / max(len(rhs), 1) * 0.16))
|
||||||
|
if rhs in lhs:
|
||||||
|
return min(0.94, 0.78 + min(0.16, len(rhs) / max(len(lhs), 1) * 0.16))
|
||||||
|
return float(SequenceMatcher(None, lhs, rhs).ratio())
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _dedupe_lookup_terms(cls, rows: list[str], *, limit: int = 6) -> list[str]:
|
||||||
terms: list[str] = []
|
terms: list[str] = []
|
||||||
pref = str(preferred or "").strip()
|
for raw in rows:
|
||||||
if len(pref) >= 2:
|
text = str(raw or "").strip()
|
||||||
terms.append(pref[:24])
|
if len(text) < 2 or text in terms:
|
||||||
|
continue
|
||||||
|
terms.append(text[:32])
|
||||||
|
if len(terms) >= max(1, int(limit)):
|
||||||
|
break
|
||||||
|
return terms
|
||||||
|
|
||||||
clean = re.sub(r"[??,,。!!::;;()()\"“”']", " ", base)
|
@classmethod
|
||||||
clean = QUESTION_NOISE_RE.sub(" ", clean)
|
def _collect_lookup_terms(
|
||||||
clean = re.sub(r"(项目|系统|进度|状态|阶段|完成率|层面)", " ", clean)
|
cls,
|
||||||
clean = re.sub(r"\s+", " ", clean).strip()
|
*,
|
||||||
|
question: str = "",
|
||||||
|
query_semantics: Optional[dict[str, Any]] = None,
|
||||||
|
operation_contract: Optional[dict[str, Any]] = None,
|
||||||
|
target_resolution: Optional[dict[str, Any]] = None,
|
||||||
|
limit: int = 6,
|
||||||
|
) -> list[str]:
|
||||||
|
mentions = cls._collect_structured_object_mentions(
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
operation_contract=operation_contract,
|
||||||
|
target_resolution=target_resolution,
|
||||||
|
limit=limit,
|
||||||
|
)
|
||||||
|
return mentions
|
||||||
|
|
||||||
token_candidates: list[str] = []
|
async def _query_project_semantic_facts_by_lookup_terms(
|
||||||
for token in clean.split(" "):
|
self,
|
||||||
t = token.strip()
|
*,
|
||||||
if len(t) >= 2:
|
query_terms: list[str],
|
||||||
token_candidates.append(t[:24])
|
project_id: Optional[int],
|
||||||
|
limit: int = 20,
|
||||||
|
) -> tuple[list[SemanticFact], str]:
|
||||||
|
return await self._query_semantic_facts_by_lookup_terms(
|
||||||
|
entity_type="project",
|
||||||
|
query_terms=query_terms,
|
||||||
|
entity_id=project_id,
|
||||||
|
limit=limit,
|
||||||
|
)
|
||||||
|
|
||||||
for m in re.finditer(r"[\u4e00-\u9fff]{2,10}", clean):
|
async def _query_semantic_facts_by_lookup_terms(
|
||||||
token_candidates.append(m.group(0)[:24])
|
self,
|
||||||
|
*,
|
||||||
|
entity_type: str,
|
||||||
|
query_terms: list[str],
|
||||||
|
entity_id: Optional[int],
|
||||||
|
limit: int = 20,
|
||||||
|
) -> tuple[list[SemanticFact], str]:
|
||||||
|
terms = self._dedupe_lookup_terms([str(item or "").strip() for item in query_terms], limit=6)
|
||||||
|
if not terms:
|
||||||
|
return ([], "")
|
||||||
|
|
||||||
for t in sorted(token_candidates, key=len, reverse=True):
|
stmt = select(SemanticFact).where(
|
||||||
if t and t not in terms:
|
and_(
|
||||||
terms.append(t)
|
SemanticFact.tenant_id == self.tenant_id,
|
||||||
return terms[:6]
|
SemanticFact.entity_type == str(entity_type or "").strip(),
|
||||||
|
SemanticFact.is_active.is_(True),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if entity_id is not None:
|
||||||
|
stmt = stmt.where(SemanticFact.entity_id == int(entity_id))
|
||||||
|
fact_rows = (
|
||||||
|
await self.db.execute(
|
||||||
|
stmt.order_by(SemanticFact.confidence.desc(), SemanticFact.updated_at.desc(), SemanticFact.id.desc()).limit(
|
||||||
|
max(40, int(limit) * 8)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
if not fact_rows:
|
||||||
|
return ([], terms[0])
|
||||||
|
|
||||||
|
ranked: list[tuple[float, SemanticFact, str]] = []
|
||||||
|
for row in fact_rows:
|
||||||
|
fact_text = str(row.fact_value_text or "").strip()
|
||||||
|
if not fact_text:
|
||||||
|
continue
|
||||||
|
best_term = ""
|
||||||
|
best_score = 0.0
|
||||||
|
for term in terms:
|
||||||
|
score = self._lookup_similarity_score(term, fact_text)
|
||||||
|
if score > best_score:
|
||||||
|
best_score = score
|
||||||
|
best_term = term
|
||||||
|
if best_score >= 0.74 and best_term:
|
||||||
|
ranked.append((best_score, row, best_term))
|
||||||
|
if not ranked:
|
||||||
|
return ([], terms[0])
|
||||||
|
|
||||||
|
ranked.sort(key=lambda item: item[0], reverse=True)
|
||||||
|
deduped: list[SemanticFact] = []
|
||||||
|
used_keyword = ranked[0][2]
|
||||||
|
seen_row_ids: set[int] = set()
|
||||||
|
for _score, row, term in ranked:
|
||||||
|
row_id = int(row.id)
|
||||||
|
if row_id in seen_row_ids:
|
||||||
|
continue
|
||||||
|
seen_row_ids.add(row_id)
|
||||||
|
deduped.append(row)
|
||||||
|
if not used_keyword:
|
||||||
|
used_keyword = term
|
||||||
|
if len(deduped) >= max(1, int(limit)):
|
||||||
|
break
|
||||||
|
return (deduped, used_keyword)
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def _query_subject_focus(cls, query_semantics: Optional[dict[str, Any]]) -> str:
|
def _query_subject_focus(cls, query_semantics: Optional[dict[str, Any]]) -> str:
|
||||||
|
|
|
||||||
|
|
@ -23,10 +23,19 @@ class AdaptiveRAGFastAnswerSemanticMixinP2:
|
||||||
rewrite_meta: Optional[dict[str, Any]] = None,
|
rewrite_meta: Optional[dict[str, Any]] = None,
|
||||||
thinking_mode: str = THINKING_MODE_NORMAL,
|
thinking_mode: str = THINKING_MODE_NORMAL,
|
||||||
answer_mode: str = ANSWER_MODE_FAST_PATH,
|
answer_mode: str = ANSWER_MODE_FAST_PATH,
|
||||||
|
route_decision: Optional[dict[str, Any]] = None,
|
||||||
|
query_semantics: Optional[dict[str, Any]] = None,
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
started = time.perf_counter()
|
started = time.perf_counter()
|
||||||
answer_text = str(fast_result.get("answer") or "未命中结果")
|
|
||||||
sources = list(fast_result.get("sources") or [])
|
sources = list(fast_result.get("sources") or [])
|
||||||
|
render_with_contract = getattr(self, "_render_answer_with_route_contract", None)
|
||||||
|
answer_text = str(fast_result.get("answer") or "未命中结果")
|
||||||
|
if callable(render_with_contract):
|
||||||
|
answer_text = render_with_contract(
|
||||||
|
answer=answer_text,
|
||||||
|
sources=sources,
|
||||||
|
route_decision=route_decision,
|
||||||
|
)
|
||||||
active_session_id = session_id
|
active_session_id = session_id
|
||||||
active_turn_index = turn_index
|
active_turn_index = turn_index
|
||||||
if active_session_id is None or active_turn_index is None:
|
if active_session_id is None or active_turn_index is None:
|
||||||
|
|
@ -44,22 +53,50 @@ class AdaptiveRAGFastAnswerSemanticMixinP2:
|
||||||
missing_fields = list(fast_result.get("missing_fields") or [])
|
missing_fields = list(fast_result.get("missing_fields") or [])
|
||||||
final_query_type = str(fast_result.get("query_type") or query_type or QUERY_TYPE_EXPLAIN)
|
final_query_type = str(fast_result.get("query_type") or query_type or QUERY_TYPE_EXPLAIN)
|
||||||
prompt_variant_id = str(fast_result.get("prompt_variant_id") or "deterministic").strip() or "deterministic"
|
prompt_variant_id = str(fast_result.get("prompt_variant_id") or "deterministic").strip() or "deterministic"
|
||||||
|
contract_verdict = self._verify_internal_answer_contract(
|
||||||
|
query_type=final_query_type,
|
||||||
|
answer=answer_text,
|
||||||
|
sources=sources,
|
||||||
|
route_decision=route_decision,
|
||||||
|
)
|
||||||
|
effective_answer_mode = answer_mode
|
||||||
|
effective_grounding_status = GROUNDING_STATUS_GROUNDED
|
||||||
|
effective_reason_code = str(fast_result.get("reason_code") or "").strip() or None
|
||||||
|
if not bool(contract_verdict.get("ok")):
|
||||||
|
guard_payload = self._build_internal_answer_contract_guard_payload(
|
||||||
|
query_type=final_query_type,
|
||||||
|
reason_code=str(contract_verdict.get("reason_code") or ""),
|
||||||
|
missing_fields=list(contract_verdict.get("missing_fields") or []),
|
||||||
|
)
|
||||||
|
answer_text = str(guard_payload.get("answer") or answer_text)
|
||||||
|
sources = list(guard_payload.get("sources") or [])
|
||||||
|
missing_fields = list(guard_payload.get("missing_fields") or [])
|
||||||
|
effective_answer_mode = str(guard_payload.get("answer_mode") or ANSWER_MODE_BLOCKED)
|
||||||
|
effective_grounding_status = str(guard_payload.get("grounding_status") or GROUNDING_STATUS_BLOCKED)
|
||||||
|
effective_reason_code = str(guard_payload.get("reason_code") or "").strip() or "internal_answer_contract_violation"
|
||||||
|
retrieval_metrics = {
|
||||||
|
"db_query_ms": int(fast_result.get("db_query_ms") or 0),
|
||||||
|
"vector_search_ms": 0,
|
||||||
|
"graph_query_ms": 0,
|
||||||
|
"db_timeout": False,
|
||||||
|
"vector_timeout": False,
|
||||||
|
"graph_timeout": False,
|
||||||
|
}
|
||||||
|
inject_runtime_summaries = getattr(self, "_inject_internal_query_runtime_summaries", None)
|
||||||
|
if callable(inject_runtime_summaries):
|
||||||
|
retrieval_metrics = inject_runtime_summaries(
|
||||||
|
retrieval_metrics=retrieval_metrics,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
)
|
||||||
runtime_fields = self._build_qa_record_runtime_fields(
|
runtime_fields = self._build_qa_record_runtime_fields(
|
||||||
answer=answer_text,
|
answer=answer_text,
|
||||||
sources=sources,
|
sources=sources,
|
||||||
missing_fields=missing_fields,
|
missing_fields=missing_fields,
|
||||||
answer_mode=answer_mode,
|
answer_mode=effective_answer_mode,
|
||||||
grounding_status=GROUNDING_STATUS_GROUNDED,
|
grounding_status=effective_grounding_status,
|
||||||
prompt_variant_id=prompt_variant_id,
|
prompt_variant_id=prompt_variant_id,
|
||||||
reason_code=str(fast_result.get("reason_code") or "").strip() or None,
|
reason_code=effective_reason_code,
|
||||||
retrieval_metrics={
|
retrieval_metrics=retrieval_metrics,
|
||||||
"db_query_ms": int(fast_result.get("db_query_ms") or 0),
|
|
||||||
"vector_search_ms": 0,
|
|
||||||
"graph_query_ms": 0,
|
|
||||||
"db_timeout": False,
|
|
||||||
"vector_timeout": False,
|
|
||||||
"graph_timeout": False,
|
|
||||||
},
|
|
||||||
)
|
)
|
||||||
|
|
||||||
qa = QARecord(
|
qa = QARecord(
|
||||||
|
|
@ -75,7 +112,7 @@ class AdaptiveRAGFastAnswerSemanticMixinP2:
|
||||||
project_id=project_id,
|
project_id=project_id,
|
||||||
session_id=active_session_id_int,
|
session_id=active_session_id_int,
|
||||||
turn_index=active_turn_index,
|
turn_index=active_turn_index,
|
||||||
**self._build_qa_record_external_fields(),
|
**self._build_qa_record_external_fields(route_decision=route_decision),
|
||||||
**runtime_fields,
|
**runtime_fields,
|
||||||
)
|
)
|
||||||
self.db.add(qa)
|
self.db.add(qa)
|
||||||
|
|
@ -91,6 +128,15 @@ class AdaptiveRAGFastAnswerSemanticMixinP2:
|
||||||
rewrite_meta=rewrite_meta or {},
|
rewrite_meta=rewrite_meta or {},
|
||||||
sources=sources,
|
sources=sources,
|
||||||
)
|
)
|
||||||
|
await self.db.flush()
|
||||||
|
persist_route_memory = getattr(self, "_persist_query_route_memory", None)
|
||||||
|
if callable(persist_route_memory):
|
||||||
|
await persist_route_memory(
|
||||||
|
qa_record_id=int(qa.id) if getattr(qa, "id", None) is not None else None,
|
||||||
|
question_text=question,
|
||||||
|
project_id=project_id,
|
||||||
|
route_decision=route_decision,
|
||||||
|
)
|
||||||
await self.db.commit()
|
await self.db.commit()
|
||||||
await self.db.refresh(qa)
|
await self.db.refresh(qa)
|
||||||
cache_session_id = active_session_id_int if active_session_id_int is not None else (
|
cache_session_id = active_session_id_int if active_session_id_int is not None else (
|
||||||
|
|
@ -120,12 +166,14 @@ class AdaptiveRAGFastAnswerSemanticMixinP2:
|
||||||
confidence=confidence,
|
confidence=confidence,
|
||||||
missing_fields=list(missing_fields or []),
|
missing_fields=list(missing_fields or []),
|
||||||
thinking_mode=thinking_mode,
|
thinking_mode=thinking_mode,
|
||||||
answer_mode=answer_mode,
|
answer_mode=effective_answer_mode,
|
||||||
grounding_status=GROUNDING_STATUS_GROUNDED,
|
grounding_status=effective_grounding_status,
|
||||||
policy_version=runtime_fields["policy_version"],
|
policy_version=runtime_fields["policy_version"],
|
||||||
citations=runtime_fields["citations"],
|
citations=runtime_fields["citations"],
|
||||||
sources_used=runtime_fields["sources_used"],
|
sources_used=runtime_fields["sources_used"],
|
||||||
evidence_pack=runtime_fields["evidence_pack"],
|
evidence_pack=runtime_fields["evidence_pack"],
|
||||||
tool_trace=runtime_fields["tool_trace"],
|
tool_trace=runtime_fields["tool_trace"],
|
||||||
prompt_variant_id=prompt_variant_id,
|
prompt_variant_id=prompt_variant_id,
|
||||||
|
reason_code=effective_reason_code,
|
||||||
|
route_decision=route_decision,
|
||||||
)
|
)
|
||||||
|
|
|
||||||
|
|
@ -579,6 +579,9 @@ class AdaptiveRAGFollowupAnchorMixin:
|
||||||
anchor_reply=chosen,
|
anchor_reply=chosen,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
if is_meta_followup and explicit_anchor is None and chosen is None and not candidates:
|
||||||
|
return FollowupAnchorResolution()
|
||||||
|
|
||||||
if not candidates:
|
if not candidates:
|
||||||
return FollowupAnchorResolution(
|
return FollowupAnchorResolution(
|
||||||
followup_mode=followup_mode,
|
followup_mode=followup_mode,
|
||||||
|
|
@ -617,6 +620,9 @@ class AdaptiveRAGFollowupAnchorMixin:
|
||||||
anchor_reply=chosen,
|
anchor_reply=chosen,
|
||||||
)
|
)
|
||||||
|
|
||||||
|
if is_meta_followup:
|
||||||
|
return FollowupAnchorResolution()
|
||||||
|
|
||||||
return FollowupAnchorResolution(
|
return FollowupAnchorResolution(
|
||||||
followup_mode=followup_mode,
|
followup_mode=followup_mode,
|
||||||
anchor_mode="ambiguous_candidates",
|
anchor_mode="ambiguous_candidates",
|
||||||
|
|
@ -764,14 +770,7 @@ class AdaptiveRAGFollowupAnchorMixin:
|
||||||
)
|
)
|
||||||
anchor_kind = str(resolution.anchor_kind or "").strip() or None
|
anchor_kind = str(resolution.anchor_kind or "").strip() or None
|
||||||
if anchor_row is None:
|
if anchor_row is None:
|
||||||
return {
|
return None
|
||||||
"answer": "当前无法追溯到本次回复的路由记录。",
|
|
||||||
"meta_followup_kind": meta_kind,
|
|
||||||
"reason_code": "meta_followup_no_route_record",
|
|
||||||
"anchor_kind": anchor_kind,
|
|
||||||
"anchor_mode": resolution.anchor_mode,
|
|
||||||
"routing_snapshot": {},
|
|
||||||
}
|
|
||||||
|
|
||||||
routing_snapshot = (
|
routing_snapshot = (
|
||||||
anchor_row.routing_snapshot
|
anchor_row.routing_snapshot
|
||||||
|
|
|
||||||
|
|
@ -101,7 +101,10 @@ class AdaptiveRAGManifestCoreMixin:
|
||||||
) -> dict[str, Any]:
|
) -> dict[str, Any]:
|
||||||
normalized = self._normalize_entity_text(question)
|
normalized = self._normalize_entity_text(question)
|
||||||
alias_terms = self._collect_alias_terms(normalized)
|
alias_terms = self._collect_alias_terms(normalized)
|
||||||
query_terms = self._extract_query_terms(question, preferred=self._extract_project_lookup_keyword(question))
|
term_collector = getattr(self, "_collect_lookup_terms", None)
|
||||||
|
query_terms = term_collector(question=question, limit=6) if callable(term_collector) else []
|
||||||
|
if normalized:
|
||||||
|
query_terms.append(normalized[:32])
|
||||||
for term in alias_terms:
|
for term in alias_terms:
|
||||||
if term not in query_terms:
|
if term not in query_terms:
|
||||||
query_terms.append(term)
|
query_terms.append(term)
|
||||||
|
|
@ -192,4 +195,3 @@ class AdaptiveRAGManifestCoreMixin:
|
||||||
"count": min(len(catalog), max_rows),
|
"count": min(len(catalog), max_rows),
|
||||||
"items": catalog[:max_rows],
|
"items": catalog[:max_rows],
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -101,8 +101,9 @@ class AdaptiveRAGManifestFeatureMixinP1:
|
||||||
catalog = self._feature_catalog_from_snapshot(system_snapshot)
|
catalog = self._feature_catalog_from_snapshot(system_snapshot)
|
||||||
if not catalog:
|
if not catalog:
|
||||||
return []
|
return []
|
||||||
query_terms = [x.lower() for x in self._extract_query_terms(question)[:8] if str(x).strip()]
|
|
||||||
matched: list[dict[str, Any]] = []
|
matched: list[dict[str, Any]] = []
|
||||||
|
normalizer = getattr(self, "_normalize_lookup_text", None)
|
||||||
|
normalized_text = normalizer(question) if callable(normalizer) else self._normalize_entity_text(question)
|
||||||
for item in catalog:
|
for item in catalog:
|
||||||
name = str(item.get("name") or "").strip()
|
name = str(item.get("name") or "").strip()
|
||||||
route = str(item.get("route") or "").strip()
|
route = str(item.get("route") or "").strip()
|
||||||
|
|
@ -116,11 +117,13 @@ class AdaptiveRAGManifestFeatureMixinP1:
|
||||||
|
|
||||||
score = 0.0
|
score = 0.0
|
||||||
for alias in alias_pool:
|
for alias in alias_pool:
|
||||||
if alias and alias in text:
|
if not alias:
|
||||||
score += 0.45
|
continue
|
||||||
for term in query_terms:
|
if alias == normalized_text:
|
||||||
if term and any(term in alias or alias in term for alias in alias_pool):
|
score = max(score, 0.96)
|
||||||
score += 0.18
|
continue
|
||||||
|
if alias in text or normalized_text in alias:
|
||||||
|
score = max(score, 0.82)
|
||||||
if score <= 0:
|
if score <= 0:
|
||||||
continue
|
continue
|
||||||
matched.append(
|
matched.append(
|
||||||
|
|
@ -223,4 +226,3 @@ class AdaptiveRAGManifestFeatureMixinP1:
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
return rows[:3]
|
return rows[:3]
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -5,6 +5,13 @@ from .shared import *
|
||||||
|
|
||||||
|
|
||||||
class AdaptiveRAGMultiStepCoreMixin:
|
class AdaptiveRAGMultiStepCoreMixin:
|
||||||
|
@staticmethod
|
||||||
|
def _skip_internal_hint_expansion(query_semantics: Optional[dict[str, Any]]) -> bool:
|
||||||
|
semantics = query_semantics if isinstance(query_semantics, dict) else {}
|
||||||
|
if bool(semantics.get("external_freshness_only")):
|
||||||
|
return True
|
||||||
|
return str(semantics.get("effective_source_policy") or semantics.get("source_policy") or "").strip().lower() == "external"
|
||||||
|
|
||||||
async def _run_single_retrieval_pass(
|
async def _run_single_retrieval_pass(
|
||||||
self,
|
self,
|
||||||
*,
|
*,
|
||||||
|
|
@ -98,8 +105,9 @@ class AdaptiveRAGMultiStepCoreMixin:
|
||||||
query_semantics: Optional[dict[str, Any]] = None,
|
query_semantics: Optional[dict[str, Any]] = None,
|
||||||
) -> tuple[str, dict[str, Any]]:
|
) -> tuple[str, dict[str, Any]]:
|
||||||
semantics = query_semantics if isinstance(query_semantics, dict) else {}
|
semantics = query_semantics if isinstance(query_semantics, dict) else {}
|
||||||
if bool(semantics.get("external_freshness_only")):
|
if self._skip_internal_hint_expansion(semantics):
|
||||||
return str(question or "").strip()[:1200], {"mode": "freshness_guard_skip_internal_hint"}
|
mode = "freshness_guard_skip_internal_hint" if bool(semantics.get("external_freshness_only")) else "external_source_policy_skip_internal_hint"
|
||||||
|
return str(question or "").strip()[:1200], {"mode": mode}
|
||||||
resolved = await self.entity_resolution(question=question, project_id=project_id, limit=5)
|
resolved = await self.entity_resolution(question=question, project_id=project_id, limit=5)
|
||||||
manifest = await self.get_project_manifest(project_id=project_id)
|
manifest = await self.get_project_manifest(project_id=project_id)
|
||||||
hints: list[str] = []
|
hints: list[str] = []
|
||||||
|
|
|
||||||
|
|
@ -19,12 +19,16 @@ class AdaptiveRAGMultiStepFlowMixin:
|
||||||
) -> tuple[list[RetrievedItem], dict[str, Any]]:
|
) -> tuple[list[RetrievedItem], dict[str, Any]]:
|
||||||
semantics = query_semantics if isinstance(query_semantics, dict) else {}
|
semantics = query_semantics if isinstance(query_semantics, dict) else {}
|
||||||
external_freshness_only = bool(semantics.get("external_freshness_only"))
|
external_freshness_only = bool(semantics.get("external_freshness_only"))
|
||||||
|
external_source_policy = str(
|
||||||
|
semantics.get("effective_source_policy") or semantics.get("source_policy") or ""
|
||||||
|
).strip().lower() == "external"
|
||||||
|
skip_internal_enrichment = external_freshness_only or external_source_policy
|
||||||
resolved: dict[str, Any] = {"matched_entities": [], "canonical_terms": []}
|
resolved: dict[str, Any] = {"matched_entities": [], "canonical_terms": []}
|
||||||
manifest: dict[str, Any] = {}
|
manifest: dict[str, Any] = {}
|
||||||
system_snapshot: dict[str, Any] = {}
|
system_snapshot: dict[str, Any] = {}
|
||||||
symbol_hits: dict[str, Any] = {"count": 0, "items": []}
|
symbol_hits: dict[str, Any] = {"count": 0, "items": []}
|
||||||
feature_hits: list[dict[str, Any]] = []
|
feature_hits: list[dict[str, Any]] = []
|
||||||
if not external_freshness_only:
|
if not skip_internal_enrichment:
|
||||||
resolved = await self.entity_resolution(question=question, project_id=project_id, limit=6)
|
resolved = await self.entity_resolution(question=question, project_id=project_id, limit=6)
|
||||||
manifest = await self.get_project_manifest(project_id=project_id)
|
manifest = await self.get_project_manifest(project_id=project_id)
|
||||||
system_snapshot = await self.context_cache.get_system_snapshot(
|
system_snapshot = await self.context_cache.get_system_snapshot(
|
||||||
|
|
@ -87,6 +91,14 @@ class AdaptiveRAGMultiStepFlowMixin:
|
||||||
"mode": "external_freshness_only",
|
"mode": "external_freshness_only",
|
||||||
}
|
}
|
||||||
)
|
)
|
||||||
|
elif external_source_policy:
|
||||||
|
tool_trace.append(
|
||||||
|
{
|
||||||
|
"step": 0,
|
||||||
|
"tool": "source_policy_guard",
|
||||||
|
"mode": "external_only",
|
||||||
|
}
|
||||||
|
)
|
||||||
if session_state is not None and isinstance(session_state.last_entity_ids, list):
|
if session_state is not None and isinstance(session_state.last_entity_ids, list):
|
||||||
anchor_ids = [int(x) for x in session_state.last_entity_ids if str(x).isdigit()][:6]
|
anchor_ids = [int(x) for x in session_state.last_entity_ids if str(x).isdigit()][:6]
|
||||||
else:
|
else:
|
||||||
|
|
@ -124,7 +136,7 @@ class AdaptiveRAGMultiStepFlowMixin:
|
||||||
vector_top_k=vector_top_k,
|
vector_top_k=vector_top_k,
|
||||||
)
|
)
|
||||||
feature_rows = []
|
feature_rows = []
|
||||||
if not external_freshness_only:
|
if not skip_internal_enrichment:
|
||||||
feature_rows = self._retrieve_feature_catalog_context(
|
feature_rows = self._retrieve_feature_catalog_context(
|
||||||
question=current_question,
|
question=current_question,
|
||||||
system_snapshot=system_snapshot,
|
system_snapshot=system_snapshot,
|
||||||
|
|
@ -190,7 +202,7 @@ class AdaptiveRAGMultiStepFlowMixin:
|
||||||
break
|
break
|
||||||
|
|
||||||
next_domains: list[str] = []
|
next_domains: list[str] = []
|
||||||
if external_freshness_only:
|
if skip_internal_enrichment:
|
||||||
next_domains = list(current_domains)
|
next_domains = list(current_domains)
|
||||||
else:
|
else:
|
||||||
for item in [*current_domains, "graph", "db", "doc", "pmflow", "feishu"]:
|
for item in [*current_domains, "graph", "db", "doc", "pmflow", "feishu"]:
|
||||||
|
|
|
||||||
|
|
@ -35,6 +35,18 @@ class AdaptiveRAGQAFeedbackRecordsMixin:
|
||||||
submitted_by=submitted_by,
|
submitted_by=submitted_by,
|
||||||
)
|
)
|
||||||
self.db.add(feedback)
|
self.db.add(feedback)
|
||||||
|
await self.db.flush()
|
||||||
|
await self._update_query_route_feedback_outcome(
|
||||||
|
qa_record_id=int(qa_id),
|
||||||
|
feedback_outcome={
|
||||||
|
"qa_feedback_id": int(feedback.id) if getattr(feedback, "id", None) is not None else None,
|
||||||
|
"rating": str(feedback.rating or "").strip().lower(),
|
||||||
|
"helpfulness": str(feedback.helpfulness or "").strip() or None,
|
||||||
|
"error_types": [str(x).strip() for x in (feedback.error_types or []) if str(x).strip()],
|
||||||
|
"correct_answer": str(feedback.correct_answer or "").strip() or None,
|
||||||
|
"submitted_by": int(submitted_by) if submitted_by is not None else None,
|
||||||
|
},
|
||||||
|
)
|
||||||
await self.db.commit()
|
await self.db.commit()
|
||||||
await self.db.refresh(feedback)
|
await self.db.refresh(feedback)
|
||||||
if str(feedback.rating or "").strip().lower() == "negative" and (
|
if str(feedback.rating or "").strip().lower() == "negative" and (
|
||||||
|
|
@ -152,4 +164,3 @@ class AdaptiveRAGQAFeedbackRecordsMixin:
|
||||||
"page_size": page_size,
|
"page_size": page_size,
|
||||||
"count": len(items),
|
"count": len(items),
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,467 @@
|
||||||
|
"""Adaptive RAG query route gateway helpers."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from .shared import *
|
||||||
|
|
||||||
|
|
||||||
|
class AdaptiveRAGQueryRouteGatewayMixin:
|
||||||
|
_QUERY_ROUTE_MODES = {"internal", "external", "uncertain"}
|
||||||
|
_INTERNAL_ALLOWED_SOURCES = {"db", "graph", "doc", "pmflow", "feishu"}
|
||||||
|
_EXTERNAL_ALLOWED_SOURCES = {"url", "mcp_search", "system"}
|
||||||
|
|
||||||
|
def _normalize_query_route_decision(self, payload: Optional[dict[str, Any]]) -> dict[str, Any]:
|
||||||
|
raw = payload if isinstance(payload, dict) else {}
|
||||||
|
reply_contract = raw.get("reply_contract") if isinstance(raw.get("reply_contract"), dict) else {}
|
||||||
|
query_profile = raw.get("query_profile") if isinstance(raw.get("query_profile"), dict) else {}
|
||||||
|
probe_snapshot_summary = (
|
||||||
|
raw.get("probe_snapshot_summary")
|
||||||
|
if isinstance(raw.get("probe_snapshot_summary"), dict)
|
||||||
|
else {}
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"route_mode": self._normalize_query_route_mode(raw.get("route_mode")),
|
||||||
|
"effective_source_policy": self._normalize_query_route_mode(
|
||||||
|
raw.get("effective_source_policy") or raw.get("route_mode")
|
||||||
|
),
|
||||||
|
"decision_origin": str(raw.get("decision_origin") or "").strip() or "unknown",
|
||||||
|
"reply_contract": dict(reply_contract),
|
||||||
|
"query_profile": self._normalize_query_profile(query_profile),
|
||||||
|
"probe_snapshot_summary": dict(probe_snapshot_summary),
|
||||||
|
"probe_snapshot": dict(raw.get("probe_snapshot") or {}) if isinstance(raw.get("probe_snapshot"), dict) else {},
|
||||||
|
"clarification_text": str(raw.get("clarification_text") or "").strip() or None,
|
||||||
|
}
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _normalize_query_route_mode(cls, value: Any) -> str:
|
||||||
|
normalized = str(value or "").strip().lower()
|
||||||
|
if normalized in cls._QUERY_ROUTE_MODES:
|
||||||
|
return normalized
|
||||||
|
return "uncertain"
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _normalize_query_profile(cls, payload: Optional[dict[str, Any]]) -> dict[str, Any]:
|
||||||
|
rows = payload if isinstance(payload, dict) else {}
|
||||||
|
referenced_objects: list[str] = []
|
||||||
|
for item in list(rows.get("referenced_objects") or []):
|
||||||
|
normalized = re.sub(r"[^a-z0-9:_\-]", "", str(item or "").strip().lower())
|
||||||
|
if normalized and normalized not in referenced_objects:
|
||||||
|
referenced_objects.append(normalized[:48])
|
||||||
|
if not referenced_objects:
|
||||||
|
referenced_objects = ["unknown"]
|
||||||
|
return {
|
||||||
|
"needs_private_context": bool(rows.get("needs_private_context")),
|
||||||
|
"needs_freshness": bool(rows.get("needs_freshness")),
|
||||||
|
"referenced_objects": referenced_objects,
|
||||||
|
"confidence": max(0.0, min(1.0, float(rows.get("confidence") or 0.0))),
|
||||||
|
}
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _build_query_route_reply_contract(
|
||||||
|
cls,
|
||||||
|
*,
|
||||||
|
route_mode: str,
|
||||||
|
effective_source_policy: str,
|
||||||
|
clarification_text: Optional[str] = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
normalized_route = cls._normalize_query_route_mode(route_mode)
|
||||||
|
normalized_effective = cls._normalize_query_route_mode(effective_source_policy)
|
||||||
|
if normalized_route == "internal" or normalized_effective == "internal":
|
||||||
|
return {
|
||||||
|
"mode": "explicit_source",
|
||||||
|
"head": "内部知识库查询:",
|
||||||
|
"tail_hint": None,
|
||||||
|
}
|
||||||
|
if normalized_route == "uncertain" and normalized_effective != "external":
|
||||||
|
return {
|
||||||
|
"mode": "clarify",
|
||||||
|
"head": None,
|
||||||
|
"tail_hint": None,
|
||||||
|
"clarification_text": str(clarification_text or "").strip() or None,
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
"mode": "explicit_source",
|
||||||
|
"head": "外部实时查询:",
|
||||||
|
"tail_hint": (
|
||||||
|
"如果你指的是系统内部配置/项目对象,请明确指出。"
|
||||||
|
if normalized_route == "uncertain"
|
||||||
|
else None
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _build_query_route_clarification_text(cls) -> str:
|
||||||
|
return "你是想问外部公开事实,还是系统内部配置/项目对象?"
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _build_contract_snapshot_summary(
|
||||||
|
cls,
|
||||||
|
*,
|
||||||
|
source_policy_hint: Optional[str],
|
||||||
|
query_semantics: Optional[dict[str, Any]],
|
||||||
|
route_mode: str,
|
||||||
|
effective_source_policy: str,
|
||||||
|
decision_origin: str,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
semantics = query_semantics if isinstance(query_semantics, dict) else {}
|
||||||
|
return {
|
||||||
|
"contract": {
|
||||||
|
"source_policy_hint": cls._normalize_query_route_mode(source_policy_hint),
|
||||||
|
"semantic_source_policy": cls._normalize_query_route_mode(semantics.get("source_policy")),
|
||||||
|
"semantic_effective_source_policy": cls._normalize_query_route_mode(
|
||||||
|
semantics.get("effective_source_policy")
|
||||||
|
),
|
||||||
|
"route_mode": cls._normalize_query_route_mode(route_mode),
|
||||||
|
"effective_source_policy": cls._normalize_query_route_mode(effective_source_policy),
|
||||||
|
"decision_origin": str(decision_origin or "").strip() or "unknown",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
def _build_query_route_audit_snapshot(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
route_decision: Optional[dict[str, Any]],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
normalized = self._normalize_query_route_decision(route_decision)
|
||||||
|
return {
|
||||||
|
"route_mode": normalized["route_mode"],
|
||||||
|
"effective_source_policy": normalized["effective_source_policy"],
|
||||||
|
"decision_origin": normalized["decision_origin"],
|
||||||
|
"reply_contract": normalized["reply_contract"],
|
||||||
|
"query_profile": normalized["query_profile"],
|
||||||
|
"probe_snapshot_summary": normalized["probe_snapshot_summary"],
|
||||||
|
}
|
||||||
|
|
||||||
|
def _resolve_route_contract(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
query_semantics: Optional[dict[str, Any]],
|
||||||
|
source_policy_hint: Optional[str],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
semantics = query_semantics if isinstance(query_semantics, dict) else {}
|
||||||
|
normalized_hint = self._normalize_query_route_mode(source_policy_hint)
|
||||||
|
if normalized_hint in {"internal", "external"}:
|
||||||
|
return {
|
||||||
|
"source_policy": normalized_hint,
|
||||||
|
"effective_source_policy": normalized_hint,
|
||||||
|
"decision_origin": "upstream_hint",
|
||||||
|
"source_reason": "upstream_source_policy_hint",
|
||||||
|
}
|
||||||
|
|
||||||
|
source_policy = self._normalize_query_route_mode(semantics.get("source_policy"))
|
||||||
|
semantic_effective = self._normalize_query_route_mode(
|
||||||
|
semantics.get("effective_source_policy") or semantics.get("source_policy")
|
||||||
|
)
|
||||||
|
effective_source_policy = (
|
||||||
|
semantic_effective
|
||||||
|
if semantic_effective in {"internal", "external"}
|
||||||
|
else ("uncertain" if source_policy == "uncertain" else source_policy)
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"source_policy": source_policy,
|
||||||
|
"effective_source_policy": effective_source_policy,
|
||||||
|
"decision_origin": str(semantics.get("decision_origin") or "query_semantics_contract").strip()
|
||||||
|
or "query_semantics_contract",
|
||||||
|
"source_reason": str(semantics.get("source_reason") or "query_semantics_contract").strip()
|
||||||
|
or "query_semantics_contract",
|
||||||
|
}
|
||||||
|
|
||||||
|
def _apply_route_decision_to_query_semantics(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
query_semantics: Optional[dict[str, Any]],
|
||||||
|
route_decision: Optional[dict[str, Any]],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
semantics = dict(query_semantics or {})
|
||||||
|
normalized = self._normalize_query_route_decision(route_decision)
|
||||||
|
effective_source_policy = normalized["effective_source_policy"]
|
||||||
|
if effective_source_policy == "uncertain":
|
||||||
|
retrieval_domains: list[str] = []
|
||||||
|
else:
|
||||||
|
retrieval_domains = self._apply_source_policy_to_domains(
|
||||||
|
domains=[
|
||||||
|
str(item).strip().lower()
|
||||||
|
for item in list(semantics.get("retrieval_domains") or [])
|
||||||
|
if str(item or "").strip()
|
||||||
|
],
|
||||||
|
effective_source_policy=effective_source_policy,
|
||||||
|
)
|
||||||
|
semantics["route_decision"] = normalized
|
||||||
|
semantics["query_profile"] = normalized["query_profile"]
|
||||||
|
semantics["source_policy"] = normalized["route_mode"]
|
||||||
|
semantics["effective_source_policy"] = effective_source_policy
|
||||||
|
semantics["decision_origin"] = normalized["decision_origin"]
|
||||||
|
semantics["source_reason"] = normalized["decision_origin"]
|
||||||
|
semantics["retrieval_domains"] = retrieval_domains
|
||||||
|
return semantics
|
||||||
|
|
||||||
|
async def _run_query_route_gateway(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
question: str,
|
||||||
|
asked_via: str,
|
||||||
|
project_id: Optional[int],
|
||||||
|
source_chat_id: Optional[str],
|
||||||
|
query_semantics: Optional[dict[str, Any]],
|
||||||
|
source_policy_hint: Optional[str],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
_ = (question, asked_via, project_id, source_chat_id)
|
||||||
|
semantics = dict(query_semantics or {})
|
||||||
|
query_profile = self._normalize_query_profile(semantics.get("query_profile"))
|
||||||
|
contract = self._resolve_route_contract(
|
||||||
|
query_semantics=semantics,
|
||||||
|
source_policy_hint=source_policy_hint,
|
||||||
|
)
|
||||||
|
route_mode = self._normalize_query_route_mode(contract.get("source_policy"))
|
||||||
|
effective_source_policy = self._normalize_query_route_mode(contract.get("effective_source_policy"))
|
||||||
|
decision_origin = str(contract.get("decision_origin") or "query_semantics_contract").strip() or "query_semantics_contract"
|
||||||
|
clarification_text = None
|
||||||
|
if route_mode == "uncertain" and effective_source_policy != "external":
|
||||||
|
clarification_text = self._build_query_route_clarification_text()
|
||||||
|
probe_snapshot = {
|
||||||
|
"contract": {
|
||||||
|
"source_policy": route_mode,
|
||||||
|
"effective_source_policy": effective_source_policy,
|
||||||
|
"source_reason": str(contract.get("source_reason") or "query_semantics_contract").strip()
|
||||||
|
or "query_semantics_contract",
|
||||||
|
"decision_origin": decision_origin,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
"route_mode": route_mode,
|
||||||
|
"effective_source_policy": effective_source_policy,
|
||||||
|
"decision_origin": decision_origin,
|
||||||
|
"reply_contract": self._build_query_route_reply_contract(
|
||||||
|
route_mode=route_mode,
|
||||||
|
effective_source_policy=effective_source_policy,
|
||||||
|
clarification_text=clarification_text,
|
||||||
|
),
|
||||||
|
"clarification_text": clarification_text,
|
||||||
|
"query_profile": query_profile,
|
||||||
|
"probe_snapshot": probe_snapshot,
|
||||||
|
"probe_snapshot_summary": self._build_contract_snapshot_summary(
|
||||||
|
source_policy_hint=source_policy_hint,
|
||||||
|
query_semantics=semantics,
|
||||||
|
route_mode=route_mode,
|
||||||
|
effective_source_policy=effective_source_policy,
|
||||||
|
decision_origin=decision_origin,
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
async def preview_query_route_decision(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
question: str,
|
||||||
|
asked_via: str,
|
||||||
|
project_id: Optional[int],
|
||||||
|
source_chat_id: Optional[str],
|
||||||
|
source_message_id: Optional[str] = None,
|
||||||
|
conversation_scope_key: Optional[str] = None,
|
||||||
|
reply_to_message_id: Optional[str] = None,
|
||||||
|
root_message_id: Optional[str] = None,
|
||||||
|
source_policy_hint: Optional[str] = None,
|
||||||
|
query_semantics: Optional[dict[str, Any]] = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
semantics = dict(query_semantics or {})
|
||||||
|
if not semantics:
|
||||||
|
semantics = await self._resolve_query_semantics(
|
||||||
|
message=question,
|
||||||
|
source_system=asked_via,
|
||||||
|
metadata={
|
||||||
|
"project_id": project_id,
|
||||||
|
"source_chat_id": source_chat_id,
|
||||||
|
"source_message_id": source_message_id,
|
||||||
|
"conversation_scope_key": conversation_scope_key,
|
||||||
|
"reply_to_message_id": reply_to_message_id,
|
||||||
|
"root_message_id": root_message_id,
|
||||||
|
},
|
||||||
|
source_policy_hint=source_policy_hint,
|
||||||
|
)
|
||||||
|
semantics = dict(semantics or {})
|
||||||
|
decision = await self._run_query_route_gateway(
|
||||||
|
question=question,
|
||||||
|
asked_via=asked_via,
|
||||||
|
project_id=project_id,
|
||||||
|
source_chat_id=source_chat_id,
|
||||||
|
query_semantics=semantics,
|
||||||
|
source_policy_hint=source_policy_hint or semantics.get("source_policy"),
|
||||||
|
)
|
||||||
|
normalized_decision = self._normalize_query_route_decision(decision)
|
||||||
|
normalized_decision["probe_snapshot"] = dict(decision.get("probe_snapshot") or {})
|
||||||
|
return {
|
||||||
|
"query_semantics": self._apply_route_decision_to_query_semantics(
|
||||||
|
query_semantics=semantics,
|
||||||
|
route_decision=normalized_decision,
|
||||||
|
),
|
||||||
|
"route_decision": normalized_decision,
|
||||||
|
}
|
||||||
|
|
||||||
|
async def _persist_query_route_memory(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
qa_record_id: Optional[int],
|
||||||
|
question_text: str,
|
||||||
|
project_id: Optional[int],
|
||||||
|
route_decision: Optional[dict[str, Any]],
|
||||||
|
) -> None:
|
||||||
|
normalized = self._normalize_query_route_decision(route_decision)
|
||||||
|
if qa_record_id is None or not str(question_text or "").strip():
|
||||||
|
return
|
||||||
|
if normalized["route_mode"] == "uncertain" and normalized["effective_source_policy"] == "uncertain":
|
||||||
|
# 澄清样本仍然保留,但不强制要求 embedding 成功。
|
||||||
|
pass
|
||||||
|
|
||||||
|
try:
|
||||||
|
row = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(QueryRouteMemory).where(
|
||||||
|
and_(
|
||||||
|
QueryRouteMemory.tenant_id == self.tenant_id,
|
||||||
|
QueryRouteMemory.qa_record_id == int(qa_record_id),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
except Exception:
|
||||||
|
row = None
|
||||||
|
|
||||||
|
question_embedding: list[float] = []
|
||||||
|
embedding_getter = getattr(getattr(self, "llm", None), "get_embedding_async", None)
|
||||||
|
if callable(embedding_getter):
|
||||||
|
try:
|
||||||
|
question_embedding = await embedding_getter(
|
||||||
|
str(question_text or "").strip(),
|
||||||
|
db=self.db,
|
||||||
|
tenant_id=self.tenant_id,
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
question_embedding = []
|
||||||
|
|
||||||
|
if row is None:
|
||||||
|
row = QueryRouteMemory(
|
||||||
|
tenant_id=self.tenant_id,
|
||||||
|
qa_record_id=int(qa_record_id),
|
||||||
|
project_id=project_id,
|
||||||
|
question_text=str(question_text or "").strip(),
|
||||||
|
route_mode_predicted=normalized["route_mode"],
|
||||||
|
route_mode_effective=normalized["effective_source_policy"],
|
||||||
|
decision_origin=normalized["decision_origin"],
|
||||||
|
reply_contract=normalized["reply_contract"],
|
||||||
|
probe_snapshot=normalized.get("probe_snapshot") or {},
|
||||||
|
query_profile=normalized["query_profile"],
|
||||||
|
feedback_outcome=None,
|
||||||
|
question_embedding=question_embedding or None,
|
||||||
|
)
|
||||||
|
self.db.add(row)
|
||||||
|
return
|
||||||
|
|
||||||
|
row.project_id = project_id
|
||||||
|
row.question_text = str(question_text or "").strip()
|
||||||
|
row.route_mode_predicted = normalized["route_mode"]
|
||||||
|
row.route_mode_effective = normalized["effective_source_policy"]
|
||||||
|
row.decision_origin = normalized["decision_origin"]
|
||||||
|
row.reply_contract = normalized["reply_contract"]
|
||||||
|
row.probe_snapshot = normalized.get("probe_snapshot") or {}
|
||||||
|
row.query_profile = normalized["query_profile"]
|
||||||
|
if question_embedding:
|
||||||
|
row.question_embedding = question_embedding
|
||||||
|
|
||||||
|
async def _update_query_route_feedback_outcome(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
qa_record_id: Optional[int],
|
||||||
|
feedback_outcome: Optional[dict[str, Any]],
|
||||||
|
) -> None:
|
||||||
|
if qa_record_id is None or not isinstance(feedback_outcome, dict):
|
||||||
|
return
|
||||||
|
row = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(QueryRouteMemory).where(
|
||||||
|
and_(
|
||||||
|
QueryRouteMemory.tenant_id == self.tenant_id,
|
||||||
|
QueryRouteMemory.qa_record_id == int(qa_record_id),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if row is None:
|
||||||
|
return
|
||||||
|
row.feedback_outcome = dict(feedback_outcome)
|
||||||
|
|
||||||
|
def _filter_sources_for_route(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
sources: list[dict[str, Any]],
|
||||||
|
route_decision: Optional[dict[str, Any]],
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
decision = route_decision if isinstance(route_decision, dict) else {}
|
||||||
|
effective_source_policy = self._normalize_query_route_mode(decision.get("effective_source_policy"))
|
||||||
|
if effective_source_policy == "internal":
|
||||||
|
allowed = self._INTERNAL_ALLOWED_SOURCES
|
||||||
|
elif effective_source_policy == "external":
|
||||||
|
allowed = self._EXTERNAL_ALLOWED_SOURCES
|
||||||
|
else:
|
||||||
|
return list(sources or [])
|
||||||
|
filtered = [
|
||||||
|
item
|
||||||
|
for item in list(sources or [])
|
||||||
|
if isinstance(item, dict) and str(item.get("source") or "").strip().lower() in allowed
|
||||||
|
]
|
||||||
|
return filtered
|
||||||
|
|
||||||
|
def _render_answer_with_route_contract(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
answer: str,
|
||||||
|
sources: list[dict[str, Any]],
|
||||||
|
route_decision: Optional[dict[str, Any]],
|
||||||
|
) -> str:
|
||||||
|
text = str(answer or "").strip()
|
||||||
|
if not text:
|
||||||
|
return text
|
||||||
|
decision = route_decision if isinstance(route_decision, dict) else {}
|
||||||
|
reply_contract = decision.get("reply_contract") if isinstance(decision.get("reply_contract"), dict) else {}
|
||||||
|
contract_mode = str(reply_contract.get("mode") or "").strip().lower()
|
||||||
|
if contract_mode == "clarify":
|
||||||
|
return str(reply_contract.get("clarification_text") or text).strip()
|
||||||
|
effective_source_policy = self._normalize_query_route_mode(decision.get("effective_source_policy"))
|
||||||
|
head = str(reply_contract.get("head") or "").strip()
|
||||||
|
tail_hint = str(reply_contract.get("tail_hint") or "").strip()
|
||||||
|
formatted = text if not head or text.startswith(head) else f"{head}\n{text}"
|
||||||
|
labels: list[str] = []
|
||||||
|
for item in self._filter_sources_for_route(sources=sources, route_decision=decision)[:3]:
|
||||||
|
if not isinstance(item, dict):
|
||||||
|
continue
|
||||||
|
source = str(item.get("source") or "").strip().lower()
|
||||||
|
if effective_source_policy == "external" and source == "system":
|
||||||
|
continue
|
||||||
|
label = str(item.get("label") or item.get("source") or "").strip()
|
||||||
|
if label and label not in labels:
|
||||||
|
labels.append(label)
|
||||||
|
if labels and "来源摘要:" not in formatted:
|
||||||
|
formatted = f"{formatted}\n\n来源摘要:{';'.join(labels[:3])}"
|
||||||
|
if tail_hint and tail_hint not in formatted:
|
||||||
|
formatted = f"{formatted}\n\n{tail_hint}"
|
||||||
|
return formatted.strip()
|
||||||
|
|
||||||
|
def _build_route_clarification_payload(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
route_decision: Optional[dict[str, Any]],
|
||||||
|
) -> Optional[dict[str, Any]]:
|
||||||
|
normalized = self._normalize_query_route_decision(route_decision)
|
||||||
|
reply_contract = (
|
||||||
|
normalized.get("reply_contract")
|
||||||
|
if isinstance(normalized.get("reply_contract"), dict)
|
||||||
|
else {}
|
||||||
|
)
|
||||||
|
if str(reply_contract.get("mode") or "").strip().lower() != "clarify":
|
||||||
|
return None
|
||||||
|
clarification_text = str(reply_contract.get("clarification_text") or "").strip()
|
||||||
|
if not clarification_text:
|
||||||
|
return None
|
||||||
|
return {
|
||||||
|
"answer": clarification_text,
|
||||||
|
"code": "query_route_clarify",
|
||||||
|
"candidates": [],
|
||||||
|
"missing_fields": ["query_route"],
|
||||||
|
}
|
||||||
|
|
@ -50,7 +50,13 @@ class AdaptiveRAGRetrievalGraphMixin:
|
||||||
).scalars().all()
|
).scalars().all()
|
||||||
|
|
||||||
if not seed_entities:
|
if not seed_entities:
|
||||||
terms = self._extract_query_terms(question, preferred=self._extract_project_lookup_keyword(question))
|
term_collector = getattr(self, "_collect_lookup_terms", None)
|
||||||
|
terms = term_collector(question=question, limit=6) if callable(term_collector) else []
|
||||||
|
if not terms:
|
||||||
|
normalizer = getattr(self, "_normalize_lookup_text", None)
|
||||||
|
normalized = normalizer(question) if callable(normalizer) else str(question or "").strip()
|
||||||
|
if normalized:
|
||||||
|
terms = [normalized[:32]]
|
||||||
if terms:
|
if terms:
|
||||||
conds = []
|
conds = []
|
||||||
for term in terms[:6]:
|
for term in terms[:6]:
|
||||||
|
|
|
||||||
|
|
@ -5,6 +5,167 @@ from .shared import *
|
||||||
|
|
||||||
|
|
||||||
class AdaptiveRAGRetrievalRouterDBMixin:
|
class AdaptiveRAGRetrievalRouterDBMixin:
|
||||||
|
async def _retrieve_generic_semantic_fact_rows(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
entity_type: str,
|
||||||
|
question: str,
|
||||||
|
query_semantics: Optional[dict[str, Any]] = None,
|
||||||
|
) -> list[RetrievedItem]:
|
||||||
|
term_collector = getattr(self, "_collect_lookup_terms", None)
|
||||||
|
query_terms = term_collector(
|
||||||
|
question=question,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
operation_contract=(
|
||||||
|
query_semantics.get("operation_contract")
|
||||||
|
if isinstance(query_semantics, dict) and isinstance(query_semantics.get("operation_contract"), dict)
|
||||||
|
else None
|
||||||
|
),
|
||||||
|
target_resolution=(
|
||||||
|
query_semantics.get("target_resolution")
|
||||||
|
if isinstance(query_semantics, dict) and isinstance(query_semantics.get("target_resolution"), dict)
|
||||||
|
else None
|
||||||
|
),
|
||||||
|
limit=4,
|
||||||
|
) if callable(term_collector) else []
|
||||||
|
if not query_terms:
|
||||||
|
return []
|
||||||
|
|
||||||
|
fact_query = getattr(self, "_query_semantic_facts_by_lookup_terms", None)
|
||||||
|
if not callable(fact_query):
|
||||||
|
return []
|
||||||
|
matched_facts, used_keyword = await fact_query(
|
||||||
|
entity_type=entity_type,
|
||||||
|
query_terms=query_terms,
|
||||||
|
entity_id=None,
|
||||||
|
limit=10,
|
||||||
|
)
|
||||||
|
if not matched_facts:
|
||||||
|
return []
|
||||||
|
|
||||||
|
grouped: dict[int, list[SemanticFact]] = {}
|
||||||
|
for row in matched_facts:
|
||||||
|
try:
|
||||||
|
entity_id = int(row.entity_id)
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
grouped.setdefault(entity_id, []).append(row)
|
||||||
|
if not grouped:
|
||||||
|
return []
|
||||||
|
|
||||||
|
preview = ";".join(
|
||||||
|
f"#{entity_id} " + " / ".join(
|
||||||
|
str(item.fact_value_text or "").strip()
|
||||||
|
for item in rows[:3]
|
||||||
|
if str(item.fact_value_text or "").strip()
|
||||||
|
)
|
||||||
|
for entity_id, rows in list(grouped.items())[:8]
|
||||||
|
)
|
||||||
|
metadata = self._normalize_source_registry_metadata(
|
||||||
|
source_type="db",
|
||||||
|
metadata={
|
||||||
|
"table": "semantic_facts",
|
||||||
|
"entity_type": entity_type,
|
||||||
|
"query_keyword": used_keyword,
|
||||||
|
"match_count": len(grouped),
|
||||||
|
"fact_keys": sorted({str(row.fact_key or "") for row in matched_facts if str(row.fact_key or "").strip()}),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
return [
|
||||||
|
RetrievedItem(
|
||||||
|
source="db",
|
||||||
|
content=f"语义事实命中{entity_type}:" + preview,
|
||||||
|
score=0.96,
|
||||||
|
source_label="Semantic facts",
|
||||||
|
metadata=metadata,
|
||||||
|
)
|
||||||
|
]
|
||||||
|
|
||||||
|
async def _retrieve_project_semantic_fact_rows(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
question: str,
|
||||||
|
project_id: Optional[int],
|
||||||
|
query_semantics: Optional[dict[str, Any]] = None,
|
||||||
|
) -> list[RetrievedItem]:
|
||||||
|
term_collector = getattr(self, "_collect_lookup_terms", None)
|
||||||
|
query_terms = term_collector(
|
||||||
|
question=question,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
operation_contract=(
|
||||||
|
query_semantics.get("operation_contract")
|
||||||
|
if isinstance(query_semantics, dict) and isinstance(query_semantics.get("operation_contract"), dict)
|
||||||
|
else None
|
||||||
|
),
|
||||||
|
target_resolution=(
|
||||||
|
query_semantics.get("target_resolution")
|
||||||
|
if isinstance(query_semantics, dict) and isinstance(query_semantics.get("target_resolution"), dict)
|
||||||
|
else None
|
||||||
|
),
|
||||||
|
limit=4,
|
||||||
|
) if callable(term_collector) else []
|
||||||
|
if not query_terms:
|
||||||
|
return []
|
||||||
|
|
||||||
|
fact_query = getattr(self, "_query_project_semantic_facts_by_lookup_terms", None)
|
||||||
|
if not callable(fact_query):
|
||||||
|
return []
|
||||||
|
matched_facts, used_keyword = await fact_query(
|
||||||
|
query_terms=query_terms,
|
||||||
|
project_id=project_id,
|
||||||
|
limit=10,
|
||||||
|
)
|
||||||
|
if not matched_facts:
|
||||||
|
return []
|
||||||
|
|
||||||
|
matched_project_ids: list[int] = []
|
||||||
|
for fact in matched_facts:
|
||||||
|
try:
|
||||||
|
entity_id = int(fact.entity_id)
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
if entity_id not in matched_project_ids:
|
||||||
|
matched_project_ids.append(entity_id)
|
||||||
|
if not matched_project_ids:
|
||||||
|
return []
|
||||||
|
|
||||||
|
projects = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(Project).where(
|
||||||
|
and_(
|
||||||
|
Project.tenant_id == self.tenant_id,
|
||||||
|
Project.id.in_(matched_project_ids),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
if not projects:
|
||||||
|
return []
|
||||||
|
|
||||||
|
metadata = self._normalize_source_registry_metadata(
|
||||||
|
source_type="db",
|
||||||
|
metadata={
|
||||||
|
"table": "semantic_facts",
|
||||||
|
"query_keyword": used_keyword,
|
||||||
|
"match_count": len(projects),
|
||||||
|
"matched_project_ids": matched_project_ids,
|
||||||
|
"fact_keys": sorted({str(row.fact_key or "") for row in matched_facts if str(row.fact_key or "").strip()}),
|
||||||
|
},
|
||||||
|
explicit_project_id=matched_project_ids[0] if len(matched_project_ids) == 1 else None,
|
||||||
|
)
|
||||||
|
return [
|
||||||
|
RetrievedItem(
|
||||||
|
source="db",
|
||||||
|
content="语义事实命中项目:" + ";".join(
|
||||||
|
f"#{int(row.id)} {str(row.title or row.name or '未命名项目')}(状态:{str(row.status or 'unknown')})"
|
||||||
|
for row in projects[:10]
|
||||||
|
),
|
||||||
|
score=0.97,
|
||||||
|
source_label="Semantic facts",
|
||||||
|
metadata=metadata,
|
||||||
|
)
|
||||||
|
]
|
||||||
|
|
||||||
async def _retrieve_structured_db(
|
async def _retrieve_structured_db(
|
||||||
self,
|
self,
|
||||||
question: str,
|
question: str,
|
||||||
|
|
@ -16,7 +177,33 @@ class AdaptiveRAGRetrievalRouterDBMixin:
|
||||||
rows: list[RetrievedItem] = []
|
rows: list[RetrievedItem] = []
|
||||||
subject_focus = self._query_subject_focus(query_semantics)
|
subject_focus = self._query_subject_focus(query_semantics)
|
||||||
scope_hint = self._query_scope_hint(query_semantics)
|
scope_hint = self._query_scope_hint(query_semantics)
|
||||||
|
target_resolution = query_semantics.get("target_resolution") if isinstance(query_semantics, dict) and isinstance(query_semantics.get("target_resolution"), dict) else {}
|
||||||
|
target_entity_type = str(target_resolution.get("target_entity_type") or "").strip().lower()
|
||||||
|
query_plan = query_semantics.get("query_plan") if isinstance(query_semantics, dict) and isinstance(query_semantics.get("query_plan"), dict) else {}
|
||||||
|
planned_entity_type = str(query_plan.get("entity_type") or "").strip().lower()
|
||||||
|
project_subject_requested = subject_focus == "project" or target_entity_type == "project" or planned_entity_type == "project"
|
||||||
project_scope_required = scope_hint == "project" or project_id is not None
|
project_scope_required = scope_hint == "project" or project_id is not None
|
||||||
|
semantic_fact_entity_type = ""
|
||||||
|
if target_entity_type in {"task", "signal", "release", "review", "project", "document", "user", "requirement"}:
|
||||||
|
semantic_fact_entity_type = target_entity_type
|
||||||
|
elif planned_entity_type in {"task", "signal", "release", "review", "project", "document", "user", "requirement"}:
|
||||||
|
semantic_fact_entity_type = planned_entity_type
|
||||||
|
elif subject_focus == "task":
|
||||||
|
semantic_fact_entity_type = "task"
|
||||||
|
elif subject_focus == "signal":
|
||||||
|
semantic_fact_entity_type = "signal"
|
||||||
|
elif subject_focus == "people":
|
||||||
|
semantic_fact_entity_type = "user"
|
||||||
|
elif subject_focus == "charter":
|
||||||
|
semantic_fact_entity_type = "requirement"
|
||||||
|
if semantic_fact_entity_type and semantic_fact_entity_type != "project":
|
||||||
|
rows.extend(
|
||||||
|
await self._retrieve_generic_semantic_fact_rows(
|
||||||
|
entity_type=semantic_fact_entity_type,
|
||||||
|
question=question,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
if subject_focus == "people":
|
if subject_focus == "people":
|
||||||
scoped_project_id: Optional[int] = None
|
scoped_project_id: Optional[int] = None
|
||||||
|
|
@ -74,9 +261,21 @@ class AdaptiveRAGRetrievalRouterDBMixin:
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
if subject_focus == "project":
|
if project_subject_requested:
|
||||||
|
semantic_fact_rows = await self._retrieve_project_semantic_fact_rows(
|
||||||
|
question=question,
|
||||||
|
project_id=project_id,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
)
|
||||||
|
rows.extend(semantic_fact_rows)
|
||||||
stmt = select(Project).where(Project.tenant_id == self.tenant_id)
|
stmt = select(Project).where(Project.tenant_id == self.tenant_id)
|
||||||
if project_id:
|
resolved_project_ref = str(target_resolution.get("target_ref") or "").strip()
|
||||||
|
if resolved_project_ref.startswith("project:"):
|
||||||
|
try:
|
||||||
|
stmt = stmt.where(Project.id == int(resolved_project_ref.split(":", 1)[1]))
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
elif project_id:
|
||||||
stmt = stmt.where(Project.id == project_id)
|
stmt = stmt.where(Project.id == project_id)
|
||||||
projects = (await self.db.execute(stmt.limit(8))).scalars().all()
|
projects = (await self.db.execute(stmt.limit(8))).scalars().all()
|
||||||
if projects:
|
if projects:
|
||||||
|
|
|
||||||
|
|
@ -176,9 +176,10 @@ class AdaptiveRAGSessionStateMixin:
|
||||||
return f"{subject_name}->{object_name}"[:255]
|
return f"{subject_name}->{object_name}"[:255]
|
||||||
|
|
||||||
text = str(question or "").strip()
|
text = str(question or "").strip()
|
||||||
keyword = self._extract_project_lookup_keyword(text)
|
normalizer = getattr(self, "_normalize_lookup_text", None)
|
||||||
if len(keyword) >= 2:
|
normalized_question = normalizer(text) if callable(normalizer) else text
|
||||||
return keyword[:255]
|
if 2 <= len(normalized_question) <= 64:
|
||||||
|
return normalized_question[:255]
|
||||||
|
|
||||||
answer_text = str(answer or "").strip()
|
answer_text = str(answer or "").strip()
|
||||||
m = re.search(r"项目[「\"]?([^」\"\n]{2,40})", answer_text)
|
m = re.search(r"项目[「\"]?([^」\"\n]{2,40})", answer_text)
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,766 @@
|
||||||
|
"""Adaptive RAG internal target resolution helpers."""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from .shared import *
|
||||||
|
|
||||||
|
|
||||||
|
class AdaptiveRAGTargetResolutionMixin:
|
||||||
|
_TARGET_ENTITY_TYPES = {"project", "feature", "config", "doc", "person", "signal", "unknown"}
|
||||||
|
_TARGET_RESOLUTION_MODES = {"deterministic", "llm_rerank", "semantic_filter", "clarify", "none"}
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _normalize_object_mentions(
|
||||||
|
cls,
|
||||||
|
raw: Any,
|
||||||
|
*,
|
||||||
|
question: str = "",
|
||||||
|
) -> list[str]:
|
||||||
|
_ = (cls, question)
|
||||||
|
mentions: list[str] = []
|
||||||
|
rows = raw if isinstance(raw, list) else []
|
||||||
|
for item in rows:
|
||||||
|
text = str(item or "").strip(" ,。,.!??::;;“”\"'()()")
|
||||||
|
text = QUESTION_NOISE_RE.sub("", text).strip()
|
||||||
|
if len(text) >= 2 and text not in mentions:
|
||||||
|
mentions.append(text[:32])
|
||||||
|
return mentions[:4]
|
||||||
|
|
||||||
|
def _build_operation_contract(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
question: str,
|
||||||
|
query_semantics: Optional[dict[str, Any]],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
_ = question
|
||||||
|
semantics = query_semantics if isinstance(query_semantics, dict) else {}
|
||||||
|
query_type = str(semantics.get("query_type") or QUERY_TYPE_UNKNOWN).strip().lower() or QUERY_TYPE_UNKNOWN
|
||||||
|
query_profile = dict(semantics.get("query_profile") or {}) if isinstance(semantics.get("query_profile"), dict) else {}
|
||||||
|
referenced_objects = {
|
||||||
|
str(item or "").strip().lower()
|
||||||
|
for item in list(query_profile.get("referenced_objects") or [])
|
||||||
|
if str(item or "").strip()
|
||||||
|
}
|
||||||
|
if query_type in {QUERY_TYPE_EXISTENCE, QUERY_TYPE_UNKNOWN} and referenced_objects.intersection(
|
||||||
|
{"model_routing", "system_config"}
|
||||||
|
):
|
||||||
|
query_type = QUERY_TYPE_STATUS
|
||||||
|
object_mentions = self._normalize_object_mentions(
|
||||||
|
semantics.get("object_mentions"),
|
||||||
|
question=question,
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"query_type": query_type,
|
||||||
|
"answer_contract": (
|
||||||
|
self._internal_answer_contract_for_query_type(query_type)
|
||||||
|
if hasattr(self, "_internal_answer_contract_for_query_type")
|
||||||
|
else "clarify_or_fail"
|
||||||
|
),
|
||||||
|
"query_profile": query_profile,
|
||||||
|
"object_mentions": object_mentions,
|
||||||
|
"clarify_policy": "ask_when_target_ambiguous",
|
||||||
|
}
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _normalize_target_candidate(cls, payload: Optional[dict[str, Any]]) -> Optional[dict[str, Any]]:
|
||||||
|
row = payload if isinstance(payload, dict) else {}
|
||||||
|
entity_type = str(row.get("target_entity_type") or row.get("entity_type") or "").strip().lower()
|
||||||
|
if entity_type not in cls._TARGET_ENTITY_TYPES:
|
||||||
|
return None
|
||||||
|
score = max(0.0, min(1.0, float(row.get("score") or 0.0)))
|
||||||
|
return {
|
||||||
|
"target_entity_type": entity_type,
|
||||||
|
"target_ref": str(row.get("target_ref") or "").strip() or None,
|
||||||
|
"resolved_label": str(row.get("resolved_label") or row.get("display_name") or "").strip() or None,
|
||||||
|
"score": round(score, 4),
|
||||||
|
"source_kind": str(row.get("source_kind") or "unknown").strip().lower() or "unknown",
|
||||||
|
"match_label": str(row.get("match_label") or "").strip() or None,
|
||||||
|
"source_confidence": round(max(0.0, min(1.0, float(row.get("source_confidence") or score or 0.0))), 4),
|
||||||
|
}
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _target_candidate_identity(row: dict[str, Any]) -> str:
|
||||||
|
return "|".join(
|
||||||
|
[
|
||||||
|
str(row.get("target_entity_type") or ""),
|
||||||
|
str(row.get("target_ref") or ""),
|
||||||
|
str(row.get("resolved_label") or ""),
|
||||||
|
]
|
||||||
|
)
|
||||||
|
|
||||||
|
async def _collect_semantic_fact_target_candidates(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
mentions: list[str],
|
||||||
|
project_id: Optional[int],
|
||||||
|
operation_contract: dict[str, Any],
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
if not mentions:
|
||||||
|
return []
|
||||||
|
candidates: list[dict[str, Any]] = []
|
||||||
|
fact_query = getattr(self, "_query_project_semantic_facts_by_lookup_terms", None)
|
||||||
|
if not callable(fact_query):
|
||||||
|
return []
|
||||||
|
for mention in mentions[:3]:
|
||||||
|
fact_rows, _used_keyword = await fact_query(
|
||||||
|
query_terms=[mention],
|
||||||
|
project_id=project_id,
|
||||||
|
limit=10,
|
||||||
|
)
|
||||||
|
project_ids: list[int] = []
|
||||||
|
fact_confidence_by_project: dict[int, float] = {}
|
||||||
|
for row in fact_rows:
|
||||||
|
try:
|
||||||
|
entity_id = int(row.entity_id)
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
if entity_id not in project_ids:
|
||||||
|
project_ids.append(entity_id)
|
||||||
|
fact_confidence_by_project[entity_id] = max(
|
||||||
|
float(fact_confidence_by_project.get(entity_id) or 0.0),
|
||||||
|
float(getattr(row, "confidence", 0.0) or 0.0),
|
||||||
|
)
|
||||||
|
if not project_ids:
|
||||||
|
continue
|
||||||
|
project_rows = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(Project).where(
|
||||||
|
and_(
|
||||||
|
Project.tenant_id == self.tenant_id,
|
||||||
|
Project.id.in_(project_ids),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
for project in project_rows:
|
||||||
|
confidence = max(0.88, min(0.99, float(fact_confidence_by_project.get(int(project.id)) or 0.0)))
|
||||||
|
candidates.append(
|
||||||
|
{
|
||||||
|
"target_entity_type": "project",
|
||||||
|
"target_ref": f"project:{int(project.id)}",
|
||||||
|
"resolved_label": str(project.title or project.name or f"项目#{int(project.id)}"),
|
||||||
|
"score": round(min(0.99, confidence + 0.03), 4),
|
||||||
|
"source_kind": "semantic_facts",
|
||||||
|
"match_label": mention,
|
||||||
|
"source_confidence": round(confidence, 4),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return candidates
|
||||||
|
|
||||||
|
async def _collect_project_target_candidates(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
mentions: list[str],
|
||||||
|
project_id: Optional[int],
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
if not mentions:
|
||||||
|
return []
|
||||||
|
candidates: list[dict[str, Any]] = []
|
||||||
|
similarity_scorer = getattr(self, "_lookup_similarity_score", None)
|
||||||
|
for mention in mentions[:3]:
|
||||||
|
matched_rows: list[Project] = []
|
||||||
|
searcher = getattr(self, "_search_projects_by_keyword", None)
|
||||||
|
if callable(searcher):
|
||||||
|
matched_rows = await searcher(keyword=mention, project_id=project_id, limit=6)
|
||||||
|
else:
|
||||||
|
stmt = select(Project).where(Project.tenant_id == self.tenant_id)
|
||||||
|
if project_id is not None:
|
||||||
|
stmt = stmt.where(Project.id == int(project_id))
|
||||||
|
like = f"%{mention}%"
|
||||||
|
stmt = stmt.where(
|
||||||
|
or_(
|
||||||
|
Project.title.ilike(like),
|
||||||
|
Project.name.ilike(like),
|
||||||
|
Project.description.ilike(like),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
matched_rows = (await self.db.execute(stmt.order_by(Project.updated_at.desc()).limit(6))).scalars().all()
|
||||||
|
if not matched_rows:
|
||||||
|
stmt = select(Project).where(Project.tenant_id == self.tenant_id)
|
||||||
|
if project_id is not None:
|
||||||
|
stmt = stmt.where(Project.id == int(project_id))
|
||||||
|
project_rows = (
|
||||||
|
await self.db.execute(
|
||||||
|
stmt.order_by(Project.updated_at.desc(), Project.id.desc()).limit(120)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
scored_rows: list[tuple[float, Project]] = []
|
||||||
|
for project in project_rows:
|
||||||
|
title = str(project.title or "").strip()
|
||||||
|
name = str(project.name or "").strip()
|
||||||
|
description = str(project.description or "").strip()
|
||||||
|
if callable(similarity_scorer):
|
||||||
|
score = max(
|
||||||
|
similarity_scorer(mention, title),
|
||||||
|
similarity_scorer(mention, name),
|
||||||
|
similarity_scorer(mention, description) * 0.8,
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
lowered = mention.lower()
|
||||||
|
title_l = title.lower()
|
||||||
|
name_l = name.lower()
|
||||||
|
description_l = description.lower()
|
||||||
|
score = 0.0
|
||||||
|
if lowered and (lowered in title_l or lowered in name_l):
|
||||||
|
score = 0.9
|
||||||
|
elif lowered and lowered in description_l:
|
||||||
|
score = 0.76
|
||||||
|
if score >= 0.62:
|
||||||
|
scored_rows.append((score, project))
|
||||||
|
scored_rows.sort(key=lambda item: item[0], reverse=True)
|
||||||
|
matched_rows = [row for _, row in scored_rows[:6]]
|
||||||
|
for project in matched_rows:
|
||||||
|
title = str(project.title or "").strip().lower()
|
||||||
|
name = str(project.name or "").strip().lower()
|
||||||
|
description = str(project.description or "").strip().lower()
|
||||||
|
lowered = mention.lower()
|
||||||
|
score = 0.82
|
||||||
|
if callable(similarity_scorer):
|
||||||
|
score = max(
|
||||||
|
0.74,
|
||||||
|
similarity_scorer(mention, title) if title else 0.0,
|
||||||
|
similarity_scorer(mention, name) if name else 0.0,
|
||||||
|
(similarity_scorer(mention, description) * 0.8) if description else 0.0,
|
||||||
|
)
|
||||||
|
elif lowered and (lowered in title or lowered in name):
|
||||||
|
score = 0.91
|
||||||
|
elif lowered and lowered in description:
|
||||||
|
score = 0.86
|
||||||
|
candidates.append(
|
||||||
|
{
|
||||||
|
"target_entity_type": "project",
|
||||||
|
"target_ref": f"project:{int(project.id)}",
|
||||||
|
"resolved_label": str(project.title or project.name or f"项目#{int(project.id)}"),
|
||||||
|
"score": round(score, 4),
|
||||||
|
"source_kind": "project_db",
|
||||||
|
"match_label": mention,
|
||||||
|
"source_confidence": round(score, 4),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return candidates
|
||||||
|
|
||||||
|
def _collect_feature_target_candidates(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
question: str,
|
||||||
|
mentions: list[str],
|
||||||
|
system_snapshot: Optional[dict[str, Any]],
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
matcher = getattr(self, "_match_feature_catalog_entries", None)
|
||||||
|
if not callable(matcher):
|
||||||
|
return []
|
||||||
|
search_text = " ".join(mentions[:2]) if mentions else question
|
||||||
|
rows = matcher(
|
||||||
|
question=search_text,
|
||||||
|
system_snapshot=system_snapshot,
|
||||||
|
limit=4,
|
||||||
|
)
|
||||||
|
candidates: list[dict[str, Any]] = []
|
||||||
|
for row in rows:
|
||||||
|
name = str(row.get("name") or "").strip()
|
||||||
|
route = str(row.get("route") or "").strip()
|
||||||
|
match_score = max(0.0, min(1.0, float(row.get("match_score") or 0.0)))
|
||||||
|
if not name and not route:
|
||||||
|
continue
|
||||||
|
score = min(0.92, max(0.72, 0.62 + match_score * 0.25))
|
||||||
|
target_key = route or name
|
||||||
|
candidates.append(
|
||||||
|
{
|
||||||
|
"target_entity_type": "feature",
|
||||||
|
"target_ref": f"feature:{target_key}",
|
||||||
|
"resolved_label": name or target_key,
|
||||||
|
"score": round(score, 4),
|
||||||
|
"source_kind": "feature_catalog",
|
||||||
|
"match_label": name or target_key,
|
||||||
|
"source_confidence": round(match_score or score, 4),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return candidates
|
||||||
|
|
||||||
|
def _collect_config_target_candidates(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
query_semantics: Optional[dict[str, Any]],
|
||||||
|
operation_contract: dict[str, Any],
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
semantics = query_semantics if isinstance(query_semantics, dict) else {}
|
||||||
|
query_profile = semantics.get("query_profile") if isinstance(semantics.get("query_profile"), dict) else {}
|
||||||
|
referenced_objects = {
|
||||||
|
str(item or "").strip().lower()
|
||||||
|
for item in list(query_profile.get("referenced_objects") or [])
|
||||||
|
if str(item or "").strip()
|
||||||
|
}
|
||||||
|
if not {"model_routing", "system_config"}.intersection(referenced_objects):
|
||||||
|
return []
|
||||||
|
mentions = list(operation_contract.get("object_mentions") or [])
|
||||||
|
rows: list[dict[str, Any]] = []
|
||||||
|
if "model_routing" in referenced_objects:
|
||||||
|
rows.append(
|
||||||
|
{
|
||||||
|
"target_entity_type": "config",
|
||||||
|
"target_ref": "config:model_routing",
|
||||||
|
"resolved_label": mentions[0] if mentions else "模型配置",
|
||||||
|
"score": 0.9,
|
||||||
|
"source_kind": "system_config",
|
||||||
|
"match_label": mentions[0] if mentions else "模型配置",
|
||||||
|
"source_confidence": 0.9,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
elif "system_config" in referenced_objects:
|
||||||
|
rows.append(
|
||||||
|
{
|
||||||
|
"target_entity_type": "config",
|
||||||
|
"target_ref": "config:system",
|
||||||
|
"resolved_label": mentions[0] if mentions else "系统配置",
|
||||||
|
"score": 0.86,
|
||||||
|
"source_kind": "system_config",
|
||||||
|
"match_label": mentions[0] if mentions else "系统配置",
|
||||||
|
"source_confidence": 0.86,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return rows
|
||||||
|
|
||||||
|
async def _collect_kg_target_candidates(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
question: str,
|
||||||
|
mentions: list[str],
|
||||||
|
project_id: Optional[int],
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
resolver = getattr(self, "entity_resolution", None)
|
||||||
|
if not callable(resolver):
|
||||||
|
return []
|
||||||
|
resolution = await resolver(
|
||||||
|
question=" ".join(mentions[:2]) if mentions else question,
|
||||||
|
project_id=project_id,
|
||||||
|
limit=6,
|
||||||
|
)
|
||||||
|
matched_entities = resolution.get("matched_entities") if isinstance(resolution, dict) else []
|
||||||
|
if not isinstance(matched_entities, list) or not matched_entities:
|
||||||
|
return []
|
||||||
|
ids: list[int] = []
|
||||||
|
matched_score: dict[int, float] = {}
|
||||||
|
for row in matched_entities:
|
||||||
|
if not isinstance(row, dict):
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
entity_id = int(row.get("entity_id"))
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
ids.append(entity_id)
|
||||||
|
matched_score[entity_id] = max(
|
||||||
|
float(matched_score.get(entity_id) or 0.0),
|
||||||
|
float(row.get("score") or 0.0),
|
||||||
|
)
|
||||||
|
if not ids:
|
||||||
|
return []
|
||||||
|
rows = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(KGEntity).where(
|
||||||
|
and_(
|
||||||
|
KGEntity.tenant_id == self.tenant_id,
|
||||||
|
KGEntity.id.in_(ids),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
candidates: list[dict[str, Any]] = []
|
||||||
|
for row in rows:
|
||||||
|
payload = self._safe_dict(row.payload) if hasattr(self, "_safe_dict") else (row.payload if isinstance(row.payload, dict) else {})
|
||||||
|
entity_score = max(0.0, min(1.0, float(matched_score.get(int(row.id)) or 0.0)))
|
||||||
|
project_ref = payload.get("project_id")
|
||||||
|
if project_ref is not None:
|
||||||
|
try:
|
||||||
|
project_ref = int(project_ref)
|
||||||
|
except Exception:
|
||||||
|
project_ref = None
|
||||||
|
if project_ref is not None:
|
||||||
|
candidates.append(
|
||||||
|
{
|
||||||
|
"target_entity_type": "project",
|
||||||
|
"target_ref": f"project:{int(project_ref)}",
|
||||||
|
"resolved_label": str(row.display_name or row.entity_key or f"项目#{int(project_ref)}"),
|
||||||
|
"score": round(min(0.89, max(0.68, 0.58 + entity_score * 0.25)), 4),
|
||||||
|
"source_kind": "knowledge_graph",
|
||||||
|
"match_label": str(row.display_name or row.entity_key or "").strip() or None,
|
||||||
|
"source_confidence": round(entity_score or 0.68, 4),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return candidates
|
||||||
|
|
||||||
|
async def _collect_internal_target_candidates(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
question: str,
|
||||||
|
project_id: Optional[int],
|
||||||
|
query_semantics: Optional[dict[str, Any]],
|
||||||
|
system_snapshot: Optional[dict[str, Any]],
|
||||||
|
operation_contract: dict[str, Any],
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
_ = query_semantics
|
||||||
|
mentions = list(operation_contract.get("object_mentions") or [])
|
||||||
|
providers = await asyncio.gather(
|
||||||
|
self._collect_semantic_fact_target_candidates(
|
||||||
|
mentions=mentions,
|
||||||
|
project_id=project_id,
|
||||||
|
operation_contract=operation_contract,
|
||||||
|
),
|
||||||
|
self._collect_project_target_candidates(
|
||||||
|
mentions=mentions,
|
||||||
|
project_id=project_id,
|
||||||
|
),
|
||||||
|
self._collect_kg_target_candidates(
|
||||||
|
question=question,
|
||||||
|
mentions=mentions,
|
||||||
|
project_id=project_id,
|
||||||
|
),
|
||||||
|
)
|
||||||
|
rows: list[dict[str, Any]] = []
|
||||||
|
for bucket in providers:
|
||||||
|
for item in bucket:
|
||||||
|
normalized = self._normalize_target_candidate(item)
|
||||||
|
if normalized is not None:
|
||||||
|
rows.append(normalized)
|
||||||
|
for item in self._collect_feature_target_candidates(
|
||||||
|
question=question,
|
||||||
|
mentions=mentions,
|
||||||
|
system_snapshot=system_snapshot,
|
||||||
|
):
|
||||||
|
normalized = self._normalize_target_candidate(item)
|
||||||
|
if normalized is not None:
|
||||||
|
rows.append(normalized)
|
||||||
|
for item in self._collect_config_target_candidates(
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
operation_contract=operation_contract,
|
||||||
|
):
|
||||||
|
normalized = self._normalize_target_candidate(item)
|
||||||
|
if normalized is not None:
|
||||||
|
rows.append(normalized)
|
||||||
|
|
||||||
|
deduped: dict[str, dict[str, Any]] = {}
|
||||||
|
for row in rows:
|
||||||
|
key = self._target_candidate_identity(row)
|
||||||
|
existing = deduped.get(key)
|
||||||
|
if existing is None or float(row.get("score") or 0.0) > float(existing.get("score") or 0.0):
|
||||||
|
deduped[key] = row
|
||||||
|
return sorted(deduped.values(), key=lambda item: float(item.get("score") or 0.0), reverse=True)[:8]
|
||||||
|
|
||||||
|
async def _rerank_internal_target_candidates(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
question: str,
|
||||||
|
operation_contract: dict[str, Any],
|
||||||
|
candidates: list[dict[str, Any]],
|
||||||
|
) -> Optional[dict[str, Any]]:
|
||||||
|
if len(candidates) < 2 or not hasattr(self, "llm"):
|
||||||
|
return None
|
||||||
|
prompt = PROMPT_INTERNAL_TARGET_RESOLUTION.format(
|
||||||
|
question=str(question or "")[:600],
|
||||||
|
operation_contract=json.dumps(operation_contract, ensure_ascii=False)[:1000],
|
||||||
|
candidates=json.dumps(
|
||||||
|
[
|
||||||
|
{
|
||||||
|
"index": idx,
|
||||||
|
"target_entity_type": str(item.get("target_entity_type") or ""),
|
||||||
|
"target_ref": str(item.get("target_ref") or ""),
|
||||||
|
"resolved_label": str(item.get("resolved_label") or ""),
|
||||||
|
"score": float(item.get("score") or 0.0),
|
||||||
|
"source_kind": str(item.get("source_kind") or ""),
|
||||||
|
}
|
||||||
|
for idx, item in enumerate(candidates[:4])
|
||||||
|
],
|
||||||
|
ensure_ascii=False,
|
||||||
|
),
|
||||||
|
).strip()
|
||||||
|
try:
|
||||||
|
raw = await self.llm.generate(
|
||||||
|
prompt,
|
||||||
|
db=self.db,
|
||||||
|
tenant_id=self.tenant_id,
|
||||||
|
function_name="internal_target_resolution",
|
||||||
|
temperature=0.0,
|
||||||
|
max_tokens=220,
|
||||||
|
use_prompt_profile=False,
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
parser = getattr(self, "_parse_json_object_safe", None)
|
||||||
|
parsed = parser(raw) if callable(parser) else {}
|
||||||
|
if not isinstance(parsed, dict):
|
||||||
|
return None
|
||||||
|
if bool(parsed.get("clarify")):
|
||||||
|
return {
|
||||||
|
"target_entity_type": "unknown",
|
||||||
|
"target_ref": None,
|
||||||
|
"resolved_label": None,
|
||||||
|
"resolution_mode": "clarify",
|
||||||
|
"confidence": round(max(0.0, min(1.0, float(parsed.get("confidence") or 0.0))), 4),
|
||||||
|
"candidate_summary": candidates[:4],
|
||||||
|
}
|
||||||
|
try:
|
||||||
|
selected_index = int(parsed.get("selected_index"))
|
||||||
|
except Exception:
|
||||||
|
return None
|
||||||
|
if selected_index < 0 or selected_index >= len(candidates[:4]):
|
||||||
|
return None
|
||||||
|
selected = dict(candidates[selected_index])
|
||||||
|
return {
|
||||||
|
"target_entity_type": str(selected.get("target_entity_type") or "unknown"),
|
||||||
|
"target_ref": selected.get("target_ref"),
|
||||||
|
"resolved_label": selected.get("resolved_label"),
|
||||||
|
"resolution_mode": "llm_rerank",
|
||||||
|
"confidence": round(max(0.0, min(1.0, float(parsed.get("confidence") or selected.get("score") or 0.0))), 4),
|
||||||
|
"candidate_summary": candidates[:4],
|
||||||
|
}
|
||||||
|
|
||||||
|
def _build_semantic_filter_target(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
query_semantics: Optional[dict[str, Any]],
|
||||||
|
operation_contract: dict[str, Any],
|
||||||
|
) -> Optional[dict[str, Any]]:
|
||||||
|
semantics = query_semantics if isinstance(query_semantics, dict) else {}
|
||||||
|
subject_focus = str(semantics.get("subject_focus") or "unknown").strip().lower()
|
||||||
|
query_profile = semantics.get("query_profile") if isinstance(semantics.get("query_profile"), dict) else {}
|
||||||
|
mentions = list(operation_contract.get("object_mentions") or [])
|
||||||
|
resolved_label = mentions[0] if mentions else None
|
||||||
|
referenced_objects = {
|
||||||
|
str(item or "").strip().lower()
|
||||||
|
for item in list(query_profile.get("referenced_objects") or [])
|
||||||
|
if str(item or "").strip()
|
||||||
|
}
|
||||||
|
|
||||||
|
if {"model_routing", "system_config"}.intersection(referenced_objects):
|
||||||
|
target_ref = "config:model_routing" if "model_routing" in referenced_objects else "config:system"
|
||||||
|
return {
|
||||||
|
"target_entity_type": "config",
|
||||||
|
"target_ref": target_ref,
|
||||||
|
"resolved_label": resolved_label or ("模型配置" if target_ref.endswith("model_routing") else "系统配置"),
|
||||||
|
"resolution_mode": "deterministic",
|
||||||
|
"confidence": 0.84,
|
||||||
|
"candidate_summary": [],
|
||||||
|
}
|
||||||
|
if subject_focus == "project":
|
||||||
|
return {
|
||||||
|
"target_entity_type": "project",
|
||||||
|
"target_ref": None,
|
||||||
|
"resolved_label": resolved_label,
|
||||||
|
"resolution_mode": "semantic_filter",
|
||||||
|
"confidence": 0.78 if resolved_label else 0.64,
|
||||||
|
"candidate_summary": [],
|
||||||
|
}
|
||||||
|
if subject_focus == "feature" or "feature_catalog" in referenced_objects:
|
||||||
|
return {
|
||||||
|
"target_entity_type": "feature",
|
||||||
|
"target_ref": None,
|
||||||
|
"resolved_label": resolved_label,
|
||||||
|
"resolution_mode": "semantic_filter",
|
||||||
|
"confidence": 0.74 if resolved_label else 0.62,
|
||||||
|
"candidate_summary": [],
|
||||||
|
}
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def _resolve_internal_target(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
question: str,
|
||||||
|
project_id: Optional[int],
|
||||||
|
query_semantics: Optional[dict[str, Any]],
|
||||||
|
system_snapshot: Optional[dict[str, Any]],
|
||||||
|
operation_contract: Optional[dict[str, Any]],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
semantics = query_semantics if isinstance(query_semantics, dict) else {}
|
||||||
|
operation = operation_contract if isinstance(operation_contract, dict) else {}
|
||||||
|
source_policy = str(
|
||||||
|
semantics.get("effective_source_policy")
|
||||||
|
or semantics.get("source_policy")
|
||||||
|
or ""
|
||||||
|
).strip().lower()
|
||||||
|
if source_policy != "internal":
|
||||||
|
return {
|
||||||
|
"target_entity_type": "unknown",
|
||||||
|
"target_ref": None,
|
||||||
|
"resolved_label": None,
|
||||||
|
"resolution_mode": "none",
|
||||||
|
"confidence": 0.0,
|
||||||
|
"candidate_summary": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
candidates = await self._collect_internal_target_candidates(
|
||||||
|
question=question,
|
||||||
|
project_id=project_id,
|
||||||
|
query_semantics=semantics,
|
||||||
|
system_snapshot=system_snapshot,
|
||||||
|
operation_contract=operation,
|
||||||
|
)
|
||||||
|
if not candidates:
|
||||||
|
fallback = self._build_semantic_filter_target(
|
||||||
|
query_semantics=semantics,
|
||||||
|
operation_contract=operation,
|
||||||
|
)
|
||||||
|
if fallback is not None:
|
||||||
|
return fallback
|
||||||
|
return {
|
||||||
|
"target_entity_type": "unknown",
|
||||||
|
"target_ref": None,
|
||||||
|
"resolved_label": None,
|
||||||
|
"resolution_mode": "clarify",
|
||||||
|
"confidence": 0.0,
|
||||||
|
"candidate_summary": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
top = candidates[0]
|
||||||
|
second = candidates[1] if len(candidates) > 1 else None
|
||||||
|
margin = float(top.get("score") or 0.0) - float(second.get("score") or 0.0) if second else 1.0
|
||||||
|
if second and str(top.get("target_entity_type") or "") != str(second.get("target_entity_type") or "") and margin < 0.1:
|
||||||
|
reranked = await self._rerank_internal_target_candidates(
|
||||||
|
question=question,
|
||||||
|
operation_contract=operation,
|
||||||
|
candidates=candidates,
|
||||||
|
)
|
||||||
|
if reranked is not None:
|
||||||
|
return reranked
|
||||||
|
if float(top.get("score") or 0.0) >= 0.88 or margin >= 0.12 or second is None:
|
||||||
|
return {
|
||||||
|
"target_entity_type": str(top.get("target_entity_type") or "unknown"),
|
||||||
|
"target_ref": top.get("target_ref"),
|
||||||
|
"resolved_label": top.get("resolved_label"),
|
||||||
|
"resolution_mode": "deterministic",
|
||||||
|
"confidence": round(max(0.0, min(1.0, float(top.get("score") or 0.0))), 4),
|
||||||
|
"candidate_summary": candidates[:4],
|
||||||
|
}
|
||||||
|
if str(top.get("target_entity_type") or "") == str(second.get("target_entity_type") or ""):
|
||||||
|
return {
|
||||||
|
"target_entity_type": str(top.get("target_entity_type") or "unknown"),
|
||||||
|
"target_ref": top.get("target_ref"),
|
||||||
|
"resolved_label": top.get("resolved_label"),
|
||||||
|
"resolution_mode": "deterministic",
|
||||||
|
"confidence": round(max(0.0, min(1.0, float(top.get("score") or 0.0))), 4),
|
||||||
|
"candidate_summary": candidates[:4],
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
"target_entity_type": "unknown",
|
||||||
|
"target_ref": None,
|
||||||
|
"resolved_label": None,
|
||||||
|
"resolution_mode": "clarify",
|
||||||
|
"confidence": round(max(0.0, min(1.0, float(top.get("score") or 0.0))), 4),
|
||||||
|
"candidate_summary": candidates[:4],
|
||||||
|
}
|
||||||
|
|
||||||
|
def _patch_scene_semantics_with_target_resolution(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
scene_semantics: Optional[dict[str, Any]],
|
||||||
|
operation_contract: Optional[dict[str, Any]],
|
||||||
|
target_resolution: Optional[dict[str, Any]],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
normalized = dict(scene_semantics or {})
|
||||||
|
query_type = str((operation_contract or {}).get("query_type") or QUERY_TYPE_UNKNOWN).strip().lower()
|
||||||
|
target_entity_type = str((target_resolution or {}).get("target_entity_type") or "unknown").strip().lower()
|
||||||
|
snapshot_focus = str(normalized.get("snapshot_focus") or "none").strip().lower()
|
||||||
|
if query_type == QUERY_TYPE_EXISTENCE and target_entity_type == "project":
|
||||||
|
normalized["snapshot_focus"] = "none"
|
||||||
|
return normalized
|
||||||
|
if target_entity_type not in {"feature", "config"} and snapshot_focus == "feature":
|
||||||
|
normalized["snapshot_focus"] = "none"
|
||||||
|
return normalized
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
def _should_allow_snapshot_direct_answer(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
scene_semantics: Optional[dict[str, Any]],
|
||||||
|
operation_contract: Optional[dict[str, Any]],
|
||||||
|
target_resolution: Optional[dict[str, Any]],
|
||||||
|
) -> bool:
|
||||||
|
snapshot_focus = str((scene_semantics or {}).get("snapshot_focus") or "none").strip().lower()
|
||||||
|
query_type = str((operation_contract or {}).get("query_type") or QUERY_TYPE_UNKNOWN).strip().lower()
|
||||||
|
target_entity_type = str((target_resolution or {}).get("target_entity_type") or "unknown").strip().lower()
|
||||||
|
if snapshot_focus in {"identity", "architecture"}:
|
||||||
|
return query_type in {QUERY_TYPE_EXPLAIN, QUERY_TYPE_STATUS}
|
||||||
|
if snapshot_focus == "feature":
|
||||||
|
return query_type in {QUERY_TYPE_EXPLAIN, QUERY_TYPE_STATUS, QUERY_TYPE_LIST} and target_entity_type in {
|
||||||
|
"feature",
|
||||||
|
"config",
|
||||||
|
"unknown",
|
||||||
|
}
|
||||||
|
return False
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _build_operation_contract_summary(operation_contract: Optional[dict[str, Any]]) -> dict[str, Any]:
|
||||||
|
payload = operation_contract if isinstance(operation_contract, dict) else {}
|
||||||
|
mentions = [
|
||||||
|
str(item or "").strip()
|
||||||
|
for item in list(payload.get("object_mentions") or [])
|
||||||
|
if str(item or "").strip()
|
||||||
|
]
|
||||||
|
return {
|
||||||
|
"query_type": str(payload.get("query_type") or QUERY_TYPE_UNKNOWN).strip().lower() or QUERY_TYPE_UNKNOWN,
|
||||||
|
"answer_contract": str(payload.get("answer_contract") or "clarify_or_fail").strip() or "clarify_or_fail",
|
||||||
|
"object_mentions": mentions[:4],
|
||||||
|
"clarify_policy": str(payload.get("clarify_policy") or "").strip() or None,
|
||||||
|
}
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _build_target_resolution_summary(target_resolution: Optional[dict[str, Any]]) -> dict[str, Any]:
|
||||||
|
payload = target_resolution if isinstance(target_resolution, dict) else {}
|
||||||
|
candidates: list[dict[str, Any]] = []
|
||||||
|
for item in list(payload.get("candidate_summary") or [])[:4]:
|
||||||
|
if not isinstance(item, dict):
|
||||||
|
continue
|
||||||
|
candidates.append(
|
||||||
|
{
|
||||||
|
"target_entity_type": str(item.get("target_entity_type") or "unknown").strip().lower() or "unknown",
|
||||||
|
"target_ref": str(item.get("target_ref") or "").strip() or None,
|
||||||
|
"resolved_label": str(item.get("resolved_label") or "").strip() or None,
|
||||||
|
"score": round(max(0.0, min(1.0, float(item.get("score") or 0.0))), 4),
|
||||||
|
"source_kind": str(item.get("source_kind") or "unknown").strip().lower() or "unknown",
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"target_entity_type": str(payload.get("target_entity_type") or "unknown").strip().lower() or "unknown",
|
||||||
|
"target_ref": str(payload.get("target_ref") or "").strip() or None,
|
||||||
|
"resolved_label": str(payload.get("resolved_label") or "").strip() or None,
|
||||||
|
"resolution_mode": str(payload.get("resolution_mode") or "none").strip().lower() or "none",
|
||||||
|
"confidence": round(max(0.0, min(1.0, float(payload.get("confidence") or 0.0))), 4),
|
||||||
|
"candidate_summary": candidates,
|
||||||
|
}
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _build_query_plan_summary(query_plan: Optional[dict[str, Any]]) -> dict[str, Any]:
|
||||||
|
payload = query_plan if isinstance(query_plan, dict) else {}
|
||||||
|
filters: list[dict[str, Any]] = []
|
||||||
|
for item in list(payload.get("filters") or [])[:4]:
|
||||||
|
if not isinstance(item, dict):
|
||||||
|
continue
|
||||||
|
filters.append(
|
||||||
|
{
|
||||||
|
"field": str(item.get("field") or "").strip() or None,
|
||||||
|
"operator": str(item.get("operator") or "").strip() or None,
|
||||||
|
"value": item.get("value"),
|
||||||
|
"value_source": str(item.get("value_source") or "").strip() or None,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return {
|
||||||
|
"plan_version": str(payload.get("plan_version") or "unknown").strip() or "unknown",
|
||||||
|
"query_type": str(payload.get("query_type") or QUERY_TYPE_UNKNOWN).strip().lower() or QUERY_TYPE_UNKNOWN,
|
||||||
|
"entity_type": str(payload.get("entity_type") or "unknown").strip().lower() or "unknown",
|
||||||
|
"target_ref": str(payload.get("target_ref") or "").strip() or None,
|
||||||
|
"answer_contract": str(payload.get("answer_contract") or "").strip() or None,
|
||||||
|
"filters": filters,
|
||||||
|
}
|
||||||
|
|
||||||
|
def _inject_internal_query_runtime_summaries(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
retrieval_metrics: Optional[dict[str, Any]],
|
||||||
|
query_semantics: Optional[dict[str, Any]],
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
metrics = dict(retrieval_metrics or {})
|
||||||
|
semantics = query_semantics if isinstance(query_semantics, dict) else {}
|
||||||
|
metrics["operation_contract_summary"] = self._build_operation_contract_summary(semantics.get("operation_contract"))
|
||||||
|
metrics["target_resolution_summary"] = self._build_target_resolution_summary(semantics.get("target_resolution"))
|
||||||
|
metrics["query_plan_summary"] = self._build_query_plan_summary(semantics.get("query_plan"))
|
||||||
|
return metrics
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["AdaptiveRAGTargetResolutionMixin"]
|
||||||
|
|
@ -40,12 +40,15 @@ from .mixin_conversation_utils_a_p1 import AdaptiveRAGConversationUtilsAMixinP1
|
||||||
from .mixin_conversation_utils_a_p2 import AdaptiveRAGConversationUtilsAMixinP2
|
from .mixin_conversation_utils_a_p2 import AdaptiveRAGConversationUtilsAMixinP2
|
||||||
from .mixin_conversation_utils_b import AdaptiveRAGConversationUtilsBMixin
|
from .mixin_conversation_utils_b import AdaptiveRAGConversationUtilsBMixin
|
||||||
from .mixin_followup_anchor import AdaptiveRAGFollowupAnchorMixin
|
from .mixin_followup_anchor import AdaptiveRAGFollowupAnchorMixin
|
||||||
|
from .mixin_query_route_gateway import AdaptiveRAGQueryRouteGatewayMixin
|
||||||
|
from .mixin_target_resolution import AdaptiveRAGTargetResolutionMixin
|
||||||
|
|
||||||
|
|
||||||
class AdaptiveRAGService(
|
class AdaptiveRAGService(
|
||||||
AdaptiveRAGCoreIntentMixin,
|
AdaptiveRAGCoreIntentMixin,
|
||||||
AdaptiveRAGAnswerEntryMixin,
|
AdaptiveRAGAnswerEntryMixin,
|
||||||
AdaptiveRAGAnswerMainlineMixin,
|
AdaptiveRAGAnswerMainlineMixin,
|
||||||
|
AdaptiveRAGTargetResolutionMixin,
|
||||||
AdaptiveRAGAnswerPrecheckMixin,
|
AdaptiveRAGAnswerPrecheckMixin,
|
||||||
AdaptiveRAGAnswerRetrievalMixin,
|
AdaptiveRAGAnswerRetrievalMixin,
|
||||||
AdaptiveRAGAnswerPayloadMixin,
|
AdaptiveRAGAnswerPayloadMixin,
|
||||||
|
|
@ -82,6 +85,7 @@ class AdaptiveRAGService(
|
||||||
AdaptiveRAGConversationUtilsAMixinP2,
|
AdaptiveRAGConversationUtilsAMixinP2,
|
||||||
AdaptiveRAGConversationUtilsBMixin,
|
AdaptiveRAGConversationUtilsBMixin,
|
||||||
AdaptiveRAGFollowupAnchorMixin,
|
AdaptiveRAGFollowupAnchorMixin,
|
||||||
|
AdaptiveRAGQueryRouteGatewayMixin,
|
||||||
):
|
):
|
||||||
"""项目内统一 RAG 服务。"""
|
"""项目内统一 RAG 服务。"""
|
||||||
_symbol_snapshot_cache: dict[str, tuple[float, dict[str, Any]]] = {}
|
_symbol_snapshot_cache: dict[str, tuple[float, dict[str, Any]]] = {}
|
||||||
|
|
|
||||||
|
|
@ -12,6 +12,7 @@ from __future__ import annotations
|
||||||
|
|
||||||
import ast
|
import ast
|
||||||
import asyncio
|
import asyncio
|
||||||
|
from difflib import SequenceMatcher
|
||||||
import hashlib
|
import hashlib
|
||||||
import json
|
import json
|
||||||
import math
|
import math
|
||||||
|
|
@ -19,6 +20,7 @@ import os
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
import re
|
import re
|
||||||
import time
|
import time
|
||||||
|
from urllib.parse import urlparse
|
||||||
from dataclasses import dataclass
|
from dataclasses import dataclass
|
||||||
from datetime import datetime, timezone
|
from datetime import datetime, timezone
|
||||||
from typing import Any, Optional
|
from typing import Any, Optional
|
||||||
|
|
@ -41,6 +43,7 @@ from ...config.ai_prompts import (
|
||||||
PROMPT_RAG_FOLLOWUP_ANCHOR_SEMANTICS,
|
PROMPT_RAG_FOLLOWUP_ANCHOR_SEMANTICS,
|
||||||
PROMPT_RAG_QUERY_SEMANTICS,
|
PROMPT_RAG_QUERY_SEMANTICS,
|
||||||
PROMPT_RAG_ANSWER_SCENE_SEMANTICS,
|
PROMPT_RAG_ANSWER_SCENE_SEMANTICS,
|
||||||
|
PROMPT_INTERNAL_TARGET_RESOLUTION,
|
||||||
)
|
)
|
||||||
from ...core.database import AsyncSessionLocal
|
from ...core.database import AsyncSessionLocal
|
||||||
from ...models import (
|
from ...models import (
|
||||||
|
|
@ -56,8 +59,10 @@ from ...models import (
|
||||||
ProjectSignalLink,
|
ProjectSignalLink,
|
||||||
QAFeedback,
|
QAFeedback,
|
||||||
QARecord,
|
QARecord,
|
||||||
|
QueryRouteMemory,
|
||||||
QASession,
|
QASession,
|
||||||
QASessionState,
|
QASessionState,
|
||||||
|
SemanticFact,
|
||||||
Signal,
|
Signal,
|
||||||
Task,
|
Task,
|
||||||
TruthNode,
|
TruthNode,
|
||||||
|
|
|
||||||
|
|
@ -24,8 +24,11 @@ DEEP_CONTEXT_BLOCKS = _env_int("RAG_DEEP_CONTEXT_BLOCKS", 16, minimum=8, maximum
|
||||||
DEEP_ANSWER_MAX_TOKENS = _env_int("RAG_DEEP_ANSWER_MAX_TOKENS", 1400, minimum=900, maximum=3000)
|
DEEP_ANSWER_MAX_TOKENS = _env_int("RAG_DEEP_ANSWER_MAX_TOKENS", 1400, minimum=900, maximum=3000)
|
||||||
QUERY_TYPE_COUNT = "count"
|
QUERY_TYPE_COUNT = "count"
|
||||||
QUERY_TYPE_LIST = "list"
|
QUERY_TYPE_LIST = "list"
|
||||||
|
QUERY_TYPE_EXISTENCE = "existence"
|
||||||
QUERY_TYPE_STATUS = "status"
|
QUERY_TYPE_STATUS = "status"
|
||||||
QUERY_TYPE_EXPLAIN = "explain"
|
QUERY_TYPE_EXPLAIN = "explain"
|
||||||
|
QUERY_TYPE_DIAGNOSE = "diagnose"
|
||||||
|
QUERY_TYPE_COMPARE = "compare"
|
||||||
QUERY_TYPE_UNKNOWN = "unknown"
|
QUERY_TYPE_UNKNOWN = "unknown"
|
||||||
ANSWER_MODE_FAST_PATH = "fast_path"
|
ANSWER_MODE_FAST_PATH = "fast_path"
|
||||||
ANSWER_MODE_RETRIEVAL_NORMAL = "retrieval_normal"
|
ANSWER_MODE_RETRIEVAL_NORMAL = "retrieval_normal"
|
||||||
|
|
|
||||||
|
|
@ -477,10 +477,15 @@ class ReplyRenderer:
|
||||||
}
|
}
|
||||||
if intent.name == INTENT_QUERY:
|
if intent.name == INTENT_QUERY:
|
||||||
answer = str(execution_result.get("answer") or "当前暂无可用回答。").strip()
|
answer = str(execution_result.get("answer") or "当前暂无可用回答。").strip()
|
||||||
|
template_key = str(execution_result.get("reply_template_key") or "").strip() or "action_succeeded"
|
||||||
return {
|
return {
|
||||||
"receipt_phase": self.receipt_phase_action,
|
"receipt_phase": self.receipt_phase_action,
|
||||||
"template_key": "action_succeeded",
|
"template_key": template_key,
|
||||||
"payload": {"intent": intent.name, "answer": answer},
|
"payload": {
|
||||||
|
"intent": intent.name,
|
||||||
|
"answer": answer,
|
||||||
|
"reply_template_key": template_key,
|
||||||
|
},
|
||||||
"text": answer,
|
"text": answer,
|
||||||
}
|
}
|
||||||
if intent.name == INTENT_PROJECT_CREATE:
|
if intent.name == INTENT_PROJECT_CREATE:
|
||||||
|
|
|
||||||
|
|
@ -179,12 +179,6 @@ class KernelIntentResolveMixin:
|
||||||
"requires_confirmation": False,
|
"requires_confirmation": False,
|
||||||
"source_system": str(source_system or "").strip().lower() or None,
|
"source_system": str(source_system or "").strip().lower() or None,
|
||||||
"source_message_context": "reply_anchor" if reply_anchor_present else "cold_start",
|
"source_message_context": "reply_anchor" if reply_anchor_present else "cold_start",
|
||||||
# Planner outage should not hard-fail the thread. Degrade to the
|
|
||||||
# controlled requirement consultation chain so downstream logic can
|
|
||||||
# keep the user inside the requirement workflow instead of emitting
|
|
||||||
# a generic error template.
|
|
||||||
"message_intent": "consult_requirement",
|
|
||||||
"consultant_mode_preference": "clarify",
|
|
||||||
"semantic_planner_degraded": True,
|
"semantic_planner_degraded": True,
|
||||||
"semantic_planner_error": str(planner_error)[:240],
|
"semantic_planner_error": str(planner_error)[:240],
|
||||||
}
|
}
|
||||||
|
|
|
||||||
|
|
@ -32,97 +32,20 @@ class KernelIngressFlowEntryMixin:
|
||||||
"requirement_gathering",
|
"requirement_gathering",
|
||||||
"requirement_convergence",
|
"requirement_convergence",
|
||||||
})
|
})
|
||||||
_EXTERNAL_FRESHNESS_FOLLOWUP_TOKENS = frozenset({
|
_EXTERNAL_FRESHNESS_GUARD_SOURCE_LABEL = "externalfreshnessguard"
|
||||||
"继续核实",
|
|
||||||
"继续查",
|
|
||||||
"继续验证",
|
|
||||||
"继续确认",
|
|
||||||
"接入外部来源",
|
|
||||||
"外部来源继续核实",
|
|
||||||
"联网查",
|
|
||||||
"联网核实",
|
|
||||||
"外网查",
|
|
||||||
"实时来源",
|
|
||||||
"查最新",
|
|
||||||
"再核实",
|
|
||||||
})
|
|
||||||
_INTERNAL_EXTERNAL_SOURCE_TARGET_MARKERS = frozenset({
|
|
||||||
"我们系统",
|
|
||||||
"系统里",
|
|
||||||
"系统内",
|
|
||||||
"pmflow",
|
|
||||||
"/ai-config",
|
|
||||||
"ai-config",
|
|
||||||
"aiconfig",
|
|
||||||
"配置页",
|
|
||||||
"模型供应商",
|
|
||||||
"给项目",
|
|
||||||
"项目id",
|
|
||||||
"项目#",
|
|
||||||
"project",
|
|
||||||
"模块",
|
|
||||||
"功能",
|
|
||||||
"接口",
|
|
||||||
"通道",
|
|
||||||
"租户",
|
|
||||||
"环境",
|
|
||||||
"服务",
|
|
||||||
})
|
|
||||||
_INTERNAL_EXTERNAL_SOURCE_ACTION_MARKERS = frozenset({
|
|
||||||
"出需求",
|
|
||||||
"需求草稿",
|
|
||||||
"起草",
|
|
||||||
"立项",
|
|
||||||
"工单",
|
|
||||||
"排期",
|
|
||||||
"任务",
|
|
||||||
"入库",
|
|
||||||
"接一个",
|
|
||||||
"新增",
|
|
||||||
"加一个",
|
|
||||||
"配置",
|
|
||||||
"接入通道",
|
|
||||||
"集成",
|
|
||||||
"实现",
|
|
||||||
"改造",
|
|
||||||
})
|
|
||||||
_EXTERNAL_FRESHNESS_GUARD_MARKERS = frozenset({
|
|
||||||
"externalfreshnessguard",
|
|
||||||
"未接入实时外部数据",
|
|
||||||
"实时外部来源",
|
|
||||||
"无法确认最新结论",
|
|
||||||
})
|
|
||||||
|
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _normalize_compact_text(value: Any) -> str:
|
def _normalize_compact_text(value: Any) -> str:
|
||||||
return re.sub(r"\s+", "", str(value or "")).strip().lower()
|
return re.sub(r"\s+", "", str(value or "")).strip().lower()
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def _looks_like_external_freshness_followup(cls, message_text: str) -> bool:
|
def _payload_has_external_freshness_guard_source(cls, value: Any) -> bool:
|
||||||
normalized = cls._normalize_compact_text(message_text)
|
|
||||||
if not normalized:
|
|
||||||
return False
|
|
||||||
return any(token in normalized for token in cls._EXTERNAL_FRESHNESS_FOLLOWUP_TOKENS)
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def _looks_like_explicit_internal_external_source_request(cls, message_text: str) -> bool:
|
|
||||||
normalized = cls._normalize_compact_text(message_text)
|
|
||||||
if not normalized:
|
|
||||||
return False
|
|
||||||
return (
|
|
||||||
any(token in normalized for token in cls._INTERNAL_EXTERNAL_SOURCE_TARGET_MARKERS)
|
|
||||||
and any(token in normalized for token in cls._INTERNAL_EXTERNAL_SOURCE_ACTION_MARKERS)
|
|
||||||
)
|
|
||||||
|
|
||||||
@classmethod
|
|
||||||
def _payload_mentions_external_freshness_guard(cls, value: Any) -> bool:
|
|
||||||
if isinstance(value, str):
|
if isinstance(value, str):
|
||||||
normalized = cls._normalize_compact_text(value)
|
return cls._normalize_compact_text(value) == cls._EXTERNAL_FRESHNESS_GUARD_SOURCE_LABEL
|
||||||
return any(marker in normalized for marker in cls._EXTERNAL_FRESHNESS_GUARD_MARKERS)
|
|
||||||
if isinstance(value, dict):
|
if isinstance(value, dict):
|
||||||
return any(cls._payload_mentions_external_freshness_guard(item) for item in value.values())
|
return any(cls._payload_has_external_freshness_guard_source(item) for item in value.values())
|
||||||
if isinstance(value, list):
|
if isinstance(value, list):
|
||||||
return any(cls._payload_mentions_external_freshness_guard(item) for item in value)
|
return any(cls._payload_has_external_freshness_guard_source(item) for item in value)
|
||||||
return False
|
return False
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
|
|
@ -135,12 +58,13 @@ class KernelIngressFlowEntryMixin:
|
||||||
return (
|
return (
|
||||||
(
|
(
|
||||||
grounding_status in {"insufficient_evidence", "partially_grounded"}
|
grounding_status in {"insufficient_evidence", "partially_grounded"}
|
||||||
and cls._payload_mentions_external_freshness_guard(getattr(qa_record, "sources_used", None))
|
and cls._payload_has_external_freshness_guard_source(getattr(qa_record, "sources_used", None))
|
||||||
)
|
)
|
||||||
or (
|
or (
|
||||||
"prompt:industry_trend" in policy_version or "prompt:external_trend" in policy_version
|
"prompt:industry_trend" in policy_version or "prompt:external_trend" in policy_version
|
||||||
)
|
)
|
||||||
and cls._payload_mentions_external_freshness_guard(answer_text)
|
and cls._payload_has_external_freshness_guard_source(getattr(qa_record, "sources_used", None))
|
||||||
|
and "无法确认最新结论" in answer_text
|
||||||
)
|
)
|
||||||
|
|
||||||
async def _load_recent_external_freshness_guard_record(
|
async def _load_recent_external_freshness_guard_record(
|
||||||
|
|
@ -183,34 +107,47 @@ class KernelIngressFlowEntryMixin:
|
||||||
).scalars().first()
|
).scalars().first()
|
||||||
if self._is_external_freshness_guard_record(anchored_row):
|
if self._is_external_freshness_guard_record(anchored_row):
|
||||||
return anchored_row
|
return anchored_row
|
||||||
|
|
||||||
if normalized_source != "feishu":
|
|
||||||
return None
|
|
||||||
conversation_scope_key = build_feishu_conversation_scope_key(
|
|
||||||
source_chat_id=channel_ref,
|
|
||||||
source_sender_open_id=source_sender_open_id,
|
|
||||||
sender=None,
|
|
||||||
)
|
|
||||||
if not conversation_scope_key:
|
|
||||||
return None
|
|
||||||
rows = (
|
|
||||||
await self.db.execute(
|
|
||||||
select(QARecord)
|
|
||||||
.where(
|
|
||||||
*criteria,
|
|
||||||
QARecord.conversation_scope_key == conversation_scope_key,
|
|
||||||
)
|
|
||||||
.order_by(QARecord.updated_at.desc(), QARecord.id.desc())
|
|
||||||
.limit(3)
|
|
||||||
)
|
|
||||||
).scalars().all()
|
|
||||||
for row in rows:
|
|
||||||
if str(getattr(row, "source_message_id", "") or "").strip() == str(source_message_id or "").strip():
|
|
||||||
continue
|
|
||||||
if self._is_external_freshness_guard_record(row):
|
|
||||||
return row
|
|
||||||
return None
|
return None
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _is_query_answer_only_bundle(cls, bundle: Any) -> bool:
|
||||||
|
workflows = list(getattr(bundle, "workflows", []) or [])
|
||||||
|
primary_workflow = str(getattr(bundle, "primary_workflow", "") or "").strip().lower()
|
||||||
|
if primary_workflow and primary_workflow != "query_flow":
|
||||||
|
return False
|
||||||
|
if not workflows:
|
||||||
|
return primary_workflow in {"", "query_flow"}
|
||||||
|
for workflow in workflows:
|
||||||
|
workflow_type = str(getattr(workflow, "workflow_type", "") or "").strip().lower()
|
||||||
|
if workflow_type and workflow_type != "query_flow":
|
||||||
|
return False
|
||||||
|
operation_codes = [str(code or "").strip().lower() for code in list(getattr(workflow, "operation_codes", []) or []) if str(code or "").strip()]
|
||||||
|
if operation_codes and any(code != "query.answer" for code in operation_codes):
|
||||||
|
return False
|
||||||
|
return True
|
||||||
|
|
||||||
|
async def _should_keep_external_freshness_followup_in_query_answer(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
message_text: str,
|
||||||
|
source_system: str,
|
||||||
|
prior_question: Optional[str],
|
||||||
|
thread_context: Optional[dict[str, Any]],
|
||||||
|
) -> bool:
|
||||||
|
try:
|
||||||
|
bundle = await WorkflowSemanticPlanner.aplan_message(
|
||||||
|
message_text=str(message_text or "").strip(),
|
||||||
|
source_system=str(source_system or "").strip().lower() or "adaptive_rag",
|
||||||
|
metadata={
|
||||||
|
"external_freshness_followup": True,
|
||||||
|
"prior_question": str(prior_question or "").strip()[:240] or None,
|
||||||
|
"thread_context": dict(thread_context or {}),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
return True
|
||||||
|
return self._is_query_answer_only_bundle(bundle)
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
def _build_external_freshness_followup_reply(
|
def _build_external_freshness_followup_reply(
|
||||||
cls,
|
cls,
|
||||||
|
|
@ -237,7 +174,7 @@ class KernelIngressFlowEntryMixin:
|
||||||
if not base_question:
|
if not base_question:
|
||||||
return followup_text
|
return followup_text
|
||||||
normalized_followup = cls._normalize_compact_text(followup_text)
|
normalized_followup = cls._normalize_compact_text(followup_text)
|
||||||
if not normalized_followup or cls._looks_like_external_freshness_followup(followup_text):
|
if not normalized_followup or normalized_followup == cls._normalize_compact_text(base_question):
|
||||||
return base_question
|
return base_question
|
||||||
return f"{base_question}。补充要求:{followup_text}"[:1200]
|
return f"{base_question}。补充要求:{followup_text}"[:1200]
|
||||||
|
|
||||||
|
|
@ -251,10 +188,6 @@ class KernelIngressFlowEntryMixin:
|
||||||
message_text: str,
|
message_text: str,
|
||||||
thread_context: Optional[dict[str, Any]],
|
thread_context: Optional[dict[str, Any]],
|
||||||
) -> Optional[dict[str, Any]]:
|
) -> Optional[dict[str, Any]]:
|
||||||
if not self._looks_like_external_freshness_followup(message_text):
|
|
||||||
return None
|
|
||||||
if self._looks_like_explicit_internal_external_source_request(message_text):
|
|
||||||
return None
|
|
||||||
try:
|
try:
|
||||||
guard_record = await self._load_recent_external_freshness_guard_record(
|
guard_record = await self._load_recent_external_freshness_guard_record(
|
||||||
source_system=source_system,
|
source_system=source_system,
|
||||||
|
|
@ -276,6 +209,13 @@ class KernelIngressFlowEntryMixin:
|
||||||
return None
|
return None
|
||||||
|
|
||||||
prior_question = str(getattr(guard_record, "question", "") or "").strip() or None
|
prior_question = str(getattr(guard_record, "question", "") or "").strip() or None
|
||||||
|
if not await self._should_keep_external_freshness_followup_in_query_answer(
|
||||||
|
message_text=message_text,
|
||||||
|
source_system=source_system,
|
||||||
|
prior_question=prior_question,
|
||||||
|
thread_context=thread_context,
|
||||||
|
):
|
||||||
|
return None
|
||||||
followup_question = self._build_external_freshness_followup_question(
|
followup_question = self._build_external_freshness_followup_question(
|
||||||
prior_question=prior_question,
|
prior_question=prior_question,
|
||||||
message_text=message_text,
|
message_text=message_text,
|
||||||
|
|
@ -975,37 +915,6 @@ class KernelIngressFlowEntryMixin:
|
||||||
ingest_event=ingest_event,
|
ingest_event=ingest_event,
|
||||||
ingress_options=ingress_options,
|
ingress_options=ingress_options,
|
||||||
)
|
)
|
||||||
terminated_requirement_scope = await mounted_runtime.terminate_requirement_plan_scope(
|
|
||||||
tenant_id=int(self.tenant_id),
|
|
||||||
source_system=source_system,
|
|
||||||
channel_ref=channel_ref,
|
|
||||||
sender=sender,
|
|
||||||
source_sender_open_id=source_sender_open_id,
|
|
||||||
source_message_id=synthetic_message_id,
|
|
||||||
)
|
|
||||||
if int(terminated_requirement_scope.get("terminated_count") or 0) > 0:
|
|
||||||
try:
|
|
||||||
await self._update_session(
|
|
||||||
tenant_id=int(self.tenant_id),
|
|
||||||
sender=sender,
|
|
||||||
session=session,
|
|
||||||
source_chat_id=channel_ref,
|
|
||||||
source_sender_open_id=source_sender_open_id,
|
|
||||||
source_message_id=synthetic_message_id,
|
|
||||||
thread_context=thread_context,
|
|
||||||
current_project_id=None,
|
|
||||||
clear_current_project_id=True,
|
|
||||||
last_intent=ROUTE_QUERY_ANSWER,
|
|
||||||
last_message_text=clean_text,
|
|
||||||
)
|
|
||||||
except Exception as session_exc:
|
|
||||||
logger.warning(
|
|
||||||
"mounted_ai_runtime_terminate_requirement_scope_session_update_failed",
|
|
||||||
tenant_id=self.tenant_id,
|
|
||||||
source_system=source_system,
|
|
||||||
source_message_id=synthetic_message_id,
|
|
||||||
error=str(session_exc),
|
|
||||||
)
|
|
||||||
active_context_followup_result = await self._maybe_handle_active_context_followup(
|
active_context_followup_result = await self._maybe_handle_active_context_followup(
|
||||||
source_system=source_system,
|
source_system=source_system,
|
||||||
source_message_id=synthetic_message_id,
|
source_message_id=synthetic_message_id,
|
||||||
|
|
@ -1847,6 +1756,9 @@ class KernelIngressFlowEntryMixin:
|
||||||
return topic, draft, self._derive_active_context_kind(topic=topic, draft=draft)
|
return topic, draft, self._derive_active_context_kind(topic=topic, draft=draft)
|
||||||
|
|
||||||
finder = getattr(self, "_find_active_requirement_plan_workspace", None)
|
finder = getattr(self, "_find_active_requirement_plan_workspace", None)
|
||||||
|
if not callable(finder):
|
||||||
|
executor = getattr(self, "executor", None)
|
||||||
|
finder = getattr(executor, "_find_active_requirement_plan_workspace", None)
|
||||||
if not callable(finder):
|
if not callable(finder):
|
||||||
return None
|
return None
|
||||||
topic, draft = await finder(
|
topic, draft = await finder(
|
||||||
|
|
@ -2022,9 +1934,49 @@ class KernelIngressFlowEntryMixin:
|
||||||
except Exception:
|
except Exception:
|
||||||
return False
|
return False
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _looks_like_requirement_plan_exit_request(message_text: str) -> bool:
|
||||||
|
try:
|
||||||
|
from ..ai_pmo_typed_executor_execute.mixin_execute_query_project import ExecuteQueryProjectMixin
|
||||||
|
|
||||||
|
return ExecuteQueryProjectMixin._is_requirement_plan_exit_request(message_text)
|
||||||
|
except Exception:
|
||||||
|
return False
|
||||||
|
|
||||||
|
async def _build_active_requirement_mode_exit_result(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
topic: Any,
|
||||||
|
source_message_id: str,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
target_topic_id = _safe_int(getattr(topic, "id", None))
|
||||||
|
topic.status = "suspended"
|
||||||
|
topic.plan_status = "suspended"
|
||||||
|
topic.pending_actions = ["resume_requirement_plan"]
|
||||||
|
topic.metadata_json = {
|
||||||
|
**(topic.metadata_json if isinstance(getattr(topic, "metadata_json", None), dict) else {}),
|
||||||
|
"plan_mode": True,
|
||||||
|
"suspended_by_message_id": str(source_message_id or "").strip() or None,
|
||||||
|
}
|
||||||
|
flusher = getattr(self.db, "flush", None)
|
||||||
|
if callable(flusher):
|
||||||
|
await flusher()
|
||||||
|
return self._build_active_context_direct_reply_result(
|
||||||
|
answer="好,这条需求先暂停。你后面引用原消息或说“继续需求”我再接上;现在可以聊别的。",
|
||||||
|
reason_code="requirement_mode_suspended",
|
||||||
|
payload={
|
||||||
|
"workflow_route": ROUTE_REQUIREMENT_PLAN_COLLECT,
|
||||||
|
"work_item_id": target_topic_id,
|
||||||
|
"topic_workspace_id": target_topic_id,
|
||||||
|
"group_reply_allowed_via_bound_context": True,
|
||||||
|
"query_type": "plan",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
async def _build_active_requirement_followup_result(
|
async def _build_active_requirement_followup_result(
|
||||||
self,
|
self,
|
||||||
*,
|
*,
|
||||||
|
topic_workspace_id: Optional[int],
|
||||||
message_text: str,
|
message_text: str,
|
||||||
source_system: str,
|
source_system: str,
|
||||||
source_message_id: str,
|
source_message_id: str,
|
||||||
|
|
@ -2054,6 +2006,16 @@ class KernelIngressFlowEntryMixin:
|
||||||
if not answer:
|
if not answer:
|
||||||
return None
|
return None
|
||||||
direct_payload = dict(direct_result) if isinstance(direct_result, dict) else {}
|
direct_payload = dict(direct_result) if isinstance(direct_result, dict) else {}
|
||||||
|
target_topic_id = _safe_int(topic_workspace_id)
|
||||||
|
if (
|
||||||
|
target_topic_id is not None
|
||||||
|
and _safe_int(direct_payload.get("work_item_id")) is None
|
||||||
|
and str(direct_payload.get("reply_template_key") or "").strip().lower() == "requirement_plan_followup"
|
||||||
|
):
|
||||||
|
# Requirement-plan collect cards need a stable positive target id for button round-trips.
|
||||||
|
# At collect stage there is often no dedicated work item yet, so we reuse the mounted topic id.
|
||||||
|
direct_payload["work_item_id"] = int(target_topic_id)
|
||||||
|
direct_payload.setdefault("topic_workspace_id", int(target_topic_id))
|
||||||
direct_payload["group_reply_allowed_via_bound_context"] = True
|
direct_payload["group_reply_allowed_via_bound_context"] = True
|
||||||
return self._build_active_context_direct_reply_result(
|
return self._build_active_context_direct_reply_result(
|
||||||
answer=answer,
|
answer=answer,
|
||||||
|
|
@ -2098,11 +2060,20 @@ class KernelIngressFlowEntryMixin:
|
||||||
if bound_context is None:
|
if bound_context is None:
|
||||||
return None
|
return None
|
||||||
topic, draft, active_context_kind = bound_context
|
topic, draft, active_context_kind = bound_context
|
||||||
|
if (
|
||||||
|
str(active_context_kind or "").strip().lower() == "requirement_plan"
|
||||||
|
and self._looks_like_requirement_plan_exit_request(message_text)
|
||||||
|
):
|
||||||
|
return await self._build_active_requirement_mode_exit_result(
|
||||||
|
topic=topic,
|
||||||
|
source_message_id=source_message_id,
|
||||||
|
)
|
||||||
if (
|
if (
|
||||||
str(active_context_kind or "").strip().lower() == "requirement_plan"
|
str(active_context_kind or "").strip().lower() == "requirement_plan"
|
||||||
and self._looks_like_requirement_plan_choice_protocol_followup(message_text)
|
and self._looks_like_requirement_plan_choice_protocol_followup(message_text)
|
||||||
):
|
):
|
||||||
direct_protocol_result = await self._build_active_requirement_followup_result(
|
direct_protocol_result = await self._build_active_requirement_followup_result(
|
||||||
|
topic_workspace_id=_safe_int(getattr(topic, "id", None)),
|
||||||
message_text=message_text,
|
message_text=message_text,
|
||||||
source_system=source_system,
|
source_system=source_system,
|
||||||
source_message_id=source_message_id,
|
source_message_id=source_message_id,
|
||||||
|
|
@ -2168,6 +2139,7 @@ class KernelIngressFlowEntryMixin:
|
||||||
if "context.followup.current_requirement" not in allowed_actions:
|
if "context.followup.current_requirement" not in allowed_actions:
|
||||||
return None
|
return None
|
||||||
direct_result = await self._build_active_requirement_followup_result(
|
direct_result = await self._build_active_requirement_followup_result(
|
||||||
|
topic_workspace_id=_safe_int(getattr(topic, "id", None)),
|
||||||
message_text=message_text,
|
message_text=message_text,
|
||||||
source_system=source_system,
|
source_system=source_system,
|
||||||
source_message_id=source_message_id,
|
source_message_id=source_message_id,
|
||||||
|
|
|
||||||
|
|
@ -103,7 +103,10 @@ class ExecuteQueryProjectMixin:
|
||||||
message_text: str,
|
message_text: str,
|
||||||
project_id: Optional[int],
|
project_id: Optional[int],
|
||||||
message_intent: Optional[str] = None,
|
message_intent: Optional[str] = None,
|
||||||
|
source_policy: Optional[str] = None,
|
||||||
) -> Optional[dict[str, Any]]:
|
) -> Optional[dict[str, Any]]:
|
||||||
|
if str(source_policy or "").strip().lower() == "external":
|
||||||
|
return None
|
||||||
normalized_intent = str(message_intent or "").strip().lower()
|
normalized_intent = str(message_intent or "").strip().lower()
|
||||||
keyword_extractor = getattr(rag_service, "_extract_project_lookup_keyword", None)
|
keyword_extractor = getattr(rag_service, "_extract_project_lookup_keyword", None)
|
||||||
extracted_keyword = ""
|
extracted_keyword = ""
|
||||||
|
|
@ -146,8 +149,11 @@ class ExecuteQueryProjectMixin:
|
||||||
*,
|
*,
|
||||||
message_text: str,
|
message_text: str,
|
||||||
message_intent: Optional[str],
|
message_intent: Optional[str],
|
||||||
|
source_policy: Optional[str] = None,
|
||||||
) -> bool:
|
) -> bool:
|
||||||
_ = message_text
|
_ = message_text
|
||||||
|
if str(source_policy or "").strip().lower() == "external":
|
||||||
|
return False
|
||||||
return str(message_intent or "").strip().lower() == "casual_chat"
|
return str(message_intent or "").strip().lower() == "casual_chat"
|
||||||
|
|
||||||
@classmethod
|
@classmethod
|
||||||
|
|
@ -3999,8 +4005,11 @@ class ExecuteQueryProjectMixin:
|
||||||
]
|
]
|
||||||
if not plan_candidates:
|
if not plan_candidates:
|
||||||
return None, None
|
return None, None
|
||||||
|
preferred_candidates = [
|
||||||
|
row for row in plan_candidates if int(row.id) in draft_by_topic_id
|
||||||
|
] or plan_candidates
|
||||||
topic = self._select_requirement_plan_topic(
|
topic = self._select_requirement_plan_topic(
|
||||||
candidates=plan_candidates,
|
candidates=preferred_candidates,
|
||||||
topic_key=topic_key,
|
topic_key=topic_key,
|
||||||
root_message_id=root_message_id,
|
root_message_id=root_message_id,
|
||||||
reply_to_message_id=reply_to_message_id,
|
reply_to_message_id=reply_to_message_id,
|
||||||
|
|
@ -4186,6 +4195,7 @@ class ExecuteQueryProjectMixin:
|
||||||
self,
|
self,
|
||||||
*,
|
*,
|
||||||
message_text: str,
|
message_text: str,
|
||||||
|
message_intent: Optional[str],
|
||||||
topic_switch: Optional[str],
|
topic_switch: Optional[str],
|
||||||
has_requirement_anchor: bool,
|
has_requirement_anchor: bool,
|
||||||
topic: Optional[MountedTopicWorkspace],
|
topic: Optional[MountedTopicWorkspace],
|
||||||
|
|
@ -4197,6 +4207,16 @@ class ExecuteQueryProjectMixin:
|
||||||
return False
|
return False
|
||||||
if self._extract_requirement_plan_choice(message_text):
|
if self._extract_requirement_plan_choice(message_text):
|
||||||
return False
|
return False
|
||||||
|
normalized_message_intent = str(message_intent or "").strip().lower()
|
||||||
|
if normalized_message_intent not in {
|
||||||
|
"requirement_plan",
|
||||||
|
"requirement_gathering",
|
||||||
|
"requirement_convergence",
|
||||||
|
"consult_requirement",
|
||||||
|
"request_draft_generation",
|
||||||
|
"request_draft_generation_from_context",
|
||||||
|
}:
|
||||||
|
return False
|
||||||
|
|
||||||
normalized_topic_switch = str(topic_switch or "").strip().lower()
|
normalized_topic_switch = str(topic_switch or "").strip().lower()
|
||||||
nontrivial_seed = self._sanitize_requirement_plan_text(message_text).strip()
|
nontrivial_seed = self._sanitize_requirement_plan_text(message_text).strip()
|
||||||
|
|
@ -5074,6 +5094,10 @@ class ExecuteQueryProjectMixin:
|
||||||
metadata = intent.metadata if isinstance(intent.metadata, dict) else {}
|
metadata = intent.metadata if isinstance(intent.metadata, dict) else {}
|
||||||
semantic_bundle = metadata.get("semantic_bundle") if isinstance(metadata.get("semantic_bundle"), dict) else {}
|
semantic_bundle = metadata.get("semantic_bundle") if isinstance(metadata.get("semantic_bundle"), dict) else {}
|
||||||
semantic_diagnostics = semantic_bundle.get("diagnostics") if isinstance(semantic_bundle.get("diagnostics"), dict) else {}
|
semantic_diagnostics = semantic_bundle.get("diagnostics") if isinstance(semantic_bundle.get("diagnostics"), dict) else {}
|
||||||
|
semantic_planner_degraded = bool(
|
||||||
|
metadata.get("semantic_planner_degraded") or semantic_diagnostics.get("semantic_planner_degraded")
|
||||||
|
)
|
||||||
|
degraded_context_source = str(metadata.get("context_source") or "").strip().lower() == "semantic_planner_degraded"
|
||||||
rag_request_context = (
|
rag_request_context = (
|
||||||
metadata.get("rag_request_context")
|
metadata.get("rag_request_context")
|
||||||
if isinstance(metadata.get("rag_request_context"), dict)
|
if isinstance(metadata.get("rag_request_context"), dict)
|
||||||
|
|
@ -5169,6 +5193,12 @@ class ExecuteQueryProjectMixin:
|
||||||
or semantic_diagnostics.get("force_thinking_mode")
|
or semantic_diagnostics.get("force_thinking_mode")
|
||||||
or ""
|
or ""
|
||||||
).strip() or None
|
).strip() or None
|
||||||
|
if semantic_planner_degraded and degraded_context_source:
|
||||||
|
# Planner failure metadata is only an audit marker. Do not let stale
|
||||||
|
# degraded scene labels drive requirement routing for a fresh query.
|
||||||
|
message_intent = None
|
||||||
|
consultant_mode_preference = None
|
||||||
|
advise_subtype = None
|
||||||
normalized_scene_metadata = WorkflowSemanticPlanner.normalize_scene_metadata(
|
normalized_scene_metadata = WorkflowSemanticPlanner.normalize_scene_metadata(
|
||||||
{
|
{
|
||||||
"message_intent": message_intent,
|
"message_intent": message_intent,
|
||||||
|
|
@ -5250,6 +5280,8 @@ class ExecuteQueryProjectMixin:
|
||||||
or needs_legacy_business_review_refresh
|
or needs_legacy_business_review_refresh
|
||||||
)
|
)
|
||||||
)
|
)
|
||||||
|
if semantic_planner_degraded:
|
||||||
|
needs_semantic_backfill = True
|
||||||
if not normalized_message_intent:
|
if not normalized_message_intent:
|
||||||
needs_semantic_backfill = True
|
needs_semantic_backfill = True
|
||||||
elif normalized_message_intent in {"business_review", "requirement_plan"} and not str(
|
elif normalized_message_intent in {"business_review", "requirement_plan"} and not str(
|
||||||
|
|
@ -5269,52 +5301,74 @@ class ExecuteQueryProjectMixin:
|
||||||
has_active_requirement_plan_workspace = (
|
has_active_requirement_plan_workspace = (
|
||||||
active_requirement_plan_topic is not None and active_requirement_plan_draft is not None
|
active_requirement_plan_topic is not None and active_requirement_plan_draft is not None
|
||||||
)
|
)
|
||||||
semantic_bundle = await self._backfill_query_semantic_bundle(
|
try:
|
||||||
message_text=message_text,
|
semantic_bundle = await self._backfill_query_semantic_bundle(
|
||||||
source_system=source_system,
|
message_text=message_text,
|
||||||
metadata=metadata,
|
source_system=source_system,
|
||||||
requested_project_id=requested_project_id if requested_project_id is not None else intent.project_id,
|
metadata=metadata,
|
||||||
source_chat_id=source_chat_id,
|
requested_project_id=requested_project_id if requested_project_id is not None else intent.project_id,
|
||||||
source_sender_open_id=source_sender_open_id,
|
source_chat_id=source_chat_id,
|
||||||
thread_context=thread_context,
|
source_sender_open_id=source_sender_open_id,
|
||||||
conversation_key=conversation_key,
|
thread_context=thread_context,
|
||||||
active_requirement_plan_topic=active_requirement_plan_topic if has_requirement_anchor else None,
|
conversation_key=conversation_key,
|
||||||
active_requirement_plan_draft=active_requirement_plan_draft if has_requirement_anchor else None,
|
active_requirement_plan_topic=active_requirement_plan_topic if has_requirement_anchor else None,
|
||||||
)
|
active_requirement_plan_draft=active_requirement_plan_draft if has_requirement_anchor else None,
|
||||||
semantic_diagnostics = (
|
)
|
||||||
semantic_bundle.get("diagnostics") if isinstance(semantic_bundle.get("diagnostics"), dict) else {}
|
except Exception as exc:
|
||||||
)
|
logger.warning(
|
||||||
refreshed_message_intent = str(semantic_diagnostics.get("message_intent") or "").strip() or None
|
"query_semantic_backfill_failed",
|
||||||
refreshed_consultant_mode = (
|
tenant_id=self.tenant_id,
|
||||||
str(semantic_diagnostics.get("consultant_mode_preference") or "").strip().lower() or None
|
source_system=source_system,
|
||||||
)
|
source_message_id=source_message_id,
|
||||||
refreshed_advise_subtype = (
|
semantic_planner_degraded=semantic_planner_degraded,
|
||||||
str(semantic_diagnostics.get("advise_subtype") or "").strip().lower() or None
|
error=str(exc),
|
||||||
)
|
)
|
||||||
refreshed_topic_switch = str(semantic_diagnostics.get("topic_switch") or "").strip().lower() or None
|
if isinstance(intent.metadata, dict):
|
||||||
refreshed_thinking_mode = str(semantic_diagnostics.get("force_thinking_mode") or "").strip() or None
|
intent.metadata["semantic_backfill_failed"] = True
|
||||||
if refreshed_message_intent:
|
intent.metadata["semantic_backfill_error"] = str(exc)[:240]
|
||||||
message_intent = refreshed_message_intent
|
if semantic_planner_degraded and degraded_context_source and not has_active_requirement_plan_workspace:
|
||||||
if refreshed_consultant_mode:
|
message_intent = None
|
||||||
consultant_mode_preference = refreshed_consultant_mode
|
consultant_mode_preference = None
|
||||||
if refreshed_advise_subtype:
|
advise_subtype = None
|
||||||
advise_subtype = refreshed_advise_subtype
|
else:
|
||||||
if refreshed_topic_switch:
|
semantic_diagnostics = (
|
||||||
topic_switch = refreshed_topic_switch
|
semantic_bundle.get("diagnostics") if isinstance(semantic_bundle.get("diagnostics"), dict) else {}
|
||||||
if refreshed_thinking_mode:
|
)
|
||||||
thinking_mode = refreshed_thinking_mode
|
refreshed_message_intent = str(semantic_diagnostics.get("message_intent") or "").strip() or None
|
||||||
metadata = {**metadata, "semantic_bundle": semantic_bundle}
|
refreshed_consultant_mode = (
|
||||||
normalized_scene_metadata = WorkflowSemanticPlanner.normalize_scene_metadata(
|
str(semantic_diagnostics.get("consultant_mode_preference") or "").strip().lower() or None
|
||||||
{
|
)
|
||||||
"message_intent": message_intent,
|
refreshed_advise_subtype = (
|
||||||
"consultant_mode_preference": consultant_mode_preference,
|
str(semantic_diagnostics.get("advise_subtype") or "").strip().lower() or None
|
||||||
}
|
)
|
||||||
)
|
refreshed_topic_switch = str(semantic_diagnostics.get("topic_switch") or "").strip().lower() or None
|
||||||
message_intent = str(normalized_scene_metadata.get("message_intent") or message_intent or "").strip() or None
|
refreshed_thinking_mode = str(semantic_diagnostics.get("force_thinking_mode") or "").strip() or None
|
||||||
consultant_mode_preference = (
|
if refreshed_message_intent:
|
||||||
str(normalized_scene_metadata.get("consultant_mode_preference") or consultant_mode_preference or "").strip().lower()
|
message_intent = refreshed_message_intent
|
||||||
or None
|
if refreshed_consultant_mode:
|
||||||
)
|
consultant_mode_preference = refreshed_consultant_mode
|
||||||
|
if refreshed_advise_subtype:
|
||||||
|
advise_subtype = refreshed_advise_subtype
|
||||||
|
if refreshed_topic_switch:
|
||||||
|
topic_switch = refreshed_topic_switch
|
||||||
|
if refreshed_thinking_mode:
|
||||||
|
thinking_mode = refreshed_thinking_mode
|
||||||
|
metadata = {**metadata, "semantic_bundle": semantic_bundle}
|
||||||
|
normalized_scene_metadata = WorkflowSemanticPlanner.normalize_scene_metadata(
|
||||||
|
{
|
||||||
|
"message_intent": message_intent,
|
||||||
|
"consultant_mode_preference": consultant_mode_preference,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
message_intent = str(normalized_scene_metadata.get("message_intent") or message_intent or "").strip() or None
|
||||||
|
consultant_mode_preference = (
|
||||||
|
str(
|
||||||
|
normalized_scene_metadata.get("consultant_mode_preference")
|
||||||
|
or consultant_mode_preference
|
||||||
|
or ""
|
||||||
|
).strip().lower()
|
||||||
|
or None
|
||||||
|
)
|
||||||
normalized_message_intent = str(message_intent or "").strip().lower()
|
normalized_message_intent = str(message_intent or "").strip().lower()
|
||||||
if not has_active_requirement_plan_workspace and normalized_message_intent == "requirement_plan":
|
if not has_active_requirement_plan_workspace and normalized_message_intent == "requirement_plan":
|
||||||
active_requirement_plan_topic, active_requirement_plan_draft = await self._find_active_requirement_plan_workspace(
|
active_requirement_plan_topic, active_requirement_plan_draft = await self._find_active_requirement_plan_workspace(
|
||||||
|
|
@ -5346,6 +5400,7 @@ class ExecuteQueryProjectMixin:
|
||||||
)
|
)
|
||||||
if self._should_force_new_same_scope_requirement_plan(
|
if self._should_force_new_same_scope_requirement_plan(
|
||||||
message_text=message_text,
|
message_text=message_text,
|
||||||
|
message_intent=message_intent,
|
||||||
topic_switch=topic_switch,
|
topic_switch=topic_switch,
|
||||||
has_requirement_anchor=has_requirement_anchor,
|
has_requirement_anchor=has_requirement_anchor,
|
||||||
topic=active_requirement_plan_topic,
|
topic=active_requirement_plan_topic,
|
||||||
|
|
@ -5489,7 +5544,11 @@ class ExecuteQueryProjectMixin:
|
||||||
)
|
)
|
||||||
direct_result["project_id"] = intent.project_id
|
direct_result["project_id"] = intent.project_id
|
||||||
return direct_result
|
return direct_result
|
||||||
if requirement_mode_locked and normalized_message_intent == "casual_chat":
|
if requirement_mode_locked and normalized_message_intent not in {
|
||||||
|
"requirement_plan",
|
||||||
|
"requirement_gathering",
|
||||||
|
"requirement_convergence",
|
||||||
|
}:
|
||||||
direct_result = self._build_direct_query_result(
|
direct_result = self._build_direct_query_result(
|
||||||
answer=self._build_requirement_mode_locked_answer(card=locked_card),
|
answer=self._build_requirement_mode_locked_answer(card=locked_card),
|
||||||
scene_label="Mounted Runtime Plan",
|
scene_label="Mounted Runtime Plan",
|
||||||
|
|
@ -5506,9 +5565,48 @@ class ExecuteQueryProjectMixin:
|
||||||
consultant_mode_preference = (
|
consultant_mode_preference = (
|
||||||
str(consultant_mode_preference or "exploratory").strip().lower() or "exploratory"
|
str(consultant_mode_preference or "exploratory").strip().lower() or "exploratory"
|
||||||
)
|
)
|
||||||
|
route_preview: dict[str, Any] = {}
|
||||||
|
route_preview_resolver = getattr(rag_service, "preview_query_route_decision", None)
|
||||||
|
if callable(route_preview_resolver):
|
||||||
|
try:
|
||||||
|
route_preview = await route_preview_resolver(
|
||||||
|
question=message_text,
|
||||||
|
asked_via=asked_via,
|
||||||
|
project_id=requested_project_id if requested_project_id is not None else intent.project_id,
|
||||||
|
source_chat_id=source_chat_id,
|
||||||
|
source_message_id=source_message_id,
|
||||||
|
conversation_scope_key=conversation_key,
|
||||||
|
reply_to_message_id=str((thread_context or {}).get("reply_to_message_id") or "").strip() or None,
|
||||||
|
root_message_id=str((thread_context or {}).get("root_message_id") or "").strip() or None,
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
route_preview = {}
|
||||||
|
source_policy_hint = (
|
||||||
|
route_preview.get("query_semantics")
|
||||||
|
if isinstance(route_preview.get("query_semantics"), dict)
|
||||||
|
else {}
|
||||||
|
)
|
||||||
|
route_decision = (
|
||||||
|
route_preview.get("route_decision")
|
||||||
|
if isinstance(route_preview.get("route_decision"), dict)
|
||||||
|
else {}
|
||||||
|
)
|
||||||
|
source_policy = (
|
||||||
|
str(source_policy_hint.get("source_policy") or route_decision.get("route_mode") or "").strip().lower() or None
|
||||||
|
)
|
||||||
|
effective_source_policy = (
|
||||||
|
str(
|
||||||
|
source_policy_hint.get("effective_source_policy")
|
||||||
|
or route_decision.get("effective_source_policy")
|
||||||
|
or source_policy
|
||||||
|
or ""
|
||||||
|
).strip().lower()
|
||||||
|
or None
|
||||||
|
)
|
||||||
if self._should_use_lightweight_chat_answer(
|
if self._should_use_lightweight_chat_answer(
|
||||||
message_text=message_text,
|
message_text=message_text,
|
||||||
message_intent=message_intent,
|
message_intent=message_intent,
|
||||||
|
source_policy=effective_source_policy,
|
||||||
):
|
):
|
||||||
direct_result = self._build_direct_query_result(
|
direct_result = self._build_direct_query_result(
|
||||||
answer=self._build_lightweight_chat_answer(message_text=message_text),
|
answer=self._build_lightweight_chat_answer(message_text=message_text),
|
||||||
|
|
@ -5587,6 +5685,7 @@ class ExecuteQueryProjectMixin:
|
||||||
message_text=message_text,
|
message_text=message_text,
|
||||||
project_id=requested_project_id if requested_project_id is not None else intent.project_id,
|
project_id=requested_project_id if requested_project_id is not None else intent.project_id,
|
||||||
message_intent=message_intent,
|
message_intent=message_intent,
|
||||||
|
source_policy=effective_source_policy,
|
||||||
)
|
)
|
||||||
if project_lookup_result is not None:
|
if project_lookup_result is not None:
|
||||||
return project_lookup_result
|
return project_lookup_result
|
||||||
|
|
@ -5613,6 +5712,9 @@ class ExecuteQueryProjectMixin:
|
||||||
"message_intent": message_intent,
|
"message_intent": message_intent,
|
||||||
"mixed_signal_part": mixed_signal_part,
|
"mixed_signal_part": mixed_signal_part,
|
||||||
"consultant_mode_preference": consultant_mode_preference,
|
"consultant_mode_preference": consultant_mode_preference,
|
||||||
|
"source_policy": source_policy,
|
||||||
|
"precomputed_query_semantics": source_policy_hint if isinstance(source_policy_hint, dict) and source_policy_hint else None,
|
||||||
|
"route_decision": route_decision if isinstance(route_decision, dict) and route_decision else None,
|
||||||
}
|
}
|
||||||
if session_id is not None:
|
if session_id is not None:
|
||||||
rag_kwargs["session_id"] = int(session_id)
|
rag_kwargs["session_id"] = int(session_id)
|
||||||
|
|
|
||||||
|
|
@ -1504,6 +1504,12 @@ class DraftConfirmHandler:
|
||||||
mounted=(draft_row, topic_row),
|
mounted=(draft_row, topic_row),
|
||||||
)
|
)
|
||||||
continue
|
continue
|
||||||
|
if anchor_context and not self._matches_mounted_draft_anchor(
|
||||||
|
draft=draft_row,
|
||||||
|
topic=topic_row,
|
||||||
|
anchor_context=anchor_context,
|
||||||
|
):
|
||||||
|
continue
|
||||||
score = 0
|
score = 0
|
||||||
if normalized_channel and str(getattr(topic_row, "channel_ref", "") or "").strip() == normalized_channel:
|
if normalized_channel and str(getattr(topic_row, "channel_ref", "") or "").strip() == normalized_channel:
|
||||||
score += 30
|
score += 30
|
||||||
|
|
|
||||||
|
|
@ -40,6 +40,8 @@ class WorkflowSemanticPlanner:
|
||||||
}
|
}
|
||||||
_SCENE_CONSULTANT_MODE_ALIASES = {
|
_SCENE_CONSULTANT_MODE_ALIASES = {
|
||||||
"structured_analysis": "structured_analysis_first",
|
"structured_analysis": "structured_analysis_first",
|
||||||
|
"requirement_analysis": "structured_analysis_first",
|
||||||
|
"requirements_analysis": "structured_analysis_first",
|
||||||
"collaborative": "exploratory",
|
"collaborative": "exploratory",
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,21 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from .gateway import ExternalQueryGateway
|
||||||
|
from .planner import ExternalQueryPlanner
|
||||||
|
from .registry import ExternalProviderRegistry
|
||||||
|
from .schemas import (
|
||||||
|
ExternalAdapterResult,
|
||||||
|
ExternalEvidenceItem,
|
||||||
|
ExternalQueryExecution,
|
||||||
|
ExternalQueryPlan,
|
||||||
|
)
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"ExternalAdapterResult",
|
||||||
|
"ExternalEvidenceItem",
|
||||||
|
"ExternalProviderRegistry",
|
||||||
|
"ExternalQueryExecution",
|
||||||
|
"ExternalQueryGateway",
|
||||||
|
"ExternalQueryPlan",
|
||||||
|
"ExternalQueryPlanner",
|
||||||
|
]
|
||||||
|
|
@ -0,0 +1,6 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from .live_search import LiveSearchFallbackAdapter
|
||||||
|
from .mcp import MCPExternalAdapter
|
||||||
|
|
||||||
|
__all__ = ["LiveSearchFallbackAdapter", "MCPExternalAdapter"]
|
||||||
|
|
@ -0,0 +1,151 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from typing import Any, Awaitable, Callable
|
||||||
|
|
||||||
|
from ..schemas import ExternalAdapterResult, ExternalEvidenceItem, ExternalQueryPlan
|
||||||
|
|
||||||
|
|
||||||
|
class LiveSearchFallbackAdapter:
|
||||||
|
adapter_name = "live_search"
|
||||||
|
supported_query_kinds = {"weather", "news", "market", "generic_web", "unknown"}
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
build_queries_callable: Callable[[str], list[str]] | None = None,
|
||||||
|
search_results_callable: Callable[..., Awaitable[list[dict[str, str]]]] | None = None,
|
||||||
|
fetch_text_callable: Callable[[str], Awaitable[str]] | None = None,
|
||||||
|
chunk_text_callable: Callable[..., list[str]] | None = None,
|
||||||
|
score_search_result_callable: Callable[..., float] | None = None,
|
||||||
|
score_chunk_callable: Callable[..., float] | None = None,
|
||||||
|
max_pages: int = 3,
|
||||||
|
) -> None:
|
||||||
|
self.build_queries_callable = build_queries_callable
|
||||||
|
self.search_results_callable = search_results_callable
|
||||||
|
self.fetch_text_callable = fetch_text_callable
|
||||||
|
self.chunk_text_callable = chunk_text_callable
|
||||||
|
self.score_search_result_callable = score_search_result_callable
|
||||||
|
self.score_chunk_callable = score_chunk_callable
|
||||||
|
self.max_pages = max(1, int(max_pages or 3))
|
||||||
|
|
||||||
|
async def execute(self, plan: ExternalQueryPlan) -> ExternalAdapterResult:
|
||||||
|
if not all(
|
||||||
|
callable(item)
|
||||||
|
for item in (
|
||||||
|
self.search_results_callable,
|
||||||
|
self.fetch_text_callable,
|
||||||
|
self.chunk_text_callable,
|
||||||
|
self.score_search_result_callable,
|
||||||
|
self.score_chunk_callable,
|
||||||
|
)
|
||||||
|
):
|
||||||
|
return ExternalAdapterResult(
|
||||||
|
adapter_name=self.adapter_name,
|
||||||
|
status="unavailable",
|
||||||
|
evidence=[],
|
||||||
|
meta={"reason": "live_search_callbacks_missing"},
|
||||||
|
)
|
||||||
|
|
||||||
|
queries = [str(item or "").strip() for item in list(plan.search_queries or []) if str(item or "").strip()]
|
||||||
|
if not queries and callable(self.build_queries_callable):
|
||||||
|
try:
|
||||||
|
queries = [
|
||||||
|
str(item or "").strip()
|
||||||
|
for item in list(self.build_queries_callable(plan.topic or "") or [])
|
||||||
|
if str(item or "").strip()
|
||||||
|
]
|
||||||
|
except Exception:
|
||||||
|
queries = []
|
||||||
|
if not queries:
|
||||||
|
return ExternalAdapterResult(adapter_name=self.adapter_name, status="empty", evidence=[], meta={"query": ""})
|
||||||
|
|
||||||
|
search_results: list[dict[str, str]] = []
|
||||||
|
selected_query = ""
|
||||||
|
for query in queries:
|
||||||
|
try:
|
||||||
|
search_results = await self.search_results_callable(
|
||||||
|
query=query,
|
||||||
|
max_results=max(3, int(plan.result_limit) * 2),
|
||||||
|
)
|
||||||
|
except Exception:
|
||||||
|
search_results = []
|
||||||
|
if search_results:
|
||||||
|
selected_query = query
|
||||||
|
break
|
||||||
|
|
||||||
|
ranked = sorted(
|
||||||
|
search_results,
|
||||||
|
key=lambda row: float(
|
||||||
|
self.score_search_result_callable(
|
||||||
|
question=plan.topic or selected_query or queries[0],
|
||||||
|
title=str(row.get("title") or ""),
|
||||||
|
url=str(row.get("url") or ""),
|
||||||
|
)
|
||||||
|
or 0.0
|
||||||
|
),
|
||||||
|
reverse=True,
|
||||||
|
)
|
||||||
|
|
||||||
|
evidence: list[ExternalEvidenceItem] = []
|
||||||
|
fetch_attempts = 0
|
||||||
|
candidate_limit = min(len(ranked), self.max_pages + 2)
|
||||||
|
for row in ranked[:candidate_limit]:
|
||||||
|
url = str(row.get("url") or "").strip()
|
||||||
|
title = str(row.get("title") or "").strip()
|
||||||
|
if not url:
|
||||||
|
continue
|
||||||
|
fetch_attempts += 1
|
||||||
|
try:
|
||||||
|
text = await self.fetch_text_callable(url)
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
chunks = list(self.chunk_text_callable(text, chunk_size=900, overlap=120) or [])
|
||||||
|
if not chunks:
|
||||||
|
continue
|
||||||
|
best_chunk = ""
|
||||||
|
best_score = 0.0
|
||||||
|
for chunk in chunks[:6]:
|
||||||
|
score = float(
|
||||||
|
self.score_chunk_callable(
|
||||||
|
question=plan.topic or selected_query or queries[0],
|
||||||
|
title=title,
|
||||||
|
content=chunk,
|
||||||
|
url=url,
|
||||||
|
)
|
||||||
|
or 0.0
|
||||||
|
)
|
||||||
|
if score > best_score:
|
||||||
|
best_score = score
|
||||||
|
best_chunk = str(chunk or "").strip()
|
||||||
|
if not best_chunk:
|
||||||
|
continue
|
||||||
|
evidence.append(
|
||||||
|
ExternalEvidenceItem(
|
||||||
|
source_type="url",
|
||||||
|
source_label=title or url,
|
||||||
|
title=title[:255],
|
||||||
|
target_url=url,
|
||||||
|
snippet=best_chunk[:1800],
|
||||||
|
retrieved_at=datetime.now(timezone.utc).isoformat(),
|
||||||
|
freshness_score=round(best_score, 4),
|
||||||
|
authority_score=round(best_score, 4),
|
||||||
|
grounded_fields=["snippet"],
|
||||||
|
raw_metadata={
|
||||||
|
"query": selected_query or queries[0],
|
||||||
|
"provider": self.adapter_name,
|
||||||
|
"title": title[:255],
|
||||||
|
},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
return ExternalAdapterResult(
|
||||||
|
adapter_name=self.adapter_name,
|
||||||
|
status="success" if evidence else "empty",
|
||||||
|
evidence=evidence[: max(1, int(plan.result_limit))],
|
||||||
|
meta={
|
||||||
|
"query": selected_query or (queries[0] if queries else ""),
|
||||||
|
"search_result_count": len(search_results),
|
||||||
|
"fetched_pages": fetch_attempts,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
@ -0,0 +1,129 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any, Awaitable, Callable
|
||||||
|
|
||||||
|
from ..schemas import ExternalAdapterResult, ExternalEvidenceItem, ExternalQueryPlan
|
||||||
|
|
||||||
|
|
||||||
|
class MCPExternalAdapter:
|
||||||
|
adapter_name = "mcp"
|
||||||
|
supported_query_kinds = {"model", "company", "generic_web", "unknown"}
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
search_callable: Callable[[ExternalQueryPlan], Awaitable[dict[str, Any] | ExternalAdapterResult]] | None = None,
|
||||||
|
fetch_text_callable: Callable[[str], Awaitable[str]] | None = None,
|
||||||
|
chunk_text_callable: Callable[..., list[str]] | None = None,
|
||||||
|
score_chunk_callable: Callable[..., float] | None = None,
|
||||||
|
max_pages: int = 3,
|
||||||
|
) -> None:
|
||||||
|
self.search_callable = search_callable
|
||||||
|
self.fetch_text_callable = fetch_text_callable
|
||||||
|
self.chunk_text_callable = chunk_text_callable
|
||||||
|
self.score_chunk_callable = score_chunk_callable
|
||||||
|
self.max_pages = max(1, int(max_pages or 3))
|
||||||
|
|
||||||
|
async def execute(self, plan: ExternalQueryPlan) -> ExternalAdapterResult:
|
||||||
|
if not callable(self.search_callable):
|
||||||
|
return ExternalAdapterResult(
|
||||||
|
adapter_name=self.adapter_name,
|
||||||
|
status="unavailable",
|
||||||
|
evidence=[],
|
||||||
|
meta={"reason": "mcp_adapter_unavailable"},
|
||||||
|
)
|
||||||
|
result = await self.search_callable(plan)
|
||||||
|
if isinstance(result, ExternalAdapterResult):
|
||||||
|
validated = result
|
||||||
|
else:
|
||||||
|
payload = dict(result or {}) if isinstance(result, dict) else {}
|
||||||
|
payload.setdefault("adapter_name", self.adapter_name)
|
||||||
|
validated = ExternalAdapterResult.model_validate(payload)
|
||||||
|
evidence, fetched_pages = await self._hydrate_evidence_from_target_pages(
|
||||||
|
plan=plan,
|
||||||
|
evidence=list(validated.evidence or []),
|
||||||
|
)
|
||||||
|
meta = dict(validated.meta or {})
|
||||||
|
if fetched_pages:
|
||||||
|
meta["fetched_pages"] = int(fetched_pages)
|
||||||
|
return validated.model_copy(update={"evidence": evidence, "meta": meta})
|
||||||
|
|
||||||
|
def _page_fetch_ready(self) -> bool:
|
||||||
|
return all(
|
||||||
|
callable(item)
|
||||||
|
for item in (
|
||||||
|
self.fetch_text_callable,
|
||||||
|
self.chunk_text_callable,
|
||||||
|
self.score_chunk_callable,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
async def _hydrate_evidence_from_target_pages(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
plan: ExternalQueryPlan,
|
||||||
|
evidence: list[ExternalEvidenceItem],
|
||||||
|
) -> tuple[list[ExternalEvidenceItem], int]:
|
||||||
|
if not evidence or not self._page_fetch_ready():
|
||||||
|
return evidence, 0
|
||||||
|
hydrated: list[ExternalEvidenceItem] = []
|
||||||
|
fetch_attempts = 0
|
||||||
|
query = str((plan.search_queries[:1] or [plan.topic])[0] or "").strip() or str(plan.topic or "").strip()
|
||||||
|
for index, item in enumerate(evidence):
|
||||||
|
target_url = str(item.target_url or "").strip()
|
||||||
|
if index >= self.max_pages or (
|
||||||
|
not target_url.startswith("http://") and not target_url.startswith("https://")
|
||||||
|
):
|
||||||
|
hydrated.append(item)
|
||||||
|
continue
|
||||||
|
fetch_attempts += 1
|
||||||
|
try:
|
||||||
|
text = await self.fetch_text_callable(target_url)
|
||||||
|
except Exception:
|
||||||
|
hydrated.append(item)
|
||||||
|
continue
|
||||||
|
chunks = list(self.chunk_text_callable(text, chunk_size=900, overlap=120) or [])
|
||||||
|
if not chunks:
|
||||||
|
hydrated.append(item)
|
||||||
|
continue
|
||||||
|
|
||||||
|
best_chunk = ""
|
||||||
|
best_score = 0.0
|
||||||
|
title = str(item.title or "").strip()
|
||||||
|
for chunk in chunks[:6]:
|
||||||
|
score = float(
|
||||||
|
self.score_chunk_callable(
|
||||||
|
question=query or plan.topic or "",
|
||||||
|
title=title,
|
||||||
|
content=chunk,
|
||||||
|
url=target_url,
|
||||||
|
)
|
||||||
|
or 0.0
|
||||||
|
)
|
||||||
|
if score > best_score:
|
||||||
|
best_score = score
|
||||||
|
best_chunk = str(chunk or "").strip()
|
||||||
|
if not best_chunk:
|
||||||
|
hydrated.append(item)
|
||||||
|
continue
|
||||||
|
|
||||||
|
grounded_fields = [str(field or "").strip() for field in list(item.grounded_fields or []) if str(field or "").strip()]
|
||||||
|
if "page_content_snippet" not in grounded_fields:
|
||||||
|
grounded_fields.append("page_content_snippet")
|
||||||
|
raw_metadata = dict(item.raw_metadata or {})
|
||||||
|
original_snippet = str(item.snippet or "").strip()
|
||||||
|
if original_snippet and not str(raw_metadata.get("search_snippet") or "").strip():
|
||||||
|
raw_metadata["search_snippet"] = original_snippet[:1800]
|
||||||
|
raw_metadata["page_chunk_score"] = round(best_score, 4)
|
||||||
|
raw_metadata["target_page_fetched"] = True
|
||||||
|
hydrated.append(
|
||||||
|
item.model_copy(
|
||||||
|
update={
|
||||||
|
"snippet": best_chunk[:1800],
|
||||||
|
"freshness_score": round(max(float(item.freshness_score or 0.0), best_score), 4),
|
||||||
|
"grounded_fields": grounded_fields,
|
||||||
|
"raw_metadata": raw_metadata,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return hydrated, fetch_attempts
|
||||||
|
|
@ -0,0 +1,271 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
from urllib.parse import urlparse
|
||||||
|
|
||||||
|
from .registry import ExternalProviderRegistry
|
||||||
|
from .schemas import ExternalAdapterResult, ExternalEvidenceItem, ExternalQueryExecution, ExternalQueryPlan
|
||||||
|
|
||||||
|
|
||||||
|
class ExternalQueryGateway:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
planner: Any,
|
||||||
|
registry: ExternalProviderRegistry,
|
||||||
|
) -> None:
|
||||||
|
self.planner = planner
|
||||||
|
self.registry = registry
|
||||||
|
|
||||||
|
async def execute(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
question: str,
|
||||||
|
query_semantics: dict[str, Any] | None = None,
|
||||||
|
) -> ExternalQueryExecution:
|
||||||
|
planned = await self.planner.build_plan(
|
||||||
|
question=question,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
)
|
||||||
|
planner_meta: dict[str, Any] = {}
|
||||||
|
if isinstance(planned, tuple) and len(planned) == 2:
|
||||||
|
plan, planner_meta = planned
|
||||||
|
else:
|
||||||
|
plan = planned
|
||||||
|
if not isinstance(plan, ExternalQueryPlan):
|
||||||
|
plan = ExternalQueryPlan.model_validate(plan)
|
||||||
|
|
||||||
|
adapter_attempts: list[dict[str, Any]] = []
|
||||||
|
selected = ExternalAdapterResult(adapter_name="none", status="empty", evidence=[], meta={})
|
||||||
|
selected_verifier = self._verify_evidence_bundle(plan=plan, evidence=selected.evidence)
|
||||||
|
fallback_used = False
|
||||||
|
for index, adapter in enumerate(self._rank_adapters(plan=plan)):
|
||||||
|
adapter_name = str(getattr(adapter, "adapter_name", "") or f"adapter_{index}").strip() or f"adapter_{index}"
|
||||||
|
result = await adapter.execute(plan)
|
||||||
|
if not isinstance(result, ExternalAdapterResult):
|
||||||
|
result = ExternalAdapterResult.model_validate(result)
|
||||||
|
verifier = self._verify_evidence_bundle(plan=plan, evidence=result.evidence)
|
||||||
|
adapter_attempts.append(
|
||||||
|
{
|
||||||
|
"adapter": adapter_name,
|
||||||
|
"status": str(result.status or "").strip() or "empty",
|
||||||
|
"evidence_count": len(result.evidence),
|
||||||
|
"verifier_ok": bool(verifier.get("ok")),
|
||||||
|
"verifier_reason": verifier.get("reason_code"),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
if result.evidence and bool(verifier.get("ok")):
|
||||||
|
selected = result
|
||||||
|
selected_verifier = verifier
|
||||||
|
fallback_used = index > 0
|
||||||
|
break
|
||||||
|
if selected.adapter_name == "none":
|
||||||
|
selected = result
|
||||||
|
selected_verifier = verifier
|
||||||
|
|
||||||
|
meta = dict(selected.meta or {})
|
||||||
|
meta["fallback_used"] = bool(fallback_used)
|
||||||
|
meta["adapter_attempts"] = adapter_attempts
|
||||||
|
return ExternalQueryExecution(
|
||||||
|
plan=plan,
|
||||||
|
adapter_name=selected.adapter_name,
|
||||||
|
evidence=list(selected.evidence or []),
|
||||||
|
verifier=selected_verifier,
|
||||||
|
planner_meta=planner_meta,
|
||||||
|
meta=meta,
|
||||||
|
)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _normalize_match_text(value: Any) -> str:
|
||||||
|
raw = str(value or "").strip().lower()
|
||||||
|
if not raw:
|
||||||
|
return ""
|
||||||
|
chars: list[str] = []
|
||||||
|
for char in raw:
|
||||||
|
if char.isalnum():
|
||||||
|
chars.append(char)
|
||||||
|
return "".join(chars)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _normalize_anchor_term(cls, value: Any) -> str:
|
||||||
|
return cls._normalize_match_text(value)[:48]
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _common_prefix_size(left: str, right: str) -> int:
|
||||||
|
limit = min(len(left), len(right))
|
||||||
|
idx = 0
|
||||||
|
while idx < limit and left[idx] == right[idx]:
|
||||||
|
idx += 1
|
||||||
|
return idx
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _build_char_windows(cls, value: str, *, size: int = 2) -> set[str]:
|
||||||
|
normalized = cls._normalize_match_text(value)
|
||||||
|
if not normalized:
|
||||||
|
return set()
|
||||||
|
if len(normalized) <= size:
|
||||||
|
return {normalized}
|
||||||
|
return {normalized[idx : idx + size] for idx in range(len(normalized) - size + 1)}
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _tokenize_match_text(cls, value: Any) -> list[str]:
|
||||||
|
raw = str(value or "").strip().lower()
|
||||||
|
if not raw:
|
||||||
|
return []
|
||||||
|
tokens: list[str] = []
|
||||||
|
current: list[str] = []
|
||||||
|
for char in raw:
|
||||||
|
if char.isalnum():
|
||||||
|
current.append(char)
|
||||||
|
continue
|
||||||
|
if current:
|
||||||
|
token = "".join(current)
|
||||||
|
if token:
|
||||||
|
tokens.append(token)
|
||||||
|
current = []
|
||||||
|
if current:
|
||||||
|
token = "".join(current)
|
||||||
|
if token:
|
||||||
|
tokens.append(token)
|
||||||
|
normalized = cls._normalize_match_text(raw)
|
||||||
|
if normalized and normalized not in tokens:
|
||||||
|
tokens.append(normalized)
|
||||||
|
return tokens
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _score_anchor_against_token(cls, anchor: str, token: str) -> float:
|
||||||
|
if not anchor or not token:
|
||||||
|
return 0.0
|
||||||
|
if anchor == token:
|
||||||
|
return 1.0
|
||||||
|
if anchor in token:
|
||||||
|
return max(0.82, min(0.98, len(anchor) / max(1, len(token))))
|
||||||
|
if token in anchor and len(token) >= 2:
|
||||||
|
return max(0.58, len(token) / max(1, len(anchor)))
|
||||||
|
|
||||||
|
prefix_size = cls._common_prefix_size(anchor, token)
|
||||||
|
prefix_ratio = prefix_size / max(1, min(len(anchor), len(token)))
|
||||||
|
|
||||||
|
anchor_windows = cls._build_char_windows(anchor)
|
||||||
|
token_windows = cls._build_char_windows(token)
|
||||||
|
window_overlap = (
|
||||||
|
len(anchor_windows & token_windows) / max(1, len(anchor_windows))
|
||||||
|
if anchor_windows and token_windows
|
||||||
|
else 0.0
|
||||||
|
)
|
||||||
|
|
||||||
|
score = 0.0
|
||||||
|
if prefix_size >= 2:
|
||||||
|
score = max(score, 0.42 + min(0.3, prefix_ratio * 0.4))
|
||||||
|
if window_overlap > 0:
|
||||||
|
score = max(score, 0.35 + min(0.35, window_overlap * 0.5))
|
||||||
|
return round(min(score, 0.98), 4)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _evidence_match_tokens(cls, item: ExternalEvidenceItem) -> list[str]:
|
||||||
|
values = [
|
||||||
|
str(item.source_label or ""),
|
||||||
|
str(item.title or ""),
|
||||||
|
str(item.snippet or ""),
|
||||||
|
str(item.target_url or ""),
|
||||||
|
str(urlparse(str(item.target_url or "").strip()).path or ""),
|
||||||
|
]
|
||||||
|
tokens: list[str] = []
|
||||||
|
seen: set[str] = set()
|
||||||
|
for value in values:
|
||||||
|
for token in cls._tokenize_match_text(value):
|
||||||
|
if token and token not in seen:
|
||||||
|
seen.add(token)
|
||||||
|
tokens.append(token)
|
||||||
|
return tokens
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _score_anchor_against_evidence(cls, anchor: str, evidence: list[ExternalEvidenceItem]) -> float:
|
||||||
|
best = 0.0
|
||||||
|
for item in evidence:
|
||||||
|
for token in cls._evidence_match_tokens(item):
|
||||||
|
best = max(best, cls._score_anchor_against_token(anchor, token))
|
||||||
|
return round(best, 4)
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _adapter_support_level(cls, *, plan: ExternalQueryPlan, adapter: Any) -> int:
|
||||||
|
supported = getattr(adapter, "supported_query_kinds", None)
|
||||||
|
if not supported:
|
||||||
|
return 1
|
||||||
|
query_kind = str(plan.query_kind or "").strip().lower()
|
||||||
|
normalized = {str(item or "").strip().lower() for item in supported if str(item or "").strip()}
|
||||||
|
return 0 if query_kind in normalized else 2
|
||||||
|
|
||||||
|
def _rank_adapters(self, *, plan: ExternalQueryPlan) -> list[Any]:
|
||||||
|
ranked: list[tuple[int, int, Any]] = []
|
||||||
|
for index, adapter in enumerate(self._ensure_registry_items(plan=plan)):
|
||||||
|
ranked.append((self._adapter_support_level(plan=plan, adapter=adapter), index, adapter))
|
||||||
|
ranked.sort(key=lambda item: (item[0], item[1]))
|
||||||
|
return [item[2] for item in ranked]
|
||||||
|
|
||||||
|
def _ensure_registry_items(self, *, plan: ExternalQueryPlan) -> list[Any]:
|
||||||
|
_ = plan
|
||||||
|
return list(self.registry.iter_adapters())
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _build_plan_anchor_terms(cls, plan: ExternalQueryPlan) -> list[str]:
|
||||||
|
terms: list[str] = []
|
||||||
|
for item in list(plan.subject_entities or []):
|
||||||
|
normalized = cls._normalize_anchor_term(item)
|
||||||
|
if len(normalized) >= 2 and normalized not in terms:
|
||||||
|
terms.append(normalized)
|
||||||
|
location_hint = cls._normalize_anchor_term(plan.location_hint)
|
||||||
|
if len(location_hint) >= 2 and location_hint not in terms:
|
||||||
|
terms.append(location_hint)
|
||||||
|
return terms[:6]
|
||||||
|
|
||||||
|
@classmethod
|
||||||
|
def _verify_evidence_bundle(cls, *, plan: ExternalQueryPlan, evidence: list[ExternalEvidenceItem]) -> dict[str, Any]:
|
||||||
|
if not evidence:
|
||||||
|
return {
|
||||||
|
"ok": False,
|
||||||
|
"reason_code": "missing_external_grounding",
|
||||||
|
"missing_fields": ["external_evidence"],
|
||||||
|
}
|
||||||
|
anchor_terms = cls._build_plan_anchor_terms(plan)
|
||||||
|
anchor_hit = not anchor_terms
|
||||||
|
for item in evidence:
|
||||||
|
target_url = str(item.target_url or "").strip().lower()
|
||||||
|
source_type = str(item.source_type or "").strip().lower()
|
||||||
|
if source_type not in {"url", "mcp_search"}:
|
||||||
|
return {
|
||||||
|
"ok": False,
|
||||||
|
"reason_code": "external_source_type_invalid",
|
||||||
|
"missing_fields": ["external_source_type"],
|
||||||
|
}
|
||||||
|
if not target_url.startswith("http://") and not target_url.startswith("https://"):
|
||||||
|
return {
|
||||||
|
"ok": False,
|
||||||
|
"reason_code": "external_target_url_missing",
|
||||||
|
"missing_fields": ["target_url"],
|
||||||
|
}
|
||||||
|
anchor_scores = [cls._score_anchor_against_evidence(anchor, evidence) for anchor in anchor_terms]
|
||||||
|
strong_hits = sum(1 for score in anchor_scores if score >= 0.8)
|
||||||
|
medium_hits = sum(1 for score in anchor_scores if score >= 0.55)
|
||||||
|
bundle_score = (
|
||||||
|
round(sum(anchor_scores) / max(1, len(anchor_scores)), 4)
|
||||||
|
if anchor_scores
|
||||||
|
else 1.0
|
||||||
|
)
|
||||||
|
if anchor_scores:
|
||||||
|
anchor_hit = strong_hits >= 1 or medium_hits == len(anchor_scores) or bundle_score >= 0.6
|
||||||
|
if not anchor_hit:
|
||||||
|
return {
|
||||||
|
"ok": False,
|
||||||
|
"reason_code": "external_evidence_off_topic",
|
||||||
|
"missing_fields": ["plan_anchor_terms"],
|
||||||
|
"bundle_score": bundle_score,
|
||||||
|
"anchor_scores": anchor_scores,
|
||||||
|
}
|
||||||
|
return {
|
||||||
|
"ok": True,
|
||||||
|
"reason_code": None,
|
||||||
|
"missing_fields": [],
|
||||||
|
"bundle_score": bundle_score,
|
||||||
|
"anchor_scores": anchor_scores,
|
||||||
|
}
|
||||||
|
|
@ -0,0 +1,163 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
from typing import Any, Callable
|
||||||
|
|
||||||
|
from ...config.ai_prompts import PROMPT_EXTERNAL_QUERY_PLAN
|
||||||
|
from .schemas import ExternalQueryPlan
|
||||||
|
|
||||||
|
|
||||||
|
class ExternalQueryPlanner:
|
||||||
|
FUNCTION_NAME = "external_query_plan"
|
||||||
|
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
llm_service: Any,
|
||||||
|
db: Any,
|
||||||
|
tenant_id: int,
|
||||||
|
query_builder: Callable[[str], list[str]] | None = None,
|
||||||
|
) -> None:
|
||||||
|
self.llm_service = llm_service
|
||||||
|
self.db = db
|
||||||
|
self.tenant_id = int(tenant_id or 1)
|
||||||
|
self.query_builder = query_builder
|
||||||
|
|
||||||
|
async def build_plan(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
question: str,
|
||||||
|
query_semantics: dict[str, Any] | None = None,
|
||||||
|
) -> tuple[ExternalQueryPlan, dict[str, Any]]:
|
||||||
|
fallback_plan = self._build_fallback_plan(
|
||||||
|
question=question,
|
||||||
|
query_semantics=query_semantics,
|
||||||
|
)
|
||||||
|
generator = getattr(self.llm_service, "generate_structured_object", None)
|
||||||
|
if not callable(generator):
|
||||||
|
return fallback_plan, {"planner": "heuristic", "fallback_taken": True}
|
||||||
|
|
||||||
|
prompt = PROMPT_EXTERNAL_QUERY_PLAN.format(
|
||||||
|
question=str(question or "").strip()[:1200],
|
||||||
|
query_semantics=json.dumps(query_semantics or {}, ensure_ascii=False)[:4000],
|
||||||
|
fallback_queries=json.dumps(fallback_plan.search_queries, ensure_ascii=False),
|
||||||
|
).strip()
|
||||||
|
try:
|
||||||
|
structured = await generator(
|
||||||
|
prompt=prompt,
|
||||||
|
db=self.db,
|
||||||
|
tenant_id=self.tenant_id,
|
||||||
|
function_name=self.FUNCTION_NAME,
|
||||||
|
use_prompt_profile=False,
|
||||||
|
temperature=0.0,
|
||||||
|
response_mode="fast",
|
||||||
|
)
|
||||||
|
payload = getattr(structured, "payload", None)
|
||||||
|
if payload is None and hasattr(structured, "model_dump"):
|
||||||
|
payload = structured.model_dump()
|
||||||
|
if not isinstance(payload, dict):
|
||||||
|
raise ValueError("external_query_plan_payload_not_dict")
|
||||||
|
merged = fallback_plan.model_dump()
|
||||||
|
for key, value in payload.items():
|
||||||
|
if value in (None, "", [], {}):
|
||||||
|
continue
|
||||||
|
merged[key] = value
|
||||||
|
plan = ExternalQueryPlan.model_validate(merged)
|
||||||
|
if not plan.search_queries:
|
||||||
|
plan.search_queries = list(fallback_plan.search_queries)
|
||||||
|
if not plan.preferred_domains:
|
||||||
|
plan.preferred_domains = list(fallback_plan.preferred_domains)
|
||||||
|
if not plan.topic:
|
||||||
|
plan.topic = fallback_plan.topic
|
||||||
|
return plan, {"planner": "llm_structured", "fallback_taken": False}
|
||||||
|
except Exception as exc:
|
||||||
|
return fallback_plan, {
|
||||||
|
"planner": "heuristic",
|
||||||
|
"fallback_taken": True,
|
||||||
|
"reason": str(exc)[:200],
|
||||||
|
}
|
||||||
|
|
||||||
|
def _build_fallback_plan(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
question: str,
|
||||||
|
query_semantics: dict[str, Any] | None = None,
|
||||||
|
) -> ExternalQueryPlan:
|
||||||
|
text = re.sub(r"\s+", " ", str(question or "")).strip()
|
||||||
|
semantics = query_semantics if isinstance(query_semantics, dict) else {}
|
||||||
|
query_profile = semantics.get("query_profile") if isinstance(semantics.get("query_profile"), dict) else {}
|
||||||
|
object_mentions = [
|
||||||
|
str(item or "").strip()
|
||||||
|
for item in list(semantics.get("object_mentions") or [])
|
||||||
|
if str(item or "").strip()
|
||||||
|
]
|
||||||
|
|
||||||
|
query_type = str(semantics.get("query_type") or "").strip().lower()
|
||||||
|
if query_type == "compare":
|
||||||
|
answer_contract = "compare_with_sources"
|
||||||
|
elif bool(query_profile.get("needs_freshness")):
|
||||||
|
answer_contract = "summary_with_sources"
|
||||||
|
else:
|
||||||
|
answer_contract = "direct_fact"
|
||||||
|
|
||||||
|
topic = text[:120] or "外部查询"
|
||||||
|
search_queries = self._build_search_queries(text)
|
||||||
|
preferred_domains = self._infer_preferred_domains(text)
|
||||||
|
subject_entities = object_mentions[:6]
|
||||||
|
location_hint = None
|
||||||
|
time_range_hint = self._infer_time_range_hint(text)
|
||||||
|
requires_multi_source = answer_contract == "compare_with_sources"
|
||||||
|
clarify_on_ambiguity = False
|
||||||
|
result_limit = 3 if requires_multi_source or bool(query_profile.get("needs_freshness")) else 2
|
||||||
|
|
||||||
|
return ExternalQueryPlan(
|
||||||
|
plan_version="v1",
|
||||||
|
query_kind="generic_web" if text else "unknown",
|
||||||
|
answer_contract=answer_contract,
|
||||||
|
freshness_required=bool(query_profile.get("needs_freshness")),
|
||||||
|
topic=topic,
|
||||||
|
subject_entities=subject_entities,
|
||||||
|
location_hint=location_hint,
|
||||||
|
time_range_hint=time_range_hint,
|
||||||
|
preferred_domains=preferred_domains,
|
||||||
|
search_queries=search_queries,
|
||||||
|
result_limit=result_limit,
|
||||||
|
requires_multi_source=requires_multi_source,
|
||||||
|
clarify_on_ambiguity=clarify_on_ambiguity,
|
||||||
|
)
|
||||||
|
|
||||||
|
def _build_search_queries(self, question: str) -> list[str]:
|
||||||
|
if callable(self.query_builder):
|
||||||
|
try:
|
||||||
|
queries = self.query_builder(question)
|
||||||
|
normalized = [str(item or "").strip() for item in list(queries or []) if str(item or "").strip()]
|
||||||
|
if normalized:
|
||||||
|
return normalized[:5]
|
||||||
|
except Exception:
|
||||||
|
pass
|
||||||
|
text = re.sub(r"\s+", " ", str(question or "")).strip()
|
||||||
|
return [text] if text else []
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _infer_preferred_domains(question: str) -> list[str]:
|
||||||
|
normalized = str(question or "").lower()
|
||||||
|
domains: list[str] = []
|
||||||
|
if any(token in normalized for token in ("codex", "gpt", "chatgpt", "openai")):
|
||||||
|
domains.extend(["openai.com", "developers.openai.com"])
|
||||||
|
if any(token in normalized for token in ("copilot", "github")):
|
||||||
|
domains.extend(["github.com", "docs.github.com"])
|
||||||
|
uniq: list[str] = []
|
||||||
|
for domain in domains:
|
||||||
|
if domain not in uniq:
|
||||||
|
uniq.append(domain)
|
||||||
|
return uniq[:4]
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _infer_time_range_hint(question: str) -> str | None:
|
||||||
|
text = str(question or "")
|
||||||
|
for pattern in (r"未来\d+天", r"未来一周", r"今天", r"截至今天", r"截至现在", r"目前", r"最新"):
|
||||||
|
match = re.search(pattern, text)
|
||||||
|
if match:
|
||||||
|
return str(match.group(0) or "").strip() or None
|
||||||
|
return None
|
||||||
|
|
@ -0,0 +1,11 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
class ExternalProviderRegistry:
|
||||||
|
def __init__(self, *, adapters: list[Any] | None = None) -> None:
|
||||||
|
self._adapters = list(adapters or [])
|
||||||
|
|
||||||
|
def iter_adapters(self) -> list[Any]:
|
||||||
|
return list(self._adapters)
|
||||||
|
|
@ -0,0 +1,62 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Any, Literal
|
||||||
|
|
||||||
|
from pydantic import BaseModel, ConfigDict, Field
|
||||||
|
|
||||||
|
|
||||||
|
ExternalQueryKind = Literal["weather", "news", "company", "model", "market", "generic_web", "unknown"]
|
||||||
|
ExternalAnswerContract = Literal["direct_fact", "summary_with_sources", "compare_with_sources", "clarify"]
|
||||||
|
|
||||||
|
|
||||||
|
class ExternalQueryPlan(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="ignore")
|
||||||
|
|
||||||
|
plan_version: str = "v1"
|
||||||
|
query_kind: ExternalQueryKind = "unknown"
|
||||||
|
answer_contract: ExternalAnswerContract = "summary_with_sources"
|
||||||
|
freshness_required: bool = False
|
||||||
|
topic: str = ""
|
||||||
|
subject_entities: list[str] = Field(default_factory=list)
|
||||||
|
location_hint: str | None = None
|
||||||
|
time_range_hint: str | None = None
|
||||||
|
preferred_domains: list[str] = Field(default_factory=list)
|
||||||
|
search_queries: list[str] = Field(default_factory=list)
|
||||||
|
result_limit: int = Field(default=3, ge=1, le=5)
|
||||||
|
requires_multi_source: bool = False
|
||||||
|
clarify_on_ambiguity: bool = False
|
||||||
|
|
||||||
|
|
||||||
|
class ExternalEvidenceItem(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="ignore")
|
||||||
|
|
||||||
|
source_type: str = "url"
|
||||||
|
source_label: str = ""
|
||||||
|
title: str = ""
|
||||||
|
target_url: str = ""
|
||||||
|
snippet: str = ""
|
||||||
|
retrieved_at: str | None = None
|
||||||
|
freshness_score: float = 0.0
|
||||||
|
authority_score: float = 0.0
|
||||||
|
grounded_fields: list[str] = Field(default_factory=list)
|
||||||
|
raw_metadata: dict[str, Any] = Field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
|
class ExternalAdapterResult(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="ignore")
|
||||||
|
|
||||||
|
adapter_name: str
|
||||||
|
status: str = "empty"
|
||||||
|
evidence: list[ExternalEvidenceItem] = Field(default_factory=list)
|
||||||
|
meta: dict[str, Any] = Field(default_factory=dict)
|
||||||
|
|
||||||
|
|
||||||
|
class ExternalQueryExecution(BaseModel):
|
||||||
|
model_config = ConfigDict(extra="ignore")
|
||||||
|
|
||||||
|
plan: ExternalQueryPlan
|
||||||
|
adapter_name: str
|
||||||
|
evidence: list[ExternalEvidenceItem] = Field(default_factory=list)
|
||||||
|
verifier: dict[str, Any] = Field(default_factory=dict)
|
||||||
|
planner_meta: dict[str, Any] = Field(default_factory=dict)
|
||||||
|
meta: dict[str, Any] = Field(default_factory=dict)
|
||||||
|
|
@ -200,6 +200,7 @@ def map_qa_feedback_to_record_payload(*, qa_feedback: Any, qa_record: Any) -> di
|
||||||
"sources": _jsonable(getattr(qa_record, "sources", None) or []),
|
"sources": _jsonable(getattr(qa_record, "sources", None) or []),
|
||||||
"citations": _jsonable(getattr(qa_record, "citations", None) or []),
|
"citations": _jsonable(getattr(qa_record, "citations", None) or []),
|
||||||
"sources_used": _jsonable(getattr(qa_record, "sources_used", None) or []),
|
"sources_used": _jsonable(getattr(qa_record, "sources_used", None) or []),
|
||||||
|
"routing_snapshot": _jsonable(getattr(qa_record, "routing_snapshot", None) or {}),
|
||||||
}
|
}
|
||||||
after_value = {
|
after_value = {
|
||||||
"rating": rating,
|
"rating": rating,
|
||||||
|
|
@ -233,6 +234,7 @@ def map_qa_feedback_to_record_payload(*, qa_feedback: Any, qa_record: Any) -> di
|
||||||
"answer_mode": getattr(qa_record, "answer_mode", None),
|
"answer_mode": getattr(qa_record, "answer_mode", None),
|
||||||
"grounding_status": getattr(qa_record, "grounding_status", None),
|
"grounding_status": getattr(qa_record, "grounding_status", None),
|
||||||
"policy_version": getattr(qa_record, "policy_version", None),
|
"policy_version": getattr(qa_record, "policy_version", None),
|
||||||
|
"routing_snapshot": _jsonable(getattr(qa_record, "routing_snapshot", None) or {}),
|
||||||
"citations": _jsonable(getattr(qa_record, "citations", None) or []),
|
"citations": _jsonable(getattr(qa_record, "citations", None) or []),
|
||||||
"sources_used": _jsonable(getattr(qa_record, "sources_used", None) or []),
|
"sources_used": _jsonable(getattr(qa_record, "sources_used", None) or []),
|
||||||
"evidence_pack": _jsonable(getattr(qa_record, "evidence_pack", None) or {}),
|
"evidence_pack": _jsonable(getattr(qa_record, "evidence_pack", None) or {}),
|
||||||
|
|
|
||||||
|
|
@ -379,13 +379,28 @@ class FeishuCardActionMixin:
|
||||||
synthetic_text = "重出一版"
|
synthetic_text = "重出一版"
|
||||||
if not synthetic_text:
|
if not synthetic_text:
|
||||||
return {"code": 0, "msg": "card_action_ignored", "data": {"reason": "missing_synthetic_text"}}
|
return {"code": 0, "msg": "card_action_ignored", "data": {"reason": "missing_synthetic_text"}}
|
||||||
kernel = AIPMOKernelService(self.db, tenant_id=self.tenant_id)
|
use_feishu_context = bool(chat_id and source_message_id)
|
||||||
result = await kernel.process_ingress(
|
ingress_source_system = "feishu" if use_feishu_context else "ai_pmo_ui"
|
||||||
source_system="ai_pmo_ui",
|
ingress_source_message_id = (
|
||||||
source_message_id=(
|
f"feishu-card-plan-{int(work_item_id)}-{action}-{int(time.time() * 1000)}"
|
||||||
|
if use_feishu_context
|
||||||
|
else (
|
||||||
source_message_id
|
source_message_id
|
||||||
or f"feishu-card-plan-{int(work_item_id)}-{action}-{int(time.time() * 1000)}"
|
or f"feishu-card-plan-{int(work_item_id)}-{action}-{int(time.time() * 1000)}"
|
||||||
),
|
)
|
||||||
|
)
|
||||||
|
ingress_options = None
|
||||||
|
if use_feishu_context:
|
||||||
|
ingress_options = {
|
||||||
|
"thread_context": {
|
||||||
|
"reply_to_message_id": str(source_message_id),
|
||||||
|
"root_message_id": str(source_message_id),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
kernel = AIPMOKernelService(self.db, tenant_id=self.tenant_id)
|
||||||
|
result = await kernel.process_ingress(
|
||||||
|
source_system=ingress_source_system,
|
||||||
|
source_message_id=ingress_source_message_id,
|
||||||
message_text=synthetic_text,
|
message_text=synthetic_text,
|
||||||
sender=f"{operator_sender}(card)",
|
sender=f"{operator_sender}(card)",
|
||||||
channel_ref=chat_id,
|
channel_ref=chat_id,
|
||||||
|
|
@ -393,6 +408,7 @@ class FeishuCardActionMixin:
|
||||||
captured_at=None,
|
captured_at=None,
|
||||||
chat_type="manual",
|
chat_type="manual",
|
||||||
project_hint_id=None,
|
project_hint_id=None,
|
||||||
|
ingress_options=ingress_options,
|
||||||
)
|
)
|
||||||
if hasattr(self.db, "commit") and callable(getattr(self.db, "commit")):
|
if hasattr(self.db, "commit") and callable(getattr(self.db, "commit")):
|
||||||
await self.db.commit()
|
await self.db.commit()
|
||||||
|
|
|
||||||
|
|
@ -181,9 +181,13 @@ class LLMCacheCircuitMixin:
|
||||||
return
|
return
|
||||||
now = time.monotonic()
|
now = time.monotonic()
|
||||||
state = dict(cls._provider_circuit_state.get(str(circuit_key)) or {})
|
state = dict(cls._provider_circuit_state.get(str(circuit_key)) or {})
|
||||||
consecutive = int(state.get("consecutive_failures") or 0) + 1
|
fail_fast = cls._is_fail_fast_exception(exc)
|
||||||
|
if fail_fast:
|
||||||
|
consecutive = CIRCUIT_BREAKER_FAILURE_THRESHOLD
|
||||||
|
else:
|
||||||
|
consecutive = int(state.get("consecutive_failures") or 0) + 1
|
||||||
open_until = float(state.get("open_until") or 0.0)
|
open_until = float(state.get("open_until") or 0.0)
|
||||||
if consecutive >= CIRCUIT_BREAKER_FAILURE_THRESHOLD:
|
if fail_fast or consecutive >= CIRCUIT_BREAKER_FAILURE_THRESHOLD:
|
||||||
open_until = now + CIRCUIT_BREAKER_COOLDOWN_SECONDS
|
open_until = now + CIRCUIT_BREAKER_COOLDOWN_SECONDS
|
||||||
cls._provider_circuit_state[str(circuit_key)] = {
|
cls._provider_circuit_state[str(circuit_key)] = {
|
||||||
"consecutive_failures": consecutive,
|
"consecutive_failures": consecutive,
|
||||||
|
|
|
||||||
|
|
@ -9,8 +9,30 @@ class LLMEmptyContentError(ValueError):
|
||||||
|
|
||||||
class LLMChatOpenAIMixin:
|
class LLMChatOpenAIMixin:
|
||||||
@staticmethod
|
@staticmethod
|
||||||
def _append_direct_answer_retry_instruction(messages: list[dict[str, str]]) -> None:
|
def _is_provider_billing_blocked_http_error(*, status_code: int, body_preview: str) -> bool:
|
||||||
retry_instruction = "请直接输出答案"
|
if int(status_code or 0) not in {400, 401, 402, 403}:
|
||||||
|
return False
|
||||||
|
lowered = str(body_preview or "").strip().lower()
|
||||||
|
if not lowered:
|
||||||
|
return False
|
||||||
|
return (
|
||||||
|
"arrearage" in lowered
|
||||||
|
or "overdue-payment" in lowered
|
||||||
|
or ("access denied" in lowered and "good standing" in lowered)
|
||||||
|
or "provider_billing_blocked" in lowered
|
||||||
|
)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _append_direct_answer_retry_instruction(
|
||||||
|
messages: list[dict[str, str]],
|
||||||
|
*,
|
||||||
|
structured_output: bool = False,
|
||||||
|
) -> None:
|
||||||
|
retry_instruction = (
|
||||||
|
"请直接输出最终 JSON 对象,不要解释,不要 Markdown,不要把正文放在 reasoning_content。"
|
||||||
|
if structured_output
|
||||||
|
else "请直接输出答案"
|
||||||
|
)
|
||||||
for idx in range(len(messages) - 1, -1, -1):
|
for idx in range(len(messages) - 1, -1, -1):
|
||||||
row = messages[idx]
|
row = messages[idx]
|
||||||
if not isinstance(row, dict):
|
if not isinstance(row, dict):
|
||||||
|
|
@ -45,6 +67,7 @@ class LLMChatOpenAIMixin:
|
||||||
request_started_at_perf: float,
|
request_started_at_perf: float,
|
||||||
request_trace_id: str,
|
request_trace_id: str,
|
||||||
control_plane_agent_timeout: Optional[int],
|
control_plane_agent_timeout: Optional[int],
|
||||||
|
routing_meta: dict[str, Any] | None = None,
|
||||||
) -> ChatCompletionResult:
|
) -> ChatCompletionResult:
|
||||||
endpoint = f"{base_url.rstrip('/')}/chat/completions"
|
endpoint = f"{base_url.rstrip('/')}/chat/completions"
|
||||||
headers = {
|
headers = {
|
||||||
|
|
@ -127,6 +150,11 @@ class LLMChatOpenAIMixin:
|
||||||
),
|
),
|
||||||
)
|
)
|
||||||
status_code = int(resp.status_code)
|
status_code = int(resp.status_code)
|
||||||
|
if self._is_provider_billing_blocked_http_error(
|
||||||
|
status_code=status_code,
|
||||||
|
body_preview=body_preview,
|
||||||
|
):
|
||||||
|
raise ValueError(f"provider_billing_blocked:{provider_name}:{body_preview}")
|
||||||
can_retry_http_status = (
|
can_retry_http_status = (
|
||||||
http_status_retry_budget > 0
|
http_status_retry_budget > 0
|
||||||
and self._is_retryable_http_status(status_code)
|
and self._is_retryable_http_status(status_code)
|
||||||
|
|
@ -185,14 +213,27 @@ class LLMChatOpenAIMixin:
|
||||||
if self._contains_json_container(normalized_reasoning):
|
if self._contains_json_container(normalized_reasoning):
|
||||||
candidate = self._extract_json_candidate(normalized_reasoning)
|
candidate = self._extract_json_candidate(normalized_reasoning)
|
||||||
if candidate:
|
if candidate:
|
||||||
_runtime_log(
|
if self._call_is_invalid_structured_output(
|
||||||
"LLM",
|
function_name=function_label,
|
||||||
(
|
response_text=candidate,
|
||||||
"结构化reasoning_content回退:"
|
routing_meta=routing_meta,
|
||||||
f"function={function_label}, provider={provider_name}"
|
):
|
||||||
),
|
_runtime_log(
|
||||||
)
|
"LLM",
|
||||||
text = candidate
|
(
|
||||||
|
"结构化reasoning_content候选无效,继续空content重试:"
|
||||||
|
f"function={function_label}, provider={provider_name}"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
_runtime_log(
|
||||||
|
"LLM",
|
||||||
|
(
|
||||||
|
"结构化reasoning_content回退:"
|
||||||
|
f"function={function_label}, provider={provider_name}"
|
||||||
|
),
|
||||||
|
)
|
||||||
|
text = candidate
|
||||||
if text:
|
if text:
|
||||||
break
|
break
|
||||||
|
|
||||||
|
|
@ -209,7 +250,10 @@ class LLMChatOpenAIMixin:
|
||||||
)
|
)
|
||||||
if not direct_answer_retry_used:
|
if not direct_answer_retry_used:
|
||||||
direct_answer_retry_used = True
|
direct_answer_retry_used = True
|
||||||
self._append_direct_answer_retry_instruction(messages)
|
self._append_direct_answer_retry_instruction(
|
||||||
|
messages,
|
||||||
|
structured_output=structured_output,
|
||||||
|
)
|
||||||
_runtime_log(
|
_runtime_log(
|
||||||
"LLM",
|
"LLM",
|
||||||
(
|
(
|
||||||
|
|
|
||||||
|
|
@ -129,6 +129,7 @@ class LLMChatTransportMixin:
|
||||||
request_started_at_perf=request_started_at_perf,
|
request_started_at_perf=request_started_at_perf,
|
||||||
request_trace_id=request_trace_id,
|
request_trace_id=request_trace_id,
|
||||||
control_plane_agent_timeout=control_plane_agent_timeout,
|
control_plane_agent_timeout=control_plane_agent_timeout,
|
||||||
|
routing_meta=normalized_routing_meta,
|
||||||
)
|
)
|
||||||
if use_provider_circuit_breaker:
|
if use_provider_circuit_breaker:
|
||||||
self._record_provider_success(circuit_key)
|
self._record_provider_success(circuit_key)
|
||||||
|
|
|
||||||
|
|
@ -3,6 +3,47 @@ from __future__ import annotations
|
||||||
from .shared import *
|
from .shared import *
|
||||||
|
|
||||||
class LLMTaskHelpersMixin:
|
class LLMTaskHelpersMixin:
|
||||||
|
@staticmethod
|
||||||
|
def _extract_embedding_vector(payload: Any) -> list[float]:
|
||||||
|
if not isinstance(payload, dict):
|
||||||
|
return []
|
||||||
|
if isinstance(payload.get("embedding"), list):
|
||||||
|
return [float(item) for item in payload.get("embedding") or []]
|
||||||
|
embeddings = payload.get("embeddings")
|
||||||
|
if isinstance(embeddings, list) and embeddings and isinstance(embeddings[0], list):
|
||||||
|
return [float(item) for item in embeddings[0]]
|
||||||
|
data = payload.get("data")
|
||||||
|
if isinstance(data, list) and data and isinstance(data[0], dict):
|
||||||
|
embedding = data[0].get("embedding")
|
||||||
|
if isinstance(embedding, list):
|
||||||
|
return [float(item) for item in embedding]
|
||||||
|
return []
|
||||||
|
|
||||||
|
async def _request_openai_embedding(self, *, endpoint: str, headers: dict[str, str], payload: dict[str, Any]) -> list[float]:
|
||||||
|
async with httpx.AsyncClient(timeout=self.request_timeout, trust_env=False) as client:
|
||||||
|
resp = await client.post(endpoint, headers=headers, json=payload)
|
||||||
|
resp.raise_for_status()
|
||||||
|
return self._extract_embedding_vector(resp.json())
|
||||||
|
|
||||||
|
async def _request_ollama_embedding(self, *, host: str, model: str, text: str) -> list[float]:
|
||||||
|
base_url = str(host or os.getenv("OLLAMA_HOST", "http://127.0.0.1:11434")).rstrip("/")
|
||||||
|
headers = {"Content-Type": "application/json"}
|
||||||
|
candidates = [
|
||||||
|
(f"{base_url}/api/embed", {"model": model, "input": text}),
|
||||||
|
(f"{base_url}/api/embeddings", {"model": model, "prompt": text}),
|
||||||
|
]
|
||||||
|
for endpoint, payload in candidates:
|
||||||
|
try:
|
||||||
|
async with httpx.AsyncClient(timeout=self.request_timeout, trust_env=False) as client:
|
||||||
|
resp = await client.post(endpoint, headers=headers, json=payload)
|
||||||
|
resp.raise_for_status()
|
||||||
|
vector = self._extract_embedding_vector(resp.json())
|
||||||
|
if vector:
|
||||||
|
return vector
|
||||||
|
except Exception:
|
||||||
|
continue
|
||||||
|
return []
|
||||||
|
|
||||||
async def generate_task_breakdown_async(self, description: str, **kwargs: Any) -> list[dict]:
|
async def generate_task_breakdown_async(self, description: str, **kwargs: Any) -> list[dict]:
|
||||||
if not description.strip():
|
if not description.strip():
|
||||||
return []
|
return []
|
||||||
|
|
@ -27,14 +68,18 @@ class LLMTaskHelpersMixin:
|
||||||
]
|
]
|
||||||
|
|
||||||
async def get_embedding_async(self, text: str, **kwargs: Any) -> list[float]:
|
async def get_embedding_async(self, text: str, **kwargs: Any) -> list[float]:
|
||||||
|
clean_text = str(text or "").strip()
|
||||||
|
if not clean_text:
|
||||||
|
return []
|
||||||
db = kwargs.get("db") or kwargs.get("db_session")
|
db = kwargs.get("db") or kwargs.get("db_session")
|
||||||
tenant_id = int(kwargs.get("tenant_id") or 1)
|
tenant_id = int(kwargs.get("tenant_id") or 1)
|
||||||
if db is None:
|
if db is None:
|
||||||
async with AsyncSessionLocal() as session:
|
async with AsyncSessionLocal() as session:
|
||||||
return await self.get_embedding_async(text, db=session, tenant_id=tenant_id)
|
return await self.get_embedding_async(clean_text, db=session, tenant_id=tenant_id)
|
||||||
|
engine: dict[str, Any] = {}
|
||||||
try:
|
try:
|
||||||
engine = await self._resolve_active_engine(db, tenant_id=tenant_id)
|
engine = await self._resolve_active_engine(db, tenant_id=tenant_id)
|
||||||
provider = str(engine.get("provider") or "")
|
provider = str(engine.get("source_provider") or engine.get("provider") or "").strip().lower()
|
||||||
if provider == "openai":
|
if provider == "openai":
|
||||||
endpoint = f"{str(engine.get('base_url') or '').rstrip('/')}/embeddings"
|
endpoint = f"{str(engine.get('base_url') or '').rstrip('/')}/embeddings"
|
||||||
headers = {
|
headers = {
|
||||||
|
|
@ -43,18 +88,29 @@ class LLMTaskHelpersMixin:
|
||||||
}
|
}
|
||||||
payload = {
|
payload = {
|
||||||
"model": str(engine.get("model") or "text-embedding-3-small"),
|
"model": str(engine.get("model") or "text-embedding-3-small"),
|
||||||
"input": text,
|
"input": clean_text,
|
||||||
}
|
}
|
||||||
async with httpx.AsyncClient(timeout=self.request_timeout, trust_env=False) as client:
|
vector = await self._request_openai_embedding(endpoint=endpoint, headers=headers, payload=payload)
|
||||||
resp = await client.post(endpoint, headers=headers, json=payload)
|
if vector:
|
||||||
resp.raise_for_status()
|
return vector
|
||||||
data = resp.json()
|
if provider == "ollama":
|
||||||
return list(data.get("data", [{}])[0].get("embedding") or [])
|
vector = await self._request_ollama_embedding(
|
||||||
|
host=str(engine.get("base_url") or os.getenv("OLLAMA_HOST", "http://127.0.0.1:11434")),
|
||||||
|
model=str(engine.get("model") or os.getenv("OLLAMA_EMBED_MODEL", "nomic-embed-text:latest")),
|
||||||
|
text=clean_text,
|
||||||
|
)
|
||||||
|
if vector:
|
||||||
|
return vector
|
||||||
except Exception:
|
except Exception:
|
||||||
pass
|
engine = dict(engine) if isinstance(engine, dict) else {}
|
||||||
# DeepSeek/GLM 及失败场景:回退伪向量,保持功能可用。
|
vector = await self._request_ollama_embedding(
|
||||||
seed = hashlib.md5(text.encode("utf-8")).digest()
|
host=str(os.getenv("OLLAMA_HOST", "http://127.0.0.1:11434")),
|
||||||
return [((seed[i % 16] / 255.0) * 2 - 1) for i in range(64)]
|
model=str(os.getenv("OLLAMA_EMBED_MODEL", "nomic-embed-text:latest")),
|
||||||
|
text=clean_text,
|
||||||
|
)
|
||||||
|
if vector:
|
||||||
|
return vector
|
||||||
|
return []
|
||||||
|
|
||||||
async def extract_context_insights_async(self, text: str, **kwargs: Any) -> dict:
|
async def extract_context_insights_async(self, text: str, **kwargs: Any) -> dict:
|
||||||
prompt = (
|
prompt = (
|
||||||
|
|
@ -102,4 +158,3 @@ class LLMTaskHelpersMixin:
|
||||||
"passed": passed,
|
"passed": passed,
|
||||||
"reason": "basic_stub_check_passed" if passed else "invalid_input",
|
"reason": "basic_stub_check_passed" if passed else "invalid_input",
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|
|
||||||
|
|
@ -122,6 +122,10 @@ FAIL_FAST_ERROR_MARKERS = (
|
||||||
"返回空内容",
|
"返回空内容",
|
||||||
"empty_response",
|
"empty_response",
|
||||||
"provider_circuit_open",
|
"provider_circuit_open",
|
||||||
|
"provider_billing_blocked",
|
||||||
|
"arrearage",
|
||||||
|
"overdue-payment",
|
||||||
|
"account is in good standing",
|
||||||
)
|
)
|
||||||
CIRCUIT_BREAKER_FAILURE_THRESHOLD = 2
|
CIRCUIT_BREAKER_FAILURE_THRESHOLD = 2
|
||||||
CIRCUIT_BREAKER_COOLDOWN_SECONDS = 300.0
|
CIRCUIT_BREAKER_COOLDOWN_SECONDS = 300.0
|
||||||
|
|
@ -190,6 +194,36 @@ STRUCTURED_OBJECT_SCHEMAS: dict[str, dict[str, Any]] = {
|
||||||
"uncertainties": ("list",),
|
"uncertainties": ("list",),
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
|
"external_query_plan": {
|
||||||
|
"required_keys": (
|
||||||
|
"plan_version",
|
||||||
|
"query_kind",
|
||||||
|
"answer_contract",
|
||||||
|
"freshness_required",
|
||||||
|
"topic",
|
||||||
|
"search_queries",
|
||||||
|
"result_limit",
|
||||||
|
"requires_multi_source",
|
||||||
|
"clarify_on_ambiguity",
|
||||||
|
),
|
||||||
|
"type_checks": {
|
||||||
|
"plan_version": "str",
|
||||||
|
"query_kind": "str",
|
||||||
|
"answer_contract": "str",
|
||||||
|
"freshness_required": "bool",
|
||||||
|
"topic": "str",
|
||||||
|
"search_queries": "list",
|
||||||
|
"result_limit": "number",
|
||||||
|
"requires_multi_source": "bool",
|
||||||
|
"clarify_on_ambiguity": "bool",
|
||||||
|
},
|
||||||
|
"optional_type_checks": {
|
||||||
|
"subject_entities": ("list",),
|
||||||
|
"preferred_domains": ("list",),
|
||||||
|
"location_hint": ("str_or_none",),
|
||||||
|
"time_range_hint": ("str_or_none",),
|
||||||
|
},
|
||||||
|
},
|
||||||
"signal_planner_analyze": {
|
"signal_planner_analyze": {
|
||||||
"required_keys": ("summary", "priority", "execution_plan", "risks", "confidence"),
|
"required_keys": ("summary", "priority", "execution_plan", "risks", "confidence"),
|
||||||
"type_checks": {
|
"type_checks": {
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,3 @@
|
||||||
|
from .service import McpConfigService
|
||||||
|
|
||||||
|
__all__ = ["McpConfigService"]
|
||||||
|
|
@ -0,0 +1,87 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from .schemas import McpFieldSpec, McpTemplateSpec
|
||||||
|
|
||||||
|
|
||||||
|
def build_template_catalog() -> list[McpTemplateSpec]:
|
||||||
|
return [
|
||||||
|
McpTemplateSpec(
|
||||||
|
id="microsoft_learn",
|
||||||
|
name="Microsoft Learn MCP",
|
||||||
|
description="官方远程 MCP,无需 Token,可直接接入 Microsoft Learn 文档检索。",
|
||||||
|
transport="streamable_http",
|
||||||
|
execution_mode="backend_direct",
|
||||||
|
icon="book-open",
|
||||||
|
recommended=True,
|
||||||
|
docs_url="https://github.com/MicrosoftDocs/mcp",
|
||||||
|
fields=[],
|
||||||
|
command_hint="保存后由 PMFlow 后端直接连接远程 MCP,不要求客户机器安装额外 MCP host。",
|
||||||
|
verify_hint="重新校验时,PMFlow 会从后端执行 MCP 握手与 tools/list 探测。",
|
||||||
|
),
|
||||||
|
McpTemplateSpec(
|
||||||
|
id="github",
|
||||||
|
name="GitHub MCP",
|
||||||
|
description="官方 GitHub MCP,面向仓库、Issue、PR 等开发资产。",
|
||||||
|
transport="stdio",
|
||||||
|
execution_mode="local_bridge",
|
||||||
|
icon="github",
|
||||||
|
docs_url="https://github.com/modelcontextprotocol/servers",
|
||||||
|
fields=[
|
||||||
|
McpFieldSpec(
|
||||||
|
key="token_env_var_name",
|
||||||
|
label="本地 Token 环境变量名",
|
||||||
|
field_type="text",
|
||||||
|
required=True,
|
||||||
|
placeholder="GITHUB_PAT",
|
||||||
|
description="PMFlow 只保存环境变量名,不保存明文 Token。",
|
||||||
|
default_value="GITHUB_PAT",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
command_hint="需要 PMFlow Local Bridge 或兼容 MCP host 在客户本机承载 stdio server。",
|
||||||
|
verify_hint="确认本机 bridge/host 已接入后,再回到这里做手动确认。",
|
||||||
|
),
|
||||||
|
McpTemplateSpec(
|
||||||
|
id="filesystem",
|
||||||
|
name="Filesystem MCP",
|
||||||
|
description="官方 reference server,面向受控目录的本地文件访问。",
|
||||||
|
transport="stdio",
|
||||||
|
execution_mode="local_bridge",
|
||||||
|
icon="folder-open",
|
||||||
|
docs_url="https://github.com/modelcontextprotocol/servers",
|
||||||
|
fields=[
|
||||||
|
McpFieldSpec(
|
||||||
|
key="allowed_paths",
|
||||||
|
label="允许目录",
|
||||||
|
field_type="path_list",
|
||||||
|
required=True,
|
||||||
|
placeholder="/absolute/path/one",
|
||||||
|
description="每行一个绝对路径,只暴露给 MCP 允许访问的目录。",
|
||||||
|
default_value=[],
|
||||||
|
),
|
||||||
|
],
|
||||||
|
command_hint="需要 PMFlow Local Bridge 或兼容 MCP host 在客户本机启动 filesystem server。",
|
||||||
|
verify_hint="确认本机 bridge/host 已接入并暴露目录后,再回到这里做手动确认。",
|
||||||
|
),
|
||||||
|
McpTemplateSpec(
|
||||||
|
id="playwright",
|
||||||
|
name="Playwright MCP",
|
||||||
|
description="官方 Playwright MCP,适合浏览器自动化和页面抓取。",
|
||||||
|
transport="stdio",
|
||||||
|
execution_mode="local_bridge",
|
||||||
|
icon="monitor-play",
|
||||||
|
docs_url="https://playwright.dev/java/agents/network-and-security",
|
||||||
|
fields=[
|
||||||
|
McpFieldSpec(
|
||||||
|
key="output_dir",
|
||||||
|
label="输出目录",
|
||||||
|
field_type="text",
|
||||||
|
required=False,
|
||||||
|
placeholder="~/.pmflow/mcp/playwright-output",
|
||||||
|
description="用于保存 Playwright MCP 输出工件的目录。",
|
||||||
|
default_value="~/.pmflow/mcp/playwright-output",
|
||||||
|
),
|
||||||
|
],
|
||||||
|
command_hint="需要 PMFlow Local Bridge 或兼容 MCP host 在客户本机启动 Playwright MCP。",
|
||||||
|
verify_hint="确认本机 bridge/host 已接入并能产出浏览器工件后,再回到这里做手动确认。",
|
||||||
|
),
|
||||||
|
]
|
||||||
|
|
@ -0,0 +1,149 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
from typing import Any, Callable
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
|
||||||
|
from ...utils.timezone import now_cn_naive
|
||||||
|
from ..external_query_service.schemas import ExternalEvidenceItem
|
||||||
|
|
||||||
|
|
||||||
|
class MicrosoftLearnMCPRuntime:
|
||||||
|
def __init__(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
endpoint_url: str = "https://learn.microsoft.com/api/mcp",
|
||||||
|
client_factory: Callable[[], httpx.AsyncClient] | None = None,
|
||||||
|
) -> None:
|
||||||
|
self.endpoint_url = str(endpoint_url or "https://learn.microsoft.com/api/mcp").strip()
|
||||||
|
self.client_factory = client_factory or (
|
||||||
|
lambda: httpx.AsyncClient(timeout=20.0, trust_env=False, follow_redirects=True)
|
||||||
|
)
|
||||||
|
|
||||||
|
async def probe(self) -> dict[str, Any]:
|
||||||
|
checked_at = now_cn_naive().isoformat()
|
||||||
|
try:
|
||||||
|
session_id, _ = await self._initialize()
|
||||||
|
_, tools_payload = await self._post_jsonrpc(
|
||||||
|
{
|
||||||
|
"jsonrpc": "2.0",
|
||||||
|
"id": 2,
|
||||||
|
"method": "tools/list",
|
||||||
|
"params": {},
|
||||||
|
},
|
||||||
|
session_id=session_id,
|
||||||
|
)
|
||||||
|
tools = list((tools_payload.get("result") or {}).get("tools") or [])
|
||||||
|
tool_names = [str(item.get("name") or "").strip() for item in tools if str(item.get("name") or "").strip()]
|
||||||
|
return {
|
||||||
|
"status": "ready",
|
||||||
|
"checked_at": checked_at,
|
||||||
|
"message": "PMFlow backend can directly reach the remote MCP server.",
|
||||||
|
"tool_names": tool_names,
|
||||||
|
}
|
||||||
|
except Exception as exc:
|
||||||
|
return {
|
||||||
|
"status": "error",
|
||||||
|
"checked_at": checked_at,
|
||||||
|
"message": f"PMFlow backend probe failed: {exc}",
|
||||||
|
"tool_names": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
async def search(self, *, query: str, result_limit: int = 3) -> list[ExternalEvidenceItem]:
|
||||||
|
session_id, _ = await self._initialize()
|
||||||
|
_, payload = await self._post_jsonrpc(
|
||||||
|
{
|
||||||
|
"jsonrpc": "2.0",
|
||||||
|
"id": 3,
|
||||||
|
"method": "tools/call",
|
||||||
|
"params": {
|
||||||
|
"name": "microsoft_docs_search",
|
||||||
|
"arguments": {
|
||||||
|
"query": str(query or "").strip(),
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
session_id=session_id,
|
||||||
|
)
|
||||||
|
text_payload = ""
|
||||||
|
for item in list((payload.get("result") or {}).get("content") or []):
|
||||||
|
if str(item.get("type") or "").strip() == "text":
|
||||||
|
text_payload = str(item.get("text") or "").strip()
|
||||||
|
if text_payload:
|
||||||
|
break
|
||||||
|
parsed = json.loads(text_payload) if text_payload else {}
|
||||||
|
results = list(parsed.get("results") or [])
|
||||||
|
retrieved_at = now_cn_naive().isoformat()
|
||||||
|
evidence: list[ExternalEvidenceItem] = []
|
||||||
|
for raw in results[: max(1, int(result_limit or 3))]:
|
||||||
|
target_url = str(raw.get("contentUrl") or "").strip()
|
||||||
|
if not target_url.startswith("http://") and not target_url.startswith("https://"):
|
||||||
|
continue
|
||||||
|
title = str(raw.get("title") or "").strip()
|
||||||
|
snippet = str(raw.get("content") or "").strip()
|
||||||
|
evidence.append(
|
||||||
|
ExternalEvidenceItem(
|
||||||
|
source_type="mcp_search",
|
||||||
|
source_label="Microsoft Learn MCP",
|
||||||
|
title=title,
|
||||||
|
target_url=target_url,
|
||||||
|
snippet=snippet[:1800],
|
||||||
|
retrieved_at=retrieved_at,
|
||||||
|
freshness_score=0.86,
|
||||||
|
authority_score=0.98,
|
||||||
|
grounded_fields=["microsoft_docs_search"],
|
||||||
|
raw_metadata={
|
||||||
|
"provider": "microsoft_learn_mcp",
|
||||||
|
"result_id": str(raw.get("id") or "").strip() or None,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return evidence
|
||||||
|
|
||||||
|
async def _initialize(self) -> tuple[str | None, dict[str, Any]]:
|
||||||
|
return await self._post_jsonrpc(
|
||||||
|
{
|
||||||
|
"jsonrpc": "2.0",
|
||||||
|
"id": 1,
|
||||||
|
"method": "initialize",
|
||||||
|
"params": {
|
||||||
|
"protocolVersion": "2025-03-26",
|
||||||
|
"capabilities": {},
|
||||||
|
"clientInfo": {
|
||||||
|
"name": "pmflow",
|
||||||
|
"version": "0.1.0",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
async def _post_jsonrpc(
|
||||||
|
self,
|
||||||
|
payload: dict[str, Any],
|
||||||
|
*,
|
||||||
|
session_id: str | None = None,
|
||||||
|
) -> tuple[str | None, dict[str, Any]]:
|
||||||
|
headers = {
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
"Accept": "application/json, text/event-stream",
|
||||||
|
}
|
||||||
|
if session_id:
|
||||||
|
headers["mcp-session-id"] = session_id
|
||||||
|
async with self.client_factory() as client:
|
||||||
|
response = await client.post(self.endpoint_url, headers=headers, json=payload)
|
||||||
|
response.raise_for_status()
|
||||||
|
next_session = str(response.headers.get("mcp-session-id") or session_id or "").strip() or None
|
||||||
|
body = response.text
|
||||||
|
parsed = self._extract_payload(body)
|
||||||
|
return next_session, parsed
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _extract_payload(body: str) -> dict[str, Any]:
|
||||||
|
data_lines: list[str] = []
|
||||||
|
for line in str(body or "").splitlines():
|
||||||
|
if line.startswith("data:"):
|
||||||
|
data_lines.append(line[len("data:") :].lstrip())
|
||||||
|
if not data_lines:
|
||||||
|
raise ValueError("MCP streamable_http response missing data payload")
|
||||||
|
return json.loads("\n".join(data_lines))
|
||||||
|
|
@ -0,0 +1,62 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from typing import Literal
|
||||||
|
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
|
McpClientApplyState = Literal["pending_local_apply", "user_confirmed", "not_required"]
|
||||||
|
McpExecutionMode = Literal["backend_direct", "local_bridge"]
|
||||||
|
McpRuntimeProbeStatus = Literal["unknown", "ready", "error", "local_bridge_required"]
|
||||||
|
|
||||||
|
|
||||||
|
class McpFieldSpec(BaseModel):
|
||||||
|
key: str
|
||||||
|
label: str
|
||||||
|
field_type: Literal["text", "path_list", "readonly"] = "text"
|
||||||
|
required: bool = False
|
||||||
|
placeholder: str | None = None
|
||||||
|
description: str | None = None
|
||||||
|
default_value: str | list[str] | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class McpCommandPreview(BaseModel):
|
||||||
|
add_command: str
|
||||||
|
verify_command: str
|
||||||
|
expected_output: str
|
||||||
|
|
||||||
|
|
||||||
|
class McpRuntimeProbe(BaseModel):
|
||||||
|
status: McpRuntimeProbeStatus = "unknown"
|
||||||
|
checked_at: str | None = None
|
||||||
|
message: str | None = None
|
||||||
|
tool_names: list[str] = Field(default_factory=list)
|
||||||
|
|
||||||
|
|
||||||
|
class McpTemplateSpec(BaseModel):
|
||||||
|
id: str
|
||||||
|
name: str
|
||||||
|
description: str
|
||||||
|
transport: Literal["stdio", "streamable_http"]
|
||||||
|
execution_mode: McpExecutionMode
|
||||||
|
icon: str
|
||||||
|
recommended: bool = False
|
||||||
|
docs_url: str | None = None
|
||||||
|
fields: list[McpFieldSpec] = Field(default_factory=list)
|
||||||
|
command_hint: str | None = None
|
||||||
|
verify_hint: str | None = None
|
||||||
|
|
||||||
|
|
||||||
|
class McpServerConfigPayload(BaseModel):
|
||||||
|
id: str
|
||||||
|
name: str
|
||||||
|
display_name: str
|
||||||
|
template_id: str
|
||||||
|
transport: Literal["stdio", "streamable_http"]
|
||||||
|
execution_mode: McpExecutionMode = "local_bridge"
|
||||||
|
params: dict[str, object] = Field(default_factory=dict)
|
||||||
|
preview: McpCommandPreview
|
||||||
|
client_apply_state: McpClientApplyState = "pending_local_apply"
|
||||||
|
runtime_probe: McpRuntimeProbe = Field(default_factory=McpRuntimeProbe)
|
||||||
|
created_at: str
|
||||||
|
updated_at: str
|
||||||
|
confirmed_at: str | None = None
|
||||||
|
|
@ -0,0 +1,374 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import re
|
||||||
|
from typing import Any
|
||||||
|
from uuid import uuid4
|
||||||
|
|
||||||
|
from sqlalchemy import and_, select
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
|
||||||
|
from ...models import Integration
|
||||||
|
from ...utils.timezone import now_cn_naive
|
||||||
|
from ..external_query_service.schemas import ExternalEvidenceItem
|
||||||
|
from .catalog import build_template_catalog
|
||||||
|
from .runtime import MicrosoftLearnMCPRuntime
|
||||||
|
from .schemas import McpCommandPreview, McpRuntimeProbe, McpServerConfigPayload, McpTemplateSpec
|
||||||
|
|
||||||
|
INTEGRATION_TYPE = "mcp_template_center"
|
||||||
|
_VALID_ENV_VAR = re.compile(r"^[A-Z_][A-Z0-9_]*$")
|
||||||
|
_VALID_NAME_CHARS = re.compile(r"[^a-z0-9_-]+")
|
||||||
|
|
||||||
|
|
||||||
|
class McpConfigService:
|
||||||
|
def __init__(self, db: AsyncSession, *, tenant_id: int) -> None:
|
||||||
|
self.db = db
|
||||||
|
self.tenant_id = int(tenant_id)
|
||||||
|
self._catalog = {item.id: item for item in build_template_catalog()}
|
||||||
|
|
||||||
|
async def get_catalog(self) -> dict[str, Any]:
|
||||||
|
items = [item.model_dump(mode="json") for item in self._catalog.values()]
|
||||||
|
recommended = next((item.id for item in self._catalog.values() if item.recommended), "microsoft_learn")
|
||||||
|
return {
|
||||||
|
"recommended_template_id": recommended,
|
||||||
|
"items": items,
|
||||||
|
}
|
||||||
|
|
||||||
|
async def list_servers(self) -> list[dict[str, Any]]:
|
||||||
|
_, bundle = await self._get_or_init_bundle()
|
||||||
|
items = [
|
||||||
|
McpServerConfigPayload.model_validate(self._normalize_server_record(item)).model_dump(mode="json")
|
||||||
|
for item in bundle.get("servers", [])
|
||||||
|
if isinstance(item, dict)
|
||||||
|
]
|
||||||
|
return sorted(items, key=lambda item: str(item.get("updated_at") or ""), reverse=True)
|
||||||
|
|
||||||
|
async def create_server(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
name: str,
|
||||||
|
display_name: str | None,
|
||||||
|
template_id: str,
|
||||||
|
params: dict[str, Any] | None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
row, bundle = await self._get_or_init_bundle()
|
||||||
|
template = self._get_template(template_id)
|
||||||
|
cleaned_name = self._normalize_server_name(name)
|
||||||
|
servers = [item for item in bundle.get("servers", []) if isinstance(item, dict)]
|
||||||
|
if any(str(item.get("name") or "") == cleaned_name for item in servers):
|
||||||
|
raise ValueError(f"MCP 配置名已存在:{cleaned_name}")
|
||||||
|
cleaned_params = self._validate_params(template, params or {})
|
||||||
|
preview = self._build_preview(template=template, name=cleaned_name, params=cleaned_params)
|
||||||
|
now_text = now_cn_naive().isoformat()
|
||||||
|
item = McpServerConfigPayload(
|
||||||
|
id=f"mcp_{uuid4().hex[:12]}",
|
||||||
|
name=cleaned_name,
|
||||||
|
display_name=str(display_name or template.name).strip() or template.name,
|
||||||
|
template_id=template.id,
|
||||||
|
transport=template.transport,
|
||||||
|
execution_mode=template.execution_mode,
|
||||||
|
params=cleaned_params,
|
||||||
|
preview=preview,
|
||||||
|
client_apply_state=self._default_client_apply_state(template),
|
||||||
|
runtime_probe=self._default_runtime_probe(template),
|
||||||
|
created_at=now_text,
|
||||||
|
updated_at=now_text,
|
||||||
|
confirmed_at=None,
|
||||||
|
).model_dump(mode="json")
|
||||||
|
servers.append(item)
|
||||||
|
bundle["servers"] = servers
|
||||||
|
row.config = bundle
|
||||||
|
await self.db.flush()
|
||||||
|
return item
|
||||||
|
|
||||||
|
async def update_server(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
server_id: str,
|
||||||
|
name: str | None = None,
|
||||||
|
display_name: str | None = None,
|
||||||
|
template_id: str | None = None,
|
||||||
|
params: dict[str, Any] | None = None,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
row, bundle = await self._get_or_init_bundle()
|
||||||
|
servers = [self._normalize_server_record(item) for item in bundle.get("servers", []) if isinstance(item, dict)]
|
||||||
|
target = self._find_server(servers, server_id=server_id)
|
||||||
|
if target is None:
|
||||||
|
raise LookupError("MCP 配置不存在")
|
||||||
|
|
||||||
|
next_template = self._get_template(str(template_id or target.get("template_id") or ""))
|
||||||
|
next_name = self._normalize_server_name(str(name or target.get("name") or ""))
|
||||||
|
if any(
|
||||||
|
str(item.get("id") or "") != str(server_id)
|
||||||
|
and str(item.get("name") or "") == next_name
|
||||||
|
for item in servers
|
||||||
|
):
|
||||||
|
raise ValueError(f"MCP 配置名已存在:{next_name}")
|
||||||
|
|
||||||
|
params_input = params if params is not None else dict(target.get("params") or {})
|
||||||
|
cleaned_params = self._validate_params(next_template, params_input)
|
||||||
|
preview = self._build_preview(template=next_template, name=next_name, params=cleaned_params)
|
||||||
|
|
||||||
|
name_changed = next_name != str(target.get("name") or "")
|
||||||
|
template_changed = next_template.id != str(target.get("template_id") or "")
|
||||||
|
params_changed = cleaned_params != dict(target.get("params") or {})
|
||||||
|
|
||||||
|
target["name"] = next_name
|
||||||
|
target["display_name"] = str(display_name or target.get("display_name") or next_template.name).strip() or next_template.name
|
||||||
|
target["template_id"] = next_template.id
|
||||||
|
target["transport"] = next_template.transport
|
||||||
|
target["execution_mode"] = next_template.execution_mode
|
||||||
|
target["params"] = cleaned_params
|
||||||
|
target["preview"] = preview.model_dump(mode="json")
|
||||||
|
target["updated_at"] = now_cn_naive().isoformat()
|
||||||
|
if name_changed or template_changed or params_changed:
|
||||||
|
target["client_apply_state"] = self._default_client_apply_state(next_template)
|
||||||
|
target["confirmed_at"] = None
|
||||||
|
target["runtime_probe"] = self._default_runtime_probe(next_template).model_dump(mode="json")
|
||||||
|
|
||||||
|
bundle["servers"] = servers
|
||||||
|
row.config = bundle
|
||||||
|
await self.db.flush()
|
||||||
|
return McpServerConfigPayload.model_validate(target).model_dump(mode="json")
|
||||||
|
|
||||||
|
async def validate_server(self, *, server_id: str) -> dict[str, Any]:
|
||||||
|
row, bundle = await self._get_or_init_bundle()
|
||||||
|
servers = [self._normalize_server_record(item) for item in bundle.get("servers", []) if isinstance(item, dict)]
|
||||||
|
target = self._find_server(servers, server_id=server_id)
|
||||||
|
if target is None:
|
||||||
|
raise LookupError("MCP 配置不存在")
|
||||||
|
template = self._get_template(str(target.get("template_id") or ""))
|
||||||
|
cleaned_params = self._validate_params(template, dict(target.get("params") or {}))
|
||||||
|
preview = self._build_preview(
|
||||||
|
template=template,
|
||||||
|
name=str(target.get("name") or ""),
|
||||||
|
params=cleaned_params,
|
||||||
|
)
|
||||||
|
target["params"] = cleaned_params
|
||||||
|
target["preview"] = preview.model_dump(mode="json")
|
||||||
|
if template.execution_mode == "backend_direct":
|
||||||
|
target["runtime_probe"] = (await self._probe_backend_runtime(template=template, params=cleaned_params))
|
||||||
|
else:
|
||||||
|
target["runtime_probe"] = self._default_runtime_probe(template).model_dump(mode="json")
|
||||||
|
bundle["servers"] = servers
|
||||||
|
row.config = bundle
|
||||||
|
await self.db.flush()
|
||||||
|
return {
|
||||||
|
"server_id": str(server_id),
|
||||||
|
"is_valid": True,
|
||||||
|
"preview": preview.model_dump(mode="json"),
|
||||||
|
"runtime_probe": dict(target.get("runtime_probe") or {}),
|
||||||
|
}
|
||||||
|
|
||||||
|
async def confirm_client_apply(self, *, server_id: str) -> dict[str, Any]:
|
||||||
|
row, bundle = await self._get_or_init_bundle()
|
||||||
|
servers = [self._normalize_server_record(item) for item in bundle.get("servers", []) if isinstance(item, dict)]
|
||||||
|
target = self._find_server(servers, server_id=server_id)
|
||||||
|
if target is None:
|
||||||
|
raise LookupError("MCP 配置不存在")
|
||||||
|
template = self._get_template(str(target.get("template_id") or ""))
|
||||||
|
if template.execution_mode == "backend_direct":
|
||||||
|
raise ValueError("后端直连模板不需要“已在本机执行”确认")
|
||||||
|
confirmed_at = now_cn_naive().isoformat()
|
||||||
|
target["client_apply_state"] = "user_confirmed"
|
||||||
|
target["confirmed_at"] = confirmed_at
|
||||||
|
target["updated_at"] = confirmed_at
|
||||||
|
bundle["servers"] = servers
|
||||||
|
row.config = bundle
|
||||||
|
await self.db.flush()
|
||||||
|
return McpServerConfigPayload.model_validate(target).model_dump(mode="json")
|
||||||
|
|
||||||
|
async def get_preferred_backend_direct_server(self, *, template_id: str | None = None) -> dict[str, Any] | None:
|
||||||
|
_, bundle = await self._get_or_init_bundle()
|
||||||
|
servers = [self._normalize_server_record(item) for item in bundle.get("servers", []) if isinstance(item, dict)]
|
||||||
|
ordered = sorted(servers, key=lambda item: str(item.get("updated_at") or ""), reverse=True)
|
||||||
|
for item in ordered:
|
||||||
|
if str(item.get("execution_mode") or "") != "backend_direct":
|
||||||
|
continue
|
||||||
|
if template_id and str(item.get("template_id") or "") != str(template_id):
|
||||||
|
continue
|
||||||
|
return McpServerConfigPayload.model_validate(item).model_dump(mode="json")
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def search_external(self, *, query: str, result_limit: int = 3) -> list[ExternalEvidenceItem]:
|
||||||
|
query_text = str(query or "").strip()
|
||||||
|
if not query_text:
|
||||||
|
return []
|
||||||
|
server = await self.get_preferred_backend_direct_server(template_id="microsoft_learn")
|
||||||
|
if not server:
|
||||||
|
return []
|
||||||
|
template = self._get_template(str(server.get("template_id") or ""))
|
||||||
|
if template.id != "microsoft_learn":
|
||||||
|
return []
|
||||||
|
runtime = MicrosoftLearnMCPRuntime()
|
||||||
|
return await runtime.search(query=query_text, result_limit=max(1, int(result_limit or 3)))
|
||||||
|
|
||||||
|
async def _get_or_init_bundle(self) -> tuple[Integration, dict[str, Any]]:
|
||||||
|
row = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(Integration).where(
|
||||||
|
and_(
|
||||||
|
Integration.tenant_id == self.tenant_id,
|
||||||
|
Integration.type == INTEGRATION_TYPE,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if row is None:
|
||||||
|
row = Integration(
|
||||||
|
tenant_id=self.tenant_id,
|
||||||
|
type=INTEGRATION_TYPE,
|
||||||
|
is_active=True,
|
||||||
|
config={"version": 1, "servers": []},
|
||||||
|
)
|
||||||
|
self.db.add(row)
|
||||||
|
await self.db.flush()
|
||||||
|
|
||||||
|
bundle = dict(row.config or {}) if isinstance(row.config, dict) else {}
|
||||||
|
if not isinstance(bundle.get("servers"), list):
|
||||||
|
bundle["servers"] = []
|
||||||
|
if not isinstance(bundle.get("version"), int):
|
||||||
|
bundle["version"] = 1
|
||||||
|
if bundle != row.config:
|
||||||
|
row.config = bundle
|
||||||
|
await self.db.flush()
|
||||||
|
return row, bundle
|
||||||
|
|
||||||
|
def _get_template(self, template_id: str) -> McpTemplateSpec:
|
||||||
|
template = self._catalog.get(str(template_id or ""))
|
||||||
|
if template is None:
|
||||||
|
raise ValueError(f"未知 MCP 模板:{template_id}")
|
||||||
|
return template
|
||||||
|
|
||||||
|
def _normalize_server_record(self, item: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
normalized = dict(item or {})
|
||||||
|
template = self._get_template(str(normalized.get("template_id") or ""))
|
||||||
|
normalized["transport"] = template.transport
|
||||||
|
normalized["execution_mode"] = template.execution_mode
|
||||||
|
if template.execution_mode == "backend_direct":
|
||||||
|
normalized["client_apply_state"] = "not_required"
|
||||||
|
else:
|
||||||
|
state = str(normalized.get("client_apply_state") or "").strip()
|
||||||
|
normalized["client_apply_state"] = state if state in {"pending_local_apply", "user_confirmed"} else "pending_local_apply"
|
||||||
|
if not isinstance(normalized.get("runtime_probe"), dict):
|
||||||
|
normalized["runtime_probe"] = self._default_runtime_probe(template).model_dump(mode="json")
|
||||||
|
if not isinstance(normalized.get("preview"), dict):
|
||||||
|
normalized["preview"] = self._build_preview(
|
||||||
|
template=template,
|
||||||
|
name=str(normalized.get("name") or ""),
|
||||||
|
params=dict(normalized.get("params") or {}),
|
||||||
|
).model_dump(mode="json")
|
||||||
|
return normalized
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _default_client_apply_state(template: McpTemplateSpec) -> str:
|
||||||
|
return "not_required" if template.execution_mode == "backend_direct" else "pending_local_apply"
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _default_runtime_probe(template: McpTemplateSpec) -> McpRuntimeProbe:
|
||||||
|
if template.execution_mode == "backend_direct":
|
||||||
|
return McpRuntimeProbe(
|
||||||
|
status="unknown",
|
||||||
|
checked_at=None,
|
||||||
|
message="保存后由 PMFlow 后端直连;点击“重新校验”可执行 MCP 握手探测。",
|
||||||
|
tool_names=[],
|
||||||
|
)
|
||||||
|
return McpRuntimeProbe(
|
||||||
|
status="local_bridge_required",
|
||||||
|
checked_at=None,
|
||||||
|
message="该模板依赖 PMFlow Local Bridge 或兼容 MCP host 在客户本机承载。",
|
||||||
|
tool_names=[],
|
||||||
|
)
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _normalize_server_name(value: str) -> str:
|
||||||
|
normalized = _VALID_NAME_CHARS.sub("-", str(value or "").strip().lower())
|
||||||
|
normalized = re.sub(r"-{2,}", "-", normalized).strip("-_")
|
||||||
|
if not normalized:
|
||||||
|
raise ValueError("MCP 配置名不能为空")
|
||||||
|
return normalized[:80]
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _find_server(servers: list[dict[str, Any]], *, server_id: str) -> dict[str, Any] | None:
|
||||||
|
for item in servers:
|
||||||
|
if str(item.get("id") or "") == str(server_id or ""):
|
||||||
|
return item
|
||||||
|
return None
|
||||||
|
|
||||||
|
def _validate_params(self, template: McpTemplateSpec, params: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
raw = dict(params or {})
|
||||||
|
if template.id == "microsoft_learn":
|
||||||
|
return {}
|
||||||
|
if template.id == "github":
|
||||||
|
for forbidden_key in ("access_token", "token", "personal_access_token", "pat"):
|
||||||
|
if str(raw.get(forbidden_key) or "").strip():
|
||||||
|
raise ValueError("GitHub MCP 只接受 Token 环境变量名,不要直接填写明文 token")
|
||||||
|
env_var_name = str(raw.get("token_env_var_name") or "").strip().upper()
|
||||||
|
if not env_var_name:
|
||||||
|
raise ValueError("token_env_var_name 不能为空")
|
||||||
|
if not _VALID_ENV_VAR.match(env_var_name):
|
||||||
|
raise ValueError("token_env_var_name 必须是合法的环境变量名")
|
||||||
|
return {"token_env_var_name": env_var_name}
|
||||||
|
if template.id == "filesystem":
|
||||||
|
value = raw.get("allowed_paths")
|
||||||
|
paths: list[str] = []
|
||||||
|
if isinstance(value, str):
|
||||||
|
paths = [item.strip() for item in value.splitlines() if item.strip()]
|
||||||
|
elif isinstance(value, list):
|
||||||
|
paths = [str(item).strip() for item in value if str(item).strip()]
|
||||||
|
if not paths:
|
||||||
|
raise ValueError("allowed_paths 至少需要一个目录")
|
||||||
|
return {"allowed_paths": paths}
|
||||||
|
if template.id == "playwright":
|
||||||
|
output_dir = str(raw.get("output_dir") or "~/.pmflow/mcp/playwright-output").strip()
|
||||||
|
return {"output_dir": output_dir}
|
||||||
|
raise ValueError(f"未知模板校验规则:{template.id}")
|
||||||
|
|
||||||
|
def _build_preview(self, *, template: McpTemplateSpec, name: str, params: dict[str, Any]) -> McpCommandPreview:
|
||||||
|
if template.id == "microsoft_learn":
|
||||||
|
return McpCommandPreview(
|
||||||
|
add_command="无需本机命令。保存后由 PMFlow 后端直接连接 Microsoft Learn MCP。",
|
||||||
|
verify_command="点击“重新校验”后,PMFlow 会从后端执行 MCP initialize 和 tools/list。",
|
||||||
|
expected_output="校验成功后,你会看到后端可用工具,例如 microsoft_docs_search、microsoft_docs_fetch。",
|
||||||
|
)
|
||||||
|
if template.id == "github":
|
||||||
|
env_name = str(params["token_env_var_name"])
|
||||||
|
return McpCommandPreview(
|
||||||
|
add_command=(
|
||||||
|
f"需要在客户本机的 PMFlow Local Bridge 或兼容 MCP host 中接入 GitHub MCP,"
|
||||||
|
f"并引用环境变量 {env_name}。"
|
||||||
|
),
|
||||||
|
verify_command="本机 bridge/host 接好后,再回到这里执行“标记为已在本机执行”。",
|
||||||
|
expected_output=f"bridge/host 中应能看到 GitHub MCP 已启用,并通过环境变量 {env_name} 读取 token。",
|
||||||
|
)
|
||||||
|
if template.id == "filesystem":
|
||||||
|
quoted_paths = " ".join(f'"{item}"' for item in params["allowed_paths"])
|
||||||
|
return McpCommandPreview(
|
||||||
|
add_command=(
|
||||||
|
"需要在客户本机的 PMFlow Local Bridge 或兼容 MCP host 中启动 Filesystem MCP,"
|
||||||
|
f"允许目录:{quoted_paths}"
|
||||||
|
),
|
||||||
|
verify_command="本机 bridge/host 接好后,再回到这里执行“标记为已在本机执行”。",
|
||||||
|
expected_output=f"bridge/host 中应能看到 Filesystem MCP 已启用,并暴露目录 {quoted_paths}。",
|
||||||
|
)
|
||||||
|
if template.id == "playwright":
|
||||||
|
output_dir = str(params["output_dir"])
|
||||||
|
return McpCommandPreview(
|
||||||
|
add_command=(
|
||||||
|
"需要在客户本机的 PMFlow Local Bridge 或兼容 MCP host 中启动 Playwright MCP,"
|
||||||
|
f"输出目录:\"{output_dir}\""
|
||||||
|
),
|
||||||
|
verify_command="本机 bridge/host 接好后,再回到这里执行“标记为已在本机执行”。",
|
||||||
|
expected_output=f"bridge/host 中应能看到 Playwright MCP 已启用,输出目录为 {output_dir}。",
|
||||||
|
)
|
||||||
|
raise ValueError(f"未知模板命令生成规则:{template.id}")
|
||||||
|
|
||||||
|
async def _probe_backend_runtime(self, *, template: McpTemplateSpec, params: dict[str, Any]) -> dict[str, Any]:
|
||||||
|
_ = params
|
||||||
|
if template.id == "microsoft_learn":
|
||||||
|
return await MicrosoftLearnMCPRuntime().probe()
|
||||||
|
return {
|
||||||
|
"status": "error",
|
||||||
|
"checked_at": now_cn_naive().isoformat(),
|
||||||
|
"message": f"模板 {template.id} 暂未实现后端直连 runtime。",
|
||||||
|
"tool_names": [],
|
||||||
|
}
|
||||||
|
|
@ -6,6 +6,7 @@ _WORK_ITEM_DELTA_CHANNEL = "work_item_delta"
|
||||||
_WORK_ITEM_DELTA_OFFSET_KEY = "last_work_item_updated_at"
|
_WORK_ITEM_DELTA_OFFSET_KEY = "last_work_item_updated_at"
|
||||||
_LEGACY_FEISHU_DELTA_CHANNEL = "feishu_delta"
|
_LEGACY_FEISHU_DELTA_CHANNEL = "feishu_delta"
|
||||||
_LEGACY_FEISHU_DELTA_OFFSET_KEY = "last_feishu_item_updated_at"
|
_LEGACY_FEISHU_DELTA_OFFSET_KEY = "last_feishu_item_updated_at"
|
||||||
|
_SEMANTIC_FACT_EXTRACT_CHANNEL = "semantic_fact_extract"
|
||||||
|
|
||||||
|
|
||||||
class PKGEventQueueMixin:
|
class PKGEventQueueMixin:
|
||||||
|
|
@ -99,6 +100,8 @@ class PKGEventQueueMixin:
|
||||||
try:
|
try:
|
||||||
if str(ev.channel) in {_WORK_ITEM_DELTA_CHANNEL, _LEGACY_FEISHU_DELTA_CHANNEL}:
|
if str(ev.channel) in {_WORK_ITEM_DELTA_CHANNEL, _LEGACY_FEISHU_DELTA_CHANNEL}:
|
||||||
await self._process_delta_event(ev)
|
await self._process_delta_event(ev)
|
||||||
|
elif str(ev.channel) == _SEMANTIC_FACT_EXTRACT_CHANNEL:
|
||||||
|
await self._process_semantic_fact_extract_event(ev)
|
||||||
ev.status = "done"
|
ev.status = "done"
|
||||||
ev.error_message = None
|
ev.error_message = None
|
||||||
ev.processed_at = _utc_now_naive()
|
ev.processed_at = _utc_now_naive()
|
||||||
|
|
@ -275,3 +278,40 @@ class PKGEventQueueMixin:
|
||||||
source_ref=source_ref,
|
source_ref=source_ref,
|
||||||
metadata={"work_item_id": int(work_item.id)},
|
metadata={"work_item_id": int(work_item.id)},
|
||||||
)
|
)
|
||||||
|
await self.enqueue_ingest_event(
|
||||||
|
channel=_SEMANTIC_FACT_EXTRACT_CHANNEL,
|
||||||
|
event_key=f"project:{int(work_item.project_id)}",
|
||||||
|
payload={
|
||||||
|
"entity_type": "project",
|
||||||
|
"entity_id": int(work_item.project_id),
|
||||||
|
"operation": "extract",
|
||||||
|
"trigger_type": "work_item_delta",
|
||||||
|
"work_item_id": int(work_item.id),
|
||||||
|
"tenant_id": int(self.tenant_id),
|
||||||
|
"source_updated_at": work_item.updated_at.isoformat() if work_item.updated_at else None,
|
||||||
|
},
|
||||||
|
status="pending",
|
||||||
|
)
|
||||||
|
|
||||||
|
async def _process_semantic_fact_extract_event(self, event: IngestEvent) -> None:
|
||||||
|
from ..semantic_fact_service import SemanticFactService
|
||||||
|
|
||||||
|
payload = event.payload if isinstance(event.payload, dict) else {}
|
||||||
|
entity_type = str(payload.get("entity_type") or "").strip().lower()
|
||||||
|
entity_id = payload.get("entity_id")
|
||||||
|
operation = str(payload.get("operation") or "extract").strip().lower() or "extract"
|
||||||
|
trigger_type = str(payload.get("trigger_type") or "event_queue").strip() or "event_queue"
|
||||||
|
if entity_id is None:
|
||||||
|
return
|
||||||
|
service = SemanticFactService(self.db, tenant_id=self.tenant_id)
|
||||||
|
if operation == "deactivate":
|
||||||
|
await service.deactivate_entity_facts(
|
||||||
|
entity_type=entity_type,
|
||||||
|
entity_id=int(entity_id),
|
||||||
|
)
|
||||||
|
return
|
||||||
|
await service.extract_entity_facts(
|
||||||
|
entity_type=entity_type,
|
||||||
|
entity_id=int(entity_id),
|
||||||
|
trigger_type=trigger_type,
|
||||||
|
)
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,4 @@
|
||||||
|
from .service import SemanticFactService
|
||||||
|
from .hooks import register_semantic_fact_hooks
|
||||||
|
|
||||||
|
__all__ = ["SemanticFactService", "register_semantic_fact_hooks"]
|
||||||
|
|
@ -0,0 +1,152 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from sqlalchemy import and_, event, select
|
||||||
|
from sqlalchemy.orm import Session
|
||||||
|
|
||||||
|
from ...models.ingest_event import IngestEvent
|
||||||
|
from ...models.knowledge_base_doc import KnowledgeBaseDoc
|
||||||
|
from ...models.project import Project
|
||||||
|
from ...models.signal import Signal
|
||||||
|
from ...models.task import Task
|
||||||
|
from ...models.truth_node import TruthNode
|
||||||
|
from ...models.user import User
|
||||||
|
from .shared import (
|
||||||
|
ENTITY_TYPE_TO_TRUTH_NODE_TYPE,
|
||||||
|
SEMANTIC_FACT_EXTRACT_CHANNEL,
|
||||||
|
TRUTH_NODE_TYPE_TO_ENTITY_TYPE,
|
||||||
|
_normalized_entity_type,
|
||||||
|
_status_is_inactive,
|
||||||
|
)
|
||||||
|
|
||||||
|
_SEMANTIC_FACT_INFO_KEY = "_pmflow_semantic_fact_events"
|
||||||
|
_SEMANTIC_FACT_APPLYING_KEY = "_pmflow_semantic_fact_applying"
|
||||||
|
|
||||||
|
|
||||||
|
def _truth_node_entity_type(node: TruthNode) -> str:
|
||||||
|
return TRUTH_NODE_TYPE_TO_ENTITY_TYPE.get(str(getattr(node, "node_type", "") or "").strip().lower(), "")
|
||||||
|
|
||||||
|
|
||||||
|
def _build_payload(obj: object, *, deleted: bool) -> dict | None:
|
||||||
|
entity_type = ""
|
||||||
|
if isinstance(obj, Project):
|
||||||
|
entity_type = "project"
|
||||||
|
elif isinstance(obj, Task):
|
||||||
|
entity_type = "task"
|
||||||
|
elif isinstance(obj, Signal):
|
||||||
|
entity_type = "signal"
|
||||||
|
elif isinstance(obj, KnowledgeBaseDoc):
|
||||||
|
entity_type = "document"
|
||||||
|
elif isinstance(obj, User):
|
||||||
|
entity_type = "user"
|
||||||
|
elif isinstance(obj, TruthNode):
|
||||||
|
entity_type = _truth_node_entity_type(obj)
|
||||||
|
if not entity_type:
|
||||||
|
return None
|
||||||
|
|
||||||
|
entity_id = getattr(obj, "id", None)
|
||||||
|
tenant_id = getattr(obj, "tenant_id", None)
|
||||||
|
if entity_id is None or tenant_id is None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
operation = "deactivate" if deleted else "extract"
|
||||||
|
if not deleted:
|
||||||
|
if isinstance(obj, User):
|
||||||
|
operation = "extract" if bool(getattr(obj, "is_active", True)) and not _status_is_inactive(getattr(obj, "status", "")) else "deactivate"
|
||||||
|
elif _status_is_inactive(getattr(obj, "status", "")):
|
||||||
|
operation = "deactivate"
|
||||||
|
|
||||||
|
updated_at = getattr(obj, "updated_at", None)
|
||||||
|
return {
|
||||||
|
"event_key": f"{_normalized_entity_type(entity_type)}:{int(entity_id)}",
|
||||||
|
"payload": {
|
||||||
|
"entity_type": _normalized_entity_type(entity_type),
|
||||||
|
"entity_id": int(entity_id),
|
||||||
|
"operation": operation,
|
||||||
|
"trigger_type": "orm_commit",
|
||||||
|
"tenant_id": int(tenant_id),
|
||||||
|
"source_updated_at": updated_at.isoformat() if updated_at is not None else None,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def register_semantic_fact_hooks() -> None:
|
||||||
|
if getattr(register_semantic_fact_hooks, "_registered", False):
|
||||||
|
return
|
||||||
|
|
||||||
|
@event.listens_for(Session, "after_flush")
|
||||||
|
def _collect_semantic_fact_events(session: Session, _flush_context) -> None: # pragma: no cover
|
||||||
|
if bool(session.info.get(_SEMANTIC_FACT_APPLYING_KEY)):
|
||||||
|
return
|
||||||
|
payload_map = session.info.get(_SEMANTIC_FACT_INFO_KEY)
|
||||||
|
if not isinstance(payload_map, dict):
|
||||||
|
payload_map = {}
|
||||||
|
session.info[_SEMANTIC_FACT_INFO_KEY] = payload_map
|
||||||
|
objects = set(session.new).union(session.dirty).union(session.deleted)
|
||||||
|
deleted_objects = set(session.deleted)
|
||||||
|
for obj in objects:
|
||||||
|
if isinstance(obj, IngestEvent):
|
||||||
|
continue
|
||||||
|
payload = _build_payload(obj, deleted=obj in deleted_objects)
|
||||||
|
if not payload:
|
||||||
|
continue
|
||||||
|
payload_map[str(payload["event_key"])] = payload["payload"]
|
||||||
|
|
||||||
|
@event.listens_for(Session, "after_flush_postexec")
|
||||||
|
def _upsert_semantic_fact_events(session: Session, _flush_context) -> None: # pragma: no cover
|
||||||
|
if bool(session.info.get(_SEMANTIC_FACT_APPLYING_KEY)):
|
||||||
|
return
|
||||||
|
payload_map = session.info.get(_SEMANTIC_FACT_INFO_KEY)
|
||||||
|
if not isinstance(payload_map, dict) or not payload_map:
|
||||||
|
return
|
||||||
|
session.info[_SEMANTIC_FACT_APPLYING_KEY] = True
|
||||||
|
try:
|
||||||
|
for event_key, payload in payload_map.items():
|
||||||
|
tenant_id = int(payload.get("tenant_id") or 0)
|
||||||
|
if tenant_id <= 0:
|
||||||
|
continue
|
||||||
|
row = session.execute(
|
||||||
|
select(IngestEvent).where(
|
||||||
|
and_(
|
||||||
|
IngestEvent.tenant_id == tenant_id,
|
||||||
|
IngestEvent.channel == SEMANTIC_FACT_EXTRACT_CHANNEL,
|
||||||
|
IngestEvent.event_key == str(event_key),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if row is None:
|
||||||
|
session.add(
|
||||||
|
IngestEvent(
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
channel=SEMANTIC_FACT_EXTRACT_CHANNEL,
|
||||||
|
event_key=str(event_key),
|
||||||
|
status="pending",
|
||||||
|
payload=payload,
|
||||||
|
retry_count=0,
|
||||||
|
error_message=None,
|
||||||
|
processed_at=None,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
else:
|
||||||
|
row.status = "pending"
|
||||||
|
row.payload = payload
|
||||||
|
row.retry_count = 0
|
||||||
|
row.error_message = None
|
||||||
|
row.processed_at = None
|
||||||
|
payload_map.clear()
|
||||||
|
finally:
|
||||||
|
session.info[_SEMANTIC_FACT_APPLYING_KEY] = False
|
||||||
|
|
||||||
|
@event.listens_for(Session, "after_commit")
|
||||||
|
def _clear_semantic_fact_events_after_commit(session: Session) -> None: # pragma: no cover
|
||||||
|
session.info.pop(_SEMANTIC_FACT_INFO_KEY, None)
|
||||||
|
session.info.pop(_SEMANTIC_FACT_APPLYING_KEY, None)
|
||||||
|
|
||||||
|
@event.listens_for(Session, "after_rollback")
|
||||||
|
def _clear_semantic_fact_events_after_rollback(session: Session) -> None: # pragma: no cover
|
||||||
|
session.info.pop(_SEMANTIC_FACT_INFO_KEY, None)
|
||||||
|
session.info.pop(_SEMANTIC_FACT_APPLYING_KEY, None)
|
||||||
|
|
||||||
|
setattr(register_semantic_fact_hooks, "_registered", True)
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["register_semantic_fact_hooks"]
|
||||||
|
|
@ -0,0 +1,567 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import UTC, datetime, timedelta
|
||||||
|
|
||||||
|
from .shared import *
|
||||||
|
|
||||||
|
|
||||||
|
class SemanticFactAdminMixin:
|
||||||
|
async def _list_project_snapshots(self, *, limit: int) -> list[dict[str, Any]]:
|
||||||
|
rows = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(Project)
|
||||||
|
.where(Project.tenant_id == self.tenant_id)
|
||||||
|
.order_by(Project.updated_at.desc(), Project.id.desc())
|
||||||
|
.limit(limit)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"entity_type": "project",
|
||||||
|
"entity_id": int(row.id),
|
||||||
|
"display_name": str(row.title or row.name or f"项目#{int(row.id)}").strip(),
|
||||||
|
"object_status": str(row.status or "").strip(),
|
||||||
|
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
|
||||||
|
"is_extractable": not _status_is_inactive(row.status),
|
||||||
|
"summary": str(row.description or "").strip(),
|
||||||
|
}
|
||||||
|
for row in rows
|
||||||
|
]
|
||||||
|
|
||||||
|
async def _list_task_snapshots(self, *, limit: int) -> list[dict[str, Any]]:
|
||||||
|
rows = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(Task)
|
||||||
|
.where(Task.tenant_id == self.tenant_id)
|
||||||
|
.order_by(Task.updated_at.desc(), Task.id.desc())
|
||||||
|
.limit(limit)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"entity_type": "task",
|
||||||
|
"entity_id": int(row.id),
|
||||||
|
"display_name": str(row.title or f"任务#{int(row.id)}").strip(),
|
||||||
|
"object_status": str(row.status or "").strip(),
|
||||||
|
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
|
||||||
|
"is_extractable": not _status_is_inactive(row.status),
|
||||||
|
"summary": str(row.description or "").strip(),
|
||||||
|
}
|
||||||
|
for row in rows
|
||||||
|
]
|
||||||
|
|
||||||
|
async def _list_signal_snapshots(self, *, limit: int) -> list[dict[str, Any]]:
|
||||||
|
rows = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(Signal)
|
||||||
|
.where(Signal.tenant_id == self.tenant_id)
|
||||||
|
.order_by(Signal.updated_at.desc(), Signal.id.desc())
|
||||||
|
.limit(limit)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"entity_type": "signal",
|
||||||
|
"entity_id": int(row.id),
|
||||||
|
"display_name": str(row.title or row.summary or f"信号#{int(row.id)}").strip(),
|
||||||
|
"object_status": str(row.status or "").strip(),
|
||||||
|
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
|
||||||
|
"is_extractable": not _status_is_inactive(row.status),
|
||||||
|
"summary": str(row.summary or row.content or "").strip(),
|
||||||
|
}
|
||||||
|
for row in rows
|
||||||
|
]
|
||||||
|
|
||||||
|
async def _list_document_snapshots(self, *, limit: int) -> list[dict[str, Any]]:
|
||||||
|
rows = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(KnowledgeBaseDoc)
|
||||||
|
.where(KnowledgeBaseDoc.tenant_id == self.tenant_id)
|
||||||
|
.order_by(KnowledgeBaseDoc.updated_at.desc(), KnowledgeBaseDoc.id.desc())
|
||||||
|
.limit(limit)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"entity_type": "document",
|
||||||
|
"entity_id": int(row.id),
|
||||||
|
"display_name": str(row.title or f"文档#{int(row.id)}").strip(),
|
||||||
|
"object_status": str(row.status or "").strip(),
|
||||||
|
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
|
||||||
|
"is_extractable": not _status_is_inactive(row.status),
|
||||||
|
"summary": f"{str(row.file_type or '').strip()} / {str(row.source_type or '').strip()}",
|
||||||
|
}
|
||||||
|
for row in rows
|
||||||
|
]
|
||||||
|
|
||||||
|
async def _list_user_snapshots(self, *, limit: int) -> list[dict[str, Any]]:
|
||||||
|
rows = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(User)
|
||||||
|
.where(User.tenant_id == self.tenant_id)
|
||||||
|
.order_by(User.updated_at.desc(), User.id.desc())
|
||||||
|
.limit(limit)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"entity_type": "user",
|
||||||
|
"entity_id": int(row.id),
|
||||||
|
"display_name": str(row.name or row.username or f"成员#{int(row.id)}").strip(),
|
||||||
|
"object_status": str(row.status or "").strip(),
|
||||||
|
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
|
||||||
|
"is_extractable": bool(row.is_active) and not _status_is_inactive(row.status),
|
||||||
|
"summary": f"{str(row.role or '').strip()} / {str(row.username or '').strip()}",
|
||||||
|
}
|
||||||
|
for row in rows
|
||||||
|
]
|
||||||
|
|
||||||
|
async def _list_truth_node_snapshots(self, *, entity_type: str, limit: int) -> list[dict[str, Any]]:
|
||||||
|
truth_node_type = ENTITY_TYPE_TO_TRUTH_NODE_TYPE[entity_type]
|
||||||
|
rows = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(TruthNode)
|
||||||
|
.where(
|
||||||
|
and_(
|
||||||
|
TruthNode.tenant_id == self.tenant_id,
|
||||||
|
TruthNode.node_type == truth_node_type,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
.order_by(TruthNode.updated_at.desc(), TruthNode.id.desc())
|
||||||
|
.limit(limit)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
return [
|
||||||
|
{
|
||||||
|
"entity_type": entity_type,
|
||||||
|
"entity_id": int(row.id),
|
||||||
|
"display_name": str(row.title or f"{entity_type}#{int(row.id)}").strip(),
|
||||||
|
"object_status": str(row.status or "").strip(),
|
||||||
|
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
|
||||||
|
"is_extractable": not _status_is_inactive(row.status),
|
||||||
|
"summary": str((row.content or {}).get("summary") or "").strip() if isinstance(row.content, dict) else "",
|
||||||
|
}
|
||||||
|
for row in rows
|
||||||
|
]
|
||||||
|
|
||||||
|
async def list_object_snapshots(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
entity_types: Optional[list[str]] = None,
|
||||||
|
limit_per_type: int = 200,
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
normalized_types = [
|
||||||
|
_normalized_entity_type(item)
|
||||||
|
for item in list(entity_types or sorted(SEMANTIC_FACT_SUPPORTED_ENTITY_TYPES))
|
||||||
|
if _normalized_entity_type(item) in SEMANTIC_FACT_SUPPORTED_ENTITY_TYPES
|
||||||
|
]
|
||||||
|
if not normalized_types:
|
||||||
|
normalized_types = sorted(SEMANTIC_FACT_SUPPORTED_ENTITY_TYPES)
|
||||||
|
limit = max(1, min(int(limit_per_type), 1000))
|
||||||
|
items: list[dict[str, Any]] = []
|
||||||
|
for entity_type in normalized_types:
|
||||||
|
if entity_type == "project":
|
||||||
|
items.extend(await self._list_project_snapshots(limit=limit))
|
||||||
|
elif entity_type == "task":
|
||||||
|
items.extend(await self._list_task_snapshots(limit=limit))
|
||||||
|
elif entity_type == "signal":
|
||||||
|
items.extend(await self._list_signal_snapshots(limit=limit))
|
||||||
|
elif entity_type == "document":
|
||||||
|
items.extend(await self._list_document_snapshots(limit=limit))
|
||||||
|
elif entity_type == "user":
|
||||||
|
items.extend(await self._list_user_snapshots(limit=limit))
|
||||||
|
elif entity_type in {"requirement", "release", "review"}:
|
||||||
|
items.extend(await self._list_truth_node_snapshots(entity_type=entity_type, limit=limit))
|
||||||
|
return items
|
||||||
|
|
||||||
|
async def _load_object_metadata_maps(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
items: list[dict[str, Any]],
|
||||||
|
) -> tuple[dict[tuple[str, int], int], dict[tuple[str, int], list[str]], dict[tuple[str, int], dict[str, Any]], dict[tuple[str, int], dict[str, Any]]]:
|
||||||
|
if not items:
|
||||||
|
return {}, {}, {}, {}
|
||||||
|
keys = {(str(item["entity_type"]), int(item["entity_id"])) for item in items}
|
||||||
|
entity_types = sorted({entity_type for entity_type, _entity_id in keys})
|
||||||
|
|
||||||
|
fact_rows = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(SemanticFact).where(
|
||||||
|
and_(
|
||||||
|
SemanticFact.tenant_id == self.tenant_id,
|
||||||
|
SemanticFact.entity_type.in_(entity_types),
|
||||||
|
SemanticFact.is_active.is_(True),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
fact_counts: dict[tuple[str, int], int] = {}
|
||||||
|
fact_texts: dict[tuple[str, int], list[str]] = {}
|
||||||
|
for row in fact_rows:
|
||||||
|
key = (str(row.entity_type), int(row.entity_id))
|
||||||
|
if key not in keys:
|
||||||
|
continue
|
||||||
|
fact_counts[key] = int(fact_counts.get(key) or 0) + 1
|
||||||
|
fact_texts.setdefault(key, []).append(
|
||||||
|
" ".join(
|
||||||
|
part for part in [
|
||||||
|
str(row.fact_key or "").strip(),
|
||||||
|
str(row.fact_value_text or "").strip(),
|
||||||
|
str(row.fact_value_norm or "").strip(),
|
||||||
|
] if part
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
run_rows = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(SemanticFactExtractionRun)
|
||||||
|
.where(
|
||||||
|
and_(
|
||||||
|
SemanticFactExtractionRun.tenant_id == self.tenant_id,
|
||||||
|
SemanticFactExtractionRun.entity_type.in_(entity_types),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
.order_by(SemanticFactExtractionRun.id.desc())
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
latest_runs: dict[tuple[str, int], dict[str, Any]] = {}
|
||||||
|
for row in run_rows:
|
||||||
|
key = (str(row.entity_type), int(row.entity_id))
|
||||||
|
if key not in keys or key in latest_runs:
|
||||||
|
continue
|
||||||
|
latest_runs[key] = {
|
||||||
|
"id": int(row.id),
|
||||||
|
"status": str(row.status or "").strip(),
|
||||||
|
"trigger_type": str(row.trigger_type or "").strip(),
|
||||||
|
"facts_written": int(row.facts_written or 0),
|
||||||
|
"failure_reason": str(row.failure_reason or "").strip() or None,
|
||||||
|
"extract_version": str(row.extract_version or "").strip() or None,
|
||||||
|
"started_at": str(row.started_at or "").strip() or None,
|
||||||
|
"finished_at": str(row.finished_at or "").strip() or None,
|
||||||
|
}
|
||||||
|
|
||||||
|
event_rows = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(IngestEvent).where(
|
||||||
|
and_(
|
||||||
|
IngestEvent.tenant_id == self.tenant_id,
|
||||||
|
IngestEvent.channel == SEMANTIC_FACT_EXTRACT_CHANNEL,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
latest_events: dict[tuple[str, int], dict[str, Any]] = {}
|
||||||
|
for row in event_rows:
|
||||||
|
payload = row.payload if isinstance(row.payload, dict) else {}
|
||||||
|
key = (
|
||||||
|
_normalized_entity_type(payload.get("entity_type") or ""),
|
||||||
|
int(payload.get("entity_id") or 0),
|
||||||
|
)
|
||||||
|
if key not in keys:
|
||||||
|
continue
|
||||||
|
latest_events[key] = {
|
||||||
|
"id": int(row.id),
|
||||||
|
"status": str(row.status or "").strip(),
|
||||||
|
"retry_count": int(row.retry_count or 0),
|
||||||
|
"error_message": str(row.error_message or "").strip() or None,
|
||||||
|
"processed_at": row.processed_at.isoformat() if row.processed_at else None,
|
||||||
|
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
|
||||||
|
"created_at": row.created_at.isoformat() if row.created_at else None,
|
||||||
|
"payload": payload,
|
||||||
|
}
|
||||||
|
return fact_counts, fact_texts, latest_runs, latest_events
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _resolve_last_status(*, event: Optional[dict[str, Any]], run: Optional[dict[str, Any]]) -> str:
|
||||||
|
if event and str(event.get("status") or "").strip():
|
||||||
|
return str(event.get("status") or "").strip()
|
||||||
|
if run and str(run.get("status") or "").strip():
|
||||||
|
return str(run.get("status") or "").strip()
|
||||||
|
return "idle"
|
||||||
|
|
||||||
|
async def list_admin_objects(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
entity_types: Optional[list[str]] = None,
|
||||||
|
q: Optional[str] = None,
|
||||||
|
status: Optional[str] = None,
|
||||||
|
limit_per_type: int = 200,
|
||||||
|
) -> list[dict[str, Any]]:
|
||||||
|
items = await self.list_object_snapshots(entity_types=entity_types, limit_per_type=limit_per_type)
|
||||||
|
fact_counts, fact_texts, latest_runs, latest_events = await self._load_object_metadata_maps(items=items)
|
||||||
|
query_text = str(q or "").strip().lower()
|
||||||
|
status_filter = str(status or "").strip().lower()
|
||||||
|
results: list[dict[str, Any]] = []
|
||||||
|
for item in items:
|
||||||
|
key = (str(item["entity_type"]), int(item["entity_id"]))
|
||||||
|
last_run = latest_runs.get(key)
|
||||||
|
last_event = latest_events.get(key)
|
||||||
|
last_status = self._resolve_last_status(event=last_event, run=last_run)
|
||||||
|
result = {
|
||||||
|
**item,
|
||||||
|
"facts_count": int(fact_counts.get(key) or 0),
|
||||||
|
"last_extracted_at": (
|
||||||
|
str((last_run or {}).get("finished_at") or (last_run or {}).get("started_at") or "").strip() or None
|
||||||
|
),
|
||||||
|
"last_status": last_status,
|
||||||
|
"queue_event": last_event,
|
||||||
|
"last_run": last_run,
|
||||||
|
}
|
||||||
|
if query_text:
|
||||||
|
haystack = " ".join(
|
||||||
|
[
|
||||||
|
str(item.get("display_name") or ""),
|
||||||
|
str(item.get("summary") or ""),
|
||||||
|
str(item.get("entity_id") or ""),
|
||||||
|
" ".join(fact_texts.get(key) or []),
|
||||||
|
]
|
||||||
|
).lower()
|
||||||
|
if query_text not in haystack:
|
||||||
|
continue
|
||||||
|
if status_filter and status_filter not in {"all", last_status.lower()}:
|
||||||
|
continue
|
||||||
|
results.append(result)
|
||||||
|
results.sort(
|
||||||
|
key=lambda row: (
|
||||||
|
_parse_iso_datetime(row.get("last_extracted_at")) or _parse_iso_datetime(row.get("updated_at")) or datetime(1970, 1, 1, tzinfo=UTC),
|
||||||
|
int(row.get("entity_id") or 0),
|
||||||
|
),
|
||||||
|
reverse=True,
|
||||||
|
)
|
||||||
|
return results
|
||||||
|
|
||||||
|
async def get_admin_detail(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
entity_type: str,
|
||||||
|
entity_id: int,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
snapshot = await self._load_entity_snapshot(entity_type=entity_type, entity_id=int(entity_id))
|
||||||
|
if snapshot is None:
|
||||||
|
raise ValueError("semantic_fact_entity_not_found")
|
||||||
|
|
||||||
|
facts = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(SemanticFact).where(
|
||||||
|
and_(
|
||||||
|
SemanticFact.tenant_id == self.tenant_id,
|
||||||
|
SemanticFact.entity_type == _normalized_entity_type(entity_type),
|
||||||
|
SemanticFact.entity_id == int(entity_id),
|
||||||
|
SemanticFact.is_active.is_(True),
|
||||||
|
)
|
||||||
|
).order_by(SemanticFact.confidence.desc(), SemanticFact.updated_at.desc(), SemanticFact.id.desc())
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
runs = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(SemanticFactExtractionRun).where(
|
||||||
|
and_(
|
||||||
|
SemanticFactExtractionRun.tenant_id == self.tenant_id,
|
||||||
|
SemanticFactExtractionRun.entity_type == _normalized_entity_type(entity_type),
|
||||||
|
SemanticFactExtractionRun.entity_id == int(entity_id),
|
||||||
|
)
|
||||||
|
).order_by(SemanticFactExtractionRun.id.desc()).limit(10)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
event = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(IngestEvent).where(
|
||||||
|
and_(
|
||||||
|
IngestEvent.tenant_id == self.tenant_id,
|
||||||
|
IngestEvent.channel == SEMANTIC_FACT_EXTRACT_CHANNEL,
|
||||||
|
IngestEvent.event_key == f"{_normalized_entity_type(entity_type)}:{int(entity_id)}",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
return {
|
||||||
|
"object": {
|
||||||
|
"entity_type": str(snapshot["entity_type"]),
|
||||||
|
"entity_id": int(snapshot["entity_id"]),
|
||||||
|
"display_name": str(snapshot["display_name"]),
|
||||||
|
"object_status": str(snapshot.get("object_status") or ""),
|
||||||
|
"updated_at": snapshot.get("updated_at"),
|
||||||
|
"is_extractable": bool(snapshot.get("is_extractable")),
|
||||||
|
"source_payload": snapshot.get("source_payload") or {},
|
||||||
|
},
|
||||||
|
"facts": [
|
||||||
|
{
|
||||||
|
"id": int(row.id),
|
||||||
|
"fact_namespace": str(row.fact_namespace or "").strip(),
|
||||||
|
"fact_key": str(row.fact_key or "").strip(),
|
||||||
|
"fact_value_text": str(row.fact_value_text or "").strip(),
|
||||||
|
"fact_value_norm": str(row.fact_value_norm or "").strip(),
|
||||||
|
"value_type": str(row.value_type or "").strip(),
|
||||||
|
"confidence": float(row.confidence or 0.0),
|
||||||
|
"evidence_excerpt": str(row.evidence_excerpt or "").strip() or None,
|
||||||
|
"freshness_at": str(row.freshness_at or "").strip() or None,
|
||||||
|
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
|
||||||
|
}
|
||||||
|
for row in facts
|
||||||
|
],
|
||||||
|
"recent_runs": [
|
||||||
|
{
|
||||||
|
"id": int(row.id),
|
||||||
|
"status": str(row.status or "").strip(),
|
||||||
|
"trigger_type": str(row.trigger_type or "").strip(),
|
||||||
|
"facts_written": int(row.facts_written or 0),
|
||||||
|
"failure_reason": str(row.failure_reason or "").strip() or None,
|
||||||
|
"extract_version": str(row.extract_version or "").strip() or None,
|
||||||
|
"started_at": str(row.started_at or "").strip() or None,
|
||||||
|
"finished_at": str(row.finished_at or "").strip() or None,
|
||||||
|
}
|
||||||
|
for row in runs
|
||||||
|
],
|
||||||
|
"queue_event": (
|
||||||
|
{
|
||||||
|
"id": int(event.id),
|
||||||
|
"status": str(event.status or "").strip(),
|
||||||
|
"retry_count": int(event.retry_count or 0),
|
||||||
|
"error_message": str(event.error_message or "").strip() or None,
|
||||||
|
"processed_at": event.processed_at.isoformat() if event.processed_at else None,
|
||||||
|
"updated_at": event.updated_at.isoformat() if event.updated_at else None,
|
||||||
|
"payload": event.payload if isinstance(event.payload, dict) else {},
|
||||||
|
}
|
||||||
|
if event is not None
|
||||||
|
else None
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
async def enqueue_backfill(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
entity_types: Optional[list[str]] = None,
|
||||||
|
only_missing: bool = True,
|
||||||
|
limit_per_type: int = 200,
|
||||||
|
) -> dict[str, Any]:
|
||||||
|
items = await self.list_object_snapshots(entity_types=entity_types, limit_per_type=limit_per_type)
|
||||||
|
by_type: dict[str, int] = {}
|
||||||
|
existing_pairs: set[tuple[str, int]] = set()
|
||||||
|
if only_missing:
|
||||||
|
fact_rows = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(SemanticFact.entity_type, SemanticFact.entity_id).where(
|
||||||
|
and_(
|
||||||
|
SemanticFact.tenant_id == self.tenant_id,
|
||||||
|
SemanticFact.is_active.is_(True),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).all()
|
||||||
|
existing_pairs = {
|
||||||
|
(str(entity_type), int(entity_id))
|
||||||
|
for entity_type, entity_id in fact_rows
|
||||||
|
}
|
||||||
|
|
||||||
|
total = 0
|
||||||
|
for item in items:
|
||||||
|
key = (str(item["entity_type"]), int(item["entity_id"]))
|
||||||
|
if not bool(item.get("is_extractable")):
|
||||||
|
continue
|
||||||
|
if only_missing and key in existing_pairs:
|
||||||
|
continue
|
||||||
|
await self.enqueue_entity_event(
|
||||||
|
entity_type=str(item["entity_type"]),
|
||||||
|
entity_id=int(item["entity_id"]),
|
||||||
|
operation="extract",
|
||||||
|
trigger_type="backfill",
|
||||||
|
extra_payload={
|
||||||
|
"source_updated_at": str(item.get("updated_at") or "").strip() or None,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
by_type[str(item["entity_type"])] = int(by_type.get(str(item["entity_type"])) or 0) + 1
|
||||||
|
total += 1
|
||||||
|
|
||||||
|
return {
|
||||||
|
"enqueued": total,
|
||||||
|
"only_missing": bool(only_missing),
|
||||||
|
"by_type": by_type,
|
||||||
|
}
|
||||||
|
|
||||||
|
async def get_admin_overview(self) -> dict[str, Any]:
|
||||||
|
items = await self.list_object_snapshots(limit_per_type=500)
|
||||||
|
active_fact_total = int(
|
||||||
|
(
|
||||||
|
await self.db.execute(
|
||||||
|
select(func.count(SemanticFact.id)).where(
|
||||||
|
and_(
|
||||||
|
SemanticFact.tenant_id == self.tenant_id,
|
||||||
|
SemanticFact.is_active.is_(True),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one()
|
||||||
|
or 0
|
||||||
|
)
|
||||||
|
event_rows = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(IngestEvent).where(
|
||||||
|
and_(
|
||||||
|
IngestEvent.tenant_id == self.tenant_id,
|
||||||
|
IngestEvent.channel == SEMANTIC_FACT_EXTRACT_CHANNEL,
|
||||||
|
)
|
||||||
|
).order_by(IngestEvent.updated_at.desc(), IngestEvent.id.desc())
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
pending_rows = [row for row in event_rows if str(row.status or "").strip() in {"pending", "retry"}]
|
||||||
|
failed_rows = [row for row in event_rows if str(row.status or "").strip() == "dead_letter"]
|
||||||
|
running_count = int(
|
||||||
|
(
|
||||||
|
await self.db.execute(
|
||||||
|
select(func.count(SemanticFactExtractionRun.id)).where(
|
||||||
|
and_(
|
||||||
|
SemanticFactExtractionRun.tenant_id == self.tenant_id,
|
||||||
|
SemanticFactExtractionRun.status == "running",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one()
|
||||||
|
or 0
|
||||||
|
)
|
||||||
|
all_runs = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(SemanticFactExtractionRun).where(
|
||||||
|
SemanticFactExtractionRun.tenant_id == self.tenant_id
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
cutoff = datetime.now(UTC) - timedelta(hours=24)
|
||||||
|
success_24h = 0
|
||||||
|
failed_24h = 0
|
||||||
|
for row in all_runs:
|
||||||
|
pivot = _parse_iso_datetime(row.finished_at or row.started_at)
|
||||||
|
if pivot is None or pivot < cutoff:
|
||||||
|
continue
|
||||||
|
status = str(row.status or "").strip()
|
||||||
|
if status == "completed":
|
||||||
|
success_24h += 1
|
||||||
|
elif status == "failed":
|
||||||
|
failed_24h += 1
|
||||||
|
|
||||||
|
return {
|
||||||
|
"kpis": {
|
||||||
|
"object_total": len(items),
|
||||||
|
"active_fact_total": active_fact_total,
|
||||||
|
"success_24h": success_24h,
|
||||||
|
"failed_24h": failed_24h,
|
||||||
|
},
|
||||||
|
"queue": {
|
||||||
|
"pending": len(pending_rows),
|
||||||
|
"processing": running_count,
|
||||||
|
"failed": len(failed_rows),
|
||||||
|
"oldest_pending_at": pending_rows[-1].created_at.isoformat() if pending_rows else None,
|
||||||
|
"recent_failures": [
|
||||||
|
{
|
||||||
|
"event_key": str(row.event_key or ""),
|
||||||
|
"status": str(row.status or "").strip(),
|
||||||
|
"error_message": str(row.error_message or "").strip() or None,
|
||||||
|
"updated_at": row.updated_at.isoformat() if row.updated_at else None,
|
||||||
|
}
|
||||||
|
for row in failed_rows[:5]
|
||||||
|
],
|
||||||
|
},
|
||||||
|
"entity_types": sorted(SEMANTIC_FACT_SUPPORTED_ENTITY_TYPES),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["SemanticFactAdminMixin"]
|
||||||
|
|
@ -0,0 +1,370 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from .shared import *
|
||||||
|
|
||||||
|
|
||||||
|
class SemanticFactExtractMixin:
|
||||||
|
async def _create_extraction_run(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
entity_type: str,
|
||||||
|
entity_id: int,
|
||||||
|
trigger_type: str,
|
||||||
|
input_fingerprint: str,
|
||||||
|
) -> SemanticFactExtractionRun:
|
||||||
|
run = SemanticFactExtractionRun(
|
||||||
|
tenant_id=self.tenant_id,
|
||||||
|
entity_type=_normalized_entity_type(entity_type),
|
||||||
|
entity_id=int(entity_id),
|
||||||
|
trigger_type=str(trigger_type or "").strip() or "manual_retry",
|
||||||
|
status="running",
|
||||||
|
input_fingerprint=input_fingerprint,
|
||||||
|
facts_written=0,
|
||||||
|
extract_version=SEMANTIC_FACT_EXTRACT_VERSION,
|
||||||
|
started_at=_utc_now_iso(),
|
||||||
|
)
|
||||||
|
self.db.add(run)
|
||||||
|
await self.db.flush()
|
||||||
|
return run
|
||||||
|
|
||||||
|
async def _load_project_snapshot(self, *, entity_id: int) -> Optional[dict[str, Any]]:
|
||||||
|
project = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(Project).where(
|
||||||
|
and_(
|
||||||
|
Project.tenant_id == self.tenant_id,
|
||||||
|
Project.id == int(entity_id),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if project is None:
|
||||||
|
return None
|
||||||
|
return {
|
||||||
|
"entity_type": "project",
|
||||||
|
"entity_id": int(project.id),
|
||||||
|
"display_name": str(project.title or project.name or f"项目#{int(project.id)}").strip(),
|
||||||
|
"object_status": str(project.status or "").strip(),
|
||||||
|
"updated_at": project.updated_at.isoformat() if project.updated_at else None,
|
||||||
|
"is_extractable": not _status_is_inactive(project.status),
|
||||||
|
"source_payload": {
|
||||||
|
"project_id": int(project.id),
|
||||||
|
"name": str(project.name or "").strip(),
|
||||||
|
"title": str(project.title or "").strip(),
|
||||||
|
"description": str(project.description or "").strip(),
|
||||||
|
"status": str(project.status or "").strip(),
|
||||||
|
"assigned_role": str(project.assigned_role or "").strip(),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
async def _load_task_snapshot(self, *, entity_id: int) -> Optional[dict[str, Any]]:
|
||||||
|
task = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(Task).where(
|
||||||
|
and_(
|
||||||
|
Task.tenant_id == self.tenant_id,
|
||||||
|
Task.id == int(entity_id),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if task is None:
|
||||||
|
return None
|
||||||
|
return {
|
||||||
|
"entity_type": "task",
|
||||||
|
"entity_id": int(task.id),
|
||||||
|
"display_name": str(task.title or f"任务#{int(task.id)}").strip(),
|
||||||
|
"object_status": str(task.status or "").strip(),
|
||||||
|
"updated_at": task.updated_at.isoformat() if task.updated_at else None,
|
||||||
|
"is_extractable": not _status_is_inactive(task.status),
|
||||||
|
"source_payload": {
|
||||||
|
"task_id": int(task.id),
|
||||||
|
"project_id": int(task.project_id),
|
||||||
|
"truth_node_id": int(task.truth_node_id),
|
||||||
|
"assignee_id": int(task.assignee_id) if task.assignee_id is not None else None,
|
||||||
|
"title": str(task.title or "").strip(),
|
||||||
|
"description": str(task.description or "").strip(),
|
||||||
|
"status": str(task.status or "").strip(),
|
||||||
|
"priority": str(task.priority or "").strip(),
|
||||||
|
"required_skill_tag_ids": list(task.required_skill_tag_ids or []),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
async def _load_signal_snapshot(self, *, entity_id: int) -> Optional[dict[str, Any]]:
|
||||||
|
signal = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(Signal).where(
|
||||||
|
and_(
|
||||||
|
Signal.tenant_id == self.tenant_id,
|
||||||
|
Signal.id == int(entity_id),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if signal is None:
|
||||||
|
return None
|
||||||
|
return {
|
||||||
|
"entity_type": "signal",
|
||||||
|
"entity_id": int(signal.id),
|
||||||
|
"display_name": str(signal.title or signal.summary or f"信号#{int(signal.id)}").strip(),
|
||||||
|
"object_status": str(signal.status or "").strip(),
|
||||||
|
"updated_at": signal.updated_at.isoformat() if signal.updated_at else None,
|
||||||
|
"is_extractable": not _status_is_inactive(signal.status),
|
||||||
|
"source_payload": {
|
||||||
|
"signal_id": int(signal.id),
|
||||||
|
"title": str(signal.title or "").strip(),
|
||||||
|
"summary": str(signal.summary or "").strip(),
|
||||||
|
"content": str(signal.content or "").strip(),
|
||||||
|
"status": str(signal.status or "").strip(),
|
||||||
|
"priority": str(signal.priority or "").strip(),
|
||||||
|
"signal_type": str(signal.signal_type or "").strip(),
|
||||||
|
"source": str(signal.source or "").strip(),
|
||||||
|
"matched_project_id": int(signal.matched_project_id) if signal.matched_project_id is not None else None,
|
||||||
|
"processed": bool(signal.processed),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
async def _load_document_snapshot(self, *, entity_id: int) -> Optional[dict[str, Any]]:
|
||||||
|
doc = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(KnowledgeBaseDoc).where(
|
||||||
|
and_(
|
||||||
|
KnowledgeBaseDoc.tenant_id == self.tenant_id,
|
||||||
|
KnowledgeBaseDoc.id == int(entity_id),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if doc is None:
|
||||||
|
return None
|
||||||
|
chunk_rows = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(KnowledgeBaseChunk).where(
|
||||||
|
and_(
|
||||||
|
KnowledgeBaseChunk.tenant_id == self.tenant_id,
|
||||||
|
KnowledgeBaseChunk.doc_id == int(doc.id),
|
||||||
|
)
|
||||||
|
).order_by(KnowledgeBaseChunk.chunk_index.asc()).limit(3)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
preview_text = "\n".join(str(row.content or "").strip() for row in chunk_rows if str(row.content or "").strip())
|
||||||
|
return {
|
||||||
|
"entity_type": "document",
|
||||||
|
"entity_id": int(doc.id),
|
||||||
|
"display_name": str(doc.title or f"文档#{int(doc.id)}").strip(),
|
||||||
|
"object_status": str(doc.status or "").strip(),
|
||||||
|
"updated_at": doc.updated_at.isoformat() if doc.updated_at else None,
|
||||||
|
"is_extractable": not _status_is_inactive(doc.status),
|
||||||
|
"source_payload": {
|
||||||
|
"doc_id": int(doc.id),
|
||||||
|
"title": str(doc.title or "").strip(),
|
||||||
|
"file_type": str(doc.file_type or "").strip(),
|
||||||
|
"source_type": str(doc.source_type or "").strip(),
|
||||||
|
"file_url": str(doc.file_url or "").strip(),
|
||||||
|
"status": str(doc.status or "").strip(),
|
||||||
|
"preview_text": preview_text[:2000],
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
async def _load_user_snapshot(self, *, entity_id: int) -> Optional[dict[str, Any]]:
|
||||||
|
user = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(User).where(
|
||||||
|
and_(
|
||||||
|
User.tenant_id == self.tenant_id,
|
||||||
|
User.id == int(entity_id),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if user is None:
|
||||||
|
return None
|
||||||
|
return {
|
||||||
|
"entity_type": "user",
|
||||||
|
"entity_id": int(user.id),
|
||||||
|
"display_name": str(user.name or user.username or f"成员#{int(user.id)}").strip(),
|
||||||
|
"object_status": str(user.status or "").strip(),
|
||||||
|
"updated_at": user.updated_at.isoformat() if user.updated_at else None,
|
||||||
|
"is_extractable": bool(user.is_active) and not _status_is_inactive(user.status),
|
||||||
|
"source_payload": {
|
||||||
|
"user_id": int(user.id),
|
||||||
|
"username": str(user.username or "").strip(),
|
||||||
|
"name": str(user.name or "").strip(),
|
||||||
|
"role": str(user.role or "").strip(),
|
||||||
|
"status": str(user.status or "").strip(),
|
||||||
|
"is_active": bool(user.is_active),
|
||||||
|
"bio": str(user.bio or "").strip(),
|
||||||
|
"assigned_domains": list(user.assigned_domains or []),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
async def _load_truth_node_snapshot(self, *, entity_type: str, entity_id: int) -> Optional[dict[str, Any]]:
|
||||||
|
truth_node_type = ENTITY_TYPE_TO_TRUTH_NODE_TYPE.get(_normalized_entity_type(entity_type))
|
||||||
|
if not truth_node_type:
|
||||||
|
return None
|
||||||
|
node = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(TruthNode).where(
|
||||||
|
and_(
|
||||||
|
TruthNode.tenant_id == self.tenant_id,
|
||||||
|
TruthNode.id == int(entity_id),
|
||||||
|
TruthNode.node_type == truth_node_type,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if node is None:
|
||||||
|
return None
|
||||||
|
normalized_entity_type = TRUTH_NODE_TYPE_TO_ENTITY_TYPE.get(str(node.node_type or "").strip().lower(), "")
|
||||||
|
if not normalized_entity_type:
|
||||||
|
return None
|
||||||
|
return {
|
||||||
|
"entity_type": normalized_entity_type,
|
||||||
|
"entity_id": int(node.id),
|
||||||
|
"display_name": str(node.title or f"{normalized_entity_type}#{int(node.id)}").strip(),
|
||||||
|
"object_status": str(node.status or "").strip(),
|
||||||
|
"updated_at": node.updated_at.isoformat() if node.updated_at else None,
|
||||||
|
"is_extractable": not _status_is_inactive(node.status),
|
||||||
|
"source_payload": {
|
||||||
|
"node_id": int(node.id),
|
||||||
|
"node_type": str(node.node_type or "").strip(),
|
||||||
|
"project_id": int(node.project_id) if node.project_id is not None else None,
|
||||||
|
"signal_id": int(node.signal_id) if node.signal_id is not None else None,
|
||||||
|
"title": str(node.title or "").strip(),
|
||||||
|
"status": str(node.status or "").strip(),
|
||||||
|
"version": int(node.version or 1),
|
||||||
|
"content": node.content if isinstance(node.content, dict) else {},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
async def _load_entity_snapshot(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
entity_type: str,
|
||||||
|
entity_id: int,
|
||||||
|
) -> Optional[dict[str, Any]]:
|
||||||
|
normalized_entity_type = _normalized_entity_type(entity_type)
|
||||||
|
if normalized_entity_type == "project":
|
||||||
|
return await self._load_project_snapshot(entity_id=int(entity_id))
|
||||||
|
if normalized_entity_type == "task":
|
||||||
|
return await self._load_task_snapshot(entity_id=int(entity_id))
|
||||||
|
if normalized_entity_type == "signal":
|
||||||
|
return await self._load_signal_snapshot(entity_id=int(entity_id))
|
||||||
|
if normalized_entity_type == "document":
|
||||||
|
return await self._load_document_snapshot(entity_id=int(entity_id))
|
||||||
|
if normalized_entity_type == "user":
|
||||||
|
return await self._load_user_snapshot(entity_id=int(entity_id))
|
||||||
|
if normalized_entity_type in {"requirement", "release", "review"}:
|
||||||
|
return await self._load_truth_node_snapshot(entity_type=normalized_entity_type, entity_id=int(entity_id))
|
||||||
|
raise ValueError(f"unsupported_semantic_fact_entity_type:{normalized_entity_type}")
|
||||||
|
|
||||||
|
def _render_extraction_prompt(self, *, entity_type: str, source_payload: dict[str, Any]) -> str:
|
||||||
|
normalized_entity_type = _normalized_entity_type(entity_type)
|
||||||
|
prompt_template = SEMANTIC_FACT_PROMPTS.get(normalized_entity_type)
|
||||||
|
if prompt_template is None:
|
||||||
|
raise ValueError(f"unsupported_semantic_fact_prompt:{normalized_entity_type}")
|
||||||
|
payload_text = json.dumps(source_payload, ensure_ascii=False, sort_keys=True)
|
||||||
|
if normalized_entity_type == "project":
|
||||||
|
return prompt_template.replace("<<project_payload>>", payload_text).strip()
|
||||||
|
return prompt_template.replace("<<entity_payload>>", payload_text).strip()
|
||||||
|
|
||||||
|
async def extract_entity_facts(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
entity_type: str,
|
||||||
|
entity_id: int,
|
||||||
|
trigger_type: str,
|
||||||
|
) -> Optional[SemanticFactExtractionRun]:
|
||||||
|
from ..llm_service import llm_service
|
||||||
|
|
||||||
|
normalized_entity_type = _normalized_entity_type(entity_type)
|
||||||
|
snapshot = await self._load_entity_snapshot(
|
||||||
|
entity_type=normalized_entity_type,
|
||||||
|
entity_id=int(entity_id),
|
||||||
|
)
|
||||||
|
if snapshot is None:
|
||||||
|
await self.deactivate_entity_facts(entity_type=normalized_entity_type, entity_id=int(entity_id))
|
||||||
|
return None
|
||||||
|
if not bool(snapshot.get("is_extractable")):
|
||||||
|
await self.deactivate_entity_facts(entity_type=normalized_entity_type, entity_id=int(entity_id))
|
||||||
|
return None
|
||||||
|
|
||||||
|
source_payload = dict(snapshot.get("source_payload") or {})
|
||||||
|
fingerprint = hashlib.sha1(
|
||||||
|
json.dumps(source_payload, ensure_ascii=False, sort_keys=True).encode("utf-8")
|
||||||
|
).hexdigest()
|
||||||
|
run = await self._create_extraction_run(
|
||||||
|
entity_type=normalized_entity_type,
|
||||||
|
entity_id=int(entity_id),
|
||||||
|
trigger_type=trigger_type,
|
||||||
|
input_fingerprint=fingerprint,
|
||||||
|
)
|
||||||
|
try:
|
||||||
|
prompt = self._render_extraction_prompt(
|
||||||
|
entity_type=normalized_entity_type,
|
||||||
|
source_payload=source_payload,
|
||||||
|
)
|
||||||
|
extraction = await llm_service.generate_structured_object(
|
||||||
|
prompt,
|
||||||
|
db=self.db,
|
||||||
|
tenant_id=self.tenant_id,
|
||||||
|
function_name=SEMANTIC_FACT_FUNCTION_NAMES[normalized_entity_type],
|
||||||
|
temperature=0.0,
|
||||||
|
max_tokens=1200,
|
||||||
|
)
|
||||||
|
payload = extraction.payload if isinstance(extraction.payload, dict) else {}
|
||||||
|
fact_rows = list(payload.get("facts") or [])
|
||||||
|
await self.deactivate_entity_facts(
|
||||||
|
entity_type=normalized_entity_type,
|
||||||
|
entity_id=int(entity_id),
|
||||||
|
)
|
||||||
|
written = 0
|
||||||
|
for item in fact_rows:
|
||||||
|
if not isinstance(item, dict):
|
||||||
|
continue
|
||||||
|
fact_key = str(item.get("fact_key") or "").strip()
|
||||||
|
fact_value_text = str(item.get("fact_value_text") or "").strip()
|
||||||
|
if not fact_key or not fact_value_text:
|
||||||
|
continue
|
||||||
|
await self.upsert_fact(
|
||||||
|
entity_type=normalized_entity_type,
|
||||||
|
entity_id=int(entity_id),
|
||||||
|
fact_namespace=str(item.get("fact_namespace") or "general").strip() or "general",
|
||||||
|
fact_key=fact_key,
|
||||||
|
fact_value_text=fact_value_text,
|
||||||
|
fact_value_norm=str(item.get("fact_value_norm") or "").strip() or None,
|
||||||
|
value_type=str(item.get("value_type") or "keyword").strip() or "keyword",
|
||||||
|
confidence=float(item.get("confidence") or 0.0),
|
||||||
|
evidence_source_type=normalized_entity_type,
|
||||||
|
evidence_source_id=int(entity_id),
|
||||||
|
evidence_excerpt=str(item.get("evidence_excerpt") or "").strip() or None,
|
||||||
|
extract_version=SEMANTIC_FACT_EXTRACT_VERSION,
|
||||||
|
freshness_at=str(snapshot.get("updated_at") or "").strip() or None,
|
||||||
|
)
|
||||||
|
written += 1
|
||||||
|
|
||||||
|
run.status = "completed"
|
||||||
|
run.facts_written = written
|
||||||
|
run.failure_reason = None
|
||||||
|
run.finished_at = _utc_now_iso()
|
||||||
|
await self.db.flush()
|
||||||
|
return run
|
||||||
|
except Exception as exc:
|
||||||
|
run.status = "failed"
|
||||||
|
run.failure_reason = str(exc)[:1000]
|
||||||
|
run.finished_at = _utc_now_iso()
|
||||||
|
await self.db.flush()
|
||||||
|
raise
|
||||||
|
|
||||||
|
async def extract_project_facts(self, *, project_id: int, trigger_type: str) -> SemanticFactExtractionRun:
|
||||||
|
run = await self.extract_entity_facts(
|
||||||
|
entity_type="project",
|
||||||
|
entity_id=int(project_id),
|
||||||
|
trigger_type=trigger_type,
|
||||||
|
)
|
||||||
|
if run is None:
|
||||||
|
raise ValueError(f"project_not_found:{int(project_id)}")
|
||||||
|
return run
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["SemanticFactExtractMixin"]
|
||||||
|
|
@ -0,0 +1,198 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from .shared import *
|
||||||
|
|
||||||
|
|
||||||
|
class SemanticFactQueryMixin:
|
||||||
|
async def upsert_fact(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
entity_type: str,
|
||||||
|
entity_id: int,
|
||||||
|
fact_namespace: str,
|
||||||
|
fact_key: str,
|
||||||
|
fact_value_text: str,
|
||||||
|
fact_value_norm: Optional[str],
|
||||||
|
value_type: str,
|
||||||
|
confidence: float,
|
||||||
|
evidence_source_type: Optional[str],
|
||||||
|
evidence_source_id: Optional[int],
|
||||||
|
evidence_excerpt: Optional[str],
|
||||||
|
extract_version: Optional[str],
|
||||||
|
freshness_at: Optional[str],
|
||||||
|
) -> SemanticFact:
|
||||||
|
normalized_fact_value = _normalize_fact_value(fact_value_norm or fact_value_text)
|
||||||
|
row = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(SemanticFact).where(
|
||||||
|
and_(
|
||||||
|
SemanticFact.tenant_id == self.tenant_id,
|
||||||
|
SemanticFact.entity_type == str(entity_type or "").strip(),
|
||||||
|
SemanticFact.entity_id == int(entity_id),
|
||||||
|
SemanticFact.fact_key == str(fact_key or "").strip(),
|
||||||
|
SemanticFact.fact_value_norm == normalized_fact_value,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
|
||||||
|
if row is None:
|
||||||
|
row = SemanticFact(
|
||||||
|
tenant_id=self.tenant_id,
|
||||||
|
entity_type=str(entity_type or "").strip(),
|
||||||
|
entity_id=int(entity_id),
|
||||||
|
fact_namespace=str(fact_namespace or "").strip() or "general",
|
||||||
|
fact_key=str(fact_key or "").strip(),
|
||||||
|
fact_value_text=str(fact_value_text or "").strip(),
|
||||||
|
fact_value_norm=normalized_fact_value,
|
||||||
|
value_type=str(value_type or "").strip() or "keyword",
|
||||||
|
confidence=max(0.0, min(1.0, float(confidence or 0.0))),
|
||||||
|
evidence_source_type=str(evidence_source_type or "").strip() or None,
|
||||||
|
evidence_source_id=int(evidence_source_id) if evidence_source_id is not None else None,
|
||||||
|
evidence_excerpt=str(evidence_excerpt or "").strip() or None,
|
||||||
|
extract_version=str(extract_version or "").strip() or None,
|
||||||
|
freshness_at=str(freshness_at or "").strip() or None,
|
||||||
|
is_active=True,
|
||||||
|
)
|
||||||
|
self.db.add(row)
|
||||||
|
await self.db.flush()
|
||||||
|
return row
|
||||||
|
|
||||||
|
row.fact_namespace = str(fact_namespace or "").strip() or row.fact_namespace
|
||||||
|
row.fact_value_text = str(fact_value_text or "").strip() or row.fact_value_text
|
||||||
|
row.value_type = str(value_type or "").strip() or row.value_type
|
||||||
|
row.confidence = max(0.0, min(1.0, float(confidence or 0.0)))
|
||||||
|
row.evidence_source_type = str(evidence_source_type or "").strip() or row.evidence_source_type
|
||||||
|
row.evidence_source_id = int(evidence_source_id) if evidence_source_id is not None else row.evidence_source_id
|
||||||
|
row.evidence_excerpt = str(evidence_excerpt or "").strip() or row.evidence_excerpt
|
||||||
|
row.extract_version = str(extract_version or "").strip() or row.extract_version
|
||||||
|
row.freshness_at = str(freshness_at or "").strip() or row.freshness_at
|
||||||
|
row.is_active = True
|
||||||
|
await self.db.flush()
|
||||||
|
return row
|
||||||
|
|
||||||
|
async def deactivate_entity_facts(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
entity_type: str,
|
||||||
|
entity_id: int,
|
||||||
|
) -> int:
|
||||||
|
normalized_entity_type = _normalized_entity_type(entity_type)
|
||||||
|
rows = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(SemanticFact).where(
|
||||||
|
and_(
|
||||||
|
SemanticFact.tenant_id == self.tenant_id,
|
||||||
|
SemanticFact.entity_type == normalized_entity_type,
|
||||||
|
SemanticFact.entity_id == int(entity_id),
|
||||||
|
SemanticFact.is_active.is_(True),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
for row in rows:
|
||||||
|
row.is_active = False
|
||||||
|
await self.db.flush()
|
||||||
|
return len(rows)
|
||||||
|
|
||||||
|
async def enqueue_entity_event(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
entity_type: str,
|
||||||
|
entity_id: int,
|
||||||
|
operation: str,
|
||||||
|
trigger_type: str,
|
||||||
|
extra_payload: Optional[dict[str, Any]] = None,
|
||||||
|
) -> IngestEvent:
|
||||||
|
normalized_entity_type = _normalized_entity_type(entity_type)
|
||||||
|
event_key = f"{normalized_entity_type}:{int(entity_id)}"
|
||||||
|
payload = {
|
||||||
|
"entity_type": normalized_entity_type,
|
||||||
|
"entity_id": int(entity_id),
|
||||||
|
"operation": str(operation or "extract").strip() or "extract",
|
||||||
|
"trigger_type": str(trigger_type or "manual_retry").strip() or "manual_retry",
|
||||||
|
"tenant_id": int(self.tenant_id),
|
||||||
|
}
|
||||||
|
if isinstance(extra_payload, dict):
|
||||||
|
for key, value in extra_payload.items():
|
||||||
|
payload[str(key)] = value
|
||||||
|
|
||||||
|
row = (
|
||||||
|
await self.db.execute(
|
||||||
|
select(IngestEvent).where(
|
||||||
|
and_(
|
||||||
|
IngestEvent.tenant_id == self.tenant_id,
|
||||||
|
IngestEvent.channel == SEMANTIC_FACT_EXTRACT_CHANNEL,
|
||||||
|
IngestEvent.event_key == event_key,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
if row is None:
|
||||||
|
row = IngestEvent(
|
||||||
|
tenant_id=self.tenant_id,
|
||||||
|
channel=SEMANTIC_FACT_EXTRACT_CHANNEL,
|
||||||
|
event_key=event_key,
|
||||||
|
status="pending",
|
||||||
|
payload=payload,
|
||||||
|
retry_count=0,
|
||||||
|
error_message=None,
|
||||||
|
processed_at=None,
|
||||||
|
)
|
||||||
|
self.db.add(row)
|
||||||
|
else:
|
||||||
|
row.status = "pending"
|
||||||
|
row.payload = payload
|
||||||
|
row.retry_count = 0
|
||||||
|
row.error_message = None
|
||||||
|
row.processed_at = None
|
||||||
|
await self.db.flush()
|
||||||
|
return row
|
||||||
|
|
||||||
|
async def query_facts(
|
||||||
|
self,
|
||||||
|
*,
|
||||||
|
entity_type: str,
|
||||||
|
query_type: str,
|
||||||
|
filters: Optional[list[dict[str, Any]]] = None,
|
||||||
|
entity_id: Optional[int] = None,
|
||||||
|
limit: int = 20,
|
||||||
|
) -> list[SemanticFact]:
|
||||||
|
_ = query_type
|
||||||
|
stmt = select(SemanticFact).where(
|
||||||
|
and_(
|
||||||
|
SemanticFact.tenant_id == self.tenant_id,
|
||||||
|
SemanticFact.entity_type == _normalized_entity_type(entity_type),
|
||||||
|
SemanticFact.is_active.is_(True),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
if entity_id is not None:
|
||||||
|
stmt = stmt.where(SemanticFact.entity_id == int(entity_id))
|
||||||
|
|
||||||
|
for item in list(filters or []):
|
||||||
|
if not isinstance(item, dict):
|
||||||
|
continue
|
||||||
|
fact_key = str(item.get("fact_key") or item.get("field") or "").strip()
|
||||||
|
if fact_key:
|
||||||
|
stmt = stmt.where(SemanticFact.fact_key == fact_key)
|
||||||
|
raw_value = str(item.get("value") or item.get("normalized_value") or "").strip()
|
||||||
|
normalized_value = _normalize_fact_value(item.get("normalized_value") or raw_value)
|
||||||
|
if normalized_value:
|
||||||
|
stmt = stmt.where(
|
||||||
|
or_(
|
||||||
|
SemanticFact.fact_value_norm == normalized_value,
|
||||||
|
func.lower(SemanticFact.fact_value_text) == raw_value.lower(),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
|
||||||
|
rows = (
|
||||||
|
await self.db.execute(
|
||||||
|
stmt.order_by(SemanticFact.confidence.desc(), SemanticFact.updated_at.desc(), SemanticFact.id.desc()).limit(
|
||||||
|
max(1, int(limit))
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
return list(rows)
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["SemanticFactQueryMixin"]
|
||||||
|
|
@ -0,0 +1,19 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from .mixin_admin import SemanticFactAdminMixin
|
||||||
|
from .mixin_extract import SemanticFactExtractMixin
|
||||||
|
from .mixin_query import SemanticFactQueryMixin
|
||||||
|
from .shared import AsyncSession
|
||||||
|
|
||||||
|
|
||||||
|
class SemanticFactService(
|
||||||
|
SemanticFactAdminMixin,
|
||||||
|
SemanticFactExtractMixin,
|
||||||
|
SemanticFactQueryMixin,
|
||||||
|
):
|
||||||
|
def __init__(self, db: AsyncSession, *, tenant_id: int) -> None:
|
||||||
|
self.db = db
|
||||||
|
self.tenant_id = int(tenant_id)
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = ["SemanticFactService"]
|
||||||
|
|
@ -0,0 +1,162 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hashlib
|
||||||
|
import json
|
||||||
|
import re
|
||||||
|
from datetime import UTC, datetime
|
||||||
|
from typing import Any, Optional
|
||||||
|
|
||||||
|
from sqlalchemy import and_, func, or_, select
|
||||||
|
from sqlalchemy.ext.asyncio import AsyncSession
|
||||||
|
|
||||||
|
from ...config.ai_prompts import (
|
||||||
|
PROMPT_SEMANTIC_FACT_EXTRACT_DOCUMENT,
|
||||||
|
PROMPT_SEMANTIC_FACT_EXTRACT_PROJECT,
|
||||||
|
PROMPT_SEMANTIC_FACT_EXTRACT_RELEASE,
|
||||||
|
PROMPT_SEMANTIC_FACT_EXTRACT_REQUIREMENT,
|
||||||
|
PROMPT_SEMANTIC_FACT_EXTRACT_REVIEW,
|
||||||
|
PROMPT_SEMANTIC_FACT_EXTRACT_SIGNAL,
|
||||||
|
PROMPT_SEMANTIC_FACT_EXTRACT_TASK,
|
||||||
|
PROMPT_SEMANTIC_FACT_EXTRACT_USER,
|
||||||
|
)
|
||||||
|
from ...models.ingest_event import IngestEvent
|
||||||
|
from ...models.knowledge_base_chunk import KnowledgeBaseChunk
|
||||||
|
from ...models.knowledge_base_doc import KnowledgeBaseDoc
|
||||||
|
from ...models.project import Project
|
||||||
|
from ...models.semantic_fact import SemanticFact
|
||||||
|
from ...models.semantic_fact_extraction_run import SemanticFactExtractionRun
|
||||||
|
from ...models.signal import Signal
|
||||||
|
from ...models.task import Task
|
||||||
|
from ...models.truth_node import TruthNode
|
||||||
|
from ...models.user import User
|
||||||
|
|
||||||
|
SEMANTIC_FACT_EXTRACT_VERSION = "semantic-facts-v1"
|
||||||
|
SEMANTIC_FACT_EXTRACT_CHANNEL = "semantic_fact_extract"
|
||||||
|
SEMANTIC_FACT_SUPPORTED_ENTITY_TYPES = {
|
||||||
|
"project",
|
||||||
|
"task",
|
||||||
|
"signal",
|
||||||
|
"document",
|
||||||
|
"user",
|
||||||
|
"requirement",
|
||||||
|
"release",
|
||||||
|
"review",
|
||||||
|
}
|
||||||
|
SEMANTIC_FACT_INACTIVE_STATUSES = {
|
||||||
|
"archived",
|
||||||
|
"closed",
|
||||||
|
"deleted",
|
||||||
|
"disabled",
|
||||||
|
"inactive",
|
||||||
|
"offline",
|
||||||
|
"已关闭",
|
||||||
|
"已归档",
|
||||||
|
}
|
||||||
|
TRUTH_NODE_TYPE_TO_ENTITY_TYPE = {
|
||||||
|
"charter": "requirement",
|
||||||
|
"release": "release",
|
||||||
|
"review": "review",
|
||||||
|
}
|
||||||
|
ENTITY_TYPE_TO_TRUTH_NODE_TYPE = {
|
||||||
|
value: key for key, value in TRUTH_NODE_TYPE_TO_ENTITY_TYPE.items()
|
||||||
|
}
|
||||||
|
SEMANTIC_FACT_PROMPTS = {
|
||||||
|
"project": PROMPT_SEMANTIC_FACT_EXTRACT_PROJECT,
|
||||||
|
"task": PROMPT_SEMANTIC_FACT_EXTRACT_TASK,
|
||||||
|
"signal": PROMPT_SEMANTIC_FACT_EXTRACT_SIGNAL,
|
||||||
|
"document": PROMPT_SEMANTIC_FACT_EXTRACT_DOCUMENT,
|
||||||
|
"user": PROMPT_SEMANTIC_FACT_EXTRACT_USER,
|
||||||
|
"requirement": PROMPT_SEMANTIC_FACT_EXTRACT_REQUIREMENT,
|
||||||
|
"release": PROMPT_SEMANTIC_FACT_EXTRACT_RELEASE,
|
||||||
|
"review": PROMPT_SEMANTIC_FACT_EXTRACT_REVIEW,
|
||||||
|
}
|
||||||
|
SEMANTIC_FACT_FUNCTION_NAMES = {
|
||||||
|
entity_type: f"semantic_fact_extract_{entity_type}"
|
||||||
|
for entity_type in SEMANTIC_FACT_SUPPORTED_ENTITY_TYPES
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def _utc_now_iso() -> str:
|
||||||
|
return datetime.now(UTC).replace(microsecond=0).isoformat()
|
||||||
|
|
||||||
|
|
||||||
|
def _normalize_fact_value(value: Any) -> str:
|
||||||
|
text = str(value or "").strip().lower()
|
||||||
|
if not text:
|
||||||
|
return ""
|
||||||
|
text = text.replace("a股", "a股")
|
||||||
|
text = re.sub(r"\s+", "_", text)
|
||||||
|
text = re.sub(r"[^a-z0-9\u4e00-\u9fff_]+", "_", text)
|
||||||
|
text = re.sub(r"_+", "_", text).strip("_")
|
||||||
|
return text
|
||||||
|
|
||||||
|
|
||||||
|
def _normalized_entity_type(entity_type: Any) -> str:
|
||||||
|
value = str(entity_type or "").strip().lower()
|
||||||
|
if value == "doc":
|
||||||
|
return "document"
|
||||||
|
return value
|
||||||
|
|
||||||
|
|
||||||
|
def _normalized_status(value: Any) -> str:
|
||||||
|
return str(value or "").strip().lower()
|
||||||
|
|
||||||
|
|
||||||
|
def _status_is_inactive(value: Any) -> bool:
|
||||||
|
return _normalized_status(value) in SEMANTIC_FACT_INACTIVE_STATUSES
|
||||||
|
|
||||||
|
|
||||||
|
def _parse_iso_datetime(value: Any) -> Optional[datetime]:
|
||||||
|
text = str(value or "").strip()
|
||||||
|
if not text:
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
parsed = datetime.fromisoformat(text.replace("Z", "+00:00"))
|
||||||
|
except ValueError:
|
||||||
|
return None
|
||||||
|
if parsed.tzinfo is None:
|
||||||
|
return parsed.replace(tzinfo=UTC)
|
||||||
|
return parsed.astimezone(UTC)
|
||||||
|
|
||||||
|
|
||||||
|
__all__ = [
|
||||||
|
"AsyncSession",
|
||||||
|
"ENTITY_TYPE_TO_TRUTH_NODE_TYPE",
|
||||||
|
"IngestEvent",
|
||||||
|
"KnowledgeBaseChunk",
|
||||||
|
"KnowledgeBaseDoc",
|
||||||
|
"Project",
|
||||||
|
"PROMPT_SEMANTIC_FACT_EXTRACT_DOCUMENT",
|
||||||
|
"PROMPT_SEMANTIC_FACT_EXTRACT_PROJECT",
|
||||||
|
"PROMPT_SEMANTIC_FACT_EXTRACT_RELEASE",
|
||||||
|
"PROMPT_SEMANTIC_FACT_EXTRACT_REQUIREMENT",
|
||||||
|
"PROMPT_SEMANTIC_FACT_EXTRACT_REVIEW",
|
||||||
|
"PROMPT_SEMANTIC_FACT_EXTRACT_SIGNAL",
|
||||||
|
"PROMPT_SEMANTIC_FACT_EXTRACT_TASK",
|
||||||
|
"PROMPT_SEMANTIC_FACT_EXTRACT_USER",
|
||||||
|
"SEMANTIC_FACT_EXTRACT_VERSION",
|
||||||
|
"SEMANTIC_FACT_EXTRACT_CHANNEL",
|
||||||
|
"SEMANTIC_FACT_FUNCTION_NAMES",
|
||||||
|
"SEMANTIC_FACT_INACTIVE_STATUSES",
|
||||||
|
"SEMANTIC_FACT_PROMPTS",
|
||||||
|
"SEMANTIC_FACT_SUPPORTED_ENTITY_TYPES",
|
||||||
|
"SemanticFact",
|
||||||
|
"SemanticFactExtractionRun",
|
||||||
|
"Signal",
|
||||||
|
"Task",
|
||||||
|
"TRUTH_NODE_TYPE_TO_ENTITY_TYPE",
|
||||||
|
"TruthNode",
|
||||||
|
"User",
|
||||||
|
"_normalize_fact_value",
|
||||||
|
"_normalized_entity_type",
|
||||||
|
"_normalized_status",
|
||||||
|
"_parse_iso_datetime",
|
||||||
|
"_status_is_inactive",
|
||||||
|
"_utc_now_iso",
|
||||||
|
"and_",
|
||||||
|
"func",
|
||||||
|
"hashlib",
|
||||||
|
"json",
|
||||||
|
"or_",
|
||||||
|
"select",
|
||||||
|
]
|
||||||
File diff suppressed because it is too large
Load Diff
|
|
@ -1522,7 +1522,7 @@ async def test_terminate_requirement_plan_scope_cancels_orphan_pending_work_item
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_process_ingress_terminates_stale_requirement_scope_before_active_context_followup(monkeypatch) -> None:
|
async def test_process_ingress_does_not_terminate_requirement_scope_before_active_context_followup(monkeypatch) -> None:
|
||||||
cleanup_calls: list[dict[str, object]] = []
|
cleanup_calls: list[dict[str, object]] = []
|
||||||
|
|
||||||
class _FakeDB:
|
class _FakeDB:
|
||||||
|
|
@ -1563,8 +1563,8 @@ async def test_process_ingress_terminates_stale_requirement_scope_before_active_
|
||||||
self.active_context_called = True
|
self.active_context_called = True
|
||||||
_ = kwargs
|
_ = kwargs
|
||||||
assert self.prepared_session is not None
|
assert self.prepared_session is not None
|
||||||
assert str(getattr(self.prepared_session, "last_intent", "") or "") == "query.answer"
|
assert str(getattr(self.prepared_session, "last_intent", "") or "") == "requirement_plan"
|
||||||
assert getattr(self.prepared_session, "current_project_id", None) is None
|
assert getattr(self.prepared_session, "current_project_id", None) == 99
|
||||||
return {
|
return {
|
||||||
"handled": True,
|
"handled": True,
|
||||||
"status": "answered",
|
"status": "answered",
|
||||||
|
|
@ -1575,11 +1575,11 @@ async def test_process_ingress_terminates_stale_requirement_scope_before_active_
|
||||||
|
|
||||||
async def _maybe_handle_draft_confirmation_ingress(self, **kwargs):
|
async def _maybe_handle_draft_confirmation_ingress(self, **kwargs):
|
||||||
_ = kwargs
|
_ = kwargs
|
||||||
raise AssertionError("stale scope cleanup should happen before draft confirmation routing")
|
raise AssertionError("active context followup should short-circuit before draft confirmation routing")
|
||||||
|
|
||||||
async def _prepare_mixed_sidecar_context(self, **kwargs):
|
async def _prepare_mixed_sidecar_context(self, **kwargs):
|
||||||
_ = kwargs
|
_ = kwargs
|
||||||
raise AssertionError("stale scope cleanup should short-circuit before query sidecar")
|
raise AssertionError("active context followup should short-circuit before query sidecar")
|
||||||
|
|
||||||
async def _fake_terminate_requirement_plan_scope(self, **kwargs):
|
async def _fake_terminate_requirement_plan_scope(self, **kwargs):
|
||||||
cleanup_calls.append(kwargs)
|
cleanup_calls.append(kwargs)
|
||||||
|
|
@ -1603,9 +1603,7 @@ async def test_process_ingress_terminates_stale_requirement_scope_before_active_
|
||||||
ingress_options={"thread_context": {"reply_to_message_id": ""}},
|
ingress_options={"thread_context": {"reply_to_message_id": ""}},
|
||||||
)
|
)
|
||||||
|
|
||||||
assert cleanup_calls
|
assert cleanup_calls == []
|
||||||
assert cleanup_calls[0]["source_message_id"] == "s-msg-terminate-1"
|
|
||||||
assert cleanup_calls[0]["source_system"] == "feishu"
|
|
||||||
assert kernel.active_context_called is True
|
assert kernel.active_context_called is True
|
||||||
assert result["direct_reply_text"] == "普通问答结果"
|
assert result["direct_reply_text"] == "普通问答结果"
|
||||||
|
|
||||||
|
|
@ -2538,6 +2536,8 @@ async def test_active_requirement_followup_prefers_current_requirement_context(m
|
||||||
assert result is not None
|
assert result is not None
|
||||||
assert result["handled"] is True
|
assert result["handled"] is True
|
||||||
assert "信号输入" in result["direct_reply_text"]
|
assert "信号输入" in result["direct_reply_text"]
|
||||||
|
assert result["reply_template_key"] == "requirement_plan_followup"
|
||||||
|
assert result["work_item_id"] == 81
|
||||||
assert captured["packet_kwargs"]["latest_user_message"] == "信号源呢?"
|
assert captured["packet_kwargs"]["latest_user_message"] == "信号源呢?"
|
||||||
assert captured["packet_kwargs"]["ingress_context"]["candidate_kind"] == "active_context_followup"
|
assert captured["packet_kwargs"]["ingress_context"]["candidate_kind"] == "active_context_followup"
|
||||||
assert captured["direct_kwargs"]["message_text"] == "信号源呢?"
|
assert captured["direct_kwargs"]["message_text"] == "信号源呢?"
|
||||||
|
|
@ -2619,6 +2619,71 @@ async def test_active_requirement_choice_protocol_followup_bypasses_semantic_dec
|
||||||
assert captured["direct_kwargs"]["message_text"] == "我选C 继续"
|
assert captured["direct_kwargs"]["message_text"] == "我选C 继续"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_active_requirement_exit_request_suspends_bound_requirement_context_before_semantic_decision(
|
||||||
|
monkeypatch,
|
||||||
|
) -> None:
|
||||||
|
flush_mock = AsyncMock()
|
||||||
|
topic = SimpleNamespace(
|
||||||
|
id=181,
|
||||||
|
status="active",
|
||||||
|
plan_status="collecting",
|
||||||
|
pending_actions=[],
|
||||||
|
metadata_json={"plan_mode": True},
|
||||||
|
)
|
||||||
|
draft = SimpleNamespace(id=182, state="collecting")
|
||||||
|
|
||||||
|
async def _executor_find_active_requirement_plan_workspace(**kwargs):
|
||||||
|
_ = kwargs
|
||||||
|
return topic, draft
|
||||||
|
|
||||||
|
class FakeKernel(mixin_entry.KernelIngressFlowEntryMixin):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.db = SimpleNamespace(flush=flush_mock)
|
||||||
|
self.tenant_id = 1
|
||||||
|
self.executor = SimpleNamespace(
|
||||||
|
_find_active_requirement_plan_workspace=_executor_find_active_requirement_plan_workspace,
|
||||||
|
)
|
||||||
|
|
||||||
|
class BombSemanticContextAssembler:
|
||||||
|
def __init__(self, db) -> None:
|
||||||
|
self.db = db
|
||||||
|
|
||||||
|
async def assemble(self, **kwargs):
|
||||||
|
_ = kwargs
|
||||||
|
raise AssertionError("explicit exit should bypass semantic decision assembly")
|
||||||
|
|
||||||
|
class BombSemanticDecisionService:
|
||||||
|
def __init__(self, _llm_service) -> None:
|
||||||
|
pass
|
||||||
|
|
||||||
|
async def decide(self, *, packet):
|
||||||
|
_ = packet
|
||||||
|
raise AssertionError("explicit exit should bypass semantic decision")
|
||||||
|
|
||||||
|
monkeypatch.setattr(mixin_entry, "SemanticContextAssembler", BombSemanticContextAssembler)
|
||||||
|
monkeypatch.setattr(mixin_entry, "SemanticDecisionService", BombSemanticDecisionService)
|
||||||
|
|
||||||
|
kernel = FakeKernel()
|
||||||
|
result = await kernel._maybe_handle_active_context_followup(
|
||||||
|
source_system="feishu",
|
||||||
|
source_message_id="msg-plan-exit-1",
|
||||||
|
channel_ref="chat-active",
|
||||||
|
source_sender_open_id="ou_active",
|
||||||
|
message_text="退出需求模式",
|
||||||
|
thread_context=None,
|
||||||
|
sender="tester",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result is not None
|
||||||
|
assert result["handled"] is True
|
||||||
|
assert "先暂停" in result["direct_reply_text"]
|
||||||
|
assert result["reply_template_key"] == "requirement_mode_suspended"
|
||||||
|
assert result["workflow_route"] == mixin_entry.ROUTE_REQUIREMENT_PLAN_COLLECT
|
||||||
|
assert result["topic_workspace_id"] == 181
|
||||||
|
flush_mock.assert_awaited_once()
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_active_requirement_followup_uses_builder_result_to_persist_requirement_context(monkeypatch) -> None:
|
async def test_active_requirement_followup_uses_builder_result_to_persist_requirement_context(monkeypatch) -> None:
|
||||||
captured: dict[str, object] = {}
|
captured: dict[str, object] = {}
|
||||||
|
|
@ -3088,6 +3153,7 @@ async def test_find_active_requirement_plan_workspace_includes_draft_pending_con
|
||||||
assert found_topic is not None
|
assert found_topic is not None
|
||||||
assert found_draft is not None
|
assert found_draft is not None
|
||||||
assert int(found_topic.id) == int(topic.id)
|
assert int(found_topic.id) == int(topic.id)
|
||||||
|
assert str(found_topic.plan_status) == "draft_pending_confirm"
|
||||||
assert str(found_draft.state) == "draft_pending_confirm"
|
assert str(found_draft.state) == "draft_pending_confirm"
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -3151,8 +3217,76 @@ async def test_find_active_requirement_plan_workspace_recovers_plan_topic_marked
|
||||||
assert found_topic is not None
|
assert found_topic is not None
|
||||||
assert found_draft is not None
|
assert found_draft is not None
|
||||||
assert int(found_topic.id) == int(topic.id)
|
assert int(found_topic.id) == int(topic.id)
|
||||||
assert str(found_topic.plan_status) == "options_ready"
|
|
||||||
assert str(found_draft.state) == "collecting"
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_find_active_requirement_plan_workspace_prefers_live_draft_topic_over_newer_idle_scope_match(
|
||||||
|
requirement_plan_db_session,
|
||||||
|
) -> None:
|
||||||
|
class FakeExecutor(mixin_execute_query_project.ExecuteQueryProjectMixin):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.db = requirement_plan_db_session
|
||||||
|
self.tenant_id = 7
|
||||||
|
|
||||||
|
executor = FakeExecutor()
|
||||||
|
conversation_scope_key = build_feishu_conversation_scope_key(
|
||||||
|
source_chat_id="chat-plan-prefer-live-draft",
|
||||||
|
source_sender_open_id="ou-plan-prefer-live-draft",
|
||||||
|
sender=None,
|
||||||
|
)
|
||||||
|
live_topic = MountedTopicWorkspace(
|
||||||
|
tenant_id=7,
|
||||||
|
topic_key="feishu:chat-plan-prefer-live-draft:message:msg-plan-root",
|
||||||
|
source_system="feishu",
|
||||||
|
channel_ref="chat-plan-prefer-live-draft",
|
||||||
|
conversation_scope_key=conversation_scope_key,
|
||||||
|
mode="plan",
|
||||||
|
status="active",
|
||||||
|
plan_status="collecting",
|
||||||
|
metadata_json={"plan_mode": True},
|
||||||
|
)
|
||||||
|
requirement_plan_db_session.add(live_topic)
|
||||||
|
await requirement_plan_db_session.flush()
|
||||||
|
live_draft = MountedDraftWorkspace(
|
||||||
|
tenant_id=7,
|
||||||
|
topic_workspace_id=int(live_topic.id),
|
||||||
|
state="collecting",
|
||||||
|
draft_version=0,
|
||||||
|
requirement_card={"title": "采购库存报表助手"},
|
||||||
|
draft_payload=None,
|
||||||
|
required_confirmations=[],
|
||||||
|
)
|
||||||
|
requirement_plan_db_session.add(live_draft)
|
||||||
|
await requirement_plan_db_session.flush()
|
||||||
|
|
||||||
|
idle_query_topic = MountedTopicWorkspace(
|
||||||
|
tenant_id=7,
|
||||||
|
topic_key="feishu:chat-plan-prefer-live-draft:message:msg-later-query",
|
||||||
|
source_system="feishu",
|
||||||
|
channel_ref="chat-plan-prefer-live-draft",
|
||||||
|
conversation_scope_key=conversation_scope_key,
|
||||||
|
mode="chat",
|
||||||
|
status="active",
|
||||||
|
plan_status="idle",
|
||||||
|
metadata_json={"plan_mode": True},
|
||||||
|
)
|
||||||
|
requirement_plan_db_session.add(idle_query_topic)
|
||||||
|
await requirement_plan_db_session.flush()
|
||||||
|
|
||||||
|
found_topic, found_draft = await executor._find_active_requirement_plan_workspace(
|
||||||
|
source_system="feishu",
|
||||||
|
source_message_id="msg-followup-exit",
|
||||||
|
source_chat_id="chat-plan-prefer-live-draft",
|
||||||
|
sender=None,
|
||||||
|
source_sender_open_id="ou-plan-prefer-live-draft",
|
||||||
|
thread_context=None,
|
||||||
|
conversation_key=conversation_scope_key,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert found_topic is not None
|
||||||
|
assert found_draft is not None
|
||||||
|
assert int(found_topic.id) == int(live_topic.id)
|
||||||
|
assert int(found_draft.id) == int(live_draft.id)
|
||||||
|
|
||||||
|
|
||||||
def test_world_model_session_project_inheritance_keeps_meaningful_followup() -> None:
|
def test_world_model_session_project_inheritance_keeps_meaningful_followup() -> None:
|
||||||
|
|
@ -3382,6 +3516,19 @@ def test_mounted_runtime_query_requirement_consultation_structured_analysis_uses
|
||||||
assert mode == "plan"
|
assert mode == "plan"
|
||||||
|
|
||||||
|
|
||||||
|
def test_mounted_runtime_query_consult_requirement_requirements_analysis_uses_plan_mode() -> None:
|
||||||
|
mode = mixin_entry.KernelIngressFlowEntryMixin._resolve_mounted_runtime_mode_override(
|
||||||
|
workflow_route="query.answer",
|
||||||
|
route_metadata={
|
||||||
|
"message_intent": "consult_requirement",
|
||||||
|
"consultant_mode_preference": "requirements_analysis",
|
||||||
|
},
|
||||||
|
reply_template_key=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert mode == "plan"
|
||||||
|
|
||||||
|
|
||||||
def test_mounted_runtime_query_requirement_gathering_stepwise_converge_uses_plan_mode() -> None:
|
def test_mounted_runtime_query_requirement_gathering_stepwise_converge_uses_plan_mode() -> None:
|
||||||
mode = mixin_entry.KernelIngressFlowEntryMixin._resolve_mounted_runtime_mode_override(
|
mode = mixin_entry.KernelIngressFlowEntryMixin._resolve_mounted_runtime_mode_override(
|
||||||
workflow_route="query.answer",
|
workflow_route="query.answer",
|
||||||
|
|
@ -3972,11 +4119,11 @@ async def test_kernel_intent_resolve_degrades_when_semantic_planner_fails(
|
||||||
assert source == "semantic_planner_degraded"
|
assert source == "semantic_planner_degraded"
|
||||||
assert intent.name == "query.read_only"
|
assert intent.name == "query.read_only"
|
||||||
assert intent.metadata["workflow_route"] == "query.answer"
|
assert intent.metadata["workflow_route"] == "query.answer"
|
||||||
assert intent.metadata["message_intent"] == "consult_requirement"
|
|
||||||
assert intent.metadata["consultant_mode_preference"] == "clarify"
|
|
||||||
assert intent.metadata["context_source"] == "semantic_planner_degraded"
|
assert intent.metadata["context_source"] == "semantic_planner_degraded"
|
||||||
assert intent.metadata["semantic_planner_degraded"] is True
|
assert intent.metadata["semantic_planner_degraded"] is True
|
||||||
assert "upstream_empty_or_invalid_response" in str(intent.metadata["semantic_planner_error"])
|
assert "upstream_empty_or_invalid_response" in str(intent.metadata["semantic_planner_error"])
|
||||||
|
assert "message_intent" not in intent.metadata
|
||||||
|
assert "consultant_mode_preference" not in intent.metadata
|
||||||
assert intent.signal_title == ""
|
assert intent.signal_title == ""
|
||||||
|
|
||||||
|
|
||||||
|
|
@ -4259,6 +4406,39 @@ def test_reply_renderer_formats_project_delete_pending_and_success() -> None:
|
||||||
assert "已删除项目#12《重复项目》" in success["text"]
|
assert "已删除项目#12《重复项目》" in success["text"]
|
||||||
|
|
||||||
|
|
||||||
|
def test_reply_renderer_preserves_requirement_plan_followup_template_for_query_intent() -> None:
|
||||||
|
renderer = ReplyRenderer(
|
||||||
|
state_waiting_manual="waiting_manual",
|
||||||
|
state_failed="failed",
|
||||||
|
receipt_phase_action="action",
|
||||||
|
receipt_phase_pending="pending",
|
||||||
|
receipt_phase_error="error",
|
||||||
|
)
|
||||||
|
intent = CanonicalIntent(
|
||||||
|
name="query.read_only",
|
||||||
|
confidence=0.97,
|
||||||
|
project_id=None,
|
||||||
|
signal_title="",
|
||||||
|
signal_summary="",
|
||||||
|
metadata={},
|
||||||
|
)
|
||||||
|
|
||||||
|
rendered = renderer.render(
|
||||||
|
work_state="succeeded",
|
||||||
|
intent=intent,
|
||||||
|
decision_record={"approval_requirement": {}},
|
||||||
|
execution_result={
|
||||||
|
"status": "succeeded",
|
||||||
|
"workflow_route": "requirement_plan.collect",
|
||||||
|
"answer": "A. 轻量工具\nB. 协同平台\nC. 能力底座\n\n⭐ 当前我更推荐:A",
|
||||||
|
"reply_template_key": "requirement_plan_followup",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert rendered["template_key"] == "requirement_plan_followup"
|
||||||
|
assert rendered["text"].startswith("A. 轻量工具")
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_feishu_outbox_service_enqueue_preserves_requirement_plan_text_above_2000_chars(
|
async def test_feishu_outbox_service_enqueue_preserves_requirement_plan_text_above_2000_chars(
|
||||||
feishu_outbox_db_session,
|
feishu_outbox_db_session,
|
||||||
|
|
@ -4823,6 +5003,83 @@ async def test_execute_query_intent_short_circuits_casual_chat_without_rag(monke
|
||||||
assert str(result["rag_result"]["query_type"]) == "chat"
|
assert str(result["rag_result"]["query_type"]) == "chat"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_execute_query_intent_external_source_policy_bypasses_casual_chat_shortcut(monkeypatch) -> None:
|
||||||
|
captured: dict[str, object] = {}
|
||||||
|
|
||||||
|
class FakeAdaptiveRAGService:
|
||||||
|
def __init__(self, db, tenant_id: int):
|
||||||
|
_ = (db, tenant_id)
|
||||||
|
|
||||||
|
async def preview_query_route_decision(self, **kwargs):
|
||||||
|
captured["route_preview_kwargs"] = dict(kwargs)
|
||||||
|
return {
|
||||||
|
"query_semantics": {
|
||||||
|
"source_policy": "uncertain",
|
||||||
|
"effective_source_policy": "external",
|
||||||
|
"query_type": "status",
|
||||||
|
"retrieval_domains": ["url"],
|
||||||
|
"query_profile": {
|
||||||
|
"needs_private_context": False,
|
||||||
|
"needs_freshness": True,
|
||||||
|
"referenced_objects": ["weather"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
"route_decision": {
|
||||||
|
"route_mode": "external",
|
||||||
|
"effective_source_policy": "external",
|
||||||
|
"decision_origin": "external_probe",
|
||||||
|
"reply_contract": {"mode": "explicit_source", "head": "外部实时查询:", "tail_hint": None},
|
||||||
|
"probe_snapshot_summary": {
|
||||||
|
"internal": {"config_hit_count": 0},
|
||||||
|
"external": {"trusted_domain_hit_count": 1, "answerable_page_hit_count": 1},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
async def answer_question(self, **kwargs):
|
||||||
|
captured["answer_kwargs"] = dict(kwargs)
|
||||||
|
return {
|
||||||
|
"answer": "广州荔湾区今天多云,午后有阵雨。",
|
||||||
|
"sources": [{"source": "url", "label": "Weather", "score": 0.93, "metadata": {"url": "https://weather.example"}}],
|
||||||
|
"query_type": "status",
|
||||||
|
}
|
||||||
|
|
||||||
|
class FakeExecutor(mixin_execute_query_project.ExecuteQueryProjectMixin):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.db = object()
|
||||||
|
self.tenant_id = 3
|
||||||
|
|
||||||
|
monkeypatch.setattr(mixin_execute_query_project, "AdaptiveRAGService", FakeAdaptiveRAGService)
|
||||||
|
executor = FakeExecutor()
|
||||||
|
intent = SimpleNamespace(
|
||||||
|
project_id=None,
|
||||||
|
metadata={
|
||||||
|
"semantic_bundle": {
|
||||||
|
"diagnostics": {
|
||||||
|
"message_intent": "casual_chat",
|
||||||
|
"consultant_mode_preference": "brainstorm",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
result = await executor._execute_query_intent(
|
||||||
|
intent=intent,
|
||||||
|
message_text="今天荔湾区天气如何?",
|
||||||
|
operator_user_id=999,
|
||||||
|
source_system="feishu",
|
||||||
|
source_message_id="msg-chat-weather-1",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result["status"] == "succeeded"
|
||||||
|
assert "广州荔湾区今天多云" in str(result["answer"] or "")
|
||||||
|
answer_kwargs = captured["answer_kwargs"]
|
||||||
|
assert isinstance(answer_kwargs, dict)
|
||||||
|
assert answer_kwargs["source_policy"] == "uncertain"
|
||||||
|
assert answer_kwargs["route_decision"]["effective_source_policy"] == "external"
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_execute_query_intent_backfills_semantic_bundle_for_casual_chat_without_legacy_helpers(
|
async def test_execute_query_intent_backfills_semantic_bundle_for_casual_chat_without_legacy_helpers(
|
||||||
monkeypatch,
|
monkeypatch,
|
||||||
|
|
@ -6799,6 +7056,101 @@ async def test_execute_query_intent_locked_requirement_mode_redirects_casual_cha
|
||||||
assert str(result["rag_result"]["query_type"]) == "plan"
|
assert str(result["rag_result"]["query_type"]) == "plan"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_execute_query_intent_locked_requirement_mode_redirects_generic_query_to_requirement_notice(
|
||||||
|
monkeypatch,
|
||||||
|
requirement_plan_db_session,
|
||||||
|
) -> None:
|
||||||
|
class FakeAdaptiveRAGService:
|
||||||
|
def __init__(self, db, tenant_id: int):
|
||||||
|
_ = (db, tenant_id)
|
||||||
|
|
||||||
|
async def answer_question(self, **kwargs):
|
||||||
|
raise AssertionError(f"locked requirement mode should not fall through to generic RAG: {kwargs}")
|
||||||
|
|
||||||
|
class FakeExecutor(mixin_execute_query_project.ExecuteQueryProjectMixin):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.db = requirement_plan_db_session
|
||||||
|
self.tenant_id = 3
|
||||||
|
|
||||||
|
_install_requirement_plan_query_bundle(
|
||||||
|
monkeypatch,
|
||||||
|
message_intent="query",
|
||||||
|
consultant_mode_preference="",
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(mixin_execute_query_project, "AdaptiveRAGService", FakeAdaptiveRAGService)
|
||||||
|
|
||||||
|
conversation_scope_key = build_feishu_conversation_scope_key(
|
||||||
|
source_chat_id="chat-lock-plan-query",
|
||||||
|
source_sender_open_id="ou-lock-plan-query",
|
||||||
|
sender=None,
|
||||||
|
)
|
||||||
|
topic = MountedTopicWorkspace(
|
||||||
|
tenant_id=3,
|
||||||
|
topic_key="feishu:chat-lock-plan-query:root:om-lock-plan-query-root",
|
||||||
|
source_system="feishu",
|
||||||
|
channel_ref="chat-lock-plan-query",
|
||||||
|
conversation_scope_key=conversation_scope_key,
|
||||||
|
mode="plan",
|
||||||
|
status="active",
|
||||||
|
plan_status="collecting",
|
||||||
|
metadata_json={"plan_mode": True},
|
||||||
|
root_message_id="om-lock-plan-query-root",
|
||||||
|
)
|
||||||
|
requirement_plan_db_session.add(topic)
|
||||||
|
await requirement_plan_db_session.flush()
|
||||||
|
requirement_plan_db_session.add(
|
||||||
|
MountedDraftWorkspace(
|
||||||
|
tenant_id=3,
|
||||||
|
topic_workspace_id=int(topic.id),
|
||||||
|
state="collecting",
|
||||||
|
draft_version=0,
|
||||||
|
requirement_card={
|
||||||
|
"plan_profile": "it_project",
|
||||||
|
"title": "采购库存报表助手",
|
||||||
|
"goal": "先把采购库存报表助手的首版需求收清楚。",
|
||||||
|
"target_user": "采购经理",
|
||||||
|
"question_round": 2,
|
||||||
|
"last_round_options": {
|
||||||
|
"field": "target_user",
|
||||||
|
"options": [
|
||||||
|
{"key": "A", "title": "只服务采购经理本人"},
|
||||||
|
{"key": "B", "title": "采购经理+仓管员"},
|
||||||
|
{"key": "C", "title": "采购经理+财务复核人"},
|
||||||
|
],
|
||||||
|
"recommendation": {"key": "A"},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
required_confirmations=[],
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await requirement_plan_db_session.flush()
|
||||||
|
|
||||||
|
executor = FakeExecutor()
|
||||||
|
intent = SimpleNamespace(
|
||||||
|
project_id=None,
|
||||||
|
metadata={
|
||||||
|
"source_chat_id": "chat-lock-plan-query",
|
||||||
|
"source_sender_open_id": "ou-lock-plan-query",
|
||||||
|
"message_intent": "query",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
result = await executor._execute_query_intent(
|
||||||
|
intent=intent,
|
||||||
|
message_text="系统里面有哪些项目?",
|
||||||
|
operator_user_id=999,
|
||||||
|
source_system="feishu",
|
||||||
|
source_message_id="msg-lock-plan-query-1",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result["status"] == "succeeded"
|
||||||
|
assert "当前正在需求模式" in str(result["answer"] or "")
|
||||||
|
assert "退出需求模式" in str(result["answer"] or "")
|
||||||
|
assert str(result["workflow_route"]) == "requirement_plan.collect"
|
||||||
|
assert str(result["rag_result"]["query_type"]) == "plan"
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_execute_query_intent_converts_query_answer_requirement_contract_into_requirement_plan_result(
|
async def test_execute_query_intent_converts_query_answer_requirement_contract_into_requirement_plan_result(
|
||||||
monkeypatch,
|
monkeypatch,
|
||||||
|
|
@ -7419,6 +7771,270 @@ async def test_execute_query_intent_treats_requirement_consultation_structured_a
|
||||||
assert "Feature Catalog" not in str(result["answer"] or "")
|
assert "Feature Catalog" not in str(result["answer"] or "")
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_execute_query_intent_treats_consult_requirement_requirements_analysis_as_requirement_plan(
|
||||||
|
monkeypatch,
|
||||||
|
requirement_plan_db_session,
|
||||||
|
) -> None:
|
||||||
|
captured: dict[str, object] = {}
|
||||||
|
|
||||||
|
class FakeAdaptiveRAGService:
|
||||||
|
def __init__(self, db, tenant_id: int):
|
||||||
|
_ = (db, tenant_id)
|
||||||
|
|
||||||
|
async def answer_question(self, **kwargs):
|
||||||
|
raise AssertionError(f"requirements analysis should not enter heavy RAG: {kwargs}")
|
||||||
|
|
||||||
|
class FakeExecutor(mixin_execute_query_project.ExecuteQueryProjectMixin):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.db = requirement_plan_db_session
|
||||||
|
self.tenant_id = 3
|
||||||
|
|
||||||
|
async def _fake_build_requirement_planning_direct_result(self, **kwargs): # noqa: ANN001
|
||||||
|
_ = self
|
||||||
|
captured["direct_kwargs"] = dict(kwargs)
|
||||||
|
return {
|
||||||
|
"status": "succeeded",
|
||||||
|
"answer": "可以,我们先不创建,先把这件事收成一版需求。",
|
||||||
|
"rag_result": {
|
||||||
|
"answer": "可以,我们先不创建,先把这件事收成一版需求。",
|
||||||
|
"query_type": "plan",
|
||||||
|
},
|
||||||
|
"reply_template_key": "requirement_plan_followup",
|
||||||
|
}
|
||||||
|
|
||||||
|
monkeypatch.setattr(
|
||||||
|
mixin_execute_query_project.ExecuteQueryProjectMixin,
|
||||||
|
"_build_requirement_planning_direct_result",
|
||||||
|
_fake_build_requirement_planning_direct_result,
|
||||||
|
raising=False,
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(mixin_execute_query_project, "AdaptiveRAGService", FakeAdaptiveRAGService)
|
||||||
|
|
||||||
|
executor = FakeExecutor()
|
||||||
|
intent = SimpleNamespace(
|
||||||
|
project_id=None,
|
||||||
|
metadata={
|
||||||
|
"source_chat_id": "chat-consult-requirements-analysis",
|
||||||
|
"source_sender_open_id": "ou-consult-requirements-analysis",
|
||||||
|
"message_intent": "consult_requirement",
|
||||||
|
"consultant_mode_preference": "requirements_analysis",
|
||||||
|
"topic_switch": "new",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
result = await executor._execute_query_intent(
|
||||||
|
intent=intent,
|
||||||
|
message_text="想跟你聊个需求,关于给采购经理提供可清理库存的报表。",
|
||||||
|
operator_user_id=999,
|
||||||
|
source_system="feishu",
|
||||||
|
source_message_id="msg-consult-requirements-analysis-1",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result["status"] == "succeeded"
|
||||||
|
assert str(result["rag_result"]["query_type"]) == "plan"
|
||||||
|
direct_kwargs = captured.get("direct_kwargs")
|
||||||
|
assert isinstance(direct_kwargs, dict)
|
||||||
|
assert str(direct_kwargs["message_text"]) == "想跟你聊个需求,关于给采购经理提供可清理库存的报表。"
|
||||||
|
assert str(direct_kwargs["source_chat_id"]) == "chat-consult-requirements-analysis"
|
||||||
|
assert "Feature Catalog" not in str(result["answer"] or "")
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_execute_query_intent_semantic_planner_degraded_factual_query_falls_back_to_regular_qa(
|
||||||
|
monkeypatch,
|
||||||
|
requirement_plan_db_session,
|
||||||
|
) -> None:
|
||||||
|
captured: dict[str, object] = {}
|
||||||
|
|
||||||
|
class FakeAdaptiveRAGService:
|
||||||
|
def __init__(self, db, tenant_id: int):
|
||||||
|
_ = (db, tenant_id)
|
||||||
|
|
||||||
|
async def answer_question(self, **kwargs):
|
||||||
|
captured["rag_kwargs"] = dict(kwargs)
|
||||||
|
return {
|
||||||
|
"answer": "根据公开页面,目前走普通事实问答链。",
|
||||||
|
"query_type": "explain",
|
||||||
|
"sources": [{"source": "web", "label": "External", "score": 0.9}],
|
||||||
|
}
|
||||||
|
|
||||||
|
class FakeExecutor(mixin_execute_query_project.ExecuteQueryProjectMixin):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.db = requirement_plan_db_session
|
||||||
|
self.tenant_id = 3
|
||||||
|
|
||||||
|
async def _failing_aplan_message(*, message_text: str, source_system: str, metadata=None):
|
||||||
|
_ = (message_text, source_system, metadata)
|
||||||
|
raise RuntimeError("upstream_empty_or_invalid_response")
|
||||||
|
|
||||||
|
async def _unexpected_requirement_direct_result(self, **kwargs): # noqa: ANN001
|
||||||
|
raise AssertionError(f"degraded factual query should not enter requirement plan direct path: {kwargs}")
|
||||||
|
|
||||||
|
monkeypatch.setattr(
|
||||||
|
WorkflowSemanticPlanner,
|
||||||
|
"aplan_message",
|
||||||
|
classmethod(lambda cls, **kwargs: _failing_aplan_message(**kwargs)),
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
mixin_execute_query_project.ExecuteQueryProjectMixin,
|
||||||
|
"_build_requirement_planning_direct_result",
|
||||||
|
_unexpected_requirement_direct_result,
|
||||||
|
raising=False,
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(mixin_execute_query_project, "AdaptiveRAGService", FakeAdaptiveRAGService)
|
||||||
|
|
||||||
|
executor = FakeExecutor()
|
||||||
|
intent = SimpleNamespace(
|
||||||
|
project_id=None,
|
||||||
|
metadata={
|
||||||
|
"source_chat_id": "chat-degraded-factual",
|
||||||
|
"source_sender_open_id": "ou-degraded-factual",
|
||||||
|
"semantic_planner_degraded": True,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
result = await executor._execute_query_intent(
|
||||||
|
intent=intent,
|
||||||
|
message_text="当前claude最新的模型用的是什么?",
|
||||||
|
operator_user_id=999,
|
||||||
|
source_system="feishu",
|
||||||
|
source_message_id="msg-degraded-factual-1",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result["status"] == "succeeded"
|
||||||
|
assert str(result["rag_result"]["query_type"]) == "explain"
|
||||||
|
assert "收成一版需求" not in str(result["answer"] or "")
|
||||||
|
assert "requirement_plan_context" not in result
|
||||||
|
assert isinstance(captured.get("rag_kwargs"), dict)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_execute_query_intent_semantic_planner_degraded_active_requirement_scope_stays_in_plan(
|
||||||
|
monkeypatch,
|
||||||
|
requirement_plan_db_session,
|
||||||
|
) -> None:
|
||||||
|
captured: dict[str, object] = {}
|
||||||
|
|
||||||
|
class FakeAdaptiveRAGService:
|
||||||
|
def __init__(self, db, tenant_id: int):
|
||||||
|
_ = (db, tenant_id)
|
||||||
|
|
||||||
|
async def answer_question(self, **kwargs):
|
||||||
|
raise AssertionError(f"active requirement scope should not enter heavy RAG: {kwargs}")
|
||||||
|
|
||||||
|
class FakeExecutor(mixin_execute_query_project.ExecuteQueryProjectMixin):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.db = requirement_plan_db_session
|
||||||
|
self.tenant_id = 3
|
||||||
|
|
||||||
|
async def _failing_aplan_message(*, message_text: str, source_system: str, metadata=None):
|
||||||
|
_ = (message_text, source_system, metadata)
|
||||||
|
raise RuntimeError("upstream_empty_or_invalid_response")
|
||||||
|
|
||||||
|
async def _fake_build_requirement_planning_direct_result(self, **kwargs): # noqa: ANN001
|
||||||
|
_ = self
|
||||||
|
captured["direct_kwargs"] = dict(kwargs)
|
||||||
|
return {
|
||||||
|
"status": "succeeded",
|
||||||
|
"answer": "第二轮:继续收敛来源范围。",
|
||||||
|
"rag_result": {
|
||||||
|
"answer": "第二轮:继续收敛来源范围。",
|
||||||
|
"query_type": "plan",
|
||||||
|
},
|
||||||
|
"reply_template_key": "requirement_plan_followup",
|
||||||
|
}
|
||||||
|
|
||||||
|
monkeypatch.setattr(
|
||||||
|
WorkflowSemanticPlanner,
|
||||||
|
"aplan_message",
|
||||||
|
classmethod(lambda cls, **kwargs: _failing_aplan_message(**kwargs)),
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(
|
||||||
|
mixin_execute_query_project.ExecuteQueryProjectMixin,
|
||||||
|
"_build_requirement_planning_direct_result",
|
||||||
|
_fake_build_requirement_planning_direct_result,
|
||||||
|
raising=False,
|
||||||
|
)
|
||||||
|
monkeypatch.setattr(mixin_execute_query_project, "AdaptiveRAGService", FakeAdaptiveRAGService)
|
||||||
|
|
||||||
|
conversation_scope_key = build_feishu_conversation_scope_key(
|
||||||
|
source_chat_id="chat-degraded-plan",
|
||||||
|
source_sender_open_id="ou-degraded-plan",
|
||||||
|
sender=None,
|
||||||
|
)
|
||||||
|
topic = MountedTopicWorkspace(
|
||||||
|
tenant_id=3,
|
||||||
|
topic_key="feishu:chat-degraded-plan:root:om-degraded-plan-root",
|
||||||
|
source_system="feishu",
|
||||||
|
channel_ref="chat-degraded-plan",
|
||||||
|
conversation_scope_key=conversation_scope_key,
|
||||||
|
mode="plan",
|
||||||
|
status="active",
|
||||||
|
plan_status="collecting",
|
||||||
|
metadata_json={"plan_mode": True},
|
||||||
|
facts_json={"title": "Claude 模型信息卡片"},
|
||||||
|
)
|
||||||
|
requirement_plan_db_session.add(topic)
|
||||||
|
await requirement_plan_db_session.flush()
|
||||||
|
requirement_plan_db_session.add(
|
||||||
|
MountedDraftWorkspace(
|
||||||
|
tenant_id=3,
|
||||||
|
topic_workspace_id=int(topic.id),
|
||||||
|
state="collecting",
|
||||||
|
draft_version=0,
|
||||||
|
requirement_card={
|
||||||
|
"plan_profile": "it_project",
|
||||||
|
"title": "Claude 模型信息卡片",
|
||||||
|
"goal": "先把模型信息卡片需求收清楚。",
|
||||||
|
"project_nature": "轻量即用型工具",
|
||||||
|
"question_round": 1,
|
||||||
|
"last_round_options": {
|
||||||
|
"field": "project_nature",
|
||||||
|
"options": [
|
||||||
|
{"key": "A", "title": "轻量即用型工具"},
|
||||||
|
{"key": "B", "title": "业务协同平台"},
|
||||||
|
{"key": "C", "title": "AI能力底座模块"},
|
||||||
|
],
|
||||||
|
"recommendation": {"key": "A"},
|
||||||
|
},
|
||||||
|
"round_option_maps": {
|
||||||
|
"project_nature": {
|
||||||
|
"A": "轻量即用型工具",
|
||||||
|
"B": "业务协同平台",
|
||||||
|
"C": "AI能力底座模块",
|
||||||
|
}
|
||||||
|
},
|
||||||
|
},
|
||||||
|
required_confirmations=[],
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await requirement_plan_db_session.flush()
|
||||||
|
|
||||||
|
executor = FakeExecutor()
|
||||||
|
intent = SimpleNamespace(
|
||||||
|
project_id=None,
|
||||||
|
metadata={
|
||||||
|
"source_chat_id": "chat-degraded-plan",
|
||||||
|
"source_sender_open_id": "ou-degraded-plan",
|
||||||
|
"semantic_planner_degraded": True,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
result = await executor._execute_query_intent(
|
||||||
|
intent=intent,
|
||||||
|
message_text="A",
|
||||||
|
operator_user_id=999,
|
||||||
|
source_system="feishu",
|
||||||
|
source_message_id="msg-degraded-plan-1",
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result["status"] == "succeeded"
|
||||||
|
assert str(result["workflow_route"]) == "requirement_plan.collect"
|
||||||
|
assert str(result["rag_result"]["query_type"]) == "plan"
|
||||||
|
assert isinstance(captured.get("direct_kwargs"), dict)
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_execute_query_intent_promotes_requirement_followup_from_structured_round_context_without_render_markers(
|
async def test_execute_query_intent_promotes_requirement_followup_from_structured_round_context_without_render_markers(
|
||||||
monkeypatch,
|
monkeypatch,
|
||||||
|
|
@ -11068,6 +11684,7 @@ async def test_should_route_to_draft_confirmation_session_does_not_route_free_te
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_external_freshness_followup_stays_in_query_answer_without_internal_target(
|
async def test_external_freshness_followup_stays_in_query_answer_without_internal_target(
|
||||||
external_qa_db_session,
|
external_qa_db_session,
|
||||||
|
monkeypatch,
|
||||||
) -> None:
|
) -> None:
|
||||||
conversation_scope_key = build_feishu_conversation_scope_key(
|
conversation_scope_key = build_feishu_conversation_scope_key(
|
||||||
source_chat_id="chat-external-qa",
|
source_chat_id="chat-external-qa",
|
||||||
|
|
@ -11113,13 +11730,41 @@ async def test_external_freshness_followup_stays_in_query_answer_without_interna
|
||||||
self.db = external_qa_db_session
|
self.db = external_qa_db_session
|
||||||
self.tenant_id = 1
|
self.tenant_id = 1
|
||||||
|
|
||||||
|
async def _fake_aplan_message(cls, *, message_text: str, source_system: str, metadata=None):
|
||||||
|
_ = (cls, message_text, source_system, metadata)
|
||||||
|
return WorkflowSemanticPlanner.deserialize_bundle(
|
||||||
|
{
|
||||||
|
"bundle_id": "freshness-followup-query",
|
||||||
|
"semantic_version": "v1",
|
||||||
|
"source_system": "feishu",
|
||||||
|
"primary_workflow": "query_flow",
|
||||||
|
"workflows": [
|
||||||
|
{
|
||||||
|
"workflow_type": "query_flow",
|
||||||
|
"workflow_action": "answer",
|
||||||
|
"target_scope": "global",
|
||||||
|
"operation_codes": ["query.answer"],
|
||||||
|
"requires_confirmation": False,
|
||||||
|
"confidence": 0.91,
|
||||||
|
"clarification_needed": False,
|
||||||
|
"metadata": {},
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"message_gate": "auto",
|
||||||
|
"reply_contract": {"mode": "text", "template_key": "workflow_bundle", "summary": "query_flow"},
|
||||||
|
"diagnostics": {"message_intent": "question"},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
monkeypatch.setattr(WorkflowSemanticPlanner, "aplan_message", classmethod(_fake_aplan_message))
|
||||||
|
|
||||||
kernel = FakeKernel()
|
kernel = FakeKernel()
|
||||||
result = await kernel._maybe_handle_external_freshness_followup(
|
result = await kernel._maybe_handle_external_freshness_followup(
|
||||||
source_system="feishu",
|
source_system="feishu",
|
||||||
source_message_id="om-external-followup-1",
|
source_message_id="om-external-followup-1",
|
||||||
channel_ref="chat-external-qa",
|
channel_ref="chat-external-qa",
|
||||||
source_sender_open_id="ou-external-qa",
|
source_sender_open_id="ou-external-qa",
|
||||||
message_text="接入外部来源继续核实",
|
message_text="重新联网核实一下",
|
||||||
thread_context={"reply_to_message_id": "om-external-question-1"},
|
thread_context={"reply_to_message_id": "om-external-question-1"},
|
||||||
)
|
)
|
||||||
|
|
||||||
|
|
@ -11134,6 +11779,7 @@ async def test_external_freshness_followup_stays_in_query_answer_without_interna
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_external_freshness_followup_allows_explicit_internal_source_request(
|
async def test_external_freshness_followup_allows_explicit_internal_source_request(
|
||||||
external_qa_db_session,
|
external_qa_db_session,
|
||||||
|
monkeypatch,
|
||||||
) -> None:
|
) -> None:
|
||||||
conversation_scope_key = build_feishu_conversation_scope_key(
|
conversation_scope_key = build_feishu_conversation_scope_key(
|
||||||
source_chat_id="chat-external-qa",
|
source_chat_id="chat-external-qa",
|
||||||
|
|
@ -11179,6 +11825,34 @@ async def test_external_freshness_followup_allows_explicit_internal_source_reque
|
||||||
self.db = external_qa_db_session
|
self.db = external_qa_db_session
|
||||||
self.tenant_id = 1
|
self.tenant_id = 1
|
||||||
|
|
||||||
|
async def _fake_aplan_message(cls, *, message_text: str, source_system: str, metadata=None):
|
||||||
|
_ = (cls, message_text, source_system, metadata)
|
||||||
|
return WorkflowSemanticPlanner.deserialize_bundle(
|
||||||
|
{
|
||||||
|
"bundle_id": "freshness-followup-signal-source",
|
||||||
|
"semantic_version": "v1",
|
||||||
|
"source_system": "feishu",
|
||||||
|
"primary_workflow": "signal_source_flow",
|
||||||
|
"workflows": [
|
||||||
|
{
|
||||||
|
"workflow_type": "signal_source_flow",
|
||||||
|
"workflow_action": "onboard",
|
||||||
|
"target_scope": "global",
|
||||||
|
"operation_codes": ["signal_source.onboard"],
|
||||||
|
"requires_confirmation": False,
|
||||||
|
"confidence": 0.95,
|
||||||
|
"clarification_needed": False,
|
||||||
|
"metadata": {},
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"message_gate": "auto",
|
||||||
|
"reply_contract": {"mode": "text", "template_key": "workflow_bundle", "summary": "signal_source_flow"},
|
||||||
|
"diagnostics": {"message_intent": "question"},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
monkeypatch.setattr(WorkflowSemanticPlanner, "aplan_message", classmethod(_fake_aplan_message))
|
||||||
|
|
||||||
kernel = FakeKernel()
|
kernel = FakeKernel()
|
||||||
result = await kernel._maybe_handle_external_freshness_followup(
|
result = await kernel._maybe_handle_external_freshness_followup(
|
||||||
source_system="feishu",
|
source_system="feishu",
|
||||||
|
|
@ -11301,13 +11975,40 @@ async def test_process_ingress_short_circuits_external_freshness_followup_before
|
||||||
async def _prepare_mixed_sidecar_context(self, **kwargs):
|
async def _prepare_mixed_sidecar_context(self, **kwargs):
|
||||||
raise AssertionError("external freshness followup should not enter mixed sidecar")
|
raise AssertionError("external freshness followup should not enter mixed sidecar")
|
||||||
|
|
||||||
|
async def _fake_aplan_message(cls, *, message_text: str, source_system: str, metadata=None):
|
||||||
|
_ = (cls, message_text, source_system, metadata)
|
||||||
|
return WorkflowSemanticPlanner.deserialize_bundle(
|
||||||
|
{
|
||||||
|
"bundle_id": "freshness-followup-query-process-ingress",
|
||||||
|
"semantic_version": "v1",
|
||||||
|
"source_system": "feishu",
|
||||||
|
"primary_workflow": "query_flow",
|
||||||
|
"workflows": [
|
||||||
|
{
|
||||||
|
"workflow_type": "query_flow",
|
||||||
|
"workflow_action": "answer",
|
||||||
|
"target_scope": "global",
|
||||||
|
"operation_codes": ["query.answer"],
|
||||||
|
"requires_confirmation": False,
|
||||||
|
"confidence": 0.93,
|
||||||
|
"clarification_needed": False,
|
||||||
|
"metadata": {},
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"message_gate": "auto",
|
||||||
|
"reply_contract": {"mode": "text", "template_key": "workflow_bundle", "summary": "query_flow"},
|
||||||
|
"diagnostics": {"message_intent": "question"},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
monkeypatch.setattr(mixin_entry, "MountedAIRuntimeWriteService", FakeMountedAIRuntimeWriteService)
|
monkeypatch.setattr(mixin_entry, "MountedAIRuntimeWriteService", FakeMountedAIRuntimeWriteService)
|
||||||
|
monkeypatch.setattr(WorkflowSemanticPlanner, "aplan_message", classmethod(_fake_aplan_message))
|
||||||
|
|
||||||
kernel = FakeKernel()
|
kernel = FakeKernel()
|
||||||
result = await kernel.process_ingress(
|
result = await kernel.process_ingress(
|
||||||
source_system="feishu",
|
source_system="feishu",
|
||||||
source_message_id="om-external-followup-3",
|
source_message_id="om-external-followup-3",
|
||||||
message_text="接入外部来源继续核实",
|
message_text="继续联网查最新结果",
|
||||||
sender="林宇航",
|
sender="林宇航",
|
||||||
channel_ref="chat-external-qa",
|
channel_ref="chat-external-qa",
|
||||||
source_sender_open_id="ou-external-qa",
|
source_sender_open_id="ou-external-qa",
|
||||||
|
|
|
||||||
|
|
@ -636,6 +636,192 @@ async def test_openai_compatible_chat_accepts_structured_json_from_reasoning_con
|
||||||
assert json.loads(result.text)["title"] == "AI直播复盘助手"
|
assert json.loads(result.text)["title"] == "AI直播复盘助手"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_openai_compatible_chat_ignores_invalid_structured_candidate_from_reasoning_content_and_retries(
|
||||||
|
monkeypatch,
|
||||||
|
) -> None:
|
||||||
|
calls: list[dict[str, object]] = []
|
||||||
|
|
||||||
|
class FakeResponse:
|
||||||
|
def __init__(self, payload: dict[str, object]) -> None:
|
||||||
|
self.status_code = 200
|
||||||
|
self._payload = payload
|
||||||
|
|
||||||
|
def json(self):
|
||||||
|
return self._payload
|
||||||
|
|
||||||
|
valid_payload = {
|
||||||
|
"topic_relation": "new_topic",
|
||||||
|
"mode_decision": "answer_only",
|
||||||
|
"followup_scope": "none",
|
||||||
|
"followup_action": "none",
|
||||||
|
"revision_note": "",
|
||||||
|
"draft_patch": {},
|
||||||
|
"workflow_plans": [
|
||||||
|
{
|
||||||
|
"workflow_type": "query_flow",
|
||||||
|
"workflow_action": "answer",
|
||||||
|
"target_scope": "global",
|
||||||
|
"operation_codes": ["query.answer"],
|
||||||
|
"requires_confirmation": False,
|
||||||
|
"confidence": 0.91,
|
||||||
|
"clarification_needed": False,
|
||||||
|
"metadata": {},
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"scene_metadata": {},
|
||||||
|
"assistant_reply": "我会查询 Microsoft Learn 并返回最新官方链接。",
|
||||||
|
"requires_confirmation": False,
|
||||||
|
"requested_actions": ["workflow.bundle.compose"],
|
||||||
|
"confidence": 0.91,
|
||||||
|
"uncertainties": [],
|
||||||
|
}
|
||||||
|
|
||||||
|
class FakeClient:
|
||||||
|
def __init__(self, *args, **kwargs):
|
||||||
|
_ = (args, kwargs)
|
||||||
|
|
||||||
|
async def __aenter__(self):
|
||||||
|
return self
|
||||||
|
|
||||||
|
async def __aexit__(self, exc_type, exc, tb):
|
||||||
|
return False
|
||||||
|
|
||||||
|
async def post(self, *args, **kwargs):
|
||||||
|
calls.append(dict(kwargs.get("json") or {}))
|
||||||
|
if len(calls) == 1:
|
||||||
|
return FakeResponse(
|
||||||
|
{
|
||||||
|
"choices": [
|
||||||
|
{
|
||||||
|
"index": 0,
|
||||||
|
"message": {
|
||||||
|
"role": "assistant",
|
||||||
|
"content": "",
|
||||||
|
"reasoning_content": (
|
||||||
|
"1. 分析输入\n"
|
||||||
|
"packet.thread.recent_turns: []\n"
|
||||||
|
"packet.ingress.allowed_context_scopes: []\n"
|
||||||
|
"接下来应进入 query.answer。"
|
||||||
|
),
|
||||||
|
},
|
||||||
|
"finish_reason": "length",
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
)
|
||||||
|
return FakeResponse(
|
||||||
|
{
|
||||||
|
"choices": [
|
||||||
|
{
|
||||||
|
"index": 0,
|
||||||
|
"message": {
|
||||||
|
"role": "assistant",
|
||||||
|
"content": json.dumps(valid_payload, ensure_ascii=False),
|
||||||
|
"reasoning_content": "",
|
||||||
|
},
|
||||||
|
"finish_reason": "stop",
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
async def fake_sleep(_seconds: float):
|
||||||
|
return None
|
||||||
|
|
||||||
|
monkeypatch.setattr(mixin_chat_openai.httpx, "AsyncClient", FakeClient)
|
||||||
|
monkeypatch.setattr(mixin_chat_openai.asyncio, "sleep", fake_sleep)
|
||||||
|
|
||||||
|
result = await llm_service._chat_completion_via_openai_compatible(
|
||||||
|
key="test-key",
|
||||||
|
base_url="https://api.edgefn.net/v1",
|
||||||
|
model="GLM-5",
|
||||||
|
temperature=0.0,
|
||||||
|
max_tokens=512,
|
||||||
|
messages=[{"role": "user", "content": "帮我判断这句消息应该走哪个 workflow"}],
|
||||||
|
request_timeout=1.0,
|
||||||
|
structured_output=True,
|
||||||
|
function_label="pmflow_semantic_context_decision",
|
||||||
|
provider_name="openai",
|
||||||
|
is_control_plane_gateway_request=False,
|
||||||
|
control_plane_agents_timeout_seconds=120,
|
||||||
|
request_started_at_perf=time.perf_counter(),
|
||||||
|
request_trace_id="trace-invalid-reasoning-structured",
|
||||||
|
control_plane_agent_timeout=None,
|
||||||
|
routing_meta={"response_contract": "structured"},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert len(calls) == 2
|
||||||
|
assert json.loads(result.text)["topic_relation"] == "new_topic"
|
||||||
|
assert "最终 JSON 对象" in str(calls[1]["messages"][-1]["content"])
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_chat_completion_raw_opens_provider_circuit_immediately_on_billing_block(monkeypatch) -> None:
|
||||||
|
circuit_key = "glm|https://dashscope.aliyuncs.com/compatible-mode/v1|qwen-plus"
|
||||||
|
original_state = dict(llm_service._provider_circuit_state)
|
||||||
|
original_signature = llm_service._provider_circuit_config_signature
|
||||||
|
|
||||||
|
class FakeResponse:
|
||||||
|
status_code = 400
|
||||||
|
headers: dict[str, str] = {}
|
||||||
|
text = json.dumps(
|
||||||
|
{
|
||||||
|
"error": {
|
||||||
|
"message": "Access denied, please make sure your account is in good standing.",
|
||||||
|
"type": "Arrearage",
|
||||||
|
"code": "Arrearage",
|
||||||
|
}
|
||||||
|
},
|
||||||
|
ensure_ascii=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
def json(self):
|
||||||
|
return json.loads(self.text)
|
||||||
|
|
||||||
|
def raise_for_status(self):
|
||||||
|
request = httpx.Request("POST", "https://dashscope.aliyuncs.com/compatible-mode/v1/chat/completions")
|
||||||
|
response = httpx.Response(self.status_code, request=request, text=self.text)
|
||||||
|
raise httpx.HTTPStatusError("Client error '400 Bad Request'", request=request, response=response)
|
||||||
|
|
||||||
|
class FakeClient:
|
||||||
|
def __init__(self, *args, **kwargs):
|
||||||
|
_ = (args, kwargs)
|
||||||
|
|
||||||
|
async def __aenter__(self):
|
||||||
|
return self
|
||||||
|
|
||||||
|
async def __aexit__(self, exc_type, exc, tb):
|
||||||
|
return False
|
||||||
|
|
||||||
|
async def post(self, *args, **kwargs):
|
||||||
|
_ = (args, kwargs)
|
||||||
|
return FakeResponse()
|
||||||
|
|
||||||
|
monkeypatch.setattr(mixin_chat_openai.httpx, "AsyncClient", FakeClient)
|
||||||
|
try:
|
||||||
|
with pytest.raises(ValueError, match="provider_billing_blocked:glm"):
|
||||||
|
await llm_service._chat_completion_raw(
|
||||||
|
provider="glm",
|
||||||
|
prompt="输出 JSON",
|
||||||
|
system_prompt="你是 JSON 生成器",
|
||||||
|
api_key="glm-key",
|
||||||
|
base_url="https://dashscope.aliyuncs.com/compatible-mode/v1",
|
||||||
|
model="qwen-plus",
|
||||||
|
temperature=0.0,
|
||||||
|
max_tokens=64,
|
||||||
|
function_name="pmflow_semantic_context_decision",
|
||||||
|
response_mode="fast",
|
||||||
|
routing_meta={"response_contract": "structured"},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert llm_service._is_provider_circuit_open(circuit_key) is True
|
||||||
|
finally:
|
||||||
|
llm_service._provider_circuit_state.clear()
|
||||||
|
llm_service._provider_circuit_state.update(original_state)
|
||||||
|
llm_service._provider_circuit_config_signature = original_signature
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_openai_compatible_chat_retries_empty_reasoning_response_with_bigger_token_budget(monkeypatch) -> None:
|
async def test_openai_compatible_chat_retries_empty_reasoning_response_with_bigger_token_budget(monkeypatch) -> None:
|
||||||
calls: list[dict[str, object]] = []
|
calls: list[dict[str, object]] = []
|
||||||
|
|
|
||||||
|
|
@ -1568,6 +1568,112 @@ async def test_resolve_active_mounted_draft_keeps_retryable_pending_confirm_with
|
||||||
assert result == (draft_row, topic_row)
|
assert result == (draft_row, topic_row)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_resolve_active_mounted_draft_does_not_fall_back_to_same_scope_row_when_explicit_anchor_misses(
|
||||||
|
) -> None:
|
||||||
|
class _FakeResult:
|
||||||
|
def __init__(self, rows) -> None:
|
||||||
|
self._rows = list(rows)
|
||||||
|
|
||||||
|
def all(self):
|
||||||
|
return list(self._rows)
|
||||||
|
|
||||||
|
draft_row = SimpleNamespace(
|
||||||
|
id=18,
|
||||||
|
topic_workspace_id=177,
|
||||||
|
active_work_item_id=None,
|
||||||
|
state=STATE_DRAFT_PENDING_CONFIRM,
|
||||||
|
latest_reply_message_id="om-stale-draft-reply-1",
|
||||||
|
latest_source_message_id="src-stale-draft-1",
|
||||||
|
)
|
||||||
|
topic_row = SimpleNamespace(
|
||||||
|
id=177,
|
||||||
|
tenant_id=1,
|
||||||
|
source_system="feishu",
|
||||||
|
mode="plan",
|
||||||
|
status="active",
|
||||||
|
channel_ref="chat-1",
|
||||||
|
conversation_scope_key="feishu:chat-1:ou_xxx",
|
||||||
|
plan_status=STATE_DRAFT_PENDING_CONFIRM,
|
||||||
|
root_message_id="msg-stale-draft-root",
|
||||||
|
thread_key="",
|
||||||
|
last_message_id="src-stale-draft-1",
|
||||||
|
updated_at=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
class _FakeDB:
|
||||||
|
async def execute(self, query): # noqa: ARG002
|
||||||
|
return _FakeResult([(draft_row, topic_row)])
|
||||||
|
|
||||||
|
handler = DraftConfirmHandler(type("Kernel", (), {"db": _FakeDB(), "tenant_id": 1})())
|
||||||
|
|
||||||
|
result = await handler._resolve_active_mounted_draft(
|
||||||
|
sender="林宇航",
|
||||||
|
source_system="feishu",
|
||||||
|
channel_ref="chat-1",
|
||||||
|
source_sender_open_id="ou_xxx",
|
||||||
|
thread_context={
|
||||||
|
"reply_to_message_id": "om-current-plan-reply-1",
|
||||||
|
"root_message_id": "msg-current-plan-root",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result is None
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_has_active_draft_session_does_not_lock_same_scope_when_explicit_anchor_targets_other_thread(
|
||||||
|
) -> None:
|
||||||
|
class _FakeResult:
|
||||||
|
def __init__(self, rows) -> None:
|
||||||
|
self._rows = list(rows)
|
||||||
|
|
||||||
|
def all(self):
|
||||||
|
return list(self._rows)
|
||||||
|
|
||||||
|
stale_draft = SimpleNamespace(
|
||||||
|
id=28,
|
||||||
|
topic_workspace_id=277,
|
||||||
|
active_work_item_id=None,
|
||||||
|
state=STATE_DRAFT_PENDING_CONFIRM,
|
||||||
|
latest_reply_message_id="om-stale-draft-reply-2",
|
||||||
|
latest_source_message_id="src-stale-draft-2",
|
||||||
|
)
|
||||||
|
stale_topic = SimpleNamespace(
|
||||||
|
id=277,
|
||||||
|
tenant_id=1,
|
||||||
|
source_system="feishu",
|
||||||
|
mode="plan",
|
||||||
|
status="active",
|
||||||
|
channel_ref="chat-1",
|
||||||
|
conversation_scope_key="feishu:chat-1:ou_xxx",
|
||||||
|
plan_status=STATE_DRAFT_PENDING_CONFIRM,
|
||||||
|
root_message_id="msg-stale-draft-root-2",
|
||||||
|
thread_key="",
|
||||||
|
last_message_id="src-stale-draft-2",
|
||||||
|
updated_at=None,
|
||||||
|
)
|
||||||
|
|
||||||
|
class _FakeDB:
|
||||||
|
async def execute(self, query): # noqa: ARG002
|
||||||
|
return _FakeResult([(stale_draft, stale_topic)])
|
||||||
|
|
||||||
|
handler = DraftConfirmHandler(type("Kernel", (), {"db": _FakeDB(), "tenant_id": 1})())
|
||||||
|
|
||||||
|
has_session = await handler.has_active_draft_session(
|
||||||
|
sender="林宇航",
|
||||||
|
source_system="feishu",
|
||||||
|
channel_ref="chat-1",
|
||||||
|
source_sender_open_id="ou_xxx",
|
||||||
|
thread_context={
|
||||||
|
"reply_to_message_id": "om-current-plan-reply-2",
|
||||||
|
"root_message_id": "msg-current-plan-root-2",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert has_session is False
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_resolve_pending_draft_falls_back_to_anchor_matched_pending_work_item_when_active_mounted_draft_has_no_active_work_item_id(
|
async def test_resolve_pending_draft_falls_back_to_anchor_matched_pending_work_item_when_active_mounted_draft_has_no_active_work_item_id(
|
||||||
monkeypatch,
|
monkeypatch,
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,51 @@
|
||||||
|
from app.services.external_query_service.planner import ExternalQueryPlanner
|
||||||
|
|
||||||
|
|
||||||
|
def test_fallback_plan_stays_generic_for_fresh_weather_semantics() -> None:
|
||||||
|
planner = ExternalQueryPlanner(
|
||||||
|
llm_service=object(),
|
||||||
|
db=object(),
|
||||||
|
tenant_id=1,
|
||||||
|
)
|
||||||
|
|
||||||
|
plan = planner._build_fallback_plan(
|
||||||
|
question="帮我看看未来5天的广州天气如何",
|
||||||
|
query_semantics={
|
||||||
|
"query_type": "status",
|
||||||
|
"object_mentions": ["广州"],
|
||||||
|
"query_profile": {
|
||||||
|
"needs_freshness": True,
|
||||||
|
"referenced_objects": ["weather"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert plan.query_kind == "generic_web"
|
||||||
|
assert plan.answer_contract == "summary_with_sources"
|
||||||
|
assert plan.location_hint is None
|
||||||
|
assert plan.result_limit == 3
|
||||||
|
|
||||||
|
|
||||||
|
def test_fallback_plan_uses_compare_contract_without_object_kind_mapping() -> None:
|
||||||
|
planner = ExternalQueryPlanner(
|
||||||
|
llm_service=object(),
|
||||||
|
db=object(),
|
||||||
|
tenant_id=1,
|
||||||
|
)
|
||||||
|
|
||||||
|
plan = planner._build_fallback_plan(
|
||||||
|
question="对比 OpenAI 和 Anthropic 最新模型能力差异",
|
||||||
|
query_semantics={
|
||||||
|
"query_type": "compare",
|
||||||
|
"object_mentions": ["OpenAI", "Anthropic"],
|
||||||
|
"query_profile": {
|
||||||
|
"needs_freshness": True,
|
||||||
|
"referenced_objects": ["public_model"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert plan.query_kind == "generic_web"
|
||||||
|
assert plan.answer_contract == "compare_with_sources"
|
||||||
|
assert plan.requires_multi_source is True
|
||||||
|
assert plan.result_limit == 3
|
||||||
|
|
@ -277,7 +277,12 @@ async def test_handle_card_action_trigger_plan_choice_dispatches_synthetic_ingre
|
||||||
assert data.get("target_work_item_id") == 88
|
assert data.get("target_work_item_id") == 88
|
||||||
ingress_kwargs = called["ingress_kwargs"]
|
ingress_kwargs = called["ingress_kwargs"]
|
||||||
assert ingress_kwargs["message_text"] == "C"
|
assert ingress_kwargs["message_text"] == "C"
|
||||||
assert ingress_kwargs["source_system"] == "ai_pmo_ui"
|
assert ingress_kwargs["source_system"] == "feishu"
|
||||||
|
assert str(ingress_kwargs["source_message_id"]).startswith("feishu-card-plan-88-plan_choice-")
|
||||||
|
ingress_options = ingress_kwargs.get("ingress_options") if isinstance(ingress_kwargs.get("ingress_options"), dict) else {}
|
||||||
|
thread_context = ingress_options.get("thread_context") if isinstance(ingress_options.get("thread_context"), dict) else {}
|
||||||
|
assert thread_context["reply_to_message_id"] == "om_plan_1"
|
||||||
|
assert thread_context["root_message_id"] == "om_plan_1"
|
||||||
assert called["reply_chat"] == "oc_plan_1"
|
assert called["reply_chat"] == "oc_plan_1"
|
||||||
assert isinstance(called["reply_card"], dict)
|
assert isinstance(called["reply_card"], dict)
|
||||||
assert called["reply_card"]["header"]["title"]["content"] == "AI PMO 需求收敛"
|
assert called["reply_card"]["header"]["title"]["content"] == "AI PMO 需求收敛"
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,269 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from types import SimpleNamespace
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
import pytest_asyncio
|
||||||
|
from fastapi import FastAPI
|
||||||
|
from httpx import ASGITransport, AsyncClient
|
||||||
|
from sqlalchemy import JSON, select
|
||||||
|
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
||||||
|
|
||||||
|
from app.api.mcp_api import routes as mcp_routes
|
||||||
|
from app.core.database import get_db
|
||||||
|
from app.dependencies.auth import get_current_user
|
||||||
|
from app.models.base import Base
|
||||||
|
from app.models.integration import Integration
|
||||||
|
from app.services.mcp_config import McpConfigService
|
||||||
|
|
||||||
|
|
||||||
|
def _use_sqlite_json_types(*tables) -> None:
|
||||||
|
for table in tables:
|
||||||
|
for column in table.columns:
|
||||||
|
if column.type.__class__.__name__ in {"JSONB", "ARRAY", "Vector"}:
|
||||||
|
column.type = JSON()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest_asyncio.fixture
|
||||||
|
async def db_session():
|
||||||
|
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
|
||||||
|
tables = [Integration.__table__]
|
||||||
|
_use_sqlite_json_types(*tables)
|
||||||
|
async with engine.begin() as conn:
|
||||||
|
await conn.run_sync(lambda sync_conn: Base.metadata.create_all(sync_conn, tables=tables))
|
||||||
|
maker = async_sessionmaker(engine, expire_on_commit=False)
|
||||||
|
async with maker() as session:
|
||||||
|
yield session
|
||||||
|
await engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
def _build_app(db_session, *, role: str, tenant_id: int = 1) -> FastAPI:
|
||||||
|
app = FastAPI()
|
||||||
|
app.include_router(mcp_routes.router)
|
||||||
|
|
||||||
|
async def _fake_get_db():
|
||||||
|
yield db_session
|
||||||
|
|
||||||
|
app.dependency_overrides[get_db] = _fake_get_db
|
||||||
|
app.dependency_overrides[get_current_user] = lambda: SimpleNamespace(
|
||||||
|
id=7,
|
||||||
|
tenant_id=tenant_id,
|
||||||
|
role=role,
|
||||||
|
username="tester",
|
||||||
|
email="tester@example.com",
|
||||||
|
name="Tester",
|
||||||
|
)
|
||||||
|
return app
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_mcp_admin_api_requires_manager_role(db_session) -> None:
|
||||||
|
app = _build_app(db_session, role="AI")
|
||||||
|
transport = ASGITransport(app=app)
|
||||||
|
|
||||||
|
async with AsyncClient(transport=transport, base_url="http://test") as client:
|
||||||
|
response = await client.get("/mcp/admin/catalog")
|
||||||
|
|
||||||
|
assert response.status_code == 403
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_mcp_admin_catalog_and_server_lifecycle(db_session, monkeypatch) -> None:
|
||||||
|
async def _fake_probe(self, *, template, params): # noqa: ANN001
|
||||||
|
_ = (self, template, params)
|
||||||
|
return {
|
||||||
|
"status": "ready",
|
||||||
|
"checked_at": "2026-04-15T12:01:00+08:00",
|
||||||
|
"message": "PMFlow backend can reach Microsoft Learn MCP.",
|
||||||
|
"tool_names": ["microsoft_docs_search", "microsoft_docs_fetch"],
|
||||||
|
}
|
||||||
|
|
||||||
|
monkeypatch.setattr(McpConfigService, "_probe_backend_runtime", _fake_probe)
|
||||||
|
app = _build_app(db_session, role="ADMIN")
|
||||||
|
transport = ASGITransport(app=app)
|
||||||
|
|
||||||
|
async with AsyncClient(transport=transport, base_url="http://test") as client:
|
||||||
|
catalog_response = await client.get("/mcp/admin/catalog")
|
||||||
|
create_response = await client.post(
|
||||||
|
"/mcp/admin/servers",
|
||||||
|
json={
|
||||||
|
"name": "microsoft-learn",
|
||||||
|
"display_name": "Microsoft Learn MCP",
|
||||||
|
"template_id": "microsoft_learn",
|
||||||
|
"params": {},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
created = create_response.json()["data"]
|
||||||
|
list_response = await client.get("/mcp/admin/servers")
|
||||||
|
validate_response = await client.post(f"/mcp/admin/servers/{created['id']}/validate")
|
||||||
|
update_response = await client.put(
|
||||||
|
f"/mcp/admin/servers/{created['id']}",
|
||||||
|
json={
|
||||||
|
"display_name": "Microsoft Learn Docs",
|
||||||
|
"params": {},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
confirm_response = await client.post(f"/mcp/admin/servers/{created['id']}/confirm-client-apply")
|
||||||
|
|
||||||
|
assert catalog_response.status_code == 200
|
||||||
|
assert catalog_response.json()["code"] == 0
|
||||||
|
assert catalog_response.json()["data"]["recommended_template_id"] == "microsoft_learn"
|
||||||
|
assert catalog_response.json()["data"]["items"][0]["execution_mode"] == "backend_direct"
|
||||||
|
|
||||||
|
assert create_response.status_code == 200
|
||||||
|
assert create_response.json()["code"] == 0
|
||||||
|
assert created["execution_mode"] == "backend_direct"
|
||||||
|
assert created["client_apply_state"] == "not_required"
|
||||||
|
assert created["runtime_probe"]["status"] == "unknown"
|
||||||
|
|
||||||
|
assert list_response.status_code == 200
|
||||||
|
assert list_response.json()["data"]["items"][0]["name"] == "microsoft-learn"
|
||||||
|
|
||||||
|
assert validate_response.status_code == 200
|
||||||
|
assert validate_response.json()["code"] == 0
|
||||||
|
assert validate_response.json()["data"]["is_valid"] is True
|
||||||
|
assert validate_response.json()["data"]["preview"]["add_command"] == "无需本机命令。保存后由 PMFlow 后端直接连接 Microsoft Learn MCP。"
|
||||||
|
assert validate_response.json()["data"]["preview"]["verify_command"] == "点击“重新校验”后,PMFlow 会从后端执行 MCP initialize 和 tools/list。"
|
||||||
|
assert validate_response.json()["data"]["runtime_probe"]["status"] == "ready"
|
||||||
|
assert validate_response.json()["data"]["runtime_probe"]["tool_names"] == ["microsoft_docs_search", "microsoft_docs_fetch"]
|
||||||
|
|
||||||
|
assert update_response.status_code == 200
|
||||||
|
assert update_response.json()["code"] == 0
|
||||||
|
assert update_response.json()["data"]["display_name"] == "Microsoft Learn Docs"
|
||||||
|
|
||||||
|
assert confirm_response.status_code == 400
|
||||||
|
assert confirm_response.json()["code"] == 400
|
||||||
|
assert "后端直连" in confirm_response.json()["message"]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_mcp_admin_servers_are_tenant_scoped(db_session) -> None:
|
||||||
|
tenant1_app = _build_app(db_session, role="ADMIN", tenant_id=1)
|
||||||
|
tenant2_app = _build_app(db_session, role="ADMIN", tenant_id=2)
|
||||||
|
|
||||||
|
async with AsyncClient(transport=ASGITransport(app=tenant1_app), base_url="http://test") as client:
|
||||||
|
create_response = await client.post(
|
||||||
|
"/mcp/admin/servers",
|
||||||
|
json={
|
||||||
|
"name": "playwright",
|
||||||
|
"display_name": "Playwright MCP",
|
||||||
|
"template_id": "playwright",
|
||||||
|
"params": {"output_dir": "~/.codex/mcp/playwright-output"},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
async with AsyncClient(transport=ASGITransport(app=tenant2_app), base_url="http://test") as client:
|
||||||
|
list_response = await client.get("/mcp/admin/servers")
|
||||||
|
|
||||||
|
assert create_response.status_code == 200
|
||||||
|
assert create_response.json()["code"] == 0
|
||||||
|
assert list_response.status_code == 200
|
||||||
|
assert list_response.json()["code"] == 0
|
||||||
|
assert list_response.json()["data"]["items"] == []
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_mcp_admin_github_template_rejects_raw_token_and_only_persists_env_var_name(db_session) -> None:
|
||||||
|
app = _build_app(db_session, role="ADMIN")
|
||||||
|
transport = ASGITransport(app=app)
|
||||||
|
|
||||||
|
async with AsyncClient(transport=transport, base_url="http://test") as client:
|
||||||
|
invalid_response = await client.post(
|
||||||
|
"/mcp/admin/servers",
|
||||||
|
json={
|
||||||
|
"name": "github",
|
||||||
|
"display_name": "GitHub MCP",
|
||||||
|
"template_id": "github",
|
||||||
|
"params": {
|
||||||
|
"token_env_var_name": "GITHUB_PAT",
|
||||||
|
"access_token": "ghp_secret_should_not_be_saved",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
valid_response = await client.post(
|
||||||
|
"/mcp/admin/servers",
|
||||||
|
json={
|
||||||
|
"name": "github",
|
||||||
|
"display_name": "GitHub MCP",
|
||||||
|
"template_id": "github",
|
||||||
|
"params": {
|
||||||
|
"token_env_var_name": "GITHUB_PAT",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
row = (
|
||||||
|
await db_session.execute(
|
||||||
|
select(Integration).where(
|
||||||
|
Integration.tenant_id == 1,
|
||||||
|
Integration.type == "mcp_template_center",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one()
|
||||||
|
server = row.config["servers"][0]
|
||||||
|
|
||||||
|
assert invalid_response.status_code == 400
|
||||||
|
assert invalid_response.json()["code"] == 400
|
||||||
|
assert "token" in invalid_response.json()["message"].lower()
|
||||||
|
|
||||||
|
assert valid_response.status_code == 200
|
||||||
|
assert valid_response.json()["code"] == 0
|
||||||
|
assert valid_response.json()["data"]["execution_mode"] == "local_bridge"
|
||||||
|
assert server["params"]["token_env_var_name"] == "GITHUB_PAT"
|
||||||
|
assert "access_token" not in server["params"]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_mcp_admin_filesystem_requires_allowed_paths(db_session) -> None:
|
||||||
|
app = _build_app(db_session, role="ADMIN")
|
||||||
|
transport = ASGITransport(app=app)
|
||||||
|
|
||||||
|
async with AsyncClient(transport=transport, base_url="http://test") as client:
|
||||||
|
response = await client.post(
|
||||||
|
"/mcp/admin/servers",
|
||||||
|
json={
|
||||||
|
"name": "filesystem",
|
||||||
|
"display_name": "Filesystem MCP",
|
||||||
|
"template_id": "filesystem",
|
||||||
|
"params": {},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert response.status_code == 400
|
||||||
|
assert response.json()["code"] == 400
|
||||||
|
assert "allowed_paths" in response.json()["message"]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_local_bridge_template_uses_manual_instructions_and_can_be_confirmed(db_session) -> None:
|
||||||
|
app = _build_app(db_session, role="ADMIN")
|
||||||
|
transport = ASGITransport(app=app)
|
||||||
|
|
||||||
|
async with AsyncClient(transport=transport, base_url="http://test") as client:
|
||||||
|
create_response = await client.post(
|
||||||
|
"/mcp/admin/servers",
|
||||||
|
json={
|
||||||
|
"name": "filesystem",
|
||||||
|
"display_name": "Filesystem MCP",
|
||||||
|
"template_id": "filesystem",
|
||||||
|
"params": {
|
||||||
|
"allowed_paths": ["/Users/yuuuu/Documents"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
created = create_response.json()["data"]
|
||||||
|
validate_response = await client.post(f"/mcp/admin/servers/{created['id']}/validate")
|
||||||
|
confirm_response = await client.post(f"/mcp/admin/servers/{created['id']}/confirm-client-apply")
|
||||||
|
|
||||||
|
assert create_response.status_code == 200
|
||||||
|
assert created["execution_mode"] == "local_bridge"
|
||||||
|
assert created["client_apply_state"] == "pending_local_apply"
|
||||||
|
assert "codex mcp add" not in created["preview"]["add_command"].lower()
|
||||||
|
assert "local bridge" in created["preview"]["add_command"].lower()
|
||||||
|
|
||||||
|
assert validate_response.status_code == 200
|
||||||
|
assert validate_response.json()["data"]["runtime_probe"]["status"] == "local_bridge_required"
|
||||||
|
|
||||||
|
assert confirm_response.status_code == 200
|
||||||
|
assert confirm_response.json()["data"]["client_apply_state"] == "user_confirmed"
|
||||||
|
|
@ -0,0 +1,448 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from types import SimpleNamespace
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
import pytest_asyncio
|
||||||
|
from sqlalchemy import JSON
|
||||||
|
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
||||||
|
|
||||||
|
from app.models.base import Base
|
||||||
|
from app.models.integration import Integration
|
||||||
|
from app.services.adaptive_rag_service.mixin_answer_retrieval import AdaptiveRAGAnswerRetrievalMixin
|
||||||
|
from app.services.adaptive_rag_service.mixin_deep_guard_rules import AdaptiveRAGDeepGuardRulesMixin
|
||||||
|
from app.services.adaptive_rag_service.shared_part_02 import RetrievedItem
|
||||||
|
from app.services.external_query_service.adapters import MCPExternalAdapter
|
||||||
|
from app.services.external_query_service.gateway import ExternalQueryGateway
|
||||||
|
from app.services.external_query_service.registry import ExternalProviderRegistry
|
||||||
|
from app.services.external_query_service.schemas import ExternalAdapterResult, ExternalEvidenceItem, ExternalQueryPlan
|
||||||
|
from app.services.mcp_config import McpConfigService
|
||||||
|
from app.services.mcp_config.runtime import MicrosoftLearnMCPRuntime
|
||||||
|
|
||||||
|
|
||||||
|
def _use_sqlite_json_types(*tables) -> None:
|
||||||
|
for table in tables:
|
||||||
|
for column in table.columns:
|
||||||
|
if column.type.__class__.__name__ in {"JSONB", "ARRAY", "Vector"}:
|
||||||
|
column.type = JSON()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest_asyncio.fixture
|
||||||
|
async def db_session():
|
||||||
|
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
|
||||||
|
tables = [Integration.__table__]
|
||||||
|
_use_sqlite_json_types(*tables)
|
||||||
|
async with engine.begin() as conn:
|
||||||
|
await conn.run_sync(lambda sync_conn: Base.metadata.create_all(sync_conn, tables=tables))
|
||||||
|
maker = async_sessionmaker(engine, expire_on_commit=False)
|
||||||
|
async with maker() as session:
|
||||||
|
yield session
|
||||||
|
await engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
async def _seed_backend_direct_server(db_session) -> None:
|
||||||
|
service = McpConfigService(db_session, tenant_id=1)
|
||||||
|
await service.create_server(
|
||||||
|
name="microsoft-learn",
|
||||||
|
display_name="Microsoft Learn MCP",
|
||||||
|
template_id="microsoft_learn",
|
||||||
|
params={},
|
||||||
|
)
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_mcp_config_service_search_external_uses_backend_direct_server(db_session, monkeypatch) -> None:
|
||||||
|
await _seed_backend_direct_server(db_session)
|
||||||
|
|
||||||
|
async def _fake_search(self, *, query: str, result_limit: int = 3): # noqa: ANN001
|
||||||
|
_ = (self, query, result_limit)
|
||||||
|
return [
|
||||||
|
ExternalEvidenceItem(
|
||||||
|
source_type="mcp_search",
|
||||||
|
source_label="Microsoft Learn MCP",
|
||||||
|
title="Azure OpenAI quickstart",
|
||||||
|
target_url="https://learn.microsoft.com/azure/ai-foundry/openai/chatgpt-quickstart",
|
||||||
|
snippet="Official Microsoft quickstart.",
|
||||||
|
retrieved_at="2026-04-15T12:30:00+08:00",
|
||||||
|
freshness_score=0.9,
|
||||||
|
authority_score=0.98,
|
||||||
|
grounded_fields=["microsoft_docs_search"],
|
||||||
|
raw_metadata={"provider": "microsoft_learn_mcp"},
|
||||||
|
)
|
||||||
|
]
|
||||||
|
|
||||||
|
monkeypatch.setattr(MicrosoftLearnMCPRuntime, "search", _fake_search)
|
||||||
|
|
||||||
|
evidence = await McpConfigService(db_session, tenant_id=1).search_external(
|
||||||
|
query="azure openai quickstart",
|
||||||
|
result_limit=3,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert len(evidence) == 1
|
||||||
|
assert evidence[0].source_type == "mcp_search"
|
||||||
|
assert evidence[0].target_url == "https://learn.microsoft.com/azure/ai-foundry/openai/chatgpt-quickstart"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_external_query_gateway_prefers_backend_direct_mcp_when_available(db_session, monkeypatch) -> None:
|
||||||
|
await _seed_backend_direct_server(db_session)
|
||||||
|
|
||||||
|
async def _fake_search_external(self, *, query: str, result_limit: int = 3): # noqa: ANN001
|
||||||
|
_ = (self, query, result_limit)
|
||||||
|
return [
|
||||||
|
ExternalEvidenceItem(
|
||||||
|
source_type="mcp_search",
|
||||||
|
source_label="Microsoft Learn MCP",
|
||||||
|
title="Azure OpenAI quickstart",
|
||||||
|
target_url="https://learn.microsoft.com/azure/ai-foundry/openai/chatgpt-quickstart",
|
||||||
|
snippet="Official Microsoft quickstart.",
|
||||||
|
retrieved_at="2026-04-15T12:30:00+08:00",
|
||||||
|
freshness_score=0.9,
|
||||||
|
authority_score=0.98,
|
||||||
|
grounded_fields=["microsoft_docs_search"],
|
||||||
|
raw_metadata={"provider": "microsoft_learn_mcp"},
|
||||||
|
)
|
||||||
|
]
|
||||||
|
|
||||||
|
monkeypatch.setattr(McpConfigService, "search_external", _fake_search_external)
|
||||||
|
|
||||||
|
class _FakeService(AdaptiveRAGAnswerRetrievalMixin, AdaptiveRAGDeepGuardRulesMixin):
|
||||||
|
def __init__(self, db):
|
||||||
|
self.db = db
|
||||||
|
self.tenant_id = 1
|
||||||
|
self.llm = None
|
||||||
|
|
||||||
|
service = _FakeService(db_session)
|
||||||
|
|
||||||
|
rows, meta = await service._execute_external_query_gateway(
|
||||||
|
question="Azure OpenAI quickstart",
|
||||||
|
query_semantics={},
|
||||||
|
result_limit=3,
|
||||||
|
max_pages=1,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert len(rows) == 1
|
||||||
|
assert isinstance(rows[0], RetrievedItem)
|
||||||
|
assert rows[0].source == "mcp_search"
|
||||||
|
assert rows[0].metadata["external_adapter"] == "mcp"
|
||||||
|
assert meta["adapter"] == "mcp"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_external_query_gateway_falls_back_when_mcp_evidence_is_off_topic() -> None:
|
||||||
|
plan = ExternalQueryPlan(
|
||||||
|
plan_version="v1",
|
||||||
|
query_kind="weather",
|
||||||
|
answer_contract="summary_with_sources",
|
||||||
|
freshness_required=True,
|
||||||
|
topic="广州未来5天天气",
|
||||||
|
subject_entities=["广州"],
|
||||||
|
location_hint="广州",
|
||||||
|
time_range_hint="未来5天",
|
||||||
|
preferred_domains=[],
|
||||||
|
search_queries=["广州 未来5天天气"],
|
||||||
|
result_limit=3,
|
||||||
|
requires_multi_source=False,
|
||||||
|
clarify_on_ambiguity=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
class _Planner:
|
||||||
|
async def build_plan(self, *, question: str, query_semantics: dict | None = None):
|
||||||
|
_ = (question, query_semantics)
|
||||||
|
return plan, {"planner": "test"}
|
||||||
|
|
||||||
|
class _MCPAdapter:
|
||||||
|
adapter_name = "mcp"
|
||||||
|
|
||||||
|
async def execute(self, current_plan: ExternalQueryPlan) -> ExternalAdapterResult:
|
||||||
|
_ = current_plan
|
||||||
|
return ExternalAdapterResult(
|
||||||
|
adapter_name="mcp",
|
||||||
|
status="success",
|
||||||
|
evidence=[
|
||||||
|
ExternalEvidenceItem(
|
||||||
|
source_type="mcp_search",
|
||||||
|
source_label="Microsoft Learn MCP",
|
||||||
|
title="Azure Maps Weather services",
|
||||||
|
target_url="https://learn.microsoft.com/azure/azure-maps/weather-services-concepts",
|
||||||
|
snippet="Learn how Azure Maps Weather APIs expose forecast data models and request options.",
|
||||||
|
retrieved_at="2026-04-15T12:30:00+08:00",
|
||||||
|
freshness_score=0.82,
|
||||||
|
authority_score=0.98,
|
||||||
|
grounded_fields=["microsoft_docs_search"],
|
||||||
|
raw_metadata={"provider": "microsoft_learn_mcp"},
|
||||||
|
)
|
||||||
|
],
|
||||||
|
meta={"provider": "mcp"},
|
||||||
|
)
|
||||||
|
|
||||||
|
class _LiveSearchAdapter:
|
||||||
|
adapter_name = "live_search"
|
||||||
|
|
||||||
|
async def execute(self, current_plan: ExternalQueryPlan) -> ExternalAdapterResult:
|
||||||
|
_ = current_plan
|
||||||
|
return ExternalAdapterResult(
|
||||||
|
adapter_name="live_search",
|
||||||
|
status="success",
|
||||||
|
evidence=[
|
||||||
|
ExternalEvidenceItem(
|
||||||
|
source_type="url",
|
||||||
|
source_label="Weather Example",
|
||||||
|
title="广州未来5天天气预报",
|
||||||
|
target_url="https://weather.example/guangzhou-5day",
|
||||||
|
snippet="广州未来5天有阵雨转多云,最高温31度,最低温24度。",
|
||||||
|
retrieved_at="2026-04-15T12:35:00+08:00",
|
||||||
|
freshness_score=0.91,
|
||||||
|
authority_score=0.76,
|
||||||
|
grounded_fields=["forecast"],
|
||||||
|
raw_metadata={"provider": "live_search"},
|
||||||
|
)
|
||||||
|
],
|
||||||
|
meta={"provider": "live_search"},
|
||||||
|
)
|
||||||
|
|
||||||
|
execution = await ExternalQueryGateway(
|
||||||
|
planner=_Planner(),
|
||||||
|
registry=ExternalProviderRegistry(adapters=[_MCPAdapter(), _LiveSearchAdapter()]),
|
||||||
|
).execute(
|
||||||
|
question="帮我看看未来5天的广州天气如何",
|
||||||
|
query_semantics={"query_profile": {"needs_freshness": True}},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert execution.adapter_name == "live_search"
|
||||||
|
assert execution.meta["fallback_used"] is True
|
||||||
|
assert execution.verifier["ok"] is True
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_mcp_adapter_fetches_target_page_content_after_search_result() -> None:
|
||||||
|
plan = ExternalQueryPlan(
|
||||||
|
plan_version="v1",
|
||||||
|
query_kind="weather",
|
||||||
|
answer_contract="summary_with_sources",
|
||||||
|
freshness_required=True,
|
||||||
|
topic="湛江市未来五天天气",
|
||||||
|
subject_entities=["湛江市"],
|
||||||
|
location_hint="湛江市",
|
||||||
|
time_range_hint="未来5天",
|
||||||
|
preferred_domains=["weather.example"],
|
||||||
|
search_queries=["湛江市未来五天天气预报"],
|
||||||
|
result_limit=3,
|
||||||
|
requires_multi_source=False,
|
||||||
|
clarify_on_ambiguity=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
class _Planner:
|
||||||
|
async def build_plan(self, *, question: str, query_semantics: dict | None = None):
|
||||||
|
_ = (question, query_semantics)
|
||||||
|
return plan, {"planner": "test"}
|
||||||
|
|
||||||
|
async def _search_callable(current_plan: ExternalQueryPlan) -> ExternalAdapterResult:
|
||||||
|
_ = current_plan
|
||||||
|
return ExternalAdapterResult(
|
||||||
|
adapter_name="mcp",
|
||||||
|
status="success",
|
||||||
|
evidence=[
|
||||||
|
ExternalEvidenceItem(
|
||||||
|
source_type="mcp_search",
|
||||||
|
source_label="Weather MCP",
|
||||||
|
title="Forecast detail page",
|
||||||
|
target_url="https://weather.example/forecast/zhanjiang-five-day",
|
||||||
|
snippet="Landing page",
|
||||||
|
retrieved_at="2026-04-16T01:05:00+08:00",
|
||||||
|
freshness_score=0.6,
|
||||||
|
authority_score=0.86,
|
||||||
|
grounded_fields=["mcp_search"],
|
||||||
|
raw_metadata={"provider": "weather_mcp"},
|
||||||
|
)
|
||||||
|
],
|
||||||
|
meta={"provider": "weather_mcp"},
|
||||||
|
)
|
||||||
|
|
||||||
|
async def _fetch_text(url: str) -> str:
|
||||||
|
assert url == "https://weather.example/forecast/zhanjiang-five-day"
|
||||||
|
return "页面说明。湛江未来5天有阵雨转多云,最高温33度,最低温26度。末尾。"
|
||||||
|
|
||||||
|
def _chunk_text(text: str, *, chunk_size: int = 900, overlap: int = 120) -> list[str]:
|
||||||
|
_ = (text, chunk_size, overlap)
|
||||||
|
return ["页面说明。", "湛江未来5天有阵雨转多云,最高温33度,最低温26度。"]
|
||||||
|
|
||||||
|
def _score_chunk(*, question: str, title: str, content: str, url: str) -> float:
|
||||||
|
_ = (question, title, url)
|
||||||
|
return 0.94 if "湛江未来5天" in content else 0.08
|
||||||
|
|
||||||
|
execution = await ExternalQueryGateway(
|
||||||
|
planner=_Planner(),
|
||||||
|
registry=ExternalProviderRegistry(
|
||||||
|
adapters=[
|
||||||
|
MCPExternalAdapter(
|
||||||
|
search_callable=_search_callable,
|
||||||
|
fetch_text_callable=_fetch_text,
|
||||||
|
chunk_text_callable=_chunk_text,
|
||||||
|
score_chunk_callable=_score_chunk,
|
||||||
|
max_pages=1,
|
||||||
|
)
|
||||||
|
]
|
||||||
|
),
|
||||||
|
).execute(
|
||||||
|
question="湛江市未来五天的天气",
|
||||||
|
query_semantics={"query_profile": {"needs_freshness": True}},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert execution.adapter_name == "mcp"
|
||||||
|
assert execution.verifier["ok"] is True
|
||||||
|
assert execution.evidence[0].snippet == "湛江未来5天有阵雨转多云,最高温33度,最低温26度。"
|
||||||
|
assert execution.evidence[0].raw_metadata["search_snippet"] == "Landing page"
|
||||||
|
assert "page_content_snippet" in execution.evidence[0].grounded_fields
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_external_query_gateway_accepts_bundle_with_normalized_anchor_match() -> None:
|
||||||
|
plan = ExternalQueryPlan(
|
||||||
|
plan_version="v1",
|
||||||
|
query_kind="weather",
|
||||||
|
answer_contract="summary_with_sources",
|
||||||
|
freshness_required=True,
|
||||||
|
topic="湛江市未来五天天气",
|
||||||
|
subject_entities=["湛江市"],
|
||||||
|
location_hint="湛江市",
|
||||||
|
time_range_hint="未来5天",
|
||||||
|
preferred_domains=["weather.com.cn", "nmc.cn"],
|
||||||
|
search_queries=["湛江市未来五天天气预报"],
|
||||||
|
result_limit=3,
|
||||||
|
requires_multi_source=False,
|
||||||
|
clarify_on_ambiguity=False,
|
||||||
|
)
|
||||||
|
|
||||||
|
class _Planner:
|
||||||
|
async def build_plan(self, *, question: str, query_semantics: dict | None = None):
|
||||||
|
_ = (question, query_semantics)
|
||||||
|
return plan, {"planner": "test"}
|
||||||
|
|
||||||
|
class _LiveSearchAdapter:
|
||||||
|
adapter_name = "live_search"
|
||||||
|
|
||||||
|
async def execute(self, current_plan: ExternalQueryPlan) -> ExternalAdapterResult:
|
||||||
|
_ = current_plan
|
||||||
|
return ExternalAdapterResult(
|
||||||
|
adapter_name="live_search",
|
||||||
|
status="success",
|
||||||
|
evidence=[
|
||||||
|
ExternalEvidenceItem(
|
||||||
|
source_type="url",
|
||||||
|
source_label="中国天气网",
|
||||||
|
title="湛江天气预报,湛江7天天气预报,湛江15天天气预报,湛江天气查询",
|
||||||
|
target_url="https://www.weather.com.cn/weather/101281001.shtml",
|
||||||
|
snippet="湛江未来5天有阵雨转多云,最高温33度,最低温26度。",
|
||||||
|
retrieved_at="2026-04-16T00:30:00+08:00",
|
||||||
|
freshness_score=0.92,
|
||||||
|
authority_score=0.95,
|
||||||
|
grounded_fields=["forecast"],
|
||||||
|
raw_metadata={"provider": "live_search"},
|
||||||
|
)
|
||||||
|
],
|
||||||
|
meta={"provider": "live_search"},
|
||||||
|
)
|
||||||
|
|
||||||
|
execution = await ExternalQueryGateway(
|
||||||
|
planner=_Planner(),
|
||||||
|
registry=ExternalProviderRegistry(adapters=[_LiveSearchAdapter()]),
|
||||||
|
).execute(
|
||||||
|
question="湛江市未来五天的天气",
|
||||||
|
query_semantics={"query_profile": {"needs_freshness": True}},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert execution.adapter_name == "live_search"
|
||||||
|
assert execution.verifier["ok"] is True
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_external_query_gateway_prefers_weather_capable_adapter_before_doc_mcp() -> None:
|
||||||
|
plan = ExternalQueryPlan(
|
||||||
|
plan_version="v1",
|
||||||
|
query_kind="weather",
|
||||||
|
answer_contract="summary_with_sources",
|
||||||
|
freshness_required=True,
|
||||||
|
topic="湛江市未来五天天气",
|
||||||
|
subject_entities=["湛江市"],
|
||||||
|
location_hint="湛江市",
|
||||||
|
time_range_hint="未来5天",
|
||||||
|
preferred_domains=["weather.com.cn", "nmc.cn"],
|
||||||
|
search_queries=["湛江市未来五天天气预报"],
|
||||||
|
result_limit=3,
|
||||||
|
requires_multi_source=False,
|
||||||
|
clarify_on_ambiguity=False,
|
||||||
|
)
|
||||||
|
call_order: list[str] = []
|
||||||
|
|
||||||
|
class _Planner:
|
||||||
|
async def build_plan(self, *, question: str, query_semantics: dict | None = None):
|
||||||
|
_ = (question, query_semantics)
|
||||||
|
return plan, {"planner": "test"}
|
||||||
|
|
||||||
|
class _MCPAdapter:
|
||||||
|
adapter_name = "mcp"
|
||||||
|
supported_query_kinds = {"model", "company", "generic_web", "unknown"}
|
||||||
|
|
||||||
|
async def execute(self, current_plan: ExternalQueryPlan) -> ExternalAdapterResult:
|
||||||
|
_ = current_plan
|
||||||
|
call_order.append("mcp")
|
||||||
|
return ExternalAdapterResult(
|
||||||
|
adapter_name="mcp",
|
||||||
|
status="success",
|
||||||
|
evidence=[
|
||||||
|
ExternalEvidenceItem(
|
||||||
|
source_type="mcp_search",
|
||||||
|
source_label="Microsoft Learn MCP",
|
||||||
|
title="Azure Maps Weather service coverage",
|
||||||
|
target_url="https://learn.microsoft.com/azure/azure-maps/weather-coverage",
|
||||||
|
snippet="Weather service coverage.",
|
||||||
|
retrieved_at="2026-04-16T00:30:00+08:00",
|
||||||
|
freshness_score=0.82,
|
||||||
|
authority_score=0.98,
|
||||||
|
grounded_fields=["microsoft_docs_search"],
|
||||||
|
raw_metadata={"provider": "mcp"},
|
||||||
|
)
|
||||||
|
],
|
||||||
|
meta={"provider": "mcp"},
|
||||||
|
)
|
||||||
|
|
||||||
|
class _LiveSearchAdapter:
|
||||||
|
adapter_name = "live_search"
|
||||||
|
supported_query_kinds = {"weather", "news", "market", "generic_web", "unknown"}
|
||||||
|
|
||||||
|
async def execute(self, current_plan: ExternalQueryPlan) -> ExternalAdapterResult:
|
||||||
|
_ = current_plan
|
||||||
|
call_order.append("live_search")
|
||||||
|
return ExternalAdapterResult(
|
||||||
|
adapter_name="live_search",
|
||||||
|
status="success",
|
||||||
|
evidence=[
|
||||||
|
ExternalEvidenceItem(
|
||||||
|
source_type="url",
|
||||||
|
source_label="中国天气网",
|
||||||
|
title="湛江天气预报,湛江7天天气预报,湛江15天天气预报,湛江天气查询",
|
||||||
|
target_url="https://www.weather.com.cn/weather/101281001.shtml",
|
||||||
|
snippet="湛江未来5天有阵雨转多云,最高温33度,最低温26度。",
|
||||||
|
retrieved_at="2026-04-16T00:30:00+08:00",
|
||||||
|
freshness_score=0.92,
|
||||||
|
authority_score=0.95,
|
||||||
|
grounded_fields=["forecast"],
|
||||||
|
raw_metadata={"provider": "live_search"},
|
||||||
|
)
|
||||||
|
],
|
||||||
|
meta={"provider": "live_search"},
|
||||||
|
)
|
||||||
|
|
||||||
|
execution = await ExternalQueryGateway(
|
||||||
|
planner=_Planner(),
|
||||||
|
registry=ExternalProviderRegistry(adapters=[_MCPAdapter(), _LiveSearchAdapter()]),
|
||||||
|
).execute(
|
||||||
|
question="湛江市未来五天的天气",
|
||||||
|
query_semantics={"query_profile": {"needs_freshness": True}},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert call_order == ["live_search"]
|
||||||
|
assert execution.adapter_name == "live_search"
|
||||||
|
assert execution.verifier["ok"] is True
|
||||||
|
|
@ -0,0 +1,120 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import json
|
||||||
|
|
||||||
|
import httpx
|
||||||
|
import pytest
|
||||||
|
|
||||||
|
from app.services.mcp_config.runtime import MicrosoftLearnMCPRuntime
|
||||||
|
|
||||||
|
|
||||||
|
def _sse(payload: dict) -> httpx.Response:
|
||||||
|
body = f"event: message\ndata: {json.dumps(payload, ensure_ascii=False)}\n\n"
|
||||||
|
return httpx.Response(
|
||||||
|
200,
|
||||||
|
headers={"content-type": "text/event-stream", "mcp-session-id": "test-session"},
|
||||||
|
text=body,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_microsoft_learn_runtime_probe_initializes_and_lists_tools() -> None:
|
||||||
|
calls: list[dict] = []
|
||||||
|
|
||||||
|
def _handler(request: httpx.Request) -> httpx.Response:
|
||||||
|
payload = json.loads(request.content.decode("utf-8"))
|
||||||
|
calls.append(payload)
|
||||||
|
if payload["method"] == "initialize":
|
||||||
|
return _sse(
|
||||||
|
{
|
||||||
|
"jsonrpc": "2.0",
|
||||||
|
"id": payload["id"],
|
||||||
|
"result": {
|
||||||
|
"serverInfo": {"name": "Microsoft Learn MCP Server", "version": "1.0.0"},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
if payload["method"] == "tools/list":
|
||||||
|
assert request.headers["mcp-session-id"] == "test-session"
|
||||||
|
return _sse(
|
||||||
|
{
|
||||||
|
"jsonrpc": "2.0",
|
||||||
|
"id": payload["id"],
|
||||||
|
"result": {
|
||||||
|
"tools": [
|
||||||
|
{"name": "microsoft_docs_search"},
|
||||||
|
{"name": "microsoft_docs_fetch"},
|
||||||
|
]
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
raise AssertionError(f"unexpected method: {payload['method']}")
|
||||||
|
|
||||||
|
runtime = MicrosoftLearnMCPRuntime(
|
||||||
|
endpoint_url="https://learn.microsoft.com/api/mcp",
|
||||||
|
client_factory=lambda: httpx.AsyncClient(transport=httpx.MockTransport(_handler)),
|
||||||
|
)
|
||||||
|
|
||||||
|
probe = await runtime.probe()
|
||||||
|
|
||||||
|
assert probe["status"] == "ready"
|
||||||
|
assert probe["tool_names"] == ["microsoft_docs_search", "microsoft_docs_fetch"]
|
||||||
|
assert [item["method"] for item in calls] == ["initialize", "tools/list"]
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_microsoft_learn_runtime_search_returns_external_evidence() -> None:
|
||||||
|
def _handler(request: httpx.Request) -> httpx.Response:
|
||||||
|
payload = json.loads(request.content.decode("utf-8"))
|
||||||
|
if payload["method"] == "initialize":
|
||||||
|
return _sse(
|
||||||
|
{
|
||||||
|
"jsonrpc": "2.0",
|
||||||
|
"id": payload["id"],
|
||||||
|
"result": {
|
||||||
|
"serverInfo": {"name": "Microsoft Learn MCP Server", "version": "1.0.0"},
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
if payload["method"] == "tools/call":
|
||||||
|
assert payload["params"]["name"] == "microsoft_docs_search"
|
||||||
|
assert payload["params"]["arguments"]["query"] == "azure openai quickstart"
|
||||||
|
return _sse(
|
||||||
|
{
|
||||||
|
"jsonrpc": "2.0",
|
||||||
|
"id": payload["id"],
|
||||||
|
"result": {
|
||||||
|
"content": [
|
||||||
|
{
|
||||||
|
"type": "text",
|
||||||
|
"text": json.dumps(
|
||||||
|
{
|
||||||
|
"results": [
|
||||||
|
{
|
||||||
|
"title": "Quickstart: Azure OpenAI",
|
||||||
|
"content": "Use Azure OpenAI with official quickstart guidance.",
|
||||||
|
"contentUrl": "https://learn.microsoft.com/azure/ai-foundry/openai/chatgpt-quickstart",
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
ensure_ascii=False,
|
||||||
|
),
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
}
|
||||||
|
)
|
||||||
|
raise AssertionError(f"unexpected method: {payload['method']}")
|
||||||
|
|
||||||
|
runtime = MicrosoftLearnMCPRuntime(
|
||||||
|
endpoint_url="https://learn.microsoft.com/api/mcp",
|
||||||
|
client_factory=lambda: httpx.AsyncClient(transport=httpx.MockTransport(_handler)),
|
||||||
|
)
|
||||||
|
|
||||||
|
evidence = await runtime.search(query="azure openai quickstart", result_limit=3)
|
||||||
|
|
||||||
|
assert len(evidence) == 1
|
||||||
|
assert evidence[0].source_type == "mcp_search"
|
||||||
|
assert evidence[0].title == "Quickstart: Azure OpenAI"
|
||||||
|
assert evidence[0].target_url == "https://learn.microsoft.com/azure/ai-foundry/openai/chatgpt-quickstart"
|
||||||
|
assert evidence[0].authority_score >= 0.9
|
||||||
|
|
@ -362,3 +362,107 @@ async def test_model_context_bundle_summary_uses_routing_names(db_env, monkeypat
|
||||||
first_row = summary["function_routing_template_matrix"][0]
|
first_row = summary["function_routing_template_matrix"][0]
|
||||||
assert "routing_snapshot" in first_row
|
assert "routing_snapshot" in first_row
|
||||||
assert "model_routing" not in first_row
|
assert "model_routing" not in first_row
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_get_embedding_async_falls_back_to_local_ollama_embed_when_active_engine_is_not_embedding_capable(
|
||||||
|
db_env,
|
||||||
|
monkeypatch,
|
||||||
|
) -> None:
|
||||||
|
db_session, _maker = db_env
|
||||||
|
captured: dict[str, object] = {}
|
||||||
|
|
||||||
|
class _FakeResponse:
|
||||||
|
def raise_for_status(self) -> None:
|
||||||
|
return None
|
||||||
|
|
||||||
|
def json(self) -> dict[str, object]:
|
||||||
|
return {"embeddings": [[0.11, 0.22, 0.33]]}
|
||||||
|
|
||||||
|
class _FakeAsyncClient:
|
||||||
|
def __init__(self, *args, **kwargs) -> None:
|
||||||
|
_ = args
|
||||||
|
captured["init_kwargs"] = dict(kwargs)
|
||||||
|
|
||||||
|
async def __aenter__(self):
|
||||||
|
return self
|
||||||
|
|
||||||
|
async def __aexit__(self, exc_type, exc, tb) -> None:
|
||||||
|
_ = (exc_type, exc, tb)
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def post(self, url: str, headers: dict[str, str], json: dict[str, object]):
|
||||||
|
captured["url"] = url
|
||||||
|
captured["headers"] = dict(headers)
|
||||||
|
captured["payload"] = dict(json)
|
||||||
|
return _FakeResponse()
|
||||||
|
|
||||||
|
async def _fake_resolve_active_engine(db, *, tenant_id: int):
|
||||||
|
_ = (db, tenant_id)
|
||||||
|
return {
|
||||||
|
"provider": "glm",
|
||||||
|
"model": "qwen-plus",
|
||||||
|
"base_url": "https://dashscope.aliyuncs.com/compatible-mode/v1",
|
||||||
|
"api_key": "glm-key",
|
||||||
|
}
|
||||||
|
|
||||||
|
monkeypatch.setattr(llm_service, "_resolve_active_engine", _fake_resolve_active_engine)
|
||||||
|
monkeypatch.setattr("app.services.llm_service.mixin_task_helpers.httpx.AsyncClient", _FakeAsyncClient)
|
||||||
|
monkeypatch.setenv("OLLAMA_HOST", "http://127.0.0.1:11434")
|
||||||
|
monkeypatch.setenv("OLLAMA_EMBED_MODEL", "nomic-embed-text:latest")
|
||||||
|
|
||||||
|
embedding = await llm_service.get_embedding_async(
|
||||||
|
"广州今天天气如何",
|
||||||
|
db=db_session,
|
||||||
|
tenant_id=1,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert embedding == [0.11, 0.22, 0.33]
|
||||||
|
assert captured["url"] == "http://127.0.0.1:11434/api/embed"
|
||||||
|
assert captured["payload"] == {"model": "nomic-embed-text:latest", "input": "广州今天天气如何"}
|
||||||
|
assert captured["init_kwargs"]["trust_env"] is False
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_get_embedding_async_returns_empty_when_all_real_backends_fail_instead_of_pseudo_vector(
|
||||||
|
db_env,
|
||||||
|
monkeypatch,
|
||||||
|
) -> None:
|
||||||
|
db_session, _maker = db_env
|
||||||
|
|
||||||
|
class _FailingAsyncClient:
|
||||||
|
def __init__(self, *args, **kwargs) -> None:
|
||||||
|
_ = (args, kwargs)
|
||||||
|
|
||||||
|
async def __aenter__(self):
|
||||||
|
return self
|
||||||
|
|
||||||
|
async def __aexit__(self, exc_type, exc, tb) -> None:
|
||||||
|
_ = (exc_type, exc, tb)
|
||||||
|
return None
|
||||||
|
|
||||||
|
async def post(self, url: str, headers: dict[str, str], json: dict[str, object]):
|
||||||
|
_ = (url, headers, json)
|
||||||
|
raise RuntimeError("embedding backend unavailable")
|
||||||
|
|
||||||
|
async def _fake_resolve_active_engine(db, *, tenant_id: int):
|
||||||
|
_ = (db, tenant_id)
|
||||||
|
return {
|
||||||
|
"provider": "glm",
|
||||||
|
"model": "qwen-plus",
|
||||||
|
"base_url": "https://dashscope.aliyuncs.com/compatible-mode/v1",
|
||||||
|
"api_key": "glm-key",
|
||||||
|
}
|
||||||
|
|
||||||
|
monkeypatch.setattr(llm_service, "_resolve_active_engine", _fake_resolve_active_engine)
|
||||||
|
monkeypatch.setattr("app.services.llm_service.mixin_task_helpers.httpx.AsyncClient", _FailingAsyncClient)
|
||||||
|
monkeypatch.setenv("OLLAMA_HOST", "http://127.0.0.1:11434")
|
||||||
|
monkeypatch.setenv("OLLAMA_EMBED_MODEL", "nomic-embed-text:latest")
|
||||||
|
|
||||||
|
embedding = await llm_service.get_embedding_async(
|
||||||
|
"今天荔湾区天气如何",
|
||||||
|
db=db_session,
|
||||||
|
tenant_id=1,
|
||||||
|
)
|
||||||
|
|
||||||
|
assert embedding == []
|
||||||
|
|
|
||||||
|
|
@ -17,6 +17,8 @@ from app.models.kg_evidence import KGEvidence
|
||||||
from app.models.kg_relation import KGRelation
|
from app.models.kg_relation import KGRelation
|
||||||
from app.models.kg_sync_run import KGSyncRun
|
from app.models.kg_sync_run import KGSyncRun
|
||||||
from app.models.project import Project
|
from app.models.project import Project
|
||||||
|
from app.models.semantic_fact import SemanticFact
|
||||||
|
from app.models.semantic_fact_extraction_run import SemanticFactExtractionRun
|
||||||
from app.models.signal import Signal
|
from app.models.signal import Signal
|
||||||
from app.models.user import User
|
from app.models.user import User
|
||||||
from app.models.work_item import WorkItem
|
from app.models.work_item import WorkItem
|
||||||
|
|
@ -42,6 +44,8 @@ async def db_session():
|
||||||
IngestOffset.__table__,
|
IngestOffset.__table__,
|
||||||
WorkItem.__table__,
|
WorkItem.__table__,
|
||||||
AIOrchestrationRun.__table__,
|
AIOrchestrationRun.__table__,
|
||||||
|
SemanticFact.__table__,
|
||||||
|
SemanticFactExtractionRun.__table__,
|
||||||
KGEntity.__table__,
|
KGEntity.__table__,
|
||||||
KGRelation.__table__,
|
KGRelation.__table__,
|
||||||
KGEvidence.__table__,
|
KGEvidence.__table__,
|
||||||
|
|
@ -191,6 +195,77 @@ async def test_enqueue_work_item_delta_events_reads_feishu_work_items_into_kg(db
|
||||||
assert relation is not None
|
assert relation is not None
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_process_work_item_event_enqueues_semantic_fact_extract_for_project(db_session) -> None:
|
||||||
|
project = Project(
|
||||||
|
tenant_id=1,
|
||||||
|
name="alpha",
|
||||||
|
title="Alpha 项目",
|
||||||
|
status="active",
|
||||||
|
owner_id=None,
|
||||||
|
assigned_role="backend",
|
||||||
|
)
|
||||||
|
seed_ingest_event = IngestEvent(
|
||||||
|
tenant_id=1,
|
||||||
|
channel="seed",
|
||||||
|
event_key="seed-semantic-fact",
|
||||||
|
status="done",
|
||||||
|
payload={},
|
||||||
|
)
|
||||||
|
db_session.add_all([project, seed_ingest_event])
|
||||||
|
await db_session.flush()
|
||||||
|
|
||||||
|
target_work_item = WorkItem(
|
||||||
|
tenant_id=1,
|
||||||
|
ingest_event_id=int(seed_ingest_event.id),
|
||||||
|
source_system="feishu",
|
||||||
|
source_message_id="om_work_item_semantic_fact",
|
||||||
|
state="waiting_manual",
|
||||||
|
intent="project.create",
|
||||||
|
project_id=int(project.id),
|
||||||
|
execution_result={"status": "pending_manual"},
|
||||||
|
review_summary={"verified": False, "issues": ["pending_manual"]},
|
||||||
|
)
|
||||||
|
db_session.add(target_work_item)
|
||||||
|
await db_session.flush()
|
||||||
|
|
||||||
|
run = AIOrchestrationRun(
|
||||||
|
tenant_id=1,
|
||||||
|
trace_id="trace-semantic-fact-1",
|
||||||
|
run_type="ingress",
|
||||||
|
status="completed",
|
||||||
|
source_system="feishu",
|
||||||
|
source_message_id="om_work_item_semantic_fact",
|
||||||
|
sender="测试用户",
|
||||||
|
channel_ref="chat-alpha",
|
||||||
|
message_text="请创建项目 Alpha",
|
||||||
|
work_item_id=int(target_work_item.id),
|
||||||
|
project_id=int(project.id),
|
||||||
|
)
|
||||||
|
db_session.add(run)
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
service = ProjectKnowledgeGraphService(db_session, tenant_id=1)
|
||||||
|
await service.enqueue_work_item_delta_events(limit=20)
|
||||||
|
process_result = await service.process_pending_events(max_batch=20)
|
||||||
|
|
||||||
|
semantic_event = (
|
||||||
|
await db_session.execute(
|
||||||
|
select(IngestEvent).where(
|
||||||
|
IngestEvent.tenant_id == 1,
|
||||||
|
IngestEvent.channel == "semantic_fact_extract",
|
||||||
|
IngestEvent.event_key == f"project:{int(project.id)}",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one_or_none()
|
||||||
|
|
||||||
|
assert process_result["done"] >= 1
|
||||||
|
assert semantic_event is not None
|
||||||
|
assert semantic_event.payload["entity_type"] == "project"
|
||||||
|
assert semantic_event.payload["entity_id"] == int(project.id)
|
||||||
|
assert semantic_event.payload["trigger_type"] == "work_item_delta"
|
||||||
|
|
||||||
|
|
||||||
@pytest.mark.asyncio
|
@pytest.mark.asyncio
|
||||||
async def test_enqueue_feishu_delta_events_is_compat_alias_for_work_item_delta(db_session) -> None:
|
async def test_enqueue_feishu_delta_events_is_compat_alias_for_work_item_delta(db_session) -> None:
|
||||||
project = Project(
|
project = Project(
|
||||||
|
|
|
||||||
|
|
@ -0,0 +1,178 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from types import SimpleNamespace
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
import pytest_asyncio
|
||||||
|
from fastapi import FastAPI
|
||||||
|
from httpx import ASGITransport, AsyncClient
|
||||||
|
from sqlalchemy import JSON, select
|
||||||
|
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
||||||
|
|
||||||
|
from app.api.semantic_facts_api import routes as semantic_fact_routes
|
||||||
|
from app.dependencies.auth import get_current_user, get_tenant_id
|
||||||
|
from app.core.database import get_db
|
||||||
|
from app.models.base import Base
|
||||||
|
from app.models.ingest_event import IngestEvent
|
||||||
|
from app.models.knowledge_base_doc import KnowledgeBaseDoc
|
||||||
|
from app.models.project import Project
|
||||||
|
from app.models.semantic_fact import SemanticFact
|
||||||
|
from app.models.semantic_fact_extraction_run import SemanticFactExtractionRun
|
||||||
|
from app.models.signal import Signal
|
||||||
|
from app.models.task import Task
|
||||||
|
from app.models.truth_node import TruthNode
|
||||||
|
from app.models.user import User
|
||||||
|
|
||||||
|
|
||||||
|
def _use_sqlite_json_types(*tables) -> None:
|
||||||
|
for table in tables:
|
||||||
|
for column in table.columns:
|
||||||
|
if column.type.__class__.__name__ in {"JSONB", "ARRAY", "Vector"}:
|
||||||
|
column.type = JSON()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest_asyncio.fixture
|
||||||
|
async def db_session():
|
||||||
|
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
|
||||||
|
tables = [
|
||||||
|
User.__table__,
|
||||||
|
Project.__table__,
|
||||||
|
Signal.__table__,
|
||||||
|
TruthNode.__table__,
|
||||||
|
Task.__table__,
|
||||||
|
KnowledgeBaseDoc.__table__,
|
||||||
|
SemanticFact.__table__,
|
||||||
|
SemanticFactExtractionRun.__table__,
|
||||||
|
IngestEvent.__table__,
|
||||||
|
]
|
||||||
|
_use_sqlite_json_types(*tables)
|
||||||
|
async with engine.begin() as conn:
|
||||||
|
await conn.run_sync(lambda sync_conn: Base.metadata.create_all(sync_conn, tables=tables))
|
||||||
|
maker = async_sessionmaker(engine, expire_on_commit=False)
|
||||||
|
async with maker() as session:
|
||||||
|
yield session
|
||||||
|
await engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
def _build_app(db_session, *, role: str) -> FastAPI:
|
||||||
|
app = FastAPI()
|
||||||
|
app.include_router(semantic_fact_routes.router)
|
||||||
|
|
||||||
|
async def _fake_get_db():
|
||||||
|
yield db_session
|
||||||
|
|
||||||
|
app.dependency_overrides[get_db] = _fake_get_db
|
||||||
|
app.dependency_overrides[get_tenant_id] = lambda: 1
|
||||||
|
app.dependency_overrides[get_current_user] = lambda: SimpleNamespace(
|
||||||
|
id=7,
|
||||||
|
tenant_id=1,
|
||||||
|
role=role,
|
||||||
|
username="tester",
|
||||||
|
email="tester@example.com",
|
||||||
|
name="Tester",
|
||||||
|
)
|
||||||
|
return app
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_semantic_fact_admin_api_requires_manager_role(db_session) -> None:
|
||||||
|
app = _build_app(db_session, role="AI")
|
||||||
|
transport = ASGITransport(app=app)
|
||||||
|
|
||||||
|
async with AsyncClient(transport=transport, base_url="http://test") as client:
|
||||||
|
response = await client.get("/semantic-facts/admin/overview")
|
||||||
|
|
||||||
|
assert response.status_code == 403
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_semantic_fact_admin_api_exposes_overview_objects_detail_and_actions(db_session) -> None:
|
||||||
|
project = Project(
|
||||||
|
tenant_id=1,
|
||||||
|
name="invoice-helper",
|
||||||
|
title="开票助手",
|
||||||
|
description="自动生成发票",
|
||||||
|
owner_id=None,
|
||||||
|
status="active",
|
||||||
|
assigned_role="backend",
|
||||||
|
)
|
||||||
|
db_session.add(project)
|
||||||
|
await db_session.flush()
|
||||||
|
db_session.add(
|
||||||
|
SemanticFact(
|
||||||
|
tenant_id=1,
|
||||||
|
entity_type="project",
|
||||||
|
entity_id=int(project.id),
|
||||||
|
fact_namespace="alias",
|
||||||
|
fact_key="aliases",
|
||||||
|
fact_value_text="开票系统",
|
||||||
|
fact_value_norm="invoice_system",
|
||||||
|
value_type="keyword",
|
||||||
|
confidence=0.95,
|
||||||
|
evidence_source_type="project",
|
||||||
|
evidence_source_id=int(project.id),
|
||||||
|
evidence_excerpt="自动生成发票",
|
||||||
|
extract_version="semantic-facts-v1",
|
||||||
|
freshness_at=None,
|
||||||
|
is_active=True,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
db_session.add(
|
||||||
|
SemanticFactExtractionRun(
|
||||||
|
tenant_id=1,
|
||||||
|
entity_type="project",
|
||||||
|
entity_id=int(project.id),
|
||||||
|
trigger_type="manual_retry",
|
||||||
|
status="completed",
|
||||||
|
input_fingerprint="fp-1",
|
||||||
|
facts_written=1,
|
||||||
|
failure_reason=None,
|
||||||
|
extract_version="semantic-facts-v1",
|
||||||
|
started_at="2026-04-15T10:00:00+00:00",
|
||||||
|
finished_at="2026-04-15T10:00:02+00:00",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
app = _build_app(db_session, role="ADMIN")
|
||||||
|
transport = ASGITransport(app=app)
|
||||||
|
|
||||||
|
async with AsyncClient(transport=transport, base_url="http://test") as client:
|
||||||
|
overview_response = await client.get("/semantic-facts/admin/overview")
|
||||||
|
objects_response = await client.get("/semantic-facts/admin/objects", params={"entity_type": "project", "q": "开票"})
|
||||||
|
detail_response = await client.get(f"/semantic-facts/admin/objects/project/{int(project.id)}")
|
||||||
|
retry_response = await client.post(f"/semantic-facts/admin/objects/project/{int(project.id)}/reextract")
|
||||||
|
retry_event = (
|
||||||
|
await db_session.execute(
|
||||||
|
select(IngestEvent).where(
|
||||||
|
IngestEvent.tenant_id == 1,
|
||||||
|
IngestEvent.channel == "semantic_fact_extract",
|
||||||
|
IngestEvent.event_key == f"project:{int(project.id)}",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one()
|
||||||
|
retry_trigger_type = retry_event.payload["trigger_type"]
|
||||||
|
backfill_response = await client.post(
|
||||||
|
"/semantic-facts/admin/backfill",
|
||||||
|
json={"entity_types": ["project"], "only_missing": False},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert overview_response.status_code == 200
|
||||||
|
assert overview_response.json()["code"] == 0
|
||||||
|
assert overview_response.json()["data"]["queue"]["pending"] >= 1
|
||||||
|
|
||||||
|
assert objects_response.status_code == 200
|
||||||
|
assert objects_response.json()["code"] == 0
|
||||||
|
assert objects_response.json()["data"]["items"][0]["display_name"] == "开票助手"
|
||||||
|
|
||||||
|
assert detail_response.status_code == 200
|
||||||
|
assert detail_response.json()["code"] == 0
|
||||||
|
assert detail_response.json()["data"]["facts"][0]["fact_value_text"] == "开票系统"
|
||||||
|
|
||||||
|
assert retry_response.status_code == 200
|
||||||
|
assert retry_response.json()["code"] == 0
|
||||||
|
assert retry_trigger_type == "manual_retry"
|
||||||
|
|
||||||
|
assert backfill_response.status_code == 200
|
||||||
|
assert backfill_response.json()["code"] == 0
|
||||||
|
assert backfill_response.json()["data"]["enqueued"] >= 1
|
||||||
|
|
@ -0,0 +1,250 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
import pytest_asyncio
|
||||||
|
from sqlalchemy import JSON, select
|
||||||
|
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
||||||
|
|
||||||
|
from app.models.base import Base
|
||||||
|
from app.models.ingest_event import IngestEvent
|
||||||
|
from app.models.knowledge_base_doc import KnowledgeBaseDoc
|
||||||
|
from app.models.project import Project
|
||||||
|
from app.models.signal import Signal
|
||||||
|
from app.models.task import Task
|
||||||
|
from app.models.truth_node import TruthNode
|
||||||
|
from app.models.user import User
|
||||||
|
from app.services.semantic_fact_service.hooks import register_semantic_fact_hooks
|
||||||
|
|
||||||
|
|
||||||
|
def _use_sqlite_json_types(*tables) -> None:
|
||||||
|
for table in tables:
|
||||||
|
for column in table.columns:
|
||||||
|
if column.type.__class__.__name__ in {"JSONB", "ARRAY", "Vector"}:
|
||||||
|
column.type = JSON()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest_asyncio.fixture
|
||||||
|
async def db_session():
|
||||||
|
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
|
||||||
|
tables = [
|
||||||
|
User.__table__,
|
||||||
|
Project.__table__,
|
||||||
|
Signal.__table__,
|
||||||
|
TruthNode.__table__,
|
||||||
|
Task.__table__,
|
||||||
|
KnowledgeBaseDoc.__table__,
|
||||||
|
IngestEvent.__table__,
|
||||||
|
]
|
||||||
|
_use_sqlite_json_types(*tables)
|
||||||
|
async with engine.begin() as conn:
|
||||||
|
await conn.run_sync(lambda sync_conn: Base.metadata.create_all(sync_conn, tables=tables))
|
||||||
|
maker = async_sessionmaker(engine, expire_on_commit=False)
|
||||||
|
async with maker() as session:
|
||||||
|
yield session
|
||||||
|
await engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_semantic_fact_hooks_enqueue_extract_events_for_supported_entities(db_session) -> None:
|
||||||
|
register_semantic_fact_hooks()
|
||||||
|
|
||||||
|
user = User(
|
||||||
|
tenant_id=1,
|
||||||
|
username="semantic_admin",
|
||||||
|
email="semantic_admin@example.com",
|
||||||
|
password_hash="hash",
|
||||||
|
name="管理员",
|
||||||
|
role="ADMIN",
|
||||||
|
status="active",
|
||||||
|
is_active=True,
|
||||||
|
)
|
||||||
|
db_session.add(user)
|
||||||
|
await db_session.flush()
|
||||||
|
|
||||||
|
project = Project(
|
||||||
|
tenant_id=1,
|
||||||
|
name="alpha",
|
||||||
|
title="Alpha 项目",
|
||||||
|
description="项目描述",
|
||||||
|
owner_id=int(user.id),
|
||||||
|
status="active",
|
||||||
|
assigned_role="backend",
|
||||||
|
)
|
||||||
|
signal = Signal(
|
||||||
|
tenant_id=1,
|
||||||
|
title="信号标题",
|
||||||
|
content="信号内容",
|
||||||
|
status="待处理",
|
||||||
|
processed=False,
|
||||||
|
priority="high",
|
||||||
|
)
|
||||||
|
doc = KnowledgeBaseDoc(
|
||||||
|
tenant_id=1,
|
||||||
|
title="知识文档",
|
||||||
|
file_type="md",
|
||||||
|
source_type="file",
|
||||||
|
status="complete",
|
||||||
|
uploaded_by=int(user.id),
|
||||||
|
)
|
||||||
|
db_session.add_all([project, signal, doc])
|
||||||
|
await db_session.flush()
|
||||||
|
|
||||||
|
charter = TruthNode(
|
||||||
|
tenant_id=1,
|
||||||
|
node_type="charter",
|
||||||
|
parent_id=None,
|
||||||
|
project_id=int(project.id),
|
||||||
|
signal_id=None,
|
||||||
|
created_by=int(user.id),
|
||||||
|
title="开票系统需求",
|
||||||
|
content={"summary": "建设开票系统"},
|
||||||
|
version=1,
|
||||||
|
status="draft",
|
||||||
|
)
|
||||||
|
release = TruthNode(
|
||||||
|
tenant_id=1,
|
||||||
|
node_type="release",
|
||||||
|
parent_id=None,
|
||||||
|
project_id=int(project.id),
|
||||||
|
signal_id=None,
|
||||||
|
created_by=int(user.id),
|
||||||
|
title="发布记录",
|
||||||
|
content={"summary": "上线版本"},
|
||||||
|
version=1,
|
||||||
|
status="pending",
|
||||||
|
)
|
||||||
|
review = TruthNode(
|
||||||
|
tenant_id=1,
|
||||||
|
node_type="review",
|
||||||
|
parent_id=None,
|
||||||
|
project_id=int(project.id),
|
||||||
|
signal_id=None,
|
||||||
|
created_by=int(user.id),
|
||||||
|
title="阶段复盘",
|
||||||
|
content={"summary": "复盘内容"},
|
||||||
|
version=1,
|
||||||
|
status="draft",
|
||||||
|
)
|
||||||
|
db_session.add_all([charter, release, review])
|
||||||
|
await db_session.flush()
|
||||||
|
|
||||||
|
task = Task(
|
||||||
|
tenant_id=1,
|
||||||
|
truth_node_id=int(charter.id),
|
||||||
|
project_id=int(project.id),
|
||||||
|
assignee_id=int(user.id),
|
||||||
|
ai_recommended_assignees=None,
|
||||||
|
ai_breakdown_payload=None,
|
||||||
|
manual_overrides=None,
|
||||||
|
title="实现开票接口",
|
||||||
|
description="任务描述",
|
||||||
|
required_skill_tag_ids=[],
|
||||||
|
status="todo",
|
||||||
|
priority="high",
|
||||||
|
estimated_hours=None,
|
||||||
|
actual_hours=None,
|
||||||
|
quality_score=None,
|
||||||
|
due_date=None,
|
||||||
|
completed_at=None,
|
||||||
|
)
|
||||||
|
db_session.add(task)
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
rows = (
|
||||||
|
await db_session.execute(
|
||||||
|
select(IngestEvent).where(
|
||||||
|
IngestEvent.tenant_id == 1,
|
||||||
|
IngestEvent.channel == "semantic_fact_extract",
|
||||||
|
).order_by(IngestEvent.event_key.asc())
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
payload_by_key = {str(row.event_key): row.payload for row in rows}
|
||||||
|
|
||||||
|
assert payload_by_key["document:%s" % int(doc.id)]["operation"] == "extract"
|
||||||
|
assert payload_by_key["project:%s" % int(project.id)]["operation"] == "extract"
|
||||||
|
assert payload_by_key["release:%s" % int(release.id)]["entity_type"] == "release"
|
||||||
|
assert payload_by_key["requirement:%s" % int(charter.id)]["entity_type"] == "requirement"
|
||||||
|
assert payload_by_key["review:%s" % int(review.id)]["entity_type"] == "review"
|
||||||
|
assert payload_by_key["signal:%s" % int(signal.id)]["entity_type"] == "signal"
|
||||||
|
assert payload_by_key["task:%s" % int(task.id)]["entity_type"] == "task"
|
||||||
|
assert payload_by_key["user:%s" % int(user.id)]["entity_type"] == "user"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_semantic_fact_hooks_switch_to_deactivate_for_archive_disable_and_delete(db_session) -> None:
|
||||||
|
register_semantic_fact_hooks()
|
||||||
|
|
||||||
|
user = User(
|
||||||
|
tenant_id=1,
|
||||||
|
username="semantic_admin_2",
|
||||||
|
email="semantic_admin_2@example.com",
|
||||||
|
password_hash="hash",
|
||||||
|
name="管理员2",
|
||||||
|
role="ADMIN",
|
||||||
|
status="active",
|
||||||
|
is_active=True,
|
||||||
|
)
|
||||||
|
project = Project(
|
||||||
|
tenant_id=1,
|
||||||
|
name="alpha",
|
||||||
|
title="Alpha 项目",
|
||||||
|
description="项目描述",
|
||||||
|
owner_id=None,
|
||||||
|
status="active",
|
||||||
|
assigned_role="backend",
|
||||||
|
)
|
||||||
|
signal = Signal(
|
||||||
|
tenant_id=1,
|
||||||
|
title="信号标题",
|
||||||
|
content="信号内容",
|
||||||
|
status="待处理",
|
||||||
|
processed=False,
|
||||||
|
priority="high",
|
||||||
|
)
|
||||||
|
doc = KnowledgeBaseDoc(
|
||||||
|
tenant_id=1,
|
||||||
|
title="知识文档",
|
||||||
|
file_type="md",
|
||||||
|
source_type="file",
|
||||||
|
status="complete",
|
||||||
|
uploaded_by=None,
|
||||||
|
)
|
||||||
|
db_session.add_all([user, project, signal, doc])
|
||||||
|
await db_session.flush()
|
||||||
|
|
||||||
|
review = TruthNode(
|
||||||
|
tenant_id=1,
|
||||||
|
node_type="review",
|
||||||
|
parent_id=None,
|
||||||
|
project_id=int(project.id),
|
||||||
|
signal_id=None,
|
||||||
|
created_by=int(user.id),
|
||||||
|
title="阶段复盘",
|
||||||
|
content={"summary": "复盘内容"},
|
||||||
|
version=1,
|
||||||
|
status="draft",
|
||||||
|
)
|
||||||
|
db_session.add(review)
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
user.is_active = False
|
||||||
|
user.status = "offline"
|
||||||
|
signal.status = "已关闭"
|
||||||
|
review.status = "archived"
|
||||||
|
await db_session.delete(doc)
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
rows = (
|
||||||
|
await db_session.execute(
|
||||||
|
select(IngestEvent).where(
|
||||||
|
IngestEvent.tenant_id == 1,
|
||||||
|
IngestEvent.channel == "semantic_fact_extract",
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
payload_by_key = {str(row.event_key): row.payload for row in rows}
|
||||||
|
|
||||||
|
assert payload_by_key["document:%s" % int(doc.id)]["operation"] == "deactivate"
|
||||||
|
assert payload_by_key["review:%s" % int(review.id)]["operation"] == "deactivate"
|
||||||
|
assert payload_by_key["signal:%s" % int(signal.id)]["operation"] == "deactivate"
|
||||||
|
assert payload_by_key["user:%s" % int(user.id)]["operation"] == "deactivate"
|
||||||
|
|
@ -0,0 +1,575 @@
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import pytest
|
||||||
|
import pytest_asyncio
|
||||||
|
from sqlalchemy import JSON, select
|
||||||
|
from sqlalchemy.ext.asyncio import async_sessionmaker, create_async_engine
|
||||||
|
|
||||||
|
from app.models.base import Base
|
||||||
|
from app.models.knowledge_base_chunk import KnowledgeBaseChunk
|
||||||
|
from app.models.knowledge_base_doc import KnowledgeBaseDoc
|
||||||
|
from app.models.project import Project
|
||||||
|
from app.models.semantic_fact import SemanticFact
|
||||||
|
from app.models.semantic_fact_extraction_run import SemanticFactExtractionRun
|
||||||
|
from app.models.signal import Signal
|
||||||
|
from app.models.task import Task
|
||||||
|
from app.models.truth_node import TruthNode
|
||||||
|
from app.models.user import User
|
||||||
|
from app.services.adaptive_rag_service.mixin_answer_payload import AdaptiveRAGAnswerPayloadMixin
|
||||||
|
from app.services.adaptive_rag_service.mixin_fast_answer_project_main import AdaptiveRAGFastAnswerProjectMainMixin
|
||||||
|
from app.services.adaptive_rag_service.mixin_fast_answer_project_utils import AdaptiveRAGFastAnswerProjectUtilsMixin
|
||||||
|
from app.services.adaptive_rag_service.mixin_retrieval_router_db import AdaptiveRAGRetrievalRouterDBMixin
|
||||||
|
from app.services.llm_service import StructuredExtractionResult
|
||||||
|
from app.services.semantic_fact_service.service import SemanticFactService
|
||||||
|
|
||||||
|
|
||||||
|
def _use_sqlite_json_types(*tables) -> None:
|
||||||
|
for table in tables:
|
||||||
|
for column in table.columns:
|
||||||
|
if column.type.__class__.__name__ in {"JSONB", "ARRAY", "Vector"}:
|
||||||
|
column.type = JSON()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest_asyncio.fixture
|
||||||
|
async def db_session():
|
||||||
|
engine = create_async_engine("sqlite+aiosqlite:///:memory:")
|
||||||
|
tables = [
|
||||||
|
User.__table__,
|
||||||
|
Project.__table__,
|
||||||
|
Signal.__table__,
|
||||||
|
TruthNode.__table__,
|
||||||
|
Task.__table__,
|
||||||
|
KnowledgeBaseDoc.__table__,
|
||||||
|
KnowledgeBaseChunk.__table__,
|
||||||
|
SemanticFact.__table__,
|
||||||
|
SemanticFactExtractionRun.__table__,
|
||||||
|
]
|
||||||
|
_use_sqlite_json_types(*tables)
|
||||||
|
async with engine.begin() as conn:
|
||||||
|
await conn.run_sync(lambda sync_conn: Base.metadata.create_all(sync_conn, tables=tables))
|
||||||
|
maker = async_sessionmaker(engine, expire_on_commit=False)
|
||||||
|
async with maker() as session:
|
||||||
|
yield session
|
||||||
|
await engine.dispose()
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_semantic_fact_service_upserts_and_queries_project_fact(db_session) -> None:
|
||||||
|
project = Project(
|
||||||
|
tenant_id=1,
|
||||||
|
name="a-share-analyzer",
|
||||||
|
title="A股趋势打分工具",
|
||||||
|
description="面向证券投研团队,聚焦 A 股公告解析。",
|
||||||
|
owner_id=None,
|
||||||
|
status="active",
|
||||||
|
assigned_role="backend",
|
||||||
|
)
|
||||||
|
db_session.add(project)
|
||||||
|
await db_session.flush()
|
||||||
|
|
||||||
|
service = SemanticFactService(db_session, tenant_id=1)
|
||||||
|
await service.upsert_fact(
|
||||||
|
entity_type="project",
|
||||||
|
entity_id=int(project.id),
|
||||||
|
fact_namespace="theme",
|
||||||
|
fact_key="market_theme",
|
||||||
|
fact_value_text="A股",
|
||||||
|
fact_value_norm="a_share",
|
||||||
|
value_type="keyword",
|
||||||
|
confidence=0.93,
|
||||||
|
evidence_source_type="project",
|
||||||
|
evidence_source_id=int(project.id),
|
||||||
|
evidence_excerpt="聚焦 A 股公告解析",
|
||||||
|
extract_version="semantic-facts-v1",
|
||||||
|
freshness_at=None,
|
||||||
|
)
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
rows = await service.query_facts(
|
||||||
|
entity_type="project",
|
||||||
|
query_type="existence",
|
||||||
|
filters=[{"fact_key": "market_theme", "value": "A股", "normalized_value": "a_share"}],
|
||||||
|
)
|
||||||
|
|
||||||
|
assert len(rows) == 1
|
||||||
|
assert rows[0].fact_key == "market_theme"
|
||||||
|
assert rows[0].fact_value_norm == "a_share"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_semantic_fact_service_extracts_project_facts_from_llm_payload(monkeypatch, db_session) -> None:
|
||||||
|
project = Project(
|
||||||
|
tenant_id=1,
|
||||||
|
name="invoice-helper",
|
||||||
|
title="开票助手",
|
||||||
|
description="面向财务运营,自动生成增值税发票草稿。",
|
||||||
|
owner_id=None,
|
||||||
|
status="active",
|
||||||
|
assigned_role="backend",
|
||||||
|
)
|
||||||
|
db_session.add(project)
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
async def _fake_generate_structured_object(prompt: str, **kwargs):
|
||||||
|
_ = (prompt, kwargs)
|
||||||
|
return StructuredExtractionResult(
|
||||||
|
payload={
|
||||||
|
"facts": [
|
||||||
|
{
|
||||||
|
"fact_namespace": "domain",
|
||||||
|
"fact_key": "business_domain",
|
||||||
|
"fact_value_text": "开票系统",
|
||||||
|
"fact_value_norm": "invoicing",
|
||||||
|
"value_type": "keyword",
|
||||||
|
"confidence": 0.94,
|
||||||
|
"evidence_excerpt": "自动生成增值税发票草稿",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"fact_namespace": "alias",
|
||||||
|
"fact_key": "aliases",
|
||||||
|
"fact_value_text": "开票系统",
|
||||||
|
"fact_value_norm": "invoice_system",
|
||||||
|
"value_type": "keyword",
|
||||||
|
"confidence": 0.9,
|
||||||
|
"evidence_excerpt": "开票助手",
|
||||||
|
},
|
||||||
|
]
|
||||||
|
},
|
||||||
|
source_field="payload",
|
||||||
|
provider="test",
|
||||||
|
model="test-model",
|
||||||
|
validation="valid",
|
||||||
|
)
|
||||||
|
|
||||||
|
monkeypatch.setattr(
|
||||||
|
"app.services.llm_service.llm_service.generate_structured_object",
|
||||||
|
_fake_generate_structured_object,
|
||||||
|
)
|
||||||
|
|
||||||
|
service = SemanticFactService(db_session, tenant_id=1)
|
||||||
|
run = await service.extract_project_facts(project_id=int(project.id), trigger_type="manual_retry")
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
facts = (
|
||||||
|
await db_session.execute(
|
||||||
|
select(SemanticFact).where(
|
||||||
|
SemanticFact.tenant_id == 1,
|
||||||
|
SemanticFact.entity_type == "project",
|
||||||
|
SemanticFact.entity_id == int(project.id),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
run_row = (
|
||||||
|
await db_session.execute(
|
||||||
|
select(SemanticFactExtractionRun).where(
|
||||||
|
SemanticFactExtractionRun.tenant_id == 1,
|
||||||
|
SemanticFactExtractionRun.id == int(run.id),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalar_one()
|
||||||
|
|
||||||
|
assert len(facts) == 2
|
||||||
|
assert {row.fact_key for row in facts} == {"business_domain", "aliases"}
|
||||||
|
assert str(run_row.status) == "completed"
|
||||||
|
assert int(run_row.facts_written or 0) == 2
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_semantic_fact_service_extract_entity_facts_replaces_active_rows_for_requirement(monkeypatch, db_session) -> None:
|
||||||
|
user = User(
|
||||||
|
tenant_id=1,
|
||||||
|
username="semantic_owner",
|
||||||
|
email="semantic_owner@example.com",
|
||||||
|
password_hash="hash",
|
||||||
|
name="语义负责人",
|
||||||
|
role="ADMIN",
|
||||||
|
status="active",
|
||||||
|
is_active=True,
|
||||||
|
)
|
||||||
|
db_session.add(user)
|
||||||
|
await db_session.flush()
|
||||||
|
|
||||||
|
charter = TruthNode(
|
||||||
|
tenant_id=1,
|
||||||
|
node_type="charter",
|
||||||
|
parent_id=None,
|
||||||
|
project_id=None,
|
||||||
|
signal_id=None,
|
||||||
|
created_by=int(user.id),
|
||||||
|
title="开票系统需求",
|
||||||
|
content={"summary": "为财务团队建设开票平台"},
|
||||||
|
version=1,
|
||||||
|
status="draft",
|
||||||
|
)
|
||||||
|
db_session.add(charter)
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
payloads = [
|
||||||
|
{
|
||||||
|
"facts": [
|
||||||
|
{
|
||||||
|
"fact_namespace": "alias",
|
||||||
|
"fact_key": "aliases",
|
||||||
|
"fact_value_text": "开票系统",
|
||||||
|
"fact_value_norm": "invoice_system",
|
||||||
|
"value_type": "keyword",
|
||||||
|
"confidence": 0.93,
|
||||||
|
"evidence_excerpt": "建设开票平台",
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"facts": [
|
||||||
|
{
|
||||||
|
"fact_namespace": "alias",
|
||||||
|
"fact_key": "aliases",
|
||||||
|
"fact_value_text": "财务开票平台",
|
||||||
|
"fact_value_norm": "finance_invoice_platform",
|
||||||
|
"value_type": "keyword",
|
||||||
|
"confidence": 0.95,
|
||||||
|
"evidence_excerpt": "建设开票平台",
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
]
|
||||||
|
|
||||||
|
async def _fake_generate_structured_object(prompt: str, **kwargs):
|
||||||
|
_ = (prompt, kwargs)
|
||||||
|
return StructuredExtractionResult(
|
||||||
|
payload=payloads.pop(0),
|
||||||
|
source_field="payload",
|
||||||
|
provider="test",
|
||||||
|
model="test-model",
|
||||||
|
validation="valid",
|
||||||
|
)
|
||||||
|
|
||||||
|
monkeypatch.setattr(
|
||||||
|
"app.services.llm_service.llm_service.generate_structured_object",
|
||||||
|
_fake_generate_structured_object,
|
||||||
|
)
|
||||||
|
|
||||||
|
service = SemanticFactService(db_session, tenant_id=1)
|
||||||
|
first_run = await service.extract_entity_facts(
|
||||||
|
entity_type="requirement",
|
||||||
|
entity_id=int(charter.id),
|
||||||
|
trigger_type="manual_retry",
|
||||||
|
)
|
||||||
|
second_run = await service.extract_entity_facts(
|
||||||
|
entity_type="requirement",
|
||||||
|
entity_id=int(charter.id),
|
||||||
|
trigger_type="manual_retry",
|
||||||
|
)
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
all_rows = (
|
||||||
|
await db_session.execute(
|
||||||
|
select(SemanticFact).where(
|
||||||
|
SemanticFact.tenant_id == 1,
|
||||||
|
SemanticFact.entity_type == "requirement",
|
||||||
|
SemanticFact.entity_id == int(charter.id),
|
||||||
|
).order_by(SemanticFact.id.asc())
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
active_rows = [row for row in all_rows if row.is_active]
|
||||||
|
|
||||||
|
assert str(first_run.status) == "completed"
|
||||||
|
assert str(second_run.status) == "completed"
|
||||||
|
assert len(all_rows) == 2
|
||||||
|
assert len(active_rows) == 1
|
||||||
|
assert active_rows[0].fact_value_text == "财务开票平台"
|
||||||
|
assert all_rows[0].is_active is False
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_semantic_fact_service_deactivate_entity_facts_marks_rows_inactive(db_session) -> None:
|
||||||
|
project = Project(
|
||||||
|
tenant_id=1,
|
||||||
|
name="market-tool",
|
||||||
|
title="趋势分析工具",
|
||||||
|
description="这是一个证券行业分析工具。",
|
||||||
|
owner_id=None,
|
||||||
|
status="active",
|
||||||
|
assigned_role="backend",
|
||||||
|
)
|
||||||
|
db_session.add(project)
|
||||||
|
await db_session.flush()
|
||||||
|
|
||||||
|
service = SemanticFactService(db_session, tenant_id=1)
|
||||||
|
await service.upsert_fact(
|
||||||
|
entity_type="project",
|
||||||
|
entity_id=int(project.id),
|
||||||
|
fact_namespace="theme",
|
||||||
|
fact_key="market_theme",
|
||||||
|
fact_value_text="A股",
|
||||||
|
fact_value_norm="a_share",
|
||||||
|
value_type="keyword",
|
||||||
|
confidence=0.95,
|
||||||
|
evidence_source_type="project",
|
||||||
|
evidence_source_id=int(project.id),
|
||||||
|
evidence_excerpt="A股",
|
||||||
|
extract_version="semantic-facts-v1",
|
||||||
|
freshness_at=None,
|
||||||
|
)
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
deactivated = await service.deactivate_entity_facts(entity_type="project", entity_id=int(project.id))
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
rows = (
|
||||||
|
await db_session.execute(
|
||||||
|
select(SemanticFact).where(
|
||||||
|
SemanticFact.tenant_id == 1,
|
||||||
|
SemanticFact.entity_type == "project",
|
||||||
|
SemanticFact.entity_id == int(project.id),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
).scalars().all()
|
||||||
|
|
||||||
|
assert deactivated == 1
|
||||||
|
assert rows
|
||||||
|
assert all(row.is_active is False for row in rows)
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_project_existence_answer_prefers_semantic_facts_before_text_fallback(db_session) -> None:
|
||||||
|
project = Project(
|
||||||
|
tenant_id=1,
|
||||||
|
name="market-tool",
|
||||||
|
title="趋势分析工具",
|
||||||
|
description="这是一个证券行业分析工具。",
|
||||||
|
owner_id=None,
|
||||||
|
status="active",
|
||||||
|
assigned_role="backend",
|
||||||
|
)
|
||||||
|
db_session.add(project)
|
||||||
|
await db_session.flush()
|
||||||
|
|
||||||
|
fact_service = SemanticFactService(db_session, tenant_id=1)
|
||||||
|
await fact_service.upsert_fact(
|
||||||
|
entity_type="project",
|
||||||
|
entity_id=int(project.id),
|
||||||
|
fact_namespace="theme",
|
||||||
|
fact_key="market_theme",
|
||||||
|
fact_value_text="A股",
|
||||||
|
fact_value_norm="a_share",
|
||||||
|
value_type="keyword",
|
||||||
|
confidence=0.95,
|
||||||
|
evidence_source_type="project",
|
||||||
|
evidence_source_id=int(project.id),
|
||||||
|
evidence_excerpt="A股",
|
||||||
|
extract_version="semantic-facts-v1",
|
||||||
|
freshness_at=None,
|
||||||
|
)
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
class _FakeService(
|
||||||
|
AdaptiveRAGFastAnswerProjectMainMixin,
|
||||||
|
AdaptiveRAGFastAnswerProjectUtilsMixin,
|
||||||
|
):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.db = db_session
|
||||||
|
self.tenant_id = 1
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _extract_project_lookup_keyword(question: str) -> str: # pragma: no cover - should not be called
|
||||||
|
raise AssertionError(f"lexical keyword fallback should be skipped: {question}")
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _extract_query_terms(question: str, *, preferred: str = "") -> list[str]: # pragma: no cover - should not be called
|
||||||
|
raise AssertionError(f"lexical term fallback should be skipped: {question}, {preferred}")
|
||||||
|
|
||||||
|
service = _FakeService()
|
||||||
|
result = await service._try_fast_project_existence_answer(
|
||||||
|
clean_q="系统里有没有A股项目?",
|
||||||
|
project_id=None,
|
||||||
|
query_semantics={
|
||||||
|
"subject_focus": "project",
|
||||||
|
"scope_hint": "global",
|
||||||
|
"query_type": "existence",
|
||||||
|
"operation_contract": {
|
||||||
|
"query_type": "existence",
|
||||||
|
"object_mentions": ["A股项目"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert result is not None
|
||||||
|
assert "趋势分析工具" in str(result["answer"])
|
||||||
|
assert str(result["sources"][0]["metadata"]["table"]) == "semantic_facts"
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_structured_db_retrieval_prefers_semantic_fact_context_for_project_queries(db_session) -> None:
|
||||||
|
project = Project(
|
||||||
|
tenant_id=1,
|
||||||
|
name="market-tool",
|
||||||
|
title="趋势分析工具",
|
||||||
|
description="这是一个证券行业分析工具。",
|
||||||
|
owner_id=None,
|
||||||
|
status="active",
|
||||||
|
assigned_role="backend",
|
||||||
|
)
|
||||||
|
db_session.add(project)
|
||||||
|
await db_session.flush()
|
||||||
|
|
||||||
|
fact_service = SemanticFactService(db_session, tenant_id=1)
|
||||||
|
await fact_service.upsert_fact(
|
||||||
|
entity_type="project",
|
||||||
|
entity_id=int(project.id),
|
||||||
|
fact_namespace="theme",
|
||||||
|
fact_key="market_theme",
|
||||||
|
fact_value_text="A股",
|
||||||
|
fact_value_norm="a_share",
|
||||||
|
value_type="keyword",
|
||||||
|
confidence=0.95,
|
||||||
|
evidence_source_type="project",
|
||||||
|
evidence_source_id=int(project.id),
|
||||||
|
evidence_excerpt="A股",
|
||||||
|
extract_version="semantic-facts-v1",
|
||||||
|
freshness_at=None,
|
||||||
|
)
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
class _FakeService(
|
||||||
|
AdaptiveRAGRetrievalRouterDBMixin,
|
||||||
|
AdaptiveRAGFastAnswerProjectUtilsMixin,
|
||||||
|
AdaptiveRAGAnswerPayloadMixin,
|
||||||
|
):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.db = db_session
|
||||||
|
self.tenant_id = 1
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _extract_project_lookup_keyword(question: str) -> str: # pragma: no cover - should not be called
|
||||||
|
raise AssertionError(f"lexical keyword fallback should be skipped: {question}")
|
||||||
|
|
||||||
|
@staticmethod
|
||||||
|
def _extract_query_terms(question: str, *, preferred: str = "") -> list[str]: # pragma: no cover - should not be called
|
||||||
|
raise AssertionError(f"lexical term fallback should be skipped: {question}, {preferred}")
|
||||||
|
|
||||||
|
service = _FakeService()
|
||||||
|
rows = await service._retrieve_structured_db(
|
||||||
|
"系统里有没有A股项目?",
|
||||||
|
project_id=None,
|
||||||
|
query_semantics={
|
||||||
|
"subject_focus": "project",
|
||||||
|
"scope_hint": "global",
|
||||||
|
"query_type": "existence",
|
||||||
|
"operation_contract": {
|
||||||
|
"query_type": "existence",
|
||||||
|
"object_mentions": ["A股项目"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert rows
|
||||||
|
assert rows[0].metadata["table"] == "semantic_facts"
|
||||||
|
assert "趋势分析工具" in rows[0].content
|
||||||
|
|
||||||
|
|
||||||
|
@pytest.mark.asyncio
|
||||||
|
async def test_structured_db_retrieval_prefers_semantic_fact_context_for_task_queries(db_session) -> None:
|
||||||
|
user = User(
|
||||||
|
tenant_id=1,
|
||||||
|
username="task_owner",
|
||||||
|
email="task_owner@example.com",
|
||||||
|
password_hash="hash",
|
||||||
|
name="任务负责人",
|
||||||
|
role="ADMIN",
|
||||||
|
status="active",
|
||||||
|
is_active=True,
|
||||||
|
)
|
||||||
|
project = Project(
|
||||||
|
tenant_id=1,
|
||||||
|
name="invoice-project",
|
||||||
|
title="开票平台项目",
|
||||||
|
description="项目描述",
|
||||||
|
owner_id=None,
|
||||||
|
status="active",
|
||||||
|
assigned_role="backend",
|
||||||
|
)
|
||||||
|
db_session.add_all([user, project])
|
||||||
|
await db_session.flush()
|
||||||
|
|
||||||
|
charter = TruthNode(
|
||||||
|
tenant_id=1,
|
||||||
|
node_type="charter",
|
||||||
|
parent_id=None,
|
||||||
|
project_id=int(project.id),
|
||||||
|
signal_id=None,
|
||||||
|
created_by=int(user.id),
|
||||||
|
title="开票需求",
|
||||||
|
content={"summary": "建设开票系统"},
|
||||||
|
version=1,
|
||||||
|
status="draft",
|
||||||
|
)
|
||||||
|
db_session.add(charter)
|
||||||
|
await db_session.flush()
|
||||||
|
|
||||||
|
task = Task(
|
||||||
|
tenant_id=1,
|
||||||
|
truth_node_id=int(charter.id),
|
||||||
|
project_id=int(project.id),
|
||||||
|
assignee_id=int(user.id),
|
||||||
|
ai_recommended_assignees=None,
|
||||||
|
ai_breakdown_payload=None,
|
||||||
|
manual_overrides=None,
|
||||||
|
title="实现开票接口",
|
||||||
|
description="对接发票创建能力",
|
||||||
|
required_skill_tag_ids=[],
|
||||||
|
status="todo",
|
||||||
|
priority="high",
|
||||||
|
estimated_hours=None,
|
||||||
|
actual_hours=None,
|
||||||
|
quality_score=None,
|
||||||
|
due_date=None,
|
||||||
|
completed_at=None,
|
||||||
|
)
|
||||||
|
db_session.add(task)
|
||||||
|
await db_session.flush()
|
||||||
|
|
||||||
|
fact_service = SemanticFactService(db_session, tenant_id=1)
|
||||||
|
await fact_service.upsert_fact(
|
||||||
|
entity_type="task",
|
||||||
|
entity_id=int(task.id),
|
||||||
|
fact_namespace="alias",
|
||||||
|
fact_key="aliases",
|
||||||
|
fact_value_text="开票任务",
|
||||||
|
fact_value_norm="invoice_task",
|
||||||
|
value_type="keyword",
|
||||||
|
confidence=0.96,
|
||||||
|
evidence_source_type="task",
|
||||||
|
evidence_source_id=int(task.id),
|
||||||
|
evidence_excerpt="实现开票接口",
|
||||||
|
extract_version="semantic-facts-v1",
|
||||||
|
freshness_at=None,
|
||||||
|
)
|
||||||
|
await db_session.commit()
|
||||||
|
|
||||||
|
class _FakeService(
|
||||||
|
AdaptiveRAGRetrievalRouterDBMixin,
|
||||||
|
AdaptiveRAGFastAnswerProjectUtilsMixin,
|
||||||
|
AdaptiveRAGAnswerPayloadMixin,
|
||||||
|
):
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self.db = db_session
|
||||||
|
self.tenant_id = 1
|
||||||
|
|
||||||
|
service = _FakeService()
|
||||||
|
rows = await service._retrieve_structured_db(
|
||||||
|
"有没有开票任务?",
|
||||||
|
project_id=None,
|
||||||
|
query_semantics={
|
||||||
|
"subject_focus": "task",
|
||||||
|
"scope_hint": "global",
|
||||||
|
"query_type": "existence",
|
||||||
|
"operation_contract": {
|
||||||
|
"query_type": "existence",
|
||||||
|
"object_mentions": ["开票任务"],
|
||||||
|
},
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
|
assert rows
|
||||||
|
assert rows[0].metadata["table"] == "semantic_facts"
|
||||||
|
assert "开票任务" in rows[0].content
|
||||||
|
|
@ -48,6 +48,64 @@ def test_semantic_lint_flags_workflow_semantic_planner_directory(tmp_path) -> No
|
||||||
assert problems[0].rule_id == "semantic_hint_constant"
|
assert problems[0].rule_id == "semantic_hint_constant"
|
||||||
|
|
||||||
|
|
||||||
|
def test_semantic_lint_flags_query_route_object_tables_in_adaptive_rag(tmp_path) -> None:
|
||||||
|
module = _load_semantic_lint_module()
|
||||||
|
target = tmp_path / "backend" / "app" / "services" / "adaptive_rag_service" / "route_gateway.py"
|
||||||
|
target.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
target.write_text('_INTERNAL_ROUTE_OBJECTS = {"system_config"}\n', encoding="utf-8")
|
||||||
|
|
||||||
|
problems = module.collect_problems(
|
||||||
|
root=tmp_path,
|
||||||
|
scan_roots=(Path("backend/app/services/adaptive_rag_service"),),
|
||||||
|
baseline_path=Path("scripts/semantic_lint_baseline.txt"),
|
||||||
|
)
|
||||||
|
|
||||||
|
assert len(problems) == 1
|
||||||
|
assert problems[0].rule_id == "semantic_route_object_table"
|
||||||
|
|
||||||
|
|
||||||
|
def test_semantic_lint_flags_query_route_probe_helpers_in_adaptive_rag(tmp_path) -> None:
|
||||||
|
module = _load_semantic_lint_module()
|
||||||
|
target = tmp_path / "backend" / "app" / "services" / "adaptive_rag_service" / "route_gateway.py"
|
||||||
|
target.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
target.write_text("def _external_probe_is_strong(snapshot):\n return bool(snapshot)\n", encoding="utf-8")
|
||||||
|
|
||||||
|
problems = module.collect_problems(
|
||||||
|
root=tmp_path,
|
||||||
|
scan_roots=(Path("backend/app/services/adaptive_rag_service"),),
|
||||||
|
baseline_path=Path("scripts/semantic_lint_baseline.txt"),
|
||||||
|
)
|
||||||
|
|
||||||
|
assert len(problems) == 1
|
||||||
|
assert problems[0].rule_id == "semantic_route_probe_helper"
|
||||||
|
|
||||||
|
|
||||||
|
def test_semantic_lint_flags_external_object_kind_mapping(tmp_path) -> None:
|
||||||
|
module = _load_semantic_lint_module()
|
||||||
|
target = tmp_path / "backend" / "app" / "services" / "external_query_service" / "planner.py"
|
||||||
|
target.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
target.write_text(
|
||||||
|
'if "weather" in referenced_objects:\n query_kind = "weather"\n',
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
|
||||||
|
problems = module.collect_problems(
|
||||||
|
root=tmp_path,
|
||||||
|
scan_roots=(Path("backend/app/services/external_query_service"),),
|
||||||
|
baseline_path=Path("scripts/semantic_lint_baseline.txt"),
|
||||||
|
)
|
||||||
|
|
||||||
|
assert len(problems) == 1
|
||||||
|
assert problems[0].rule_id == "semantic_object_kind_mapping"
|
||||||
|
|
||||||
|
|
||||||
|
def test_semantic_lint_default_roots_cover_adaptive_rag_and_external_query() -> None:
|
||||||
|
module = _load_semantic_lint_module()
|
||||||
|
|
||||||
|
assert Path("backend/app/services/adaptive_rag_service") in module.DEFAULT_SCAN_ROOTS
|
||||||
|
assert Path("backend/app/services/external_query_service") in module.DEFAULT_SCAN_ROOTS
|
||||||
|
|
||||||
|
|
||||||
def test_semantic_lint_repo_has_no_compat_baseline_exceptions() -> None:
|
def test_semantic_lint_repo_has_no_compat_baseline_exceptions() -> None:
|
||||||
repo_root = Path(__file__).resolve().parents[2]
|
repo_root = Path(__file__).resolve().parents[2]
|
||||||
baseline_path = repo_root / "scripts" / "semantic_lint_baseline.txt"
|
baseline_path = repo_root / "scripts" / "semantic_lint_baseline.txt"
|
||||||
|
|
|
||||||
|
|
@ -3,3 +3,5 @@
|
||||||
{"generated_at": "2026-03-28T08:52:16.466761", "case": "challenge", "source_message_id": "codex-feishu-challenge-1774659109", "question": "我要把整个系统一次性全部重构掉,这么做靠谱吗", "run_status": "missing", "processing": {"status": "success", "route": "ai_pmo.query.answer", "intent": "query.answer", "reply_source": "action_succeeded", "error_message": ""}, "outbox": [{"id": 655, "status": "sent", "attempts": 1, "sent_message_id": "om_x100b53462dfeb8a4b4a22d394e5186c", "text": "不靠谱。 \n依据:当前系统后端模块统计为全零(api/services/models/schemas均为0),但热点模块集中在`backend/app/api/sstc.py`等5个文件,说明核心逻辑实际集中、未模块化;前端仅6个路由,功能边界清晰;156个待处理信号中P0级3个、P1级46个,系统处于高负载运行态。 \n下一步:立即暂停重构决策,优先完成`sstc.py`模块拆解图谱与接口契约定义,并确认杨宇(唯一跨角色开发者,workload_ratio=796.67)是否参与重构设计。"}], "final_reply": "不靠谱。 \n依据:当前系统后端模块统计为全零(api/services/models/schemas均为0),但热点模块集中在`backend/app/api/sstc.py`等5个文件,说明核心逻辑实际集中、未模块化;前端仅6个路由,功能边界清晰;156个待处理信号中P0级3个、P1级46个,系统处于高负载运行态。 \n下一步:立即暂停重构决策,优先完成`sstc.py`模块拆解图谱与接口契约定义,并确认杨宇(唯一跨角色开发者,workload_ratio=796.67)是否参与重构设计。"}
|
{"generated_at": "2026-03-28T08:52:16.466761", "case": "challenge", "source_message_id": "codex-feishu-challenge-1774659109", "question": "我要把整个系统一次性全部重构掉,这么做靠谱吗", "run_status": "missing", "processing": {"status": "success", "route": "ai_pmo.query.answer", "intent": "query.answer", "reply_source": "action_succeeded", "error_message": ""}, "outbox": [{"id": 655, "status": "sent", "attempts": 1, "sent_message_id": "om_x100b53462dfeb8a4b4a22d394e5186c", "text": "不靠谱。 \n依据:当前系统后端模块统计为全零(api/services/models/schemas均为0),但热点模块集中在`backend/app/api/sstc.py`等5个文件,说明核心逻辑实际集中、未模块化;前端仅6个路由,功能边界清晰;156个待处理信号中P0级3个、P1级46个,系统处于高负载运行态。 \n下一步:立即暂停重构决策,优先完成`sstc.py`模块拆解图谱与接口契约定义,并确认杨宇(唯一跨角色开发者,workload_ratio=796.67)是否参与重构设计。"}], "final_reply": "不靠谱。 \n依据:当前系统后端模块统计为全零(api/services/models/schemas均为0),但热点模块集中在`backend/app/api/sstc.py`等5个文件,说明核心逻辑实际集中、未模块化;前端仅6个路由,功能边界清晰;156个待处理信号中P0级3个、P1级46个,系统处于高负载运行态。 \n下一步:立即暂停重构决策,优先完成`sstc.py`模块拆解图谱与接口契约定义,并确认杨宇(唯一跨角色开发者,workload_ratio=796.67)是否参与重构设计。"}
|
||||||
{"generated_at": "2026-03-28T08:53:35.385412", "case": "brainstorm", "source_message_id": "codex-feishu-brainstorm-1774659145", "question": "我们一起想几个 AI 劝退助手的方向", "run_status": "missing", "processing": {"status": "success", "route": "ai_pmo.draft_pending_confirm", "intent": "project_signal.create", "reply_source": "draft_pending_confirm", "error_message": ""}, "outbox": [{"id": 656, "status": "sent", "attempts": 1, "sent_message_id": "om_x100b5346298338a4b2c4f90aaa6bc9f", "text": "📋 我先按产品经理视角起草一版评审建议,请确认:\n\n📌 需求草稿\n我们一起想几个 AI 劝退助手的方向\n\n🔎 同名 / 相似项目:\n· #51 AI劝退助手:现有项目已有一定描述,建议对比当前需求的目标、范围和交付边界。;若范围重叠,可能造成重复建设或维护分散。;名称或目标高度接近,优先确认是否复用或合并。\n\n🚀 MVP 建议:\n· 明确项目目标与边界\n· 完成同名/相似项目查重\n· 交付最小可验证闭环\n\n一、定位\n· 一句话描述:我们一起想几个 AI 劝退助手的方向\n· 目标客户:待补充\n\n二、核心痛点\n· 待补充核心痛点\n\n三、MVP 功能范围(8周交付)\nPhase 1:核心功能(Week 1-4)\n· 明确项目目标与边界\n· 完成同名/相似项目查重\n· 交付最小可验证闭环\nPhase 2:差异化功能(Week 5-8)\n· 待补充:确认差异化方向后再展开\n\n四、技术架构(精简版)\n· 待补充技术架构\n\n五、商业模式\n· 待补充商业模式\n\n六、验证指标\n验证指标:\n· 待补充\n\n七、快速启动建议\n快速启动建议:\n· 待补充\n\n八、团队配置\n· 待补充团队配置\n\n📈 前景判断:\n· 该项目具备落地价值,但建议先确认范围边界和现有项目复用情况,再决定是否推进\n\n🧩 需求清单:\n· 项目定位与目标\n· MVP 功能范围\n\n❓ 还缺的信息:\n· 暂无\n\n✅ 可行性:待讨论\n信息仍需补充,建议先确认边界。\n\n👤 推荐负责人:系统管理员 · 当前负载100%\n基于当前负载与任务匹配度给出的建议。\n\n📅 跟进计划:\n· 明天:确认触发时机与验收口径\n· 一周后:复核执行效果并决定是否扩展\n\n---\n回复 确认 → 入库\n回复 修改 [具体修改意见] → 重新起草\n回复 取消 → 放弃\n⏰ 24小时内未确认自动取消"}], "final_reply": "📋 我先按产品经理视角起草一版评审建议,请确认:\n\n📌 需求草稿\n我们一起想几个 AI 劝退助手的方向\n\n🔎 同名 / 相似项目:\n· #51 AI劝退助手:现有项目已有一定描述,建议对比当前需求的目标、范围和交付边界。;若范围重叠,可能造成重复建设或维护分散。;名称或目标高度接近,优先确认是否复用或合并。\n\n🚀 MVP 建议:\n· 明确项目目标与边界\n· 完成同名/相似项目查重\n· 交付最小可验证闭环\n\n一、定位\n· 一句话描述:我们一起想几个 AI 劝退助手的方向\n· 目标客户:待补充\n\n二、核心痛点\n· 待补充核心痛点\n\n三、MVP 功能范围(8周交付)\nPhase 1:核心功能(Week 1-4)\n· 明确项目目标与边界\n· 完成同名/相似项目查重\n· 交付最小可验证闭环\nPhase 2:差异化功能(Week 5-8)\n· 待补充:确认差异化方向后再展开\n\n四、技术架构(精简版)\n· 待补充技术架构\n\n五、商业模式\n· 待补充商业模式\n\n六、验证指标\n验证指标:\n· 待补充\n\n七、快速启动建议\n快速启动建议:\n· 待补充\n\n八、团队配置\n· 待补充团队配置\n\n📈 前景判断:\n· 该项目具备落地价值,但建议先确认范围边界和现有项目复用情况,再决定是否推进\n\n🧩 需求清单:\n· 项目定位与目标\n· MVP 功能范围\n\n❓ 还缺的信息:\n· 暂无\n\n✅ 可行性:待讨论\n信息仍需补充,建议先确认边界。\n\n👤 推荐负责人:系统管理员 · 当前负载100%\n基于当前负载与任务匹配度给出的建议。\n\n📅 跟进计划:\n· 明天:确认触发时机与验收口径\n· 一周后:复核执行效果并决定是否扩展\n\n---\n回复 确认 → 入库\n回复 修改 [具体修改意见] → 重新起草\n回复 取消 → 放弃\n⏰ 24小时内未确认自动取消"}
|
{"generated_at": "2026-03-28T08:53:35.385412", "case": "brainstorm", "source_message_id": "codex-feishu-brainstorm-1774659145", "question": "我们一起想几个 AI 劝退助手的方向", "run_status": "missing", "processing": {"status": "success", "route": "ai_pmo.draft_pending_confirm", "intent": "project_signal.create", "reply_source": "draft_pending_confirm", "error_message": ""}, "outbox": [{"id": 656, "status": "sent", "attempts": 1, "sent_message_id": "om_x100b5346298338a4b2c4f90aaa6bc9f", "text": "📋 我先按产品经理视角起草一版评审建议,请确认:\n\n📌 需求草稿\n我们一起想几个 AI 劝退助手的方向\n\n🔎 同名 / 相似项目:\n· #51 AI劝退助手:现有项目已有一定描述,建议对比当前需求的目标、范围和交付边界。;若范围重叠,可能造成重复建设或维护分散。;名称或目标高度接近,优先确认是否复用或合并。\n\n🚀 MVP 建议:\n· 明确项目目标与边界\n· 完成同名/相似项目查重\n· 交付最小可验证闭环\n\n一、定位\n· 一句话描述:我们一起想几个 AI 劝退助手的方向\n· 目标客户:待补充\n\n二、核心痛点\n· 待补充核心痛点\n\n三、MVP 功能范围(8周交付)\nPhase 1:核心功能(Week 1-4)\n· 明确项目目标与边界\n· 完成同名/相似项目查重\n· 交付最小可验证闭环\nPhase 2:差异化功能(Week 5-8)\n· 待补充:确认差异化方向后再展开\n\n四、技术架构(精简版)\n· 待补充技术架构\n\n五、商业模式\n· 待补充商业模式\n\n六、验证指标\n验证指标:\n· 待补充\n\n七、快速启动建议\n快速启动建议:\n· 待补充\n\n八、团队配置\n· 待补充团队配置\n\n📈 前景判断:\n· 该项目具备落地价值,但建议先确认范围边界和现有项目复用情况,再决定是否推进\n\n🧩 需求清单:\n· 项目定位与目标\n· MVP 功能范围\n\n❓ 还缺的信息:\n· 暂无\n\n✅ 可行性:待讨论\n信息仍需补充,建议先确认边界。\n\n👤 推荐负责人:系统管理员 · 当前负载100%\n基于当前负载与任务匹配度给出的建议。\n\n📅 跟进计划:\n· 明天:确认触发时机与验收口径\n· 一周后:复核执行效果并决定是否扩展\n\n---\n回复 确认 → 入库\n回复 修改 [具体修改意见] → 重新起草\n回复 取消 → 放弃\n⏰ 24小时内未确认自动取消"}], "final_reply": "📋 我先按产品经理视角起草一版评审建议,请确认:\n\n📌 需求草稿\n我们一起想几个 AI 劝退助手的方向\n\n🔎 同名 / 相似项目:\n· #51 AI劝退助手:现有项目已有一定描述,建议对比当前需求的目标、范围和交付边界。;若范围重叠,可能造成重复建设或维护分散。;名称或目标高度接近,优先确认是否复用或合并。\n\n🚀 MVP 建议:\n· 明确项目目标与边界\n· 完成同名/相似项目查重\n· 交付最小可验证闭环\n\n一、定位\n· 一句话描述:我们一起想几个 AI 劝退助手的方向\n· 目标客户:待补充\n\n二、核心痛点\n· 待补充核心痛点\n\n三、MVP 功能范围(8周交付)\nPhase 1:核心功能(Week 1-4)\n· 明确项目目标与边界\n· 完成同名/相似项目查重\n· 交付最小可验证闭环\nPhase 2:差异化功能(Week 5-8)\n· 待补充:确认差异化方向后再展开\n\n四、技术架构(精简版)\n· 待补充技术架构\n\n五、商业模式\n· 待补充商业模式\n\n六、验证指标\n验证指标:\n· 待补充\n\n七、快速启动建议\n快速启动建议:\n· 待补充\n\n八、团队配置\n· 待补充团队配置\n\n📈 前景判断:\n· 该项目具备落地价值,但建议先确认范围边界和现有项目复用情况,再决定是否推进\n\n🧩 需求清单:\n· 项目定位与目标\n· MVP 功能范围\n\n❓ 还缺的信息:\n· 暂无\n\n✅ 可行性:待讨论\n信息仍需补充,建议先确认边界。\n\n👤 推荐负责人:系统管理员 · 当前负载100%\n基于当前负载与任务匹配度给出的建议。\n\n📅 跟进计划:\n· 明天:确认触发时机与验收口径\n· 一周后:复核执行效果并决定是否扩展\n\n---\n回复 确认 → 入库\n回复 修改 [具体修改意见] → 重新起草\n回复 取消 → 放弃\n⏰ 24小时内未确认自动取消"}
|
||||||
{"generated_at": "2026-04-07T00:07:35.291848", "idx": 9001, "source_message_id": "real-scene-p1-manual-9001-1775491344", "question": "跟你聊个新需求,关于换季系统自动推荐搭配衣服", "message": "跟你聊个新需求,关于换季系统自动推荐搭配衣服", "capture_status": "answered", "capture_direct_reply_text": "可以,我们先不创建项目,先把这件事收成一版需求。\n\n我先把结果主要给谁用定一下,这会直接影响后面的内容粒度和表达方式。\n\nA. 核心执行人 / 内部 PM\n更适合:先服务负责跟进、拆动作、推动落地的人\n\nB. 负责人 / 管理者\n更适合:先看结论、风险和待拍板项\n\nC. 整个团队 / 终端用户\n更适合:需要直接同步所有相关方,或结果就是给最终使用者看的\n\n你回 A / B / C,也可以直接说更准确的对象。", "consultant_dispatch": null, "run_status": "missing", "ack": "", "final_reply": "可以,我们先不创建项目,先把这件事收成一版需求。\n\n我先把结果主要给谁用定一下,这会直接影响后面的内容粒度和表达方式。\n\nA. 核心执行人 / 内部 PM\n更适合:先服务负责跟进、拆动作、推动落地的人\n\nB. 负责人 / 管理者\n更适合:先看结论、风险和待拍板项\n\nC. 整个团队 / 终端用户\n更适合:需要直接同步所有相关方,或结果就是给最终使用者看的\n\n你回 A / B / C,也可以直接说更准确的对象。", "prompt_scene": "unknown", "inferred_scene": "unknown", "actual_scene": "unknown", "thinking_leak": false, "has_capacity_warning": false, "processing": {"route": "ai_pmo.query.answer", "intent": "query.answer", "status": "success", "duration_ms": 9984, "error_message": "", "reply_source": "action_succeeded"}, "outbox": [{"id": 966, "status": "pending", "attempts": 0, "created_at": "2026-04-07 00:02:34.138267", "text": "可以,我们先不创建项目,先把这件事收成一版需求。\n\n我先把结果主要给谁用定一下,这会直接影响后面的内容粒度和表达方式。\n\nA. 核心执行人 / 内部 PM\n更适合:先服务负责跟进、拆动作、推动落地的人\n\nB. 负责人 / 管理者\n更适合:先看结论、风险和待拍板项\n\nC. 整个团队 / 终端用户\n更适合:需要直接同步所有相关方,或结果就是给最终使用者看的\n\n你回 A / B / C,也可以直接说更准确的对象。"}], "call_log_candidates": [], "pass": false, "pass_rule": "unknown"}
|
{"generated_at": "2026-04-07T00:07:35.291848", "idx": 9001, "source_message_id": "real-scene-p1-manual-9001-1775491344", "question": "跟你聊个新需求,关于换季系统自动推荐搭配衣服", "message": "跟你聊个新需求,关于换季系统自动推荐搭配衣服", "capture_status": "answered", "capture_direct_reply_text": "可以,我们先不创建项目,先把这件事收成一版需求。\n\n我先把结果主要给谁用定一下,这会直接影响后面的内容粒度和表达方式。\n\nA. 核心执行人 / 内部 PM\n更适合:先服务负责跟进、拆动作、推动落地的人\n\nB. 负责人 / 管理者\n更适合:先看结论、风险和待拍板项\n\nC. 整个团队 / 终端用户\n更适合:需要直接同步所有相关方,或结果就是给最终使用者看的\n\n你回 A / B / C,也可以直接说更准确的对象。", "consultant_dispatch": null, "run_status": "missing", "ack": "", "final_reply": "可以,我们先不创建项目,先把这件事收成一版需求。\n\n我先把结果主要给谁用定一下,这会直接影响后面的内容粒度和表达方式。\n\nA. 核心执行人 / 内部 PM\n更适合:先服务负责跟进、拆动作、推动落地的人\n\nB. 负责人 / 管理者\n更适合:先看结论、风险和待拍板项\n\nC. 整个团队 / 终端用户\n更适合:需要直接同步所有相关方,或结果就是给最终使用者看的\n\n你回 A / B / C,也可以直接说更准确的对象。", "prompt_scene": "unknown", "inferred_scene": "unknown", "actual_scene": "unknown", "thinking_leak": false, "has_capacity_warning": false, "processing": {"route": "ai_pmo.query.answer", "intent": "query.answer", "status": "success", "duration_ms": 9984, "error_message": "", "reply_source": "action_succeeded"}, "outbox": [{"id": 966, "status": "pending", "attempts": 0, "created_at": "2026-04-07 00:02:34.138267", "text": "可以,我们先不创建项目,先把这件事收成一版需求。\n\n我先把结果主要给谁用定一下,这会直接影响后面的内容粒度和表达方式。\n\nA. 核心执行人 / 内部 PM\n更适合:先服务负责跟进、拆动作、推动落地的人\n\nB. 负责人 / 管理者\n更适合:先看结论、风险和待拍板项\n\nC. 整个团队 / 终端用户\n更适合:需要直接同步所有相关方,或结果就是给最终使用者看的\n\n你回 A / B / C,也可以直接说更准确的对象。"}], "call_log_candidates": [], "pass": false, "pass_rule": "unknown"}
|
||||||
|
{"generated_at": "2026-04-16T00:59:02.932070", "idx": 99, "source_message_id": "feishu-weather-e2e-99-1776271963", "question": "湛江市未来五天的天气", "message": "湛江市未来五天的天气", "capture_status": "answered", "capture_direct_reply_text": "当前正在需求模式,这条线程先只处理“当前这条需求”的收敛。你可以继续回 A / B / C / D、补一句、出草稿、重出这轮;如果要聊别的,请先回复“退出需求模式”。", "consultant_dispatch": null, "run_status": "missing", "ack": "", "final_reply": "当前正在需求模式,这条线程先只处理“当前这条需求”的收敛。你可以继续回 A / B / C / D、补一句、出草稿、重出这轮;如果要聊别的,请先回复“退出需求模式”。", "prompt_scene": "unknown", "inferred_scene": "unknown", "actual_scene": "unknown", "thinking_leak": false, "has_capacity_warning": false, "processing": {"route": "ai_pmo.query.answer", "intent": "query.answer", "status": "success", "duration_ms": 41383, "error_message": "", "reply_source": "action_succeeded"}, "outbox": [{"id": 1146, "status": "pending", "attempts": 0, "created_at": "2026-04-16 00:54:01.540858", "text": "当前正在需求模式,这条线程先只处理“当前这条需求”的收敛。你可以继续回 A / B / C / D、补一句、出草稿、重出这轮;如果要聊别的,请先回复“退出需求模式”。"}], "call_log_candidates": [], "pass": false, "pass_rule": "unknown"}
|
||||||
|
{"generated_at": "2026-04-16T10:04:28.903335", "idx": 901, "source_message_id": "codex-feishu-weather-901-1776304695", "question": "湛江市未来五天的天气", "message": "湛江市未来五天的天气", "capture_status": "answered", "capture_direct_reply_text": "当前正在需求模式,这条线程先只处理“当前这条需求”的收敛。你可以继续回 A / B / C / D、补一句、出草稿、重出这轮;如果要聊别的,请先回复“退出需求模式”。", "consultant_dispatch": null, "run_status": "missing", "ack": "", "final_reply": "当前正在需求模式,这条线程先只处理“当前这条需求”的收敛。你可以继续回 A / B / C / D、补一句、出草稿、重出这轮;如果要聊别的,请先回复“退出需求模式”。", "prompt_scene": "unknown", "inferred_scene": "unknown", "actual_scene": "unknown", "thinking_leak": false, "has_capacity_warning": false, "processing": {"route": "ai_pmo.query.answer", "intent": "query.answer", "status": "success", "duration_ms": 56505, "error_message": "", "reply_source": "action_succeeded"}, "outbox": [{"id": 1151, "status": "pending", "attempts": 0, "created_at": "2026-04-16 09:59:27.962651", "text": "当前正在需求模式,这条线程先只处理“当前这条需求”的收敛。你可以继续回 A / B / C / D、补一句、出草稿、重出这轮;如果要聊别的,请先回复“退出需求模式”。"}], "call_log_candidates": [{"id": 6075, "scene": "unknown", "function": "adaptive_rag_answer_normal", "created_at": "2026-04-16 10:02:19.531104"}], "pass": false, "pass_rule": "unknown"}
|
||||||
|
|
|
||||||
File diff suppressed because one or more lines are too long
|
|
@ -25,6 +25,7 @@
|
||||||
- `docs/architecture/DATAFLOW.md`:主链与数据流说明
|
- `docs/architecture/DATAFLOW.md`:主链与数据流说明
|
||||||
- `docs/architecture/TASK_BRIEF_TEMPLATE.md`:任务包模板
|
- `docs/architecture/TASK_BRIEF_TEMPLATE.md`:任务包模板
|
||||||
- `docs/superpowers/plans/`:实现计划
|
- `docs/superpowers/plans/`:实现计划
|
||||||
|
- `docs/superpowers/specs/`:需求/方案规格
|
||||||
- `docs/task_briefs/`:任务包产物与验收截图
|
- `docs/task_briefs/`:任务包产物与验收截图
|
||||||
|
|
||||||
## 4. 后端库存
|
## 4. 后端库存
|
||||||
|
|
@ -37,6 +38,7 @@
|
||||||
- `backend/app/api/sstc_api/`
|
- `backend/app/api/sstc_api/`
|
||||||
- `backend/app/api/signals_api/`
|
- `backend/app/api/signals_api/`
|
||||||
- `backend/app/api/signal_sources_api/`
|
- `backend/app/api/signal_sources_api/`
|
||||||
|
- `backend/app/api/semantic_facts_api/`
|
||||||
- `backend/app/api/skill_tags_api/`
|
- `backend/app/api/skill_tags_api/`
|
||||||
- `backend/app/api/ai_logs_api/`
|
- `backend/app/api/ai_logs_api/`
|
||||||
|
|
||||||
|
|
@ -51,6 +53,9 @@
|
||||||
|
|
||||||
### 4.3 问答、模型与生成
|
### 4.3 问答、模型与生成
|
||||||
- `backend/app/services/adaptive_rag_service/`
|
- `backend/app/services/adaptive_rag_service/`
|
||||||
|
- `backend/app/services/adaptive_rag_service/mixin_query_route_gateway.py`
|
||||||
|
- `backend/app/services/semantic_fact_service/`
|
||||||
|
- `backend/app/services/semantic_fact_service/hooks.py`
|
||||||
- `backend/app/services/llm_service/`
|
- `backend/app/services/llm_service/`
|
||||||
- `backend/app/services/ai_steward_service/`
|
- `backend/app/services/ai_steward_service/`
|
||||||
- `backend/app/services/feedback_center/`
|
- `backend/app/services/feedback_center/`
|
||||||
|
|
@ -80,6 +85,9 @@
|
||||||
- `backend/app/models/reply_iteration.py`
|
- `backend/app/models/reply_iteration.py`
|
||||||
- `backend/app/models/training_pipeline.py`
|
- `backend/app/models/training_pipeline.py`
|
||||||
- `backend/app/models/mounted_ai_runtime.py`
|
- `backend/app/models/mounted_ai_runtime.py`
|
||||||
|
- `backend/app/models/query_route_memory.py`
|
||||||
|
- `backend/app/models/semantic_fact.py`
|
||||||
|
- `backend/app/models/semantic_fact_extraction_run.py`
|
||||||
- `backend/app/api/ai_steward_api/routes_reply_iteration.py`
|
- `backend/app/api/ai_steward_api/routes_reply_iteration.py`
|
||||||
- `backend/app/api/ai_steward_api/routes_training_pipeline.py`
|
- `backend/app/api/ai_steward_api/routes_training_pipeline.py`
|
||||||
- `backend/app/api/ai_steward_api/routes_mounted_ai.py`
|
- `backend/app/api/ai_steward_api/routes_mounted_ai.py`
|
||||||
|
|
@ -88,7 +96,16 @@
|
||||||
- `backend/app/services/` 根目录已目录化收敛,新增服务优先进入对应子包。
|
- `backend/app/services/` 根目录已目录化收敛,新增服务优先进入对应子包。
|
||||||
- API canonical 入口优先使用 `*_api/` 子包,不再优先引用顶层兼容壳。
|
- API canonical 入口优先使用 `*_api/` 子包,不再优先引用顶层兼容壳。
|
||||||
- 运行时业务动作归因优先走“对象/上下文绑定 -> `SemanticContextPacket` -> `SemanticDecision` -> gatekeeper”。
|
- 运行时业务动作归因优先走“对象/上下文绑定 -> `SemanticContextPacket` -> `SemanticDecision` -> gatekeeper”。
|
||||||
|
- non-plan `query.answer` 现已在 `AdaptiveRAG` 内增加 `Query Route Gateway`:只消费上游单次结构化 `query_profile/source_policy/effective_source_policy` 与显式 hint,归一化产出 `route_mode / decision_origin / reply_contract / probe_snapshot(contract audit)`,不再在运行时执行 internal/external probe 或 route memory 强拍板,再进入检索与回答主链。
|
||||||
|
- internal `query.answer` 的下一阶段基线已经确认:`Query Route Gateway` 之后不再直接靠快答分支拍板,而是统一进入 `Operation Contract -> Target Resolver -> Query DSL Plan -> 白名单受控执行器 -> semantic facts 优先 -> verifier` 主链;相关设计规格见 `docs/superpowers/specs/2026-04-15-internal-query-dsl-plan-design.md` 与 `docs/superpowers/specs/2026-04-15-semantic-facts-read-model-design.md`。
|
||||||
|
- external `query.answer` 已完成 first-pass 收口:`Query Route Gateway` 之后统一进入 `backend/app/services/external_query_service/`,按 `External Query Planner -> External Query Gateway -> External Provider Registry -> MCP adapter 优先 -> bounded fallback live search -> external answer verifier` 执行;AdaptiveRAG external probe、freshness fetch 与 external route retrieval 已接到这条链上,并对 external sources 执行 fail-closed 过滤;相关设计规格见 `docs/superpowers/specs/2026-04-15-external-query-gateway-design.md`。
|
||||||
|
- `query.answer` 的客户可见回答展示也已确认独立基线:运行时证据必须拆成 `customer_safe` 与 `admin_debug` 双视图,客户侧统一消费中文化 `display_label`,禁止直接暴露 `raw_source_label`、机器来源或内部术语;相关设计规格见 `docs/superpowers/specs/2026-04-15-answer-presentation-policy-design.md`。
|
||||||
|
- `semantic_fact_service` 已升级为统一基线:`semantic_facts / semantic_fact_extraction_runs` 读模型、ORM 提交钩子自动收集对象变更并 upsert `ingest_events(channel=semantic_fact_extract)`、统一 worker 抽取/失活,以及 internal query 的 facts-first 主路径。
|
||||||
|
- semantic facts 自动抽取覆盖 `project / task / signal / document / user / requirement(charter) / release / review` 八类业务对象;manual retry 与 backfill 走同一队列、同一 worker,不再开旁路。
|
||||||
|
- MCP 模板接入中心已接入 `backend/app/api/mcp_api/` + `backend/app/services/mcp_config/`:租户内统一保存 MCP 模板 catalog、server 配置 bundle、命令预览、validate 结果、runtime probe 与 `pending_local_apply / user_confirmed / not_required` 状态;远程 `streamable_http` 模板可由 PMFlow 后端直连,本地 `stdio` 模板只输出 Local Bridge/兼容 host 接入说明,服务端不直写客户机器 `~/.codex/config.toml`。
|
||||||
|
- `query_route_memories` 作为独立路由记忆表,承接 `QARecord` 的 route audit、feedback 回灌和离线评估样本,不复用原 answer-quality 资产表。
|
||||||
- `backend/tests/test_runtime_structured_semantics_contract.py` 与 `backend/tests/test_feishu_pipeline_queue_contract.py` 共同守门 runtime 结构化语义收口,锁定飞书入口、signal capture、reply/runtime routing 不得回流旧词法拍板 helper。
|
- `backend/tests/test_runtime_structured_semantics_contract.py` 与 `backend/tests/test_feishu_pipeline_queue_contract.py` 共同守门 runtime 结构化语义收口,锁定飞书入口、signal capture、reply/runtime routing 不得回流旧词法拍板 helper。
|
||||||
|
- `backend/tests/test_external_query_planner.py` 现已补进 external fallback 回归,锁定 planner fallback 只能产出 generic external plan,不得再根据 `weather/news/public_model/...` 之类对象映射在运行时硬分 query_kind。
|
||||||
|
|
||||||
## 5. 前端库存
|
## 5. 前端库存
|
||||||
|
|
||||||
|
|
@ -103,6 +120,7 @@
|
||||||
- `frontend/src/pages/feishu-config/`
|
- `frontend/src/pages/feishu-config/`
|
||||||
- `frontend/src/pages/ai-config/`
|
- `frontend/src/pages/ai-config/`
|
||||||
- `frontend/src/pages/model-routing-config/`
|
- `frontend/src/pages/model-routing-config/`
|
||||||
|
- `frontend/src/pages/ai-config/mcp-config/`
|
||||||
- `frontend/src/pages/task-brief-center/`
|
- `frontend/src/pages/task-brief-center/`
|
||||||
- `frontend/src/pages/task-brief-runs/`
|
- `frontend/src/pages/task-brief-runs/`
|
||||||
- `frontend/src/pages/task-brief-template/`
|
- `frontend/src/pages/task-brief-template/`
|
||||||
|
|
@ -110,6 +128,7 @@
|
||||||
- `frontend/src/pages/contract-sync-board/`
|
- `frontend/src/pages/contract-sync-board/`
|
||||||
- `frontend/src/pages/dashboard/`
|
- `frontend/src/pages/dashboard/`
|
||||||
- `frontend/src/pages/knowledge-graph-context-board/`
|
- `frontend/src/pages/knowledge-graph-context-board/`
|
||||||
|
- `frontend/src/pages/semantic-facts-board/`
|
||||||
- `frontend/src/pages/login/`
|
- `frontend/src/pages/login/`
|
||||||
- `frontend/src/pages/skill-tasks/`
|
- `frontend/src/pages/skill-tasks/`
|
||||||
- `frontend/src/pages/skills/`
|
- `frontend/src/pages/skills/`
|
||||||
|
|
@ -123,7 +142,9 @@
|
||||||
- `frontend/src/api/aiSteward.ts`
|
- `frontend/src/api/aiSteward.ts`
|
||||||
- `frontend/src/api/taskBriefs.ts`
|
- `frontend/src/api/taskBriefs.ts`
|
||||||
- `frontend/src/api/modelRouting.ts`
|
- `frontend/src/api/modelRouting.ts`
|
||||||
|
- `frontend/src/api/mcp.ts`
|
||||||
- `frontend/src/api/knowledgeGraph.ts`
|
- `frontend/src/api/knowledgeGraph.ts`
|
||||||
|
- `frontend/src/api/semanticFacts.ts`
|
||||||
- `frontend/src/api/mountedAi.ts`
|
- `frontend/src/api/mountedAi.ts`
|
||||||
- `frontend/src/api/feedbackCenter.ts`
|
- `frontend/src/api/feedbackCenter.ts`
|
||||||
- `frontend/src/api/sstc.ts`
|
- `frontend/src/api/sstc.ts`
|
||||||
|
|
@ -137,6 +158,7 @@
|
||||||
- 页面级编排逻辑优先收敛到 `pages/*/` 子目录。
|
- 页面级编排逻辑优先收敛到 `pages/*/` 子目录。
|
||||||
- API 调用统一走 `frontend/src/api/`,不在页面或组件内直接 `fetch`。
|
- API 调用统一走 `frontend/src/api/`,不在页面或组件内直接 `fetch`。
|
||||||
- `frontend/src/components/sstc/` 继续作为 SSTC 共享组件目录维护。
|
- `frontend/src/components/sstc/` 继续作为 SSTC 共享组件目录维护。
|
||||||
|
- `AI Config -> Governance Tools` 现新增 `/ai-config/mcp-config` 与 `/ai-config/semantic-facts-board` 入口;两类治理页都只对 `BOSS/ADMIN` 开放。MCP 页负责模板化配置、后端直连/Local Bridge 状态展示、runtime probe、命令预览与本机确认;semantic facts 页负责 KPI、队列、对象表与详情抽屉。
|
||||||
|
|
||||||
## 6. 工具与运行库存
|
## 6. 工具与运行库存
|
||||||
- `backend/scripts/repair_requirement_plan_history.py`
|
- `backend/scripts/repair_requirement_plan_history.py`
|
||||||
|
|
@ -145,6 +167,7 @@
|
||||||
- `scripts/gen_task_brief.sh`
|
- `scripts/gen_task_brief.sh`
|
||||||
- `scripts/check_contract_sync.sh`
|
- `scripts/check_contract_sync.sh`
|
||||||
- `frontend/playwright.config.ts`
|
- `frontend/playwright.config.ts`
|
||||||
|
- `frontend/vitest.setup.ts`
|
||||||
- `frontend/e2e/feishu-project-full-flow.spec.ts`
|
- `frontend/e2e/feishu-project-full-flow.spec.ts`
|
||||||
|
|
||||||
## 7. 维护建议
|
## 7. 维护建议
|
||||||
|
|
|
||||||
|
|
@ -30,8 +30,19 @@
|
||||||
|
|
||||||
### 2.3 RAG / AI PMO / Knowledge Graph
|
### 2.3 RAG / AI PMO / Knowledge Graph
|
||||||
- RAG 问答主入口:参考 `backend/app/api/rag_api/` 与 `backend/app/api/rag_ask_contract.py`
|
- RAG 问答主入口:参考 `backend/app/api/rag_api/` 与 `backend/app/api/rag_ask_contract.py`
|
||||||
|
- Query 路由网关与显式来源回复合同:参考 `backend/app/services/adaptive_rag_service/mixin_query_route_gateway.py`
|
||||||
|
- External Query Service 主链:参考 `backend/app/services/external_query_service/`(`planner.py`、`gateway.py`、`registry.py`、`adapters/`)
|
||||||
|
- 运行时语义守门与回归:参考 `scripts/semantic_lint.py`、`backend/tests/test_semantic_lint.py`、`backend/tests/test_external_query_planner.py`
|
||||||
|
- MCP backend-direct runtime / Microsoft Learn bridge:参考 `backend/app/services/mcp_config/runtime.py` 与 `backend/app/services/mcp_config/service.py`
|
||||||
|
- 路由记忆样本模型:参考 `backend/app/models/query_route_memory.py`
|
||||||
|
- Internal Query `Operation Contract / Target Resolver / DSL / verifier` 主链:参考 `backend/app/services/adaptive_rag_service/mixin_target_resolution.py`、`mixin_answer_mainline.py`、`mixin_deterministic_query_core.py`、`mixin_answer_payload.py`
|
||||||
|
- Semantic facts 读模型与抽取服务:参考 `backend/app/models/semantic_fact.py`、`backend/app/models/semantic_fact_extraction_run.py`、`backend/app/services/semantic_fact_service/`
|
||||||
|
- Semantic facts 自动入队钩子:参考 `backend/app/services/semantic_fact_service/hooks.py`
|
||||||
|
- Semantic facts 异步事件消费:参考 `backend/app/services/project_knowledge_graph_service/mixin_event_queue.py`
|
||||||
|
- Semantic facts 治理后台接口:参考 `backend/app/api/semantic_facts_api/routes.py`
|
||||||
- KG 上下文闭环接口:参考 `backend/app/api/knowledge_graph.py`
|
- KG 上下文闭环接口:参考 `backend/app/api/knowledge_graph.py`
|
||||||
- 任务包中心接口:参考 `backend/app/api/task_briefs.py`
|
- 任务包中心接口:参考 `backend/app/api/task_briefs.py`
|
||||||
|
- MCP 模板接入中心接口:参考 `backend/app/api/mcp_api/routes.py`
|
||||||
- AI PMO 恢复接口:参考 `POST /ai-pmo/work-items/{work_item_id}/resume`
|
- AI PMO 恢复接口:参考 `POST /ai-pmo/work-items/{work_item_id}/resume`
|
||||||
- AI PMO 草稿决策接口:参考 `POST /ai-pmo/work-items/{work_item_id}/draft-decision`
|
- AI PMO 草稿决策接口:参考 `POST /ai-pmo/work-items/{work_item_id}/draft-decision`
|
||||||
- AI PMO 审批接口:参考 `POST /ai-pmo/work-items/{work_item_id}/decision`
|
- AI PMO 审批接口:参考 `POST /ai-pmo/work-items/{work_item_id}/decision`
|
||||||
|
|
@ -60,6 +71,11 @@
|
||||||
- Scheduler Center / Mounted AI Console:参考 `frontend/src/pages/scheduler-center/`
|
- Scheduler Center / Mounted AI Console:参考 `frontend/src/pages/scheduler-center/`
|
||||||
- SSTC 工作流组件:参考 `frontend/src/components/sstc/`
|
- SSTC 工作流组件:参考 `frontend/src/components/sstc/`
|
||||||
- Model Routing 页面:参考 `frontend/src/pages/model-routing-config/`
|
- Model Routing 页面:参考 `frontend/src/pages/model-routing-config/`
|
||||||
|
- MCP 模板接入页:参考 `frontend/src/pages/ai-config/mcp-config/`
|
||||||
|
- Semantic facts 治理页:参考 `frontend/src/pages/semantic-facts-board/`
|
||||||
|
- MCP 前端 API:参考 `frontend/src/api/mcp.ts`
|
||||||
|
- Semantic facts 前端 API:参考 `frontend/src/api/semanticFacts.ts`
|
||||||
|
- AI Config 治理工具入口:参考 `frontend/src/pages/ai-config/constants.ts` 与 `frontend/src/router/index.tsx`
|
||||||
|
|
||||||
## 4. 模型与配置真源
|
## 4. 模型与配置真源
|
||||||
- 模型配置唯一来源:参考 `backend/app/services/model/model_config.py`
|
- 模型配置唯一来源:参考 `backend/app/services/model/model_config.py`
|
||||||
|
|
@ -67,6 +83,7 @@
|
||||||
- Prompt 聚合:`backend/app/config/ai_prompts.py`
|
- Prompt 聚合:`backend/app/config/ai_prompts.py`
|
||||||
- Prompt 场景化分片:`backend/app/config/ai_prompts_catalog/`
|
- Prompt 场景化分片:`backend/app/config/ai_prompts_catalog/`
|
||||||
- Prompt registry:`backend/app/config/ai_prompt_registry.py`
|
- Prompt registry:`backend/app/config/ai_prompt_registry.py`
|
||||||
|
- External Query Planner Prompt:`backend/app/config/ai_prompts.py` 中的 `PROMPT_EXTERNAL_QUERY_PLAN`
|
||||||
|
|
||||||
## 5. 维护建议
|
## 5. 维护建议
|
||||||
- 新增接口时:先更新本文档,再同步 `API_CONTRACTS.md`
|
- 新增接口时:先更新本文档,再同步 `API_CONTRACTS.md`
|
||||||
|
|
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Loading…
Reference in New Issue