13 KiB

Raw Permalink Blame History

PMFlow Full Adaptive RAG

Goal: Questions asked in Feishu or PMFlow get answered directly by AI. Questions never enter the decision queue.
Knowledge coverage: Project data + document library + Feishu history + PMFlow internal records + external URLs
Answer entry points: Feishu direct reply + PMFlow AI Steward chat

Progress Log

2026-03-02: 已完成文档页码来源引用（PDF 上传保留 page marker 并写入 chunk page_number，检索标签显示 p.xx）。
2026-03-02: 已完成 Feishu/PMFlow 向量检索项目组隔离（按 project_id 严格过滤，避免跨项目数据泄漏），并在 Feishu 问答链路接入群上下文项目推断。
2026-03-02: 已完成 Feishu 直答深链输出（命中 PMFlow 记录时追加 View details → /projects/...）。
2026-03-02: 已完成检索阶段时延预算控制（DB 查询超时 5s、向量检索超时 10s，超时自动降级且返回检索指标）。
2026-03-02: 已完成 QA Records 统计聚合接入（/rag/qa-records/stats 前端展示）、时间范围筛选（本周/近30天/全部）与 corrected 过滤。
2026-03-02: 已完成 Alembic 回填 add_adaptive_rag_tables，新增 knowledge_base_docs / knowledge_base_chunks / qa_records / qa_feedback 及核心索引。
2026-03-02: 已完成 Feishu question/mixed 分流直答与 AI Steward RAG 问答入口，前端已接 QA Records tab。
2026-03-02: 已完成 PMFlow 写时向量化（charter/task/review create/update）异步回填 knowledge_base_chunks(source_type=pmflow)。
2026-03-02: 已完成知识库管理首版（文件上传解析、URL新增与手动刷新、processing状态3秒轮询、删除级联向量）。
2026-03-02: 已完成 URL 周期回抓调度（默认每168小时）并提供手动触发接口。

Architecture

User asks question (Feishu or PMFlow)
        ↓
Intent classification (question / statement)
        ↓
        ├─ statement → existing signal processing flow (unchanged)
        │
        └─ question → Router selects knowledge domain(s)
                        ├─ Project structured data → direct DB query
                        ├─ Document library        → vector search
                        ├─ Feishu history          → vector search
                        ├─ PMFlow records          → vector search
                        └─ External URLs           → fetch + vector search
                            ↓
                        GLM-4 generates answer
                            ↓
                        Feishu reply / AI Steward chat display

Part 1 — Intent Classification

Add as the first step in the existing AI pipeline, before any other processing.

Classify every incoming Feishu message:
- question — interrogative, query, "does X exist", "how to", "why", "who is"
- statement — requirement description, project intent, signal content
- mixed — contains both question and statement, split and handle separately
Classify every incoming Feishu message:
- question — interrogative, query, "does X exist", "how to", "why", "who is"
- statement — requirement description, project intent, signal content
- mixed — contains both question and statement, split and handle separately
question → RAG flow, reply directly in Feishu, never enters decision queue
statement → existing signal processing flow, unchanged
mixed → RAG answers the question part, signal queue handles the statement part. Feishu reply includes: "I've answered your question and logged the requirement as a signal."

Part 2 — Knowledge Domains

Domain 1 — Project structured data (direct DB query, no vectors)

Real-time queries. Not vectorized — ensures data is always current.

Queryable content	Example questions
Project list / detail / status	"Is there a customer service project?" "What is project #5?"
Signal source	"What signals are pending?" "How many P0 signals are there?"
Charter content / versions	"What is the goal in the invoicing system charter?"
Task progress	"How many tasks are unfinished?" "Who is working on the API?"
Team workload	"What's Zhang San's current workload?"

Build db_query_tool: AI automatically determines which tables to query, generates SQL or calls existing APIs
Results fed directly to GLM-4 to generate natural language answer

Domain 2 — Document library (vector search)

User-uploaded documents, specs, templates, static knowledge:

Supported formats: PDF / Word / Markdown / TXT
On upload: auto-chunk (chunk size 500 tokens, overlap 50), auto-vectorize
Store vectors in pgvector (reuse existing PostgreSQL, no new DB)
Search returns top-5 relevant chunks + source filename + page number

New DB tables:

knowledge_base_docs: id / title / file_type / uploaded_by / created_at
knowledge_base_chunks: id / doc_id / content / embedding / chunk_index / metadata

Domain 3 — Feishu message history (vector search)

All received Feishu messages including filtered and archived ones:

Every Feishu message vectorized asynchronously on receipt (does not block main pipeline)
Search filterable by project group to prevent cross-project data leakage

Domain 4 — PMFlow internal records (vector search)

Charter content, retrospective records, task descriptions:

Auto-vectorize on charter create/update
Auto-vectorize on retrospective record write
Task description vectorization (optional, enable when volume is high)

Domain 5 — External URLs (fetch + vector search)

User can submit external URLs in knowledge base management page
On submission: web_fetch retrieves content → auto-chunk → vectorize → stored in knowledge_base_chunks with source_type = url
Re-fetch scheduled weekly to keep content fresh (or manual refresh button)
Supported: public documentation pages, Notion links, Google Docs (public), GitHub READMEs（当前按公开 URL 通用抓取）

Part 3 — RAG Query Flow

Router (domain selection)

AI determines which domain(s) apply to the question (multiple allowed):
- Contains project / signal / task / charter / progress / workload keywords → db_query_tool
- Contains document / spec / how-to / process keywords → document library search
- Contains "previously said" / "last time" / "history" → Feishu history search
- Contains retrospective / charter content / task description → PMFlow records search
- Contains URL or "according to [site]" → external URL search
- Ambiguous → parallel search across all vector domains, merge results

Retrieve → Generate flow

Question
  ↓
Router (domain selection)
  ↓
Parallel retrieval (DB query + vector searches across selected domains)
  ↓
Context assembly:
  - Original question
  - Retrieved results (with source labels)
  - Current project context (if question asked inside a project group)
  - Last 10 turns of conversation (for follow-up questions)
  ↓
GLM-4 generates answer
  ↓
Answer + source citations

If retrieval relevance score < 0.75: AI responds "I couldn't find relevant information, please provide more context"
Never fabricate answers. If uncertain, explicitly state uncertainty.
Max context window per query: 8000 tokens (trim oldest retrieved chunks if exceeded)

Part 4 — Answer Entry Points

Entry point 1 — Feishu direct reply

Feishu bot receives question → RAG processing → reply directly under original message

Reply format:

[AI answer]

Source: Project database / Document [filename] / Feishu history / [URL]

If answer references a specific PMFlow page, append deep link "View details →"
Response time target: < 5s for DB queries, < 10s for vector searches

Entry point 2 — PMFlow AI Steward chat

Reuse existing AI Steward page, add RAG capability
Multi-turn conversation, retain last 10 turns of context
Each answer shows source citation inline
Follow-up questions work: "What's the charter for that project you just mentioned?" → AI resolves reference from conversation history
Show "Searching knowledge base..." indicator while RAG is running

Part 5 — Knowledge Base Management Page

Add "知识库 / Knowledge Base" tab to existing 智能配置 (AI Config) page:

File upload zone (drag & drop or click to browse)
Uploaded files list: filename / type / upload time / vectorization status (processing / complete / failed)
Delete file → also deletes all associated chunks and vectors
URL submission: input field + "Add URL" button, shows URL list with last-fetched time + manual refresh button
Vectorization status shown in real time (poll every 3s while processing)

Implementation Order

Week	Deliverables
1	Intent classification (question/statement split). `db_query_tool`. Structured project data queries. Feishu direct reply.
2	pgvector setup. Document library: upload / chunk / vectorize / search. Knowledge base management page.
3	Feishu history + PMFlow records vectorization. External URL fetch + vectorize. Multi-domain Router. Parallel search + result merge. Source citations.
4	AI Steward multi-turn RAG integration. Follow-up context linking. End-to-end test across all 5 domains.

Out of scope (this version)

Voice-to-text input
Image / attachment content understanding (Feishu image messages)
Per-user knowledge base access control
Self-hosted embedding model (use API)
Real-time web search (not the same as indexed external URLs)

Part 6 — QA Recording & Feedback (Training Data Collection)

Placement: New "QA Records" tab in AI Steward page, between "Daily Brief" and "Engine Config".

Backend