# PMFlow Full Adaptive RAG

**Goal:** Questions asked in Feishu or PMFlow get answered directly by AI. Questions never enter the decision queue.  
**Knowledge coverage:** Project data + document library + Feishu history + PMFlow internal records + external URLs  
**Answer entry points:** Feishu direct reply + PMFlow AI Steward chat

## Progress Log

- 2026-03-02: 已完成文档页码来源引用（PDF 上传保留 page marker 并写入 chunk `page_number`，检索标签显示 `p.xx`）。
- 2026-03-02: 已完成 Feishu/PMFlow 向量检索项目组隔离（按 `project_id` 严格过滤，避免跨项目数据泄漏），并在 Feishu 问答链路接入群上下文项目推断。
- 2026-03-02: 已完成 Feishu 直答深链输出（命中 PMFlow 记录时追加 `View details → /projects/...`）。
- 2026-03-02: 已完成检索阶段时延预算控制（DB 查询超时 5s、向量检索超时 10s，超时自动降级且返回检索指标）。
- 2026-03-02: 已完成 QA Records 统计聚合接入（`/rag/qa-records/stats` 前端展示）、时间范围筛选（本周/近30天/全部）与 `corrected` 过滤。
- 2026-03-02: 已完成 Alembic 回填 `add_adaptive_rag_tables`，新增 `knowledge_base_docs / knowledge_base_chunks / qa_records / qa_feedback` 及核心索引。
- 2026-03-02: 已完成 Feishu `question/mixed` 分流直答与 AI Steward RAG 问答入口，前端已接 QA Records tab。
- 2026-03-02: 已完成 PMFlow 写时向量化（charter/task/review create/update）异步回填 `knowledge_base_chunks(source_type=pmflow)`。
- 2026-03-02: 已完成知识库管理首版（文件上传解析、URL新增与手动刷新、processing状态3秒轮询、删除级联向量）。
- 2026-03-02: 已完成 URL 周期回抓调度（默认每168小时）并提供手动触发接口。

---

## Architecture

```
User asks question (Feishu or PMFlow)
        ↓
Intent classification (question / statement)
        ↓
        ├─ statement → existing signal processing flow (unchanged)
        │
        └─ question → Router selects knowledge domain(s)
                        ├─ Project structured data → direct DB query
                        ├─ Document library        → vector search
                        ├─ Feishu history          → vector search
                        ├─ PMFlow records          → vector search
                        └─ External URLs           → fetch + vector search
                            ↓
                        GLM-4 generates answer
                            ↓
                        Feishu reply / AI Steward chat display
```

---

## Part 1 — Intent Classification

Add as the first step in the existing AI pipeline, before any other processing.

- [x] Classify every incoming Feishu message:
  - `question` — interrogative, query, "does X exist", "how to", "why", "who is"
  - `statement` — requirement description, project intent, signal content
  - `mixed` — contains both question and statement, split and handle separately
- [x] Classify every incoming Feishu message:
  - `question` — interrogative, query, "does X exist", "how to", "why", "who is"
  - `statement` — requirement description, project intent, signal content
  - `mixed` — contains both question and statement, split and handle separately
- [x] `question` → RAG flow, reply directly in Feishu, **never enters decision queue**
- [x] `statement` → existing signal processing flow, unchanged
- [x] `mixed` → RAG answers the question part, signal queue handles the statement part. Feishu reply includes: "I've answered your question and logged the requirement as a signal."

---

## Part 2 — Knowledge Domains

### Domain 1 — Project structured data (direct DB query, no vectors)

Real-time queries. Not vectorized — ensures data is always current.

| Queryable content | Example questions |
|------------------|------------------|
| Project list / detail / status | "Is there a customer service project?" "What is project #5?" |
| Signal source | "What signals are pending?" "How many P0 signals are there?" |
| Charter content / versions | "What is the goal in the invoicing system charter?" |
| Task progress | "How many tasks are unfinished?" "Who is working on the API?" |
| Team workload | "What's Zhang San's current workload?" |

- [x] Build `db_query_tool`: AI automatically determines which tables to query, generates SQL or calls existing APIs
- [x] Results fed directly to GLM-4 to generate natural language answer

### Domain 2 — Document library (vector search)

User-uploaded documents, specs, templates, static knowledge:

- [x] Supported formats: PDF / Word / Markdown / TXT
- [x] On upload: auto-chunk (chunk size 500 tokens, overlap 50), auto-vectorize
- [x] Store vectors in pgvector (reuse existing PostgreSQL, no new DB)
- [x] Search returns top-5 relevant chunks + source filename + page number

**New DB tables:**
```
knowledge_base_docs: id / title / file_type / uploaded_by / created_at
knowledge_base_chunks: id / doc_id / content / embedding / chunk_index / metadata
```

### Domain 3 — Feishu message history (vector search)

All received Feishu messages including filtered and archived ones:

- [x] Every Feishu message vectorized asynchronously on receipt (does not block main pipeline)
- [x] Search filterable by project group to prevent cross-project data leakage

### Domain 4 — PMFlow internal records (vector search)

Charter content, retrospective records, task descriptions:

- [x] Auto-vectorize on charter create/update
- [x] Auto-vectorize on retrospective record write
- [x] Task description vectorization (optional, enable when volume is high)

### Domain 5 — External URLs (fetch + vector search)

- [x] User can submit external URLs in knowledge base management page
- [x] On submission: `web_fetch` retrieves content → auto-chunk → vectorize → stored in `knowledge_base_chunks` with `source_type = url`
- [x] Re-fetch scheduled weekly to keep content fresh (or manual refresh button)
- [x] Supported: public documentation pages, Notion links, Google Docs (public), GitHub READMEs（当前按公开 URL 通用抓取）

---

## Part 3 — RAG Query Flow

### Router (domain selection)

- [x] AI determines which domain(s) apply to the question (multiple allowed):
  - Contains project / signal / task / charter / progress / workload keywords → `db_query_tool`
  - Contains document / spec / how-to / process keywords → document library search
  - Contains "previously said" / "last time" / "history" → Feishu history search
  - Contains retrospective / charter content / task description → PMFlow records search
  - Contains URL or "according to [site]" → external URL search
  - Ambiguous → parallel search across all vector domains, merge results

### Retrieve → Generate flow

```
Question
  ↓
Router (domain selection)
  ↓
Parallel retrieval (DB query + vector searches across selected domains)
  ↓
Context assembly:
  - Original question
  - Retrieved results (with source labels)
  - Current project context (if question asked inside a project group)
  - Last 10 turns of conversation (for follow-up questions)
  ↓
GLM-4 generates answer
  ↓
Answer + source citations
```

- [x] If retrieval relevance score < 0.75: AI responds "I couldn't find relevant information, please provide more context"
- [x] Never fabricate answers. If uncertain, explicitly state uncertainty.
- [x] Max context window per query: 8000 tokens (trim oldest retrieved chunks if exceeded)

---

## Part 4 — Answer Entry Points

### Entry point 1 — Feishu direct reply

- [x] Feishu bot receives question → RAG processing → reply directly under original message
- [x] Reply format:
  ```
  [AI answer]

  Source: Project database / Document [filename] / Feishu history / [URL]
  ```
- [x] If answer references a specific PMFlow page, append deep link "View details →"
- [x] Response time target: < 5s for DB queries, < 10s for vector searches

### Entry point 2 — PMFlow AI Steward chat

- [x] Reuse existing AI Steward page, add RAG capability
- [x] Multi-turn conversation, retain last 10 turns of context
- [x] Each answer shows source citation inline
- [x] Follow-up questions work: "What's the charter for that project you just mentioned?" → AI resolves reference from conversation history
- [x] Show "Searching knowledge base..." indicator while RAG is running

---

## Part 5 — Knowledge Base Management Page

Add "知识库 / Knowledge Base" tab to existing 智能配置 (AI Config) page:

- [x] File upload zone (drag & drop or click to browse)
- [x] Uploaded files list: filename / type / upload time / vectorization status (processing / complete / failed)
- [x] Delete file → also deletes all associated chunks and vectors
- [x] URL submission: input field + "Add URL" button, shows URL list with last-fetched time + manual refresh button
- [x] Vectorization status shown in real time (poll every 3s while processing)

---

## Implementation Order

| Week | Deliverables |
|------|-------------|
| 1 | Intent classification (question/statement split). `db_query_tool`. Structured project data queries. Feishu direct reply. |
| 2 | pgvector setup. Document library: upload / chunk / vectorize / search. Knowledge base management page. |
| 3 | Feishu history + PMFlow records vectorization. External URL fetch + vectorize. Multi-domain Router. Parallel search + result merge. Source citations. |
| 4 | AI Steward multi-turn RAG integration. Follow-up context linking. End-to-end test across all 5 domains. |

---

## Out of scope (this version)

- Voice-to-text input
- Image / attachment content understanding (Feishu image messages)
- Per-user knowledge base access control
- Self-hosted embedding model (use API)
- Real-time web search (not the same as indexed external URLs)

---

## Part 6 — QA Recording & Feedback (Training Data Collection)

**Placement:** New "QA Records" tab in AI Steward page, between "Daily Brief" and "Engine Config".

### Backend

**New DB tables:**
```
qa_records:
  id / question / answer / sources (JSON array) / 
  intent / domains_used (JSON array) /
  response_time_ms / asked_by / asked_via (feishu/pmflow) /
  project_id (nullable) / created_at

qa_feedback:
  id / qa_id / 
  rating (positive/negative) /
  helpfulness (fully/partially/could_be_better) — for positive ratings /
  error_types (JSON array: wrong_answer/wrong_source/incomplete/off_topic/other) — for negative /
  correct_answer (text, optional) /
  submitted_by / created_at
```

- [x] Auto-write `qa_records` on every RAG answer (both Feishu reply and AI Steward chat)
- [x] `POST /api/qa-records/{id}/feedback` — submit rating + structured feedback
- [x] `GET /api/qa-records?project_id=&date_from=&date_to=&rating=` — paginated list with filters
- [x] `GET /api/qa-records/export` — export as JSONL for model fine-tuning (admin only)

**JSONL export format:**
```json
{"prompt": "有客服工具的项目吗？", "response": "有，一个AI客服工具项目，进行中", "rating": "positive", "helpfulness": "fully"}
{"prompt": "开票系统宪章目标是什么？", "response": "...", "rating": "negative", "correct_answer": "...", "error_type": "incomplete"}
```

- [x] Export only includes records with feedback (unrated excluded by default, optional flag to include)
- [x] Admin can filter export by date range, rating, domain, project

### Frontend — QA Records Tab

**Top bar:**
```
[ Search questions... ]  [ Export JSONL ]  [ All ratings ▾ ]  [ This week ▾ ]
```

**Each record card:**
```
┌─────────────────────────────────────────────────┐
│ 14:23  有客服工具的项目吗？          飞书 · #5项目 │
│ AI：有，一个AI客服工具项目，进行中               │
│ 来源：项目数据库                    耗时 2.3s    │
│                                                 │
│ 👍 有用    👎 有问题    ✏️ 修正                   │
└─────────────────────────────────────────────────┘
```

**On 👍 click — helpfulness picker (non-blocking, slides in below card):**
```
这条回答对你有多大帮助？
● 完全解决了问题   ○ 部分有用   ○ 有用但可以更好
[跳过]  [提交]
```

**On 👎 click — structured feedback form (slides in below card):**
```
这条回答哪里有问题？（可多选）
☐ 答案错误    ☐ 来源错误    ☐ 回答不完整
☐ 答非所问    ☐ 其他

正确答案应该是：（选填，填了训练权重更高）
┌──────────────────────────────┐
│                              │
└──────────────────────────────┘
[提交]
```

- [x] After submitting feedback: card shows "✅ 已评价" badge, feedback form collapses
- [x] Cards with correct_answer filled: highlighted with green left border (highest value training data)
- [x] Filter by: all / positive / negative / corrected / unrated

### Stats summary (top of QA Records tab)

```
总问答  342   👍 已评价  89%   平均帮助度  4.2/5   可导出训练对  156
```

- [x] `可导出训练对` = records with feedback (both positive with helpfulness + negative with or without correction)