linyuhang
/
PMFlow
2
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
PMFlow/mdproject/ai-visibility-panel-plan.md

7.0 KiB
Raw Blame History

AI Processing Visibility Panel — Implementation Plan

Goal: Make AI processing fully observable. Know if AI is running, what it output, why it routed each item.
Placement: Above signal source list on SSTC board. Collapsible. Expanded by default in dev mode.
Scope: Frontend panel + backend logging endpoints. No new pages.

Backend

1. Logging (add to existing AI job)

Every Feishu message processed must write a log record:

  • Create ai_processing_logs table:

    id
feishu_message_id
feishu_item_id (FK)
raw_message          -- original Feishu text
ai_raw_output        -- full JSON from LLM
project_score        -- float 0–1
signal_confidence    -- float 0–1
level                -- "project" or "signal"
routing_decision     -- "auto_create_project" / "auto_enter_signal" / "pending_low_confidence" / "pending_cold_start" / "pending_uncertain_project" / "duplicate_filtered"
routing_reason       -- plain text, 1 sentence
reasoning_bullets    -- JSON array, 3 items
processing_status    -- "success" / "failed" / "processing"
error_message        -- null if success
duration_ms          -- processing time
created_at

  • On duplicate detected (feishu_message_id already exists): write log with routing_decision = "duplicate_filtered", do not process further

  • On AI job failure: write log with processing_status = "failed" + error_message

2. API endpoints

  • GET /api/ai-logs?project_id=&limit=20&offset=0 — paginated log list for a project
  • GET /api/ai-logs/stats?project_id=&date=today — daily stats summary
  • GET /api/ai-logs/{id} — full detail for one log record
  • GET /api/ai-logs/latest?project_id= — most recent log record (for real-time status)

Frontend

Panel structure

Collapsible section above the signal source table. Toggle button: "AI 处理状态 ▾"
Default state: collapsed in production, expanded in dev (import.meta.env.DEV).

Three blocks inside:

Block 1 — Latest processing status (real-time)

Polls GET /api/ai-logs/latest every 10 seconds.

┌─────────────────────────────────────────────────────┐
│ 最近处理                                             │
│                                                     │
│ ✅ 已处理   14:23:01   耗时 3.2s                     │
│ 原始消息：新增个需求，关于做饭食谱...（点击展开）      │
│                                                     │
│ project_score: 0.31  →  信号级别                    │
│ confidence:    0.62  →  低于阈值，进待处理队列        │
│ 路由：冷启动期 / 低置信度                            │
└─────────────────────────────────────────────────────┘

Status indicator colors:

  • ✅ green — success

  • ⏳ yellow — processing (AI job in flight)

  • ❌ red — failed, shows error message

  • Build LatestProcessingStatus component

  • Poll /api/ai-logs/latest every 10s, stop polling if tab not focused

  • Show raw message truncated to 1 line, click to expand full text

  • Show project_score, signal_confidence, routing_decision, routing_reason

  • Show duration_ms formatted as seconds

Block 2 — Today's stats

Fetches GET /api/ai-logs/stats once on mount, refetches when new log arrives.

┌─────────────────────────────────────────────────────┐
│ 今日统计                                             │
│                                                     │
│ 接收 12  │  AI处理成功 10  │  失败 2  │  重复过滤 2  │
│ 自动入库 3  │  待处理 7                              │
│ 平均 confidence: 0.71  │  平均 project_score: 0.28  │
└─────────────────────────────────────────────────────┘
  • Build DailyStats component
  • Display all 7 counters in a single row of stat chips
  • Highlight failed count in red if > 0
  • Show average scores to 2 decimal places

Block 3 — Processing log list

Fetches GET /api/ai-logs?project_id= with pagination. Shows last 20 records.

Each row:

时间        状态   路由决策              confidence  project_score  操作
14:23:01   ✅     pending_low_conf      0.62        0.31          [详情]
12:06:25   ✅     auto_enter_signal     0.83        0.29          [详情]
12:00:11   ❌     failed                —           —             [详情]
11:39:27   ✅     duplicate_filtered    —           —             [详情]
  • Build ProcessingLogTable component
  • Columns: time / status icon / routing decision (translated to Chinese label) / confidence / project_score / detail button
  • routing_decision label map:
    • auto_create_project → 🏗 自动创建项目
    • auto_enter_signal → ✅ 自动入库
    • pending_low_confidence → ⏸ 低置信度待处理
    • pending_cold_start → ⏸ 冷启动期待处理
    • pending_uncertain_project → ⏸ 项目归属不确定
    • duplicate_filtered → 🔁 重复过滤
    • failed → ❌ 处理失败
  • Click [详情] → opens a right-side drawer showing:
    • Raw Feishu message (full text)
    • AI raw output (JSON, formatted)
    • All scores and reasoning bullets
    • Error message if failed
  • Pagination: load more button at bottom (not infinite scroll)
  • Row color: red background if processing_status = failed

Routing decision labels (used in both backend logs and frontend display)

routing_decision value Meaning
auto_create_project project_score ≥ 0.85, cold start over, project auto-created
auto_enter_signal confidence ≥ 0.80, cold start over, signal auto-entered
pending_low_confidence confidence < 0.80, queued for review
pending_cold_start cold start period active, all items queued regardless of score
pending_uncertain_project project_score 0.60–0.84, queued for user to confirm
duplicate_filtered feishu_message_id already processed, dropped
failed AI job threw an error

Implementation order

Task When
ai_processing_logs table + write logs in AI job First, before anything else — need data to display
4 API endpoints After table exists
DailyStats component Quick win, shows immediately if AI is running
LatestProcessingStatus component Core visibility, shows scores + routing
ProcessingLogTable + detail drawer Last, most complex

Out of scope

  • Alerts / notifications when AI fails (add later)
  • Log retention policy / auto-delete (add later)
  • Exporting logs (add later)