139 lines
8.2 KiB
Markdown
139 lines
8.2 KiB
Markdown
# External News Evidence Upgrade Implementation Plan
|
|
|
|
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
|
|
|
**Goal:** Make external freshness/news questions answerable with article-grade evidence, while keeping the current fail-closed external contract and without adding runtime keyword tables or business regex routing.
|
|
|
|
**Architecture:** Keep `route_mode=external` and the current verifier/presenter boundary, but strengthen the evidence acquisition stage. The next pass should expand search candidates into real article pages, normalize article metadata into structured evidence, separate `attempted_sources` from `answer_evidence`, and only let the presenter see customer-safe, article-backed sources.
|
|
|
|
**Tech Stack:** Python 3.11, FastAPI, PMFlow AdaptiveRAG, Pydantic, pytest, httpx, HTMLParser/JSON-LD metadata
|
|
|
|
---
|
|
|
|
### Task 1: Lock The Next-Stage News Retrieval Contract With Tests
|
|
|
|
**Files:**
|
|
- Modify: `backend/tests/test_mcp_external_query_bridge.py`
|
|
- Modify: `backend/tests/test_external_answer_presenter_contract.py`
|
|
- Modify: `backend/tests/test_adaptive_rag_mainline_contract.py`
|
|
|
|
- [ ] **Step 1: Add a failing test for live-search candidate expansion into article pages**
|
|
- Simulate a search result that points to a category page plus one resolvable article page.
|
|
- Expect the adapter/gateway to keep only the article page as `verified_evidence`.
|
|
|
|
- [ ] **Step 2: Add a failing test that `sources_used` only includes answer evidence**
|
|
- Feed one article page and two discarded candidates.
|
|
- Expect customer-visible `sources_used` / `来源摘要` to exclude discarded candidates.
|
|
|
|
- [ ] **Step 3: Add a failing test for absolute-date news output**
|
|
- Feed `published_at`.
|
|
- Expect the presenter prompt/output contract to use `YYYY-MM-DD` or `日期未明确`, never relative time like `3天前`.
|
|
|
|
- [ ] **Step 4: Run the targeted tests to confirm they fail for the right reason**
|
|
- Run: `PYTHONPATH=backend backend/venv/bin/python -m pytest backend/tests/test_mcp_external_query_bridge.py -v`
|
|
- Run: `PYTHONPATH=backend backend/venv/bin/python -m pytest backend/tests/test_external_answer_presenter_contract.py -v`
|
|
- Run: `PYTHONPATH=backend backend/venv/bin/python -m pytest backend/tests/test_adaptive_rag_mainline_contract.py -v`
|
|
|
|
### Task 2: Add Article Expansion And Structured Candidate Hydration
|
|
|
|
**Files:**
|
|
- Create: `backend/app/services/external_query_service/article_expander.py`
|
|
- Modify: `backend/app/services/external_query_service/page_metadata.py`
|
|
- Modify: `backend/app/services/external_query_service/adapters/live_search.py`
|
|
- Modify: `backend/app/services/external_query_service/adapters/mcp.py`
|
|
- Modify: `backend/app/services/external_query_service/schemas.py`
|
|
- Modify: `backend/app/services/adaptive_rag_service/mixin_conversation_utils_a_p1.py`
|
|
|
|
- [ ] **Step 1: Add a focused article-expansion helper**
|
|
- Accept one fetched page payload.
|
|
- Extract canonical/linked article candidates from structured HTML signals only (`link rel=canonical`, JSON-LD, standard anchors), not business keyword heuristics.
|
|
|
|
- [ ] **Step 2: Extend evidence schemas for staged acquisition**
|
|
- Add fields such as `evidence_stage`, `attempted_url`, `canonical_url`, `article_url`, and `selection_reason`.
|
|
- Keep `display_label/display_summary/page_kind/published_at` as the customer-safe surface.
|
|
|
|
- [ ] **Step 3: Teach live-search hydration to do a second hop when first-page evidence is landing/category**
|
|
- Fetch the top candidate page.
|
|
- If it is not article-grade, expand one bounded hop to article candidates and fetch those instead.
|
|
- Keep the fetch count bounded.
|
|
|
|
- [ ] **Step 4: Reuse the same bounded second-hop logic for MCP target URLs**
|
|
- MCP search can still return landing/detail wrappers.
|
|
- Hydration should converge on article-grade evidence without changing upstream semantics.
|
|
|
|
- [ ] **Step 5: Re-run the targeted news retrieval tests**
|
|
- Run: `PYTHONPATH=backend backend/venv/bin/python -m pytest backend/tests/test_mcp_external_query_bridge.py -v`
|
|
|
|
### Task 3: Separate Attempted Sources From Answer Evidence End-To-End
|
|
|
|
**Files:**
|
|
- Modify: `backend/app/services/external_query_service/gateway.py`
|
|
- Modify: `backend/app/services/adaptive_rag_service/mixin_answer_retrieval.py`
|
|
- Modify: `backend/app/services/adaptive_rag_service/mixin_answer_payload.py`
|
|
- Modify: `backend/app/services/adaptive_rag_service/mixin_query_route_gateway.py`
|
|
|
|
- [ ] **Step 1: Persist `adapter_attempts` and discarded candidates as debug-only metadata**
|
|
- Do not expose them in customer-visible `sources_used`.
|
|
|
|
- [ ] **Step 2: Make `execution.evidence` represent only selected answer evidence**
|
|
- Keep attempted/discarded candidates in `meta`.
|
|
- Preserve current fail-closed semantics when no answer evidence remains.
|
|
|
|
- [ ] **Step 3: Build customer-visible sources from `display_label` only**
|
|
- `sources_used` / `来源摘要` should come from selected answer evidence.
|
|
- `raw_source_label` stays admin/debug only.
|
|
|
|
- [ ] **Step 4: Re-run the answer payload and route-contract tests**
|
|
- Run: `PYTHONPATH=backend backend/venv/bin/python -m pytest backend/tests/test_adaptive_rag_mainline_contract.py -v`
|
|
|
|
### Task 4: Tighten The News Presentation Contract Without Relaxing Safety
|
|
|
|
**Files:**
|
|
- Modify: `backend/app/services/external_query_service/presenter.py`
|
|
- Modify: `backend/app/config/ai_prompts_catalog/part_scenes_06.py`
|
|
- Modify: `backend/app/config/ai_prompts.py`
|
|
- Modify: `backend/app/config/ai_prompt_registry.py`
|
|
|
|
- [ ] **Step 1: Keep `external_news_brief` as the only news presenter contract**
|
|
- Do not fall back to generic summary style for news/freshness questions.
|
|
|
|
- [ ] **Step 2: Force date rendering through structured fields**
|
|
- Use `published_at -> YYYY-MM-DD`.
|
|
- If absent, explicitly render `日期未明确`.
|
|
|
|
- [ ] **Step 3: Keep the prompt surface customer-safe**
|
|
- Evidence blocks may include `display_label`, `display_summary`, `title`, `published_at`, `target_url`.
|
|
- Never feed `URL[...]`, `MCP[...]`, or `raw_source_label`.
|
|
|
|
- [ ] **Step 4: Re-run presenter and prompt-registry tests**
|
|
- Run: `PYTHONPATH=backend backend/venv/bin/python -m pytest backend/tests/test_external_answer_presenter_contract.py -v`
|
|
- Run: `PYTHONPATH=backend backend/venv/bin/python -m pytest backend/tests/test_ai_prompt_registry_contract.py -v`
|
|
|
|
### Task 5: Verify On The Real Backend And Sync Runtime Docs
|
|
|
|
**Files:**
|
|
- Modify: `CLAUDE.md`
|
|
- Modify: `docs/architecture/ARCHITECTURE_INVENTORY.md`
|
|
- Modify: `docs/architecture/CODE_REFERENCE_INDEX.md`
|
|
- Modify: `docs/architecture/RUNTIME_GUARDRAILS.md`
|
|
- Modify: `docs/architecture/DATAFLOW.md`
|
|
|
|
- [ ] **Step 1: Update docs for second-hop article expansion and answer-evidence separation**
|
|
- Document that `news` grounding now requires article-grade evidence.
|
|
- Document that attempted sources and answer sources are different surfaces.
|
|
|
|
- [ ] **Step 2: Run compile + targeted regression**
|
|
- Run: `PYTHONPATH=backend /opt/homebrew/bin/python3 -m py_compile backend/app/services/external_query_service/article_expander.py backend/app/services/external_query_service/page_metadata.py backend/app/services/external_query_service/adapters/live_search.py backend/app/services/external_query_service/adapters/mcp.py backend/app/services/external_query_service/gateway.py backend/app/services/external_query_service/presenter.py backend/app/services/adaptive_rag_service/mixin_answer_retrieval.py backend/app/services/adaptive_rag_service/mixin_answer_payload.py backend/app/services/adaptive_rag_service/mixin_query_route_gateway.py`
|
|
|
|
- [ ] **Step 3: Rebuild the backend container so Python changes really load**
|
|
- Run: `docker compose up -d --build backend`
|
|
|
|
- [ ] **Step 4: Live-check a real news query and capture the raw JSON**
|
|
- Ask: `最近 AI 行业有哪些值得关注的新闻？`
|
|
- Expected:
|
|
- Best case: grounded answer with article-grade sources and customer-safe labels.
|
|
- Safe fallback: explicit `insufficient_evidence`, but no machine labels and no landing-page hallucination.
|
|
|
|
- [ ] **Step 5: Record any remaining provider-gap separately**
|
|
- If live-search still cannot surface article-grade pages reliably, document the exact gap before adding a new provider.
|