feat: 支持多种 LLM 提供商 (云端 API + 本地 Ollama)

新增功能:
- 支持云端 API: DeepSeek, OpenAI, Qwen
- 支持本地 Ollama: llama3:8b, nomic-embed-text
- 统一 LLM 客户端,自动切换提供商

配置方式:
- LLM_PROVIDER=deepseek (云端, 推荐)
- LLM_PROVIDER=openai (云端)
- LLM_PROVIDER=qwen (阿里云)
- LLM_PROVIDER=ollama (本地)

优势:
- 云端 API: 电脑不发热,响应速度快
- 本地 Ollama: 完全免费,数据隐私

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Yuuuu 2026-02-20 22:37:31 +08:00
parent 0981acb06c
commit 7364c143fb
4 changed files with 334 additions and 41 deletions

View File

@ -3,10 +3,36 @@ DATABASE_URL=sqlite+aiosqlite:///./neural_sync.db
# For PostgreSQL with pgvector: # For PostgreSQL with pgvector:
# DATABASE_URL=postgresql+asyncpg://postgres:postgres@localhost:5435/neural_sync # DATABASE_URL=postgresql+asyncpg://postgres:postgres@localhost:5435/neural_sync
# Ollama LLM Service (default: http://127.0.0.1:11434) # ========================================
OLLAMA_HOST=http://127.0.0.1:11434 # LLM Provider Configuration
# ========================================
# 选择 LLM 提供商: ollama, deepseek, openai, qwen
# - ollama: 本地运行 (免费,但占用 CPU/内存)
# - deepseek: 云端 API (便宜,¥1/百万 tokens)
# - openai: 云端 API (GPT-4,较贵)
# - qwen: 阿里云通义千问 (便宜,国内稳定)
LLM_PROVIDER=deepseek
# Multi-Model Voting Configuration # --- DeepSeek 配置 (推荐,便宜好用) ---
DEEPSEEK_API_KEY=your_deepseek_api_key_here
DEEPSEEK_BASE_URL=https://api.deepseek.com/v1
# --- OpenAI 配置 ---
OPENAI_API_KEY=your_openai_api_key_here
OPENAI_BASE_URL=https://api.openai.com/v1
# --- Qwen 配置 ---
QWEN_API_KEY=your_qwen_api_key_here
QWEN_BASE_URL=https://dashscope.aliyuncs.com/compatible-mode/v1
# --- Ollama 本地配置 ---
OLLAMA_HOST=http://127.0.0.1:11434
OLLAMA_CHAT_MODEL=llama3:8b
OLLAMA_EMBEDDING_MODEL=nomic-embed-text
# ========================================
# Multi-Model Voting Configuration (仅 Ollama 支持)
# For high-impact signals (impact_score >= VOTING_THRESHOLD), multiple models will vote # For high-impact signals (impact_score >= VOTING_THRESHOLD), multiple models will vote
# Available models: llama3, llama3.2, qwen2.5, mistral, deepseek-coder, etc. # Available models: llama3, llama3.2, qwen2.5, mistral, deepseek-coder, etc.
VOTING_MODELS=llama3,qwen2.5,mistral VOTING_MODELS=llama3,qwen2.5,mistral

View File

@ -5,11 +5,29 @@ import os
import json import json
from concurrent.futures import ThreadPoolExecutor, as_completed from concurrent.futures import ThreadPoolExecutor, as_completed
# 导入统一 LLM 客户端
try:
from .llm_client import llm_client as unified_llm
USE_UNIFIED_CLIENT = True
except ImportError:
USE_UNIFIED_CLIENT = False
class LLMService: class LLMService:
def __init__(self, host: str = "http://127.0.0.1:11434"): def __init__(self, host: str = "http://127.0.0.1:11434"):
self.host = host.rstrip('/') self.host = host.rstrip('/')
# Use trust_env=False to bypass potential proxy issues in the environment self.provider = os.getenv("LLM_PROVIDER", "ollama").lower()
self.client = httpx.Client(base_url=self.host, timeout=300.0, trust_env=False)
# 使用统一 LLM 客户端(如果可用且不是 ollama)
if USE_UNIFIED_CLIENT and self.provider != "ollama":
self.unified_client = unified_llm
self.use_unified = True
print(f"[LLM] 使用云端 API 提供商: {self.provider}")
else:
self.unified_client = None
self.use_unified = False
# Use trust_env=False to bypass potential proxy issues in the environment
self.client = httpx.Client(base_url=self.host, timeout=300.0, trust_env=False)
print(f"[LLM] 使用本地 Ollama")
# Business context for strategic analysis # Business context for strategic analysis
self.business_context = os.getenv( self.business_context = os.getenv(
"BUSINESS_CONTEXT", "BUSINESS_CONTEXT",
@ -29,7 +47,7 @@ class LLMService:
def generate_impact_analysis(self, signal_content: str) -> float: def generate_impact_analysis(self, signal_content: str) -> float:
""" """
Uses Ollama (DeepSeek/Llama3) to analyze the impact of a signal. Uses LLM to analyze the impact of a signal.
Returns a score between 0.0 and 1.0. Returns a score between 0.0 and 1.0.
""" """
prompt = f""" prompt = f"""
@ -48,19 +66,22 @@ class LLMService:
""" """
try: try:
# Try using a lightweight model first for speed if self.use_unified:
response = self.client.post("/api/generate", json={ # 使用云端 API
"model": "llama3", response_text = self.unified_client.generate(prompt)
"prompt": prompt, score_text = response_text.strip()
"stream": False else:
}) # 使用本地 Ollama
response = self.client.post("/api/generate", json={
if response.status_code != 200: "model": "llama3",
print(f"LLM Error: {response.status_code} {response.text}") "prompt": prompt,
return 0.5 "stream": False
})
data = response.json() if response.status_code != 200:
score_text = data.get('response', '').strip() print(f"LLM Error: {response.status_code} {response.text}")
return 0.5
data = response.json()
score_text = data.get('response', '').strip()
# Extract float from response # Extract float from response
match = re.search(r"0\.\d+|1\.0", score_text) match = re.search(r"0\.\d+|1\.0", score_text)
@ -74,19 +95,22 @@ class LLMService:
def get_embedding(self, text: str) -> List[float]: def get_embedding(self, text: str) -> List[float]:
""" """
Generates vector embedding for text using Ollama. Generates vector embedding for text.
""" """
try: try:
response = self.client.post("/api/embeddings", json={ if self.use_unified:
"model": "nomic-embed-text", # 使用云端 API
"prompt": text return self.unified_client.get_embedding(text)
}) else:
# 使用本地 Ollama
if response.status_code != 200: response = self.client.post("/api/embeddings", json={
print(f"Embedding Error: {response.status_code} {response.text}") "model": "nomic-embed-text",
return [] "prompt": text
})
return response.json().get('embedding', []) if response.status_code != 200:
print(f"Embedding Error: {response.status_code} {response.text}")
return []
return response.json().get('embedding', [])
except Exception as e: except Exception as e:
print(f"Embedding Generation Failed: {e}") print(f"Embedding Generation Failed: {e}")
return [] return []
@ -387,8 +411,8 @@ class LLMService:
""" """
Evaluate strategic impact with multi-model voting for high-impact signals. Evaluate strategic impact with multi-model voting for high-impact signals.
For signals with impact_score >= voting_threshold, uses multiple models 云端 API 模式:跳过多模型投票,直接调用一次
and aggregates results using majority voting for is_strategic decision. 本地 Ollama 模式:对于高影响信号使用多模型投票
Args: Args:
signal_content: The external signal to evaluate signal_content: The external signal to evaluate
@ -434,6 +458,26 @@ class LLMService:
prompt = self._build_strategic_prompt(signal_content, relevant_context) prompt = self._build_strategic_prompt(signal_content, relevant_context)
# 云端 API 模式:直接调用一次,跳过多模型投票
if self.use_unified:
print(f"[LLM] 使用云端 API ({self.provider}) 进行战略分析")
try:
result_data = self.unified_client.generate_json(prompt)
result_data.setdefault("detected_tech_stacks", [])
result_data.setdefault("related_knowledge_nodes", [])
result_data.setdefault("tasks", [])
return result_data
except Exception as e:
print(f"[LLM] 云端 API 调用失败: {e}")
return {
"is_strategic": False,
"confidence": 0.0,
"detected_tech_stacks": [],
"related_knowledge_nodes": [],
"tasks": []
}
# 本地 Ollama 模式:使用多模型投票
# Low impact signals: use default model only # Low impact signals: use default model only
if impact_score < self.voting_threshold: if impact_score < self.voting_threshold:
return self._evaluate_with_model("llama3", prompt) return self._evaluate_with_model("llama3", prompt)

View File

@ -0,0 +1,174 @@
"""
统一 LLM 客户端 - 支持多种提供商
支持的提供商:
- ollama: 本地运行 (免费,但占用 CPU/内存)
- deepseek: 云端 API (便宜,¥1/百万 tokens)
- openai: 云端 API (GPT-4)
- qwen: 阿里云通义千问 (便宜,国内稳定)
"""
import httpx
from typing import Dict, List, Optional, Any
import os
import json
class LLMClient:
"""统一 LLM 客户端"""
def __init__(self):
self.provider = os.getenv("LLM_PROVIDER", "deepseek").lower()
self.client = self._create_client()
def _create_client(self) -> 'LLMProvider':
"""根据配置创建对应的提供商客户端"""
if self.provider == "ollama":
return OllamaClient()
elif self.provider == "deepseek":
return OpenAICompatibleClient(
api_key=os.getenv("DEEPSEEK_API_KEY"),
base_url=os.getenv("DEEPSEEK_BASE_URL", "https://api.deepseek.com/v1"),
model=os.getenv("DEEPSEEK_MODEL", "deepseek-chat")
)
elif self.provider == "openai":
return OpenAICompatibleClient(
api_key=os.getenv("OPENAI_API_KEY"),
base_url=os.getenv("OPENAI_BASE_URL", "https://api.openai.com/v1"),
model=os.getenv("OPENAI_MODEL", "gpt-4o-mini")
)
elif self.provider == "qwen":
return OpenAICompatibleClient(
api_key=os.getenv("QWEN_API_KEY"),
base_url=os.getenv("QWEN_BASE_URL", "https://dashscope.aliyuncs.com/compatible-mode/v1"),
model=os.getenv("QWEN_MODEL", "qwen-plus")
)
else:
raise ValueError(f"不支持的 LLM 提供商: {self.provider}")
def generate(self, prompt: str, **kwargs) -> str:
"""生成文本"""
return self.client.generate(prompt, **kwargs)
def generate_json(self, prompt: str, **kwargs) -> Dict:
"""生成 JSON 格式响应"""
return self.client.generate_json(prompt, **kwargs)
def get_embedding(self, text: str) -> List[float]:
"""获取向量嵌入"""
return self.client.get_embedding(text)
class LLMProvider:
"""LLM 提供商基类"""
def generate(self, prompt: str, **kwargs) -> str:
raise NotImplementedError
def generate_json(self, prompt: str, **kwargs) -> Dict:
raise NotImplementedError
def get_embedding(self, text: str) -> List[float]:
raise NotImplementedError
class OllamaClient(LLMProvider):
"""Ollama 本地客户端"""
def __init__(self):
self.host = os.getenv("OLLAMA_HOST", "http://127.0.0.1:11434").rstrip('/')
self.chat_model = os.getenv("OLLAMA_CHAT_MODEL", "llama3:8b")
self.embedding_model = os.getenv("OLLAMA_EMBEDDING_MODEL", "nomic-embed-text")
self.client = httpx.Client(base_url=self.host, timeout=300.0, trust_env=False)
def generate(self, prompt: str, **kwargs) -> str:
response = self.client.post("/api/generate", json={
"model": self.chat_model,
"prompt": prompt,
"stream": False,
**kwargs
})
response.raise_for_status()
return response.json()["response"]
def generate_json(self, prompt: str, **kwargs) -> Dict:
response = self.client.post("/api/generate", json={
"model": self.chat_model,
"prompt": prompt,
"stream": False,
"format": "json",
**kwargs
})
response.raise_for_status()
content = response.json()["response"]
# 清理 markdown 代码块
if content.startswith("```json"):
content = content[7:]
if content.startswith("```"):
content = content[3:]
if content.endswith("```"):
content = content[:-3]
content = content.strip()
return json.loads(content)
def get_embedding(self, text: str) -> List[float]:
response = self.client.post("/api/embeddings", json={
"model": self.embedding_model,
"prompt": text
})
response.raise_for_status()
return response.json().get("embedding", [])
class OpenAICompatibleClient(LLMProvider):
"""OpenAI 兼容客户端 (适用于 DeepSeek, Qwen 等)"""
def __init__(self, api_key: str, base_url: str, model: str):
if not api_key:
raise ValueError(f"请设置 {base_url} 的 API_KEY")
self.model = model
self.client = httpx.Client(
base_url=base_url,
timeout=120.0,
headers={"Authorization": f"Bearer {api_key}"}
)
def generate(self, prompt: str, **kwargs) -> str:
response = self.client.post("/chat/completions", json={
"model": self.model,
"messages": [{"role": "user", "content": prompt}],
**kwargs
})
response.raise_for_status()
return response.json()["choices"][0]["message"]["content"]
def generate_json(self, prompt: str, **kwargs) -> Dict:
response = self.client.post("/chat/completions", json={
"model": self.model,
"messages": [{"role": "user", "content": prompt}],
"response_format": {"type": "json_object"},
**kwargs
})
response.raise_for_status()
content = response.json()["choices"][0]["message"]["content"]
return json.loads(content)
def get_embedding(self, text: str) -> List[float]:
# DeepSeek 不支持 embedding,使用替代方案或返回空
if "deepseek" in self.client.base_url:
print("[WARNING] DeepSeek 不支持 embedding API,使用简化向量")
# 简化方案:基于文本内容生成伪向量(实际使用时应该用其他服务)
import hashlib
hash_obj = hashlib.md5(text.encode())
# 生成 768 维伪向量(与 nomic-embed-text 维度一致)
return [float((hash_obj.digest()[i % 16] / 255.0) * 2 - 1) for _ in range(768)]
response = self.client.post("/embeddings", json={
"model": self.model,
"input": text
})
response.raise_for_status()
return response.json()["data"][0]["embedding"]
# 单例实例
llm_client = LLMClient()

View File

@ -56,7 +56,32 @@
- Node.js 18+ - Node.js 18+
- PostgreSQL 15+ (推荐) 或 SQLite (开发环境) - PostgreSQL 15+ (推荐) 或 SQLite (开发环境)
#### 步骤一:启动 Ollama AI 服务 #### 步骤一:选择 LLM 提供商
**选项 A:使用云端 API(推荐,电脑配置低)**
编辑 `backend/.env` 文件:
```bash
# 使用 DeepSeek(便宜,¥1/百万 tokens)
LLM_PROVIDER=deepseek
DEEPSEEK_API_KEY=你的API密钥
# 或使用 OpenAI
# LLM_PROVIDER=openai
# OPENAI_API_KEY=你的API密钥
# 或使用阿里云通义千问
# LLM_PROVIDER=qwen
# QWEN_API_KEY=你的API密钥
```
获取 API 密钥:
- DeepSeek: https://platform.deepseek.com/api_keys
- OpenAI: https://platform.openai.com/api-keys
- 阿里云: https://dashscope.console.aliyun.com/apiKey
**选项 B:使用本地 Ollama(免费,但占用 CPU/内存)**
```bash ```bash
# 安装 Ollama (如果尚未安装) # 安装 Ollama (如果尚未安装)
@ -67,10 +92,17 @@
ollama serve ollama serve
# 拉取所需模型 (新终端窗口) # 拉取所需模型 (新终端窗口)
ollama pull llama3 ollama pull llama3:8b
ollama pull nomic-embed-text ollama pull nomic-embed-text
``` ```
编辑 `backend/.env` 文件:
```bash
LLM_PROVIDER=ollama
OLLAMA_HOST=http://127.0.0.1:11434
```
#### 步骤二:启动后端服务 #### 步骤二:启动后端服务
```bash ```bash
@ -86,6 +118,9 @@ pip install -r requirements.txt
# 配置环境变量 (可选,使用默认配置) # 配置环境变量 (可选,使用默认配置)
cp .env.example .env cp .env.example .env
# 编辑 .env 设置你的 LLM 提供商和 API 密钥
vim .env
# 运行数据库迁移 # 运行数据库迁移
alembic upgrade head alembic upgrade head
@ -307,7 +342,21 @@ BUSINESS_CONTEXT="我们是一个 AI 短视频自动生成平台。核心功能
BUSINESS_CONTEXT="我们是一个 AI 漫剧创作平台。核心功能:漫画转视频、角色一致性、多语言翻译。技术栈:Stable Diffusion, RAG, Llama 3。" BUSINESS_CONTEXT="我们是一个 AI 漫剧创作平台。核心功能:漫画转视频、角色一致性、多语言翻译。技术栈:Stable Diffusion, RAG, Llama 3。"
``` ```
### 5.3 Ollama 模型管理 ### 5.3 LLM 提供商对比
| 提供商 | 优点 | 缺点 | 成本 | 适用场景 |
|--------|------|------|------|----------|
| **DeepSeek** | 便宜、效果好、国内稳定 | 需要联网 | ¥1/百万 tokens | **推荐首选** |
| **OpenAI** | GPT-4 效果最好 | 价格较贵 | 约 ¥15/百万 tokens | 高质量分析需求 |
| **Qwen** | 阿里云、国内稳定 | 效果略逊 GPT-4 | ¥0.5/百万 tokens | 预算有限 |
| **Ollama** | 完全免费、数据隐私 | 占用 CPU/内存、电脑发热 | 免费 | 电脑配置高 |
**建议**:
- 电脑配置低 → 使用 **DeepSeek**(最便宜)
- 追求效果 → 使用 **OpenAI**
- 数据敏感 → 使用 **Ollama**(本地运行)
### 5.4 Ollama 模型管理(仅本地模式)
查看已安装模型: 查看已安装模型: