chore: opencode冗余清理 + LLM任务级模型选择 + systemd服务化

- 删除 opencode_search.py / mcp_search_server.py 及所有 MCP 引用
- 移除搜索缓存定时任务(scheduled_refresh_search_cache)
- 清理前后端所有 opencode/MCP 代码和注释
- LLM 提供商量换:opencode-go→nvidia(默认)+sensenova(合规审查)
- llm_configs 新增 is_default 字段,API 层互斥逻辑
- 所有定时任务支持独立 LLM 模型选择(LLM_TASK_PROVIDER env)
- compliance_optimizer.py 修复:import os / 解硬编码 / 关键词过滤
- Scheduler 日志修复:始终 INSERT,避免僵尸 running 行
- Systemd 服务化:Restart=always / 单 worker / Type=exec
- 搜索提供商:替换 opencode→360/搜狗/微信(免 Key)
- 更新 AGENTS.md / PROGRESS.md
This commit is contained in:
Yuzhiran Dev
2026-06-02 15:38:16 +08:00
parent abbf1468bc
commit 499c511140
25 changed files with 451 additions and 678 deletions
+2 -2
View File
@@ -4,9 +4,9 @@
- **Backend**: FastAPI 0.104 + SQLAlchemy 2.0 + PostgreSQL 16 (`yzr_nr`) - **Backend**: FastAPI 0.104 + SQLAlchemy 2.0 + PostgreSQL 16 (`yzr_nr`)
- **Frontend**: Vue 3 (CDN, no build step) + Element Plus — static HTML served by FastAPI - **Frontend**: Vue 3 (CDN, no build step) + Element Plus — static HTML served by FastAPI
- **Auth**: JWT (`python-jose` + bcrypt), default admin `admin/admin123` - **Auth**: JWT (`python-jose` + bcrypt), default admin `admin/admin123`
- **Scheduler**: APScheduler (daily cron: 01:00 searchcache, 01:10 trends, 01:30 collect, 02:00 generate, 03:00 optimize, 05:00 sources, 06:00 metrics) - **Scheduler**: APScheduler (daily cron: 01:10 trends, 01:30 collect, 02:00 generate, 03:00 optimize, 05:00 sources, 06:00 metrics)
- **Task DB**: `TaskLog` (module_id/status/error_trace/result_data/triggered_by) + `TaskConfig` (params/enabled/schedule) - **Task DB**: `TaskLog` (module_id/status/error_trace/result_data/triggered_by) + `TaskConfig` (params/enabled/schedule)
- **LLM**: Multi-provider (opencode-go primary, nvidia backup). API keys only in `.env`, not DB. - **LLM**: Multi-provider (nvidia primary, opencode-go fallback). API keys in DB (managed via admin UI) or `.env`.
## Commands ## Commands
+14 -5
View File
@@ -3,7 +3,7 @@
> 本文件为项目进度唯一真理源,所有进度信息以此为准。 > 本文件为项目进度唯一真理源,所有进度信息以此为准。
> 其他文档中的进度描述一律以本文为准。 > 其他文档中的进度描述一律以本文为准。
**最后更新**2026-05-22 (v17) **最后更新**2026-06-02 (v18)
--- ---
@@ -15,7 +15,7 @@
| 技术栈 | FastAPI + SQLAlchemy + PostgreSQL 16 + Vue 3 (CDN) + Element Plus | | 技术栈 | FastAPI + SQLAlchemy + PostgreSQL 16 + Vue 3 (CDN) + Element Plus |
| 平台服务 | 运行中 (端口 8001) | | 平台服务 | 运行中 (端口 8001) |
| 策略阶段 | 全球-本土对比研究(2026-04-15 升级) | | 策略阶段 | 全球-本土对比研究(2026-04-15 升级) |
| Git 提交 | 99 commits · 4 tags (v1.0.0~v1.0.4) · main 分支 | | Git 提交 | 165 commits · 4 tags (v1.0.0~v1.0.4) · main 分支 |
--- ---
@@ -76,19 +76,22 @@
| publish_to_wechat_mp.sh | ✅ 可用 | 微信公众号自动发布 | | publish_to_wechat_mp.sh | ✅ 可用 | 微信公众号自动发布 |
| generate_images.py / image_generator.py | ✅ 可用 | SVG + PNG 配图生成 | | generate_images.py / image_generator.py | ✅ 可用 | SVG + PNG 配图生成 |
| compliance_checker.py | ✅ 可用 | 合规审查(敏感词/平台规则/品牌规范) | | compliance_checker.py | ✅ 可用 | 合规审查(敏感词/平台规则/品牌规范) |
| compliance_optimizer.py | ✅ 重构 | 移除 manual_review,改为迭代LLM修复(最多3次),合规分回写入Topic | | compliance_optimizer.py | ✅ 重构 | 移除 manual_review,改为迭代LLM修复(最多3次),合规分回写入Topic;解硬编码 provider,改从 env 读取;关键词过滤从宽泛改为精准 |
| creator.py / writer.py / outline.py / research.py | ✅ 优化 | 全链路LLM提示词优化(SEO/平台适配/真人感) | | creator.py / writer.py / outline.py / research.py | ✅ 优化 | 全链路LLM提示词优化(SEO/平台适配/真人感) |
| collector.py / collector_db_integration.py | ✅ 可用 | 趋势采集 | | collector.py / collector_db_integration.py | ✅ 可用 | 趋势采集 |
| wecom_notifier.py | ✅ 可用 | 企业微信通知 | | wecom_notifier.py | ✅ 可用 | 企业微信通知 |
| db_helper.py | ✅ 扩展 | update_topic_status 支持保存 compliance_score | | db_helper.py | ✅ 扩展 | update_topic_status 支持保存 compliance_score |
| search_utils.py | ✅ 重构 | 移除 _call_mcpopencode MCP),新增 360/搜狗/微信搜索(免 API Key),百度千帆日限提升至 200 |
| opencode_search.py / mcp_search_server.py | ❌ 已删除 | opencode 搜索配额耗尽,替换为 360/搜狗/微信等免 Key 源 |
| web_search.py | ✅ 保留 | 本地缓存 + Bing 搜索(闲置备用) |
### 4.4 流水线流程 ### 4.4 流水线流程
| 步骤 | 触发方式 | 说明 | | 步骤 | 触发方式 | 说明 |
|------|---------|------| |------|---------|------|
| 内容采集 | 定时 01:30 | 热点趋势采集→生成选题建议→存入选题库 | | 内容采集 | 定时 01:30 | 热点趋势采集→生成选题建议→存入选题库 |
| 内容创作 | 手动点击 / 定时 03:30 | 研究→大纲→撰写文章→合规审查→存入 articles 表 | | 内容创作 | 手动点击 / 定时 03:30 (原 02:00) | 研究→大纲→撰写文章→合规审查→存入 articles 表 |
| 合规审查 | 手动点击 / 定时 04:30 | 从 articles 表读取 draft→合规检查→LLM迭代修复(最多3次)→状态→待发布 | | 合规审查 | 手动点击 / 定时 04:30 (原 03:00) | 从 articles 表读取 draft→合规检查→LLM迭代修复(最多3次)→状态→待发布 |
| 信息源优化 | 定时 05:00 | AI评估采集类别与信息源配置,给出调整建议 | | 信息源优化 | 定时 05:00 | AI评估采集类别与信息源配置,给出调整建议 |
| 指标同步 | 定时 06:00 | 同步统计数据 | | 指标同步 | 定时 06:00 | 同步统计数据 |
| 发布 | 手动点击 | 仅待发布状态可选 | | 发布 | 手动点击 | 仅待发布状态可选 |
@@ -144,6 +147,12 @@
| platforms.html 入口合并 | 2026-05-22 | uni-nav 移除"平台"独立入口;admin.html 恢复"平台配置"tab 加启用中/全部筛选 | | platforms.html 入口合并 | 2026-05-22 | uni-nav 移除"平台"独立入口;admin.html 恢复"平台配置"tab 加启用中/全部筛选 |
| opencode_search.py 日志 | 2026-05-22 | 补 FileHandler + StreamHandler,解决管理后台显示"从未运行" | | opencode_search.py 日志 | 2026-05-22 | 补 FileHandler + StreamHandler,解决管理后台显示"从未运行" |
| scheduler.json 导入修复 | 2026-05-22 | 补 import json,修复 sources(05:00) 执行时报错阻断 metrics(06:00) | | scheduler.json 导入修复 | 2026-05-22 | 补 import json,修复 sources(05:00) 执行时报错阻断 metrics(06:00) |
| Scheduler 日志修复(僵尸行) | 2026-05-28 | _log_task 始终 INSERT 新行,所有 _run_* 保存 log_id 后 UPDATE 同一行,消除重复 running 状态 |
| Systemd 服务化 | 2026-05-28 | yzr-platform.serviceRestart=always,崩溃自动恢复);单 worker(--workers 1)防调度器冲突;Type=exec + KillMode=control-group 防僵尸进程 |
| LLM 提供商量换 + 任务级模型选择 | 2026-05-28 | opencode-go 换 nvidia qwen3.5-397b-a17b(默认)+ sensenova deepseek-v4-flash(合规审查);llm_configs 加 is_default;所有 _run_* 方法开头调用 _set_task_llm_provider 设置 LLM_TASK_PROVIDER |
| 合规审查修复 | 2026-05-28 | compliance_optimizer.py 移除硬编码 provider;补 import os(之前导致 NameError);关键词过滤从宽泛改为精准,避免误拦正常内容 |
| 搜索提供商重构 | 2026-05-28 | 移除 opencode MCP(配额耗尽),新增 360/搜狗/微信搜索(免 Key),百度千帆日限 50→200 |
| opencode 冗余代码清理 | 2026-06-02 | 删除 opencode_search.py / mcp_search_server.py / _call_mcp / 前后端所有 opencode/MCP 引用;禁用搜索缓存定时任务 |
### ⏳ 待办 ### ⏳ 待办
+11
View File
@@ -9,6 +9,14 @@ from .auth import get_current_admin
router = APIRouter(prefix="/api/admin/llmconfigs", tags=["admin"]) router = APIRouter(prefix="/api/admin/llmconfigs", tags=["admin"])
def _apply_default_exclusive(config: LLMConfig, db: Session):
"""当 config.is_default=True 时,将其他所有配置的 is_default 置为 False"""
if config.is_default:
db.query(LLMConfig).filter(LLMConfig.id != config.id).update(
{"is_default": False}, synchronize_session=False
)
db.flush()
@router.get("", response_model=List[LLMConfigResponse]) @router.get("", response_model=List[LLMConfigResponse])
def list_llm_configs( def list_llm_configs(
request: Request, request: Request,
@@ -42,6 +50,8 @@ def create_llm_config(
"""创建 LLM 配置""" """创建 LLM 配置"""
config = LLMConfig(**config_data.model_dump()) config = LLMConfig(**config_data.model_dump())
db.add(config) db.add(config)
db.flush()
_apply_default_exclusive(config, db)
db.commit() db.commit()
db.refresh(config) db.refresh(config)
return config return config
@@ -61,6 +71,7 @@ def update_llm_config(
update_data = config_update.model_dump(exclude_unset=True) update_data = config_update.model_dump(exclude_unset=True)
for field, value in update_data.items(): for field, value in update_data.items():
setattr(config, field, value) setattr(config, field, value)
_apply_default_exclusive(config, db)
db.commit() db.commit()
db.refresh(config) db.refresh(config)
return config return config
@@ -119,16 +119,6 @@ def test_provider(provider_id: int, data: dict = {}, db: Session = Depends(get_d
if resp.status_code != 200: if resp.status_code != 200:
return {"ok": False, "error": f"HTTP {resp.status_code}: {resp.text[:200]}"} return {"ok": False, "error": f"HTTP {resp.status_code}: {resp.text[:200]}"}
return {"ok": True, "results": resp.json().get("webPages", {}).get("value", [])[:3]} return {"ok": True, "results": resp.json().get("webPages", {}).get("value", [])[:3]}
elif p.provider_type == "mcp":
import subprocess, json as _json
mcp_script = Path(__file__).resolve().parent.parent.parent.parent.parent / "scripts" / "mcp_search_server.py"
r = subprocess.run(
[sys.executable, str(mcp_script), "--query", query],
capture_output=True, text=True, timeout=90,
)
if r.returncode != 0:
return {"ok": False, "error": f"子进程失败: {r.stderr[:200]}"}
return {"ok": True, "results": _json.loads(r.stdout)[:3]}
return {"ok": False, "error": f"Unknown provider_type: {p.provider_type}"} return {"ok": False, "error": f"Unknown provider_type: {p.provider_type}"}
except Exception as e: except Exception as e:
return {"ok": False, "error": str(e)} return {"ok": False, "error": str(e)}
-23
View File
@@ -280,28 +280,6 @@ def trigger_metrics_sync():
except Exception as e: except Exception as e:
raise HTTPException(status_code=500, detail=str(e)) raise HTTPException(status_code=500, detail=str(e))
@router.post("/refresh-search-cache/run")
def trigger_refresh_search_cache(db: Session = Depends(get_db), current_user=Depends(get_current_user)):
try:
import sys as sys_mod
scripts_dir = PROJECT_ROOT / "scripts"
from ..database import SessionLocal as _ss
proc = subprocess.Popen(
[sys_mod.executable, str(scripts_dir / "opencode_search.py"), "--refresh-cache"],
stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True,
cwd=str(PROJECT_ROOT)
)
logger.info("Search cache refresh started (pid=%s)", proc.pid)
log = TaskLog(module_id="scheduled_refresh_search_cache", task_name="🔍 搜索缓存", status="running", message="搜索缓存刷新已启动", triggered_by="manual", started_at=datetime.now(timezone.utc), result_data={"pid": proc.pid})
db.add(log)
db.commit()
log_id = log.id
t = threading.Thread(target=_monitor_subprocess, args=(log_id, proc, "scheduled_refresh_search_cache", "🔍 搜索缓存", _ss), daemon=True)
t.start()
return {"message": "搜索缓存刷新已后台启动", "pid": proc.pid, "log_id": log_id}
except Exception as e:
raise HTTPException(status_code=500, detail=str(e))
@router.post("/trends/run") @router.post("/trends/run")
def trigger_trends_refresh(db: Session = Depends(get_db), current_user=Depends(get_current_user)): def trigger_trends_refresh(db: Session = Depends(get_db), current_user=Depends(get_current_user)):
try: try:
@@ -370,7 +348,6 @@ def get_modules_status(db: Session = Depends(get_db)):
config_map = {c.module_id: c for c in configs} config_map = {c.module_id: c for c in configs}
MODULE_META = { MODULE_META = {
"scheduled_refresh_search_cache": {"name": "🔍 搜索缓存", "cron": "01:00", "params_desc": {"refresh_queries": "搜索关键词列表"}},
"scheduled_fetch_trends": {"name": "🔥 热点趋势", "cron": "01:10", "params_desc": {}}, "scheduled_fetch_trends": {"name": "🔥 热点趋势", "cron": "01:10", "params_desc": {}},
"scheduled_collect": {"name": "📡 内容采集", "cron": "01:30", "params_desc": {"max_topics": "最大选题数", "categories": "采集类别"}}, "scheduled_collect": {"name": "📡 内容采集", "cron": "01:30", "params_desc": {"max_topics": "最大选题数", "categories": "采集类别"}},
"scheduled_generate": {"name": "🤖 内容创作", "cron": "02:00", "params_desc": {"auto_review": "自动合规审查"}}, "scheduled_generate": {"name": "🤖 内容创作", "cron": "02:00", "params_desc": {"auto_review": "自动合规审查"}},
+7 -8
View File
@@ -11,14 +11,13 @@ from .auth import get_current_admin
router = APIRouter(prefix="/api/admin/task-configs", tags=["admin"]) router = APIRouter(prefix="/api/admin/task-configs", tags=["admin"])
DEFAULT_CONFIGS = { DEFAULT_CONFIGS = {
"scheduled_refresh_search_cache": {"name": "🔍 搜索缓存", "cron": "01:00", "params": {}}, "scheduled_fetch_trends": {"name": "🔥 热点趋势", "cron": "01:10", "params": {"llm_provider": "nvidia"}},
"scheduled_fetch_trends": {"name": "🔥 热点趋势", "cron": "01:10", "params": {}}, "scheduled_collect": {"name": "📡 内容采集", "cron": "01:30", "params": {"max_topics": 20, "llm_provider": "nvidia"}},
"scheduled_collect": {"name": "📡 内容采集", "cron": "01:30", "params": {"max_topics": 20}}, "scheduled_generate": {"name": "🤖 内容创作", "cron": "02:00", "params": {"auto_review": True, "llm_provider": "nvidia"}},
"scheduled_generate": {"name": "🤖 内容创作", "cron": "02:00", "params": {"auto_review": True}}, "scheduled_optimize": {"name": "🔍 合规审查", "cron": "03:00", "params": {"auto_pass_threshold": 80, "llm_provider": "sensenova"}},
"scheduled_optimize": {"name": "🔍 合规审查", "cron": "03:00", "params": {"auto_pass_threshold": 80}}, "scheduled_optimize_sources": {"name": "📡 信息源优化", "cron": "05:00", "params": {"llm_provider": "nvidia"}},
"scheduled_optimize_sources": {"name": "📡 信息源优化", "cron": "05:00", "params": {}}, "scheduled_metrics_sync": {"name": "📊 指标同步", "cron": "06:00", "params": {"llm_provider": "nvidia"}},
"scheduled_metrics_sync": {"name": "📊 指标同步", "cron": "06:00", "params": {}}, "scheduled_task_monitor": {"name": "⏰ 任务监控", "cron": "*", "params": {"llm_provider": "nvidia"}},
"scheduled_task_monitor": {"name": "⏰ 任务监控", "cron": "*", "params": {}},
} }
def _attach_last_log(resp: TaskConfigResponse, db: Session, module_id: str) -> TaskConfigResponse: def _attach_last_log(resp: TaskConfigResponse, db: Session, module_id: str) -> TaskConfigResponse:
-2
View File
@@ -12,7 +12,6 @@ from .auth import get_current_admin
router = APIRouter(prefix="/api/admin/task-logs", tags=["admin"]) router = APIRouter(prefix="/api/admin/task-logs", tags=["admin"])
MODULES = { MODULES = {
"scheduled_refresh_search_cache": "🔍 搜索缓存",
"scheduled_fetch_trends": "🔥 热点趋势", "scheduled_fetch_trends": "🔥 热点趋势",
"scheduled_collect": "📡 内容采集", "scheduled_collect": "📡 内容采集",
"scheduled_generate": "🤖 内容创作", "scheduled_generate": "🤖 内容创作",
@@ -79,7 +78,6 @@ def list_log_types(db: Session = Depends(get_db), admin_user=Depends(get_current
for mid, name in MODULES.items(): for mid, name in MODULES.items():
if mid in used_ids or True: if mid in used_ids or True:
log_file_map = { log_file_map = {
"scheduled_refresh_search_cache": "opencode_search",
"scheduled_fetch_trends": "trends", "scheduled_fetch_trends": "trends",
"scheduled_collect": "collector", "scheduled_collect": "collector",
"scheduled_generate": "creator", "scheduled_generate": "creator",
-18
View File
@@ -236,7 +236,6 @@ def _get_module_detail_data(module_id: str, db, ROOT, DATA_DIR, LOGS_DIR, today_
import json as json_mod import json as json_mod
import re as re_mod import re as re_mod
MODULE_META = { MODULE_META = {
"scheduled_refresh_search_cache": {"name": "🔍 搜索缓存", "description": "通过 opencode webfetch 联网搜索,刷新 8 个分类的搜索缓存,供内容采集器使用"},
"scheduled_fetch_trends": {"name": "🔥 热点趋势", "description": "从百度、微博、知乎实时热搜 API 抓取当天热点,LLM 补充,存入 trends.json"}, "scheduled_fetch_trends": {"name": "🔥 热点趋势", "description": "从百度、微博、知乎实时热搜 API 抓取当天热点,LLM 补充,存入 trends.json"},
"scheduled_collect": {"name": "📡 内容采集", "description": "读取搜索缓存 + 热点趋势 + 历史表现 + AI 建议,经 LLM 分析后生成选题"}, "scheduled_collect": {"name": "📡 内容采集", "description": "读取搜索缓存 + 热点趋势 + 历史表现 + AI 建议,经 LLM 分析后生成选题"},
"scheduled_generate": {"name": "🤖 内容创作", "description": "基于选题,LLM 生成三平台文章(知乎、微信、小红书),存入 articles 表"}, "scheduled_generate": {"name": "🤖 内容创作", "description": "基于选题,LLM 生成三平台文章(知乎、微信、小红书),存入 articles 表"},
@@ -253,22 +252,6 @@ def _get_module_detail_data(module_id: str, db, ROOT, DATA_DIR, LOGS_DIR, today_
outputs = {} outputs = {}
history = [] history = []
if module_id == "scheduled_refresh_search_cache":
cache_file = DATA_DIR / "search_cache.json"
if cache_file.exists():
try:
cache = json_mod.loads(cache_file.read_text(encoding="utf-8"))
meta_ = cache.pop("_metadata", {})
for q, results in cache.items():
inputs.setdefault("搜索词", []).append(q)
outputs.setdefault("各分类结果", []).append({
"query": q, "count": len(results),
"samples": [r.get("title","")[:50] for r in results[:3]]
})
outputs["更新时间"] = meta_.get("updated_at", "")
outputs["结果总数"] = sum(len(v) for v in cache.values())
except Exception:
pass
# try reading queries from yaml # try reading queries from yaml
try: try:
import yaml import yaml
@@ -390,7 +373,6 @@ def _get_module_detail_data(module_id: str, db, ROOT, DATA_DIR, LOGS_DIR, today_
# History from log files # History from log files
log_map = { log_map = {
"scheduled_refresh_search_cache": LOGS_DIR / f"opencode_search_{today_str}.log",
"scheduled_fetch_trends": LOGS_DIR / f"trends_{today_str}.log", "scheduled_fetch_trends": LOGS_DIR / f"trends_{today_str}.log",
"scheduled_collect": LOGS_DIR / f"collector_{today_str}.log", "scheduled_collect": LOGS_DIR / f"collector_{today_str}.log",
"scheduled_generate": LOGS_DIR / f"creator_{today_str}.log", "scheduled_generate": LOGS_DIR / f"creator_{today_str}.log",
+14 -3
View File
@@ -35,11 +35,18 @@ _FALLBACK = {
} }
def _get_active_provider() -> str: def _get_active_provider() -> str:
"""从 DB 读取活跃供应商,DB 不可用时回退环境变量""" """从 DB 读取活跃供应商,优先取 is_default=TrueDB 不可用时回退环境变量"""
try: try:
from ..database import SessionLocal from ..database import SessionLocal
from ..models import LLMConfig from ..models import LLMConfig
db = SessionLocal() db = SessionLocal()
# 优先取默认
default = db.query(LLMConfig).filter(
LLMConfig.is_default == True, LLMConfig.is_active == True
).first()
if default and default.provider:
db.close()
return default.provider
active = db.query(LLMConfig).filter(LLMConfig.is_active == True).first() active = db.query(LLMConfig).filter(LLMConfig.is_active == True).first()
db.close() db.close()
if active and active.provider: if active and active.provider:
@@ -58,7 +65,7 @@ def _get_provider_config(provider: Optional[str] = None) -> dict:
from ..database import SessionLocal from ..database import SessionLocal
from ..models import LLMConfig from ..models import LLMConfig
db = SessionLocal() db = SessionLocal()
cfg = db.query(LLMConfig).filter(LLMConfig.provider == p).order_by(LLMConfig.is_active.desc()).first() cfg = db.query(LLMConfig).filter(LLMConfig.provider == p).order_by(LLMConfig.is_default.desc(), LLMConfig.is_active.desc()).first()
if cfg: if cfg:
db_model = cfg.model db_model = cfg.model
db_base_url = cfg.base_url db_base_url = cfg.base_url
@@ -107,7 +114,7 @@ def _get_provider_fallback_list() -> List[str]:
return providers return providers
except Exception: except Exception:
pass pass
return ["opencode-go", "nvidia"] return ["nvidia", "sensenova", "opencode-go"]
def call_llm( def call_llm(
prompt: str, prompt: str,
@@ -128,6 +135,10 @@ def call_llm(
system_prompt = system_prompt if system_prompt is not None else defaults["system_prompt"] system_prompt = system_prompt if system_prompt is not None else defaults["system_prompt"]
providers_to_try = [provider] if provider else _get_provider_fallback_list() providers_to_try = [provider] if provider else _get_provider_fallback_list()
# LLM_TASK_PROVIDER 环境变量可覆盖任务级别的模型选择
if not provider and os.getenv("LLM_TASK_PROVIDER"):
task_provider = os.getenv("LLM_TASK_PROVIDER")
providers_to_try = [task_provider] + [p for p in providers_to_try if p != task_provider]
last_error = None last_error = None
for p in providers_to_try: for p in providers_to_try:
try: try:
+24 -44
View File
@@ -19,8 +19,25 @@ from .collector import run_collector_blocking
logger = logging.getLogger(__name__) logger = logging.getLogger(__name__)
def _set_task_llm_provider(module_id: str):
"""从 TaskConfig 读取 llm_provider 并设为环境变量,供子进程和 call_llm 读取"""
try:
from ..database import SessionLocal
from ..models import TaskConfig
db = SessionLocal()
cfg = db.query(TaskConfig).filter(TaskConfig.module_id == module_id).first()
db.close()
if cfg and cfg.params:
provider = cfg.params.get("llm_provider")
if provider:
os.environ["LLM_TASK_PROVIDER"] = provider
logger.debug("[%s] LLM provider set to %s", module_id, provider)
return
except Exception:
pass
os.environ.pop("LLM_TASK_PROVIDER", None)
MODULES = { MODULES = {
"scheduled_refresh_search_cache": {"name": "🔍 搜索缓存", "cron": "01:00"},
"scheduled_fetch_trends": {"name": "🔥 热点趋势", "cron": "01:10"}, "scheduled_fetch_trends": {"name": "🔥 热点趋势", "cron": "01:10"},
"scheduled_collect": {"name": "📡 内容采集", "cron": "01:30"}, "scheduled_collect": {"name": "📡 内容采集", "cron": "01:30"},
"scheduled_generate": {"name": "🤖 内容创作", "cron": "02:00"}, "scheduled_generate": {"name": "🤖 内容创作", "cron": "02:00"},
@@ -32,7 +49,6 @@ MODULES = {
} }
LOG_FILE_MAP = { LOG_FILE_MAP = {
"scheduled_refresh_search_cache": "opencode_search",
"scheduled_fetch_trends": "trends", "scheduled_fetch_trends": "trends",
"scheduled_collect": "collector", "scheduled_collect": "collector",
"scheduled_generate": "creator", "scheduled_generate": "creator",
@@ -149,7 +165,6 @@ class TaskScheduler:
db.close() db.close()
MODULE_JOBS = [ MODULE_JOBS = [
("scheduled_refresh_search_cache", self._run_refresh_search_cache, "搜索缓存"),
("scheduled_fetch_trends", self._run_fetch_trends, "热点趋势"), ("scheduled_fetch_trends", self._run_fetch_trends, "热点趋势"),
("scheduled_collect", self._run_collect, "内容采集"), ("scheduled_collect", self._run_collect, "内容采集"),
("scheduled_generate", self._run_generate, "内容创作"), ("scheduled_generate", self._run_generate, "内容创作"),
@@ -200,6 +215,7 @@ class TaskScheduler:
def _run_fetch_trends(self): def _run_fetch_trends(self):
"""定时刷新热点趋势(百度/微博/知乎实时热搜 + LLM补充)""" """定时刷新热点趋势(百度/微博/知乎实时热搜 + LLM补充)"""
_set_task_llm_provider("scheduled_fetch_trends")
started = datetime.now(timezone.utc) started = datetime.now(timezone.utc)
log_id = _log_task("scheduled_fetch_trends", "running", started_at=started) log_id = _log_task("scheduled_fetch_trends", "running", started_at=started)
try: try:
@@ -229,48 +245,8 @@ class TaskScheduler:
started_at=started, finished_at=datetime.now(timezone.utc)) started_at=started, finished_at=datetime.now(timezone.utc))
logger.exception("[Scheduled] Trends refresh error: %s", e) logger.exception("[Scheduled] Trends refresh error: %s", e)
def _run_refresh_search_cache(self):
"""定时刷新搜索缓存(通过 opencode webfetch"""
started = datetime.now(timezone.utc)
log_id = _log_task("scheduled_refresh_search_cache", "running", started_at=started)
try:
logger.info("[Scheduled] Refreshing search cache via opencode...")
import subprocess
result = subprocess.run(
[sys.executable, str(PROJECT_ROOT / "scripts" / "opencode_search.py"), "--refresh-cache"],
capture_output=True, text=True, timeout=600
)
for line in result.stdout.strip().split("\n"):
if line.strip():
logger.info("[SearchCache] %s", line.strip())
for line in result.stderr.strip().split("\n"):
if line.strip():
logger.warning("[SearchCache] %s", line.strip())
if result.returncode == 0:
_log_task("scheduled_refresh_search_cache", "success", log_id=log_id,
message="搜索缓存刷新成功",
result_data={"output_lines": len(result.stdout.splitlines())},
started_at=started, finished_at=datetime.now(timezone.utc))
logger.info("[Scheduled] Search cache refreshed")
else:
_log_task("scheduled_refresh_search_cache", "failed", log_id=log_id,
message="部分失败",
error_trace=result.stderr[-500:],
started_at=started, finished_at=datetime.now(timezone.utc))
logger.warning("[Scheduled] Search cache refresh may have partial failures")
except subprocess.TimeoutExpired:
_log_task("scheduled_refresh_search_cache", "failed", log_id=log_id,
message="超时",
started_at=started, finished_at=datetime.now(timezone.utc))
logger.warning("[Scheduled] Search cache refresh timed out")
except Exception as e:
_log_task("scheduled_refresh_search_cache", "failed", log_id=log_id,
message=str(e),
error_trace=traceback.format_exc(),
started_at=started, finished_at=datetime.now(timezone.utc))
logger.exception("[Scheduled] Search cache refresh error: %s", e)
def _run_generate(self): def _run_generate(self):
_set_task_llm_provider("scheduled_generate")
started = datetime.now(timezone.utc) started = datetime.now(timezone.utc)
log_id = _log_task("scheduled_generate", "running", started_at=started) log_id = _log_task("scheduled_generate", "running", started_at=started)
try: try:
@@ -296,6 +272,7 @@ class TaskScheduler:
logger.exception("[Scheduled] Generation pipeline failed: %s", e) logger.exception("[Scheduled] Generation pipeline failed: %s", e)
def _run_optimize(self): def _run_optimize(self):
_set_task_llm_provider("scheduled_optimize")
started = datetime.now(timezone.utc) started = datetime.now(timezone.utc)
log_id = _log_task("scheduled_optimize", "running", started_at=started) log_id = _log_task("scheduled_optimize", "running", started_at=started)
try: try:
@@ -314,6 +291,7 @@ class TaskScheduler:
logger.exception("[Scheduled] Review failed: %s", e) logger.exception("[Scheduled] Review failed: %s", e)
def _run_collect(self): def _run_collect(self):
_set_task_llm_provider("scheduled_collect")
started = datetime.now(timezone.utc) started = datetime.now(timezone.utc)
log_id = _log_task("scheduled_collect", "running", started_at=started) log_id = _log_task("scheduled_collect", "running", started_at=started)
try: try:
@@ -334,6 +312,7 @@ class TaskScheduler:
def _run_optimize_sources(self, triggered_by="scheduler"): def _run_optimize_sources(self, triggered_by="scheduler"):
"""AI自动优化采集类别与信息源:对比市场热点和当前配置,给出调整建议""" """AI自动优化采集类别与信息源:对比市场热点和当前配置,给出调整建议"""
_set_task_llm_provider("scheduled_optimize_sources")
started = datetime.now(timezone.utc) started = datetime.now(timezone.utc)
log_id = _log_task("scheduled_optimize_sources", "running", started_at=started, triggered_by=triggered_by) log_id = _log_task("scheduled_optimize_sources", "running", started_at=started, triggered_by=triggered_by)
try: try:
@@ -397,6 +376,7 @@ class TaskScheduler:
def _run_metrics_sync(self, triggered_by="scheduler"): def _run_metrics_sync(self, triggered_by="scheduler"):
"""定时从各平台公开API获取发布文章的效果数据(当前仅支持知乎)""" """定时从各平台公开API获取发布文章的效果数据(当前仅支持知乎)"""
_set_task_llm_provider("scheduled_metrics_sync")
started = datetime.now(timezone.utc) started = datetime.now(timezone.utc)
log_id = _log_task("scheduled_metrics_sync", "running", started_at=started, triggered_by=triggered_by) log_id = _log_task("scheduled_metrics_sync", "running", started_at=started, triggered_by=triggered_by)
try: try:
+1
View File
@@ -47,6 +47,7 @@ def init_db():
conn.execute(text("ALTER TABLE llm_configs ADD COLUMN IF NOT EXISTS provider VARCHAR DEFAULT 'opencode-go'")) conn.execute(text("ALTER TABLE llm_configs ADD COLUMN IF NOT EXISTS provider VARCHAR DEFAULT 'opencode-go'"))
conn.execute(text("ALTER TABLE llm_configs ADD COLUMN IF NOT EXISTS base_url VARCHAR")) conn.execute(text("ALTER TABLE llm_configs ADD COLUMN IF NOT EXISTS base_url VARCHAR"))
conn.execute(text("ALTER TABLE llm_configs ADD COLUMN IF NOT EXISTS api_key VARCHAR")) conn.execute(text("ALTER TABLE llm_configs ADD COLUMN IF NOT EXISTS api_key VARCHAR"))
conn.execute(text("ALTER TABLE llm_configs ADD COLUMN IF NOT EXISTS is_default BOOLEAN DEFAULT FALSE"))
try: try:
conn.execute(text("ALTER TABLE articles ADD COLUMN IF NOT EXISTS images JSON DEFAULT '{}'::json")) conn.execute(text("ALTER TABLE articles ADD COLUMN IF NOT EXISTS images JSON DEFAULT '{}'::json"))
except Exception: except Exception:
+5 -6
View File
@@ -35,7 +35,7 @@ def import_initial_data():
db.commit() db.commit()
print(f"✅ 创建默认管理员: {DEFAULT_ADMIN_USERNAME}") print(f"✅ 创建默认管理员: {DEFAULT_ADMIN_USERNAME}")
# 补充或更新 LLM 供应商配置(opencode-go 为主,nvidia 为备) # 补充或更新 LLM 供应商配置(nvidia 为主,opencode-go 为备)
expected = { expected = {
"opencode-go": dict(provider="opencode-go", model="deepseek-v4-flash", "opencode-go": dict(provider="opencode-go", model="deepseek-v4-flash",
base_url="https://opencode.ai/zen/go/v1", temperature=0.7, max_tokens=131072, is_active=True, base_url="https://opencode.ai/zen/go/v1", temperature=0.7, max_tokens=131072, is_active=True,
@@ -74,11 +74,10 @@ def import_initial_data():
# 初始化默认搜索 API 提供商 # 初始化默认搜索 API 提供商
if db.query(SearchProvider).count() == 0: if db.query(SearchProvider).count() == 0:
providers = [ providers = [
SearchProvider(name="百度千帆", provider_type="baidu", api_key="", api_url="https://qianfan.baidubce.com/v2/ai_search/web_search", console_url="https://console.bce.baidu.com/qianfan/ais/console/onlineService", priority=1, enabled=True, daily_limit=50), SearchProvider(name="百度千帆", provider_type="baidu", api_key="", api_url="https://qianfan.baidubce.com/v2/ai_search/web_search", console_url="https://console.bce.baidu.com/qianfan/ais/console/onlineService", priority=1, enabled=True, daily_limit=200),
SearchProvider(name="opencode云搜索", provider_type="mcp", api_key="", api_url="", console_url="https://opencode.ai", priority=2, enabled=True, daily_limit=99999), SearchProvider(name="360搜索", provider_type="360", api_key="", api_url="", console_url="https://www.so.com", priority=0, enabled=True, daily_limit=99999),
SearchProvider(name="360搜索", provider_type="360", api_key="", api_url="", console_url="https://www.so.com", priority=3, enabled=True, daily_limit=200), SearchProvider(name="搜狗搜索", provider_type="sogou", api_key="", api_url="", console_url="https://sogou.com", priority=1, enabled=True, daily_limit=99999),
SearchProvider(name="狗搜索", provider_type="sogou", api_key="", api_url="", console_url="https://sogou.com", priority=4, enabled=True, daily_limit=200), SearchProvider(name="微信搜一", provider_type="wechat", api_key="", api_url="", console_url="https://wx.sogou.com/weixin", priority=2, enabled=True, daily_limit=99999),
SearchProvider(name="微信搜一搜", provider_type="wechat", api_key="", api_url="", console_url="https://wx.sogou.com/weixin", priority=5, enabled=True, daily_limit=200),
] ]
for p in providers: for p in providers:
db.add(p) db.add(p)
+2
View File
@@ -627,6 +627,7 @@ class LLMConfig(Base):
base_url = Column(String, nullable=True) base_url = Column(String, nullable=True)
api_key = Column(String, nullable=True) api_key = Column(String, nullable=True)
is_active = Column(Boolean, default=True) is_active = Column(Boolean, default=True)
is_default = Column(Boolean, default=False)
created_at = Column(DateTime(timezone=True), server_default=func.now()) created_at = Column(DateTime(timezone=True), server_default=func.now())
updated_at = Column(DateTime(timezone=True), onupdate=func.now()) updated_at = Column(DateTime(timezone=True), onupdate=func.now())
@@ -643,6 +644,7 @@ class LLMConfig(Base):
"base_url": self.base_url, "base_url": self.base_url,
"api_key": f"{self.api_key[:8]}..." if self.api_key else None, "api_key": f"{self.api_key[:8]}..." if self.api_key else None,
"is_active": self.is_active, "is_active": self.is_active,
"is_default": self.is_default,
"created_at": self.created_at.isoformat() if self.created_at else None, "created_at": self.created_at.isoformat() if self.created_at else None,
"updated_at": self.updated_at.isoformat() if self.updated_at else None, "updated_at": self.updated_at.isoformat() if self.updated_at else None,
} }
+1
View File
@@ -522,6 +522,7 @@ class LLMConfigBase(BaseModel):
base_url: Optional[str] = None base_url: Optional[str] = None
api_key: Optional[str] = None api_key: Optional[str] = None
is_active: bool = True is_active: bool = True
is_default: bool = False
class LLMConfigResponse(LLMConfigBase): class LLMConfigResponse(LLMConfigBase):
+16 -11
View File
@@ -136,10 +136,15 @@
</el-table-column> </el-table-column>
<el-table-column prop="temperature" label="温度" width="70"></el-table-column> <el-table-column prop="temperature" label="温度" width="70"></el-table-column>
<el-table-column prop="max_tokens" label="最大Token" width="100"></el-table-column> <el-table-column prop="max_tokens" label="最大Token" width="100"></el-table-column>
<el-table-column prop="is_active" label="状态" width="90"> <el-table-column prop="is_active" label="状态" width="80">
<template #default="scope"> <template #default="scope">
<el-tag v-if="scope.row.is_active" type="success" size="small" effect="dark">当前使</el-tag> <el-tag v-if="scope.row.is_active" type="success" size="small" effect="dark"></el-tag>
<el-tag v-else type="info" size="small">未启</el-tag> <el-tag v-else type="info" size="small"></el-tag>
</template>
</el-table-column>
<el-table-column prop="is_default" label="默认" width="70">
<template #default="scope">
<el-tag v-if="scope.row.is_default" type="warning" size="small" effect="dark">默认</el-tag>
</template> </template>
</el-table-column> </el-table-column>
<el-table-column label="系统提示词" min-width="150"> <el-table-column label="系统提示词" min-width="150">
@@ -556,7 +561,6 @@
<el-col :span="12"> <el-form-item label="类型"> <el-col :span="12"> <el-form-item label="类型">
<el-select v-model="searchProviderForm.provider_type" style="width:100%"> <el-select v-model="searchProviderForm.provider_type" style="width:100%">
<el-option label="百度千帆" value="baidu"></el-option> <el-option label="百度千帆" value="baidu"></el-option>
<el-option label="opencode云搜索" value="mcp"></el-option>
<el-option label="360搜索" value="360"></el-option> <el-option label="360搜索" value="360"></el-option>
<el-option label="搜狗搜索" value="sogou"></el-option> <el-option label="搜狗搜索" value="sogou"></el-option>
<el-option label="微信搜一搜" value="wechat"></el-option> <el-option label="微信搜一搜" value="wechat"></el-option>
@@ -623,7 +627,8 @@
</el-row> </el-row>
<el-row :gutter="16"> <el-row :gutter="16">
<el-col :span="12"><el-form-item label="最大Token"><el-input-number v-model="llmConfigForm.max_tokens" :min="1" :max="999999" style="width:100%"/></el-form-item></el-col> <el-col :span="12"><el-form-item label="最大Token"><el-input-number v-model="llmConfigForm.max_tokens" :min="1" :max="999999" style="width:100%"/></el-form-item></el-col>
<el-col :span="12"><el-form-item label="激活"><el-switch v-model="llmConfigForm.is_active"/></el-form-item></el-col> <el-col :span="6"><el-form-item label="激活"><el-switch v-model="llmConfigForm.is_active"/></el-form-item></el-col>
<el-col :span="6"><el-form-item label="默认"><el-switch v-model="llmConfigForm.is_default"/></el-form-item></el-col>
</el-row> </el-row>
<el-row :gutter="16"> <el-row :gutter="16">
<el-col :span="24"><el-form-item label="系统提示词"><el-input type="textarea" v-model="llmConfigForm.system_prompt" :rows="3" placeholder="你是一个专业的内容创作助手。"/></el-form-item></el-col> <el-col :span="24"><el-form-item label="系统提示词"><el-input type="textarea" v-model="llmConfigForm.system_prompt" :rows="3" placeholder="你是一个专业的内容创作助手。"/></el-form-item></el-col>
@@ -706,7 +711,7 @@ const llmConfigs = ref([]);
const llmConfigsLoading = ref(false); const llmConfigsLoading = ref(false);
const llmConfigDialogVisible = ref(false); const llmConfigDialogVisible = ref(false);
const llmConfigDialogTitle = ref('新增 LLM 配置'); const llmConfigDialogTitle = ref('新增 LLM 配置');
const llmConfigForm = reactive({ id: null, name: '', system_prompt: '', user_prompt_template: '', temperature: 0.7, max_tokens: 131072, model: '', provider: 'opencode-go', base_url: '', api_key: '', is_active: true }); const llmConfigForm = reactive({ id: null, name: '', system_prompt: '', user_prompt_template: '', temperature: 0.7, max_tokens: 131072, model: '', provider: 'opencode-go', base_url: '', api_key: '', is_active: true, is_default: false });
const editingLLMConfigId = ref(null); const editingLLMConfigId = ref(null);
const loadLLMConfigs = async () => { const loadLLMConfigs = async () => {
llmConfigsLoading.value = true; llmConfigsLoading.value = true;
@@ -721,7 +726,7 @@ const llmConfigs = ref([]);
llmConfigDialogTitle.value = '新增配置'; editingLLMConfigId.value = null; llmConfigDialogTitle.value = '新增配置'; editingLLMConfigId.value = null;
llmConfigForm.id = null; llmConfigForm.name = ''; llmConfigForm.provider = 'sensenova'; llmConfigForm.model = 'deepseek-v4-flash'; llmConfigForm.id = null; llmConfigForm.name = ''; llmConfigForm.provider = 'sensenova'; llmConfigForm.model = 'deepseek-v4-flash';
llmConfigForm.base_url = 'https://token.sensenova.cn/v1'; llmConfigForm.api_key = ''; llmConfigForm.temperature = 0.3; llmConfigForm.base_url = 'https://token.sensenova.cn/v1'; llmConfigForm.api_key = ''; llmConfigForm.temperature = 0.3;
llmConfigForm.max_tokens = 4000; llmConfigForm.system_prompt = ''; llmConfigForm.user_prompt_template = ''; llmConfigForm.is_active = true; llmConfigForm.max_tokens = 4000; llmConfigForm.system_prompt = ''; llmConfigForm.user_prompt_template = ''; llmConfigForm.is_active = true; llmConfigForm.is_default = false;
} }
llmConfigDialogVisible.value = true; llmConfigDialogVisible.value = true;
}; };
@@ -1101,12 +1106,12 @@ const llmConfigs = ref([]);
}); });
const defaultProvider = computed(() => { const defaultProvider = computed(() => {
const sorted = [...llmConfigs.value].filter(x => x.is_active).sort((a, b) => a.id - b.id); const d = [...llmConfigs.value].filter(x => x.is_default);
return sorted.length ? sorted[0].provider : '-'; return d.length ? d[0].provider : '-';
}); });
const defaultModel = computed(() => { const defaultModel = computed(() => {
const sorted = [...llmConfigs.value].filter(x => x.is_active).sort((a, b) => a.id - b.id); const d = [...llmConfigs.value].filter(x => x.is_default);
return sorted.length ? sorted[0].model : '-'; return d.length ? d[0].model : '-';
}); });
const complianceProvider = computed(() => { const complianceProvider = computed(() => {
const c = [...llmConfigs.value].filter(x => x.is_active && x.model === 'deepseek-v4-flash'); const c = [...llmConfigs.value].filter(x => x.is_active && x.model === 'deepseek-v4-flash');
-1
View File
@@ -384,7 +384,6 @@
this.runningModule = modId; this.runningModule = modId;
const endpoints = { const endpoints = {
scheduled_collect: '/api/system/collect/run', scheduled_collect: '/api/system/collect/run',
scheduled_refresh_search_cache: '/api/system/refresh-search-cache/run',
scheduled_fetch_trends: '/api/system/trends/run', scheduled_fetch_trends: '/api/system/trends/run',
scheduled_generate: '/api/system/generate/run', scheduled_generate: '/api/system/generate/run',
scheduled_optimize: '/api/system/review/run', scheduled_optimize: '/api/system/review/run',
+25 -7
View File
@@ -103,6 +103,7 @@
<div><span>最后运行</span><span>{{ mod.last_run || '从未' }}<span v-if="mod.last_status" :style="{marginLeft:'6px',padding:'1px 6px',borderRadius:'8px',fontSize:'11px',fontWeight:500}"><span v-if="mod.last_status==='success'" style="color:#67c23a;">✅成功</span><span v-else-if="mod.last_status==='failed'" style="color:#f56c6c;">❌失败</span><span v-else-if="mod.last_status==='running'" style="color:#e6a23c;">⏳运行中</span></span></span></div> <div><span>最后运行</span><span>{{ mod.last_run || '从未' }}<span v-if="mod.last_status" :style="{marginLeft:'6px',padding:'1px 6px',borderRadius:'8px',fontSize:'11px',fontWeight:500}"><span v-if="mod.last_status==='success'" style="color:#67c23a;">✅成功</span><span v-else-if="mod.last_status==='failed'" style="color:#f56c6c;">❌失败</span><span v-else-if="mod.last_status==='running'" style="color:#e6a23c;">⏳运行中</span></span></span></div>
<div><span>下次运行</span><span>{{ mod.next_run || '—' }}</span></div> <div><span>下次运行</span><span>{{ mod.next_run || '—' }}</span></div>
<div><span>累计运行</span><span>{{ mod.total_runs }} 次 <span style="color:#67c23a;">{{ mod.success_runs }} 成功</span> <span style="color:#f56c6c;">{{ mod.failed_runs }} 失败</span><span v-if="mod.running > 0" style="color:#e6a23c;"> {{ mod.running }} 运行中</span></span></div> <div><span>累计运行</span><span>{{ mod.total_runs }} 次 <span style="color:#67c23a;">{{ mod.success_runs }} 成功</span> <span style="color:#f56c6c;">{{ mod.failed_runs }} 失败</span><span v-if="mod.running > 0" style="color:#e6a23c;"> {{ mod.running }} 运行中</span></span></div>
<div><span>LLM 模型</span><span>{{ (mod.params && mod.params.llm_provider) || (defaultLlmLabel || '默认') }}</span></div>
<div style="margin-top:10px; border-bottom:none;"> <div style="margin-top:10px; border-bottom:none;">
<el-button size="small" type="primary" @click.stop="triggerModule(mod.module_id)" :loading="runningModule === mod.module_id" :disabled="!mod.enabled">立即运行</el-button> <el-button size="small" type="primary" @click.stop="triggerModule(mod.module_id)" :loading="runningModule === mod.module_id" :disabled="!mod.enabled">立即运行</el-button>
<el-button size="small" @click.stop="openModuleDetail(mod)">查看详情</el-button> <el-button size="small" @click.stop="openModuleDetail(mod)">查看详情</el-button>
@@ -346,10 +347,17 @@
<el-input v-model="drawerData.schedule" size="small" style="width:120px;" placeholder="HH:MM" :disabled="savingConfig"></el-input> <el-input v-model="drawerData.schedule" size="small" style="width:120px;" placeholder="HH:MM" :disabled="savingConfig"></el-input>
<span style="font-size:12px;color:#909399;">每日执行时间(HH:MM</span> <span style="font-size:12px;color:#909399;">每日执行时间(HH:MM</span>
</div> </div>
<div style="display:flex;gap:16px;align-items:center;margin-bottom:12px;">
<span style="font-size:13px;color:#606266;width:60px;">LLM</span>
<el-select v-model="drawerData.params.llm_provider" size="small" style="width:200px;" clearable placeholder="系统默认" @change="saveModuleConfig" :disabled="savingConfig">
<el-option v-for="c in llmConfigs.filter(x=>x.is_active)" :key="c.id" :label="c.provider + ' (' + c.model + ')' + (c.is_default ? ' (默认)' : '')" :value="c.provider"/>
</el-select>
<span style="font-size:12px;color:#909399;">留空则使用系统默认模型</span>
</div> </div>
<div v-if="Object.keys(drawerData.params || {}).length > 0"> </div>
<div v-if="Object.keys(drawerData.params || {}).filter(k => k !== 'llm_provider').length > 0">
<div style="font-size:13px;font-weight:600;margin-bottom:12px;">参数配置</div> <div style="font-size:13px;font-weight:600;margin-bottom:12px;">参数配置</div>
<div v-for="(val, key) in drawerData.params" :key="key" style="margin-bottom:12px;display:flex;align-items:center;gap:12px;"> <div v-for="(val, key) in drawerData.params" v-if="key !== 'llm_provider'" :key="key" style="margin-bottom:12px;display:flex;align-items:center;gap:12px;">
<span style="font-size:13px;color:#606266;width:120px;">{{ key }}</span> <span style="font-size:13px;color:#606266;width:120px;">{{ key }}</span>
<el-input v-if="typeof val === 'string'" v-model="drawerData.params[key]" size="small" style="flex:1;" @change="saveModuleConfig" :disabled="savingConfig"></el-input> <el-input v-if="typeof val === 'string'" v-model="drawerData.params[key]" size="small" style="flex:1;" @change="saveModuleConfig" :disabled="savingConfig"></el-input>
<el-input-number v-else-if="typeof val === 'number'" v-model="drawerData.params[key]" size="small" :disabled="savingConfig" @change="saveModuleConfig"></el-input-number> <el-input-number v-else-if="typeof val === 'number'" v-model="drawerData.params[key]" size="small" :disabled="savingConfig" @change="saveModuleConfig"></el-input-number>
@@ -552,13 +560,11 @@ const TasksApp = {
'scheduled_optimize': { icon: 'IconSearch', name: '合规审查', defaultTime: '04:30' }, 'scheduled_optimize': { icon: 'IconSearch', name: '合规审查', defaultTime: '04:30' },
'scheduled_optimize_sources': { icon: 'IconSetting', name: '信息源优化', defaultTime: '05:00' }, 'scheduled_optimize_sources': { icon: 'IconSetting', name: '信息源优化', defaultTime: '05:00' },
'scheduled_metrics_sync': { icon: 'IconDashboard', name: '指标同步', defaultTime: '06:00' }, 'scheduled_metrics_sync': { icon: 'IconDashboard', name: '指标同步', defaultTime: '06:00' },
'scheduled_refresh_search_cache': { icon: 'IconRefresh', name: '搜索缓存', defaultTime: '01:00' },
'scheduled_fetch_trends': { icon: 'IconRefresh', name: '热点趋势', defaultTime: '01:10' }, 'scheduled_fetch_trends': { icon: 'IconRefresh', name: '热点趋势', defaultTime: '01:10' },
'scheduled_task_monitor': { icon: 'IconRefresh', name: '任务监控', defaultTime: '*' }, 'scheduled_task_monitor': { icon: 'IconRefresh', name: '任务监控', defaultTime: '*' },
}; };
const MODULE_TRIGGER_ENDPOINTS = { const MODULE_TRIGGER_ENDPOINTS = {
scheduled_collect: '/api/system/collect/run', scheduled_collect: '/api/system/collect/run',
scheduled_refresh_search_cache: '/api/system/refresh-search-cache/run',
scheduled_fetch_trends: '/api/system/trends/run', scheduled_fetch_trends: '/api/system/trends/run',
scheduled_generate: '/api/system/generate/run', scheduled_generate: '/api/system/generate/run',
scheduled_optimize: '/api/system/review/run', scheduled_optimize: '/api/system/review/run',
@@ -592,6 +598,7 @@ const TasksApp = {
sourceForm: { name: '', source_type: 'web_search', query: '', credibility: 'medium', focus: '', sort_order: 0, is_active: true }, sourceForm: { name: '', source_type: 'web_search', query: '', credibility: 'medium', focus: '', sort_order: 0, is_active: true },
sourcePage: 1, sourcePageSize: 10, sourcePage: 1, sourcePageSize: 10,
SCHEDULER_JOBS, MODULE_TRIGGER_ENDPOINTS, SCHEDULER_JOBS, MODULE_TRIGGER_ENDPOINTS,
llmConfigs: [], defaultLlmLabel: '',
} }
}, },
computed: { computed: {
@@ -645,17 +652,28 @@ const TasksApp = {
async loadModules() { async loadModules() {
this.moduleLoading = true; this.moduleLoading = true;
try { try {
const data = await this.api('/api/system/modules/status'); const [data, llmConfigs] = await Promise.all([
this.api('/api/system/modules/status'),
this.loadLLMConfigs(),
]);
if (!data) return; if (!data) return;
this.modules = data.modules || []; this.modules = data.modules || [];
this.schedulerRunning = data.scheduler && data.scheduler.running === true; this.schedulerRunning = data.scheduler && data.scheduler.running === true;
} catch (e) { console.error(e); } } catch (e) { console.error(e); }
finally { this.moduleLoading = false; } finally { this.moduleLoading = false; }
}, },
async loadLLMConfigs() {
try {
const configs = await this.api('/api/admin/llmconfigs');
this.llmConfigs = configs || [];
const def = (configs || []).find(c => c.is_default);
this.defaultLlmLabel = def ? def.provider + ' (' + def.model + ')' : '';
} catch (e) { console.error('loadLLMConfigs error:', e); this.llmConfigs = []; }
},
async openModuleDetail(mod) { async openModuleDetail(mod) {
this.showDrawer = true; this.showDrawer = true;
this.drawerTitle = mod.title + ' 详情'; this.drawerTitle = mod.title + ' 详情';
this.drawerData = { ...mod }; this.drawerData = { ...mod, params: { ...(mod.params || {}) } };
this.drawerError = ''; this.drawerError = '';
this.drawerLoading = true; this.drawerLoading = true;
this.drawerTab = 'inputs'; this.drawerTab = 'inputs';
@@ -667,7 +685,7 @@ const TasksApp = {
this.api('/api/admin/task-configs/history/' + mod.module_id + '?limit=20'), this.api('/api/admin/task-configs/history/' + mod.module_id + '?limit=20'),
this.api('/api/admin/prompt-configs?module_id=' + mod.module_id), this.api('/api/admin/prompt-configs?module_id=' + mod.module_id),
]); ]);
this.drawerData = { ...mod, ...detail }; this.drawerData = { ...mod, ...detail, params: { ...((detail.params || mod.params || {})) } };
this.drawerHistory = history || []; this.drawerHistory = history || [];
this.drawerPrompts = prompts || []; this.drawerPrompts = prompts || [];
} catch (e) { this.drawerError = e.message; } } catch (e) { this.drawerError = e.message; }
+6 -6
View File
@@ -3,7 +3,7 @@
合规审查:文章合规检查 → LLM迭代修复 合规审查:文章合规检查 → LLM迭代修复
从articles表读取待审文章,进行合规评分;不合格文章由LLM修复(最多3次),通过后更新选题状态为待发布 从articles表读取待审文章,进行合规评分;不合格文章由LLM修复(最多3次),通过后更新选题状态为待发布
""" """
import json, datetime, logging, sys, re import json, os, datetime, logging, sys, re
from pathlib import Path from pathlib import Path
from typing import Dict, List, Optional, Tuple from typing import Dict, List, Optional, Tuple
from dataclasses import dataclass, asdict from dataclasses import dataclass, asdict
@@ -165,7 +165,7 @@ def polish_with_llm(html: str, platform: str, remaining_issues: Optional[List[Di
"""用 LLM 优化文章内容,返回 (html, log_message_or_None) """用 LLM 优化文章内容,返回 (html, log_message_or_None)
如果指定 remaining_issues,则针对性修复合规问题 如果指定 remaining_issues,则针对性修复合规问题
LLM 失败时自动重试一次 LLM 失败时自动重试一次
固定使用 opencode-go (deepseek-v4-flash) — 审查用更好的模型 LLM 提供者由 LLM_TASK_PROVIDER 环境变量决定(scheduler 从 TaskConfig 读取设置)
""" """
if not HAVE_LLM: if not HAVE_LLM:
return html, None return html, None
@@ -182,16 +182,15 @@ def polish_with_llm(html: str, platform: str, remaining_issues: Optional[List[Di
prompt = get_prompt("compliance_fix", issues_desc=issues_desc, html=html) prompt = get_prompt("compliance_fix", issues_desc=issues_desc, html=html)
else: else:
prompt = get_prompt("compliance_polish", html=html) prompt = get_prompt("compliance_polish", html=html)
polished = call_llm(prompt, provider="sensenova", temperature=temperature, max_tokens=max_tokens, system_prompt=system_prompt) polished = call_llm(prompt, temperature=temperature, max_tokens=max_tokens, system_prompt=system_prompt)
polished = clean_html_content(polished) polished = clean_html_content(polished)
polished = strip_ai_preface(polished) polished = strip_ai_preface(polished)
polished = strip_thinking_html(polished) polished = strip_thinking_html(polished)
if '<h2' in polished or '<p>' in polished: if '<h2' in polished or '<p>' in polished:
if len(polished) > len(html) * 0.3 and len(polished) > 100: if len(polished) > len(html) * 0.3 and len(polished) > 100:
if not any(kw in polished for kw in ['保留', '建议', '可以', '应该', '推荐', '改为', '替换为']):
tag = "针对性修复" if remaining_issues else "常规润色" tag = "针对性修复" if remaining_issues else "常规润色"
return polished, f"LLM {tag}" return polished, f"LLM {tag}"
logger.warning(f"LLM 优化输出异常(过短或含建议性文字),保留原文 (len={len(polished)})") logger.warning(f"LLM 优化输出过短,保留原文 (len={len(polished)})")
except Exception as e: except Exception as e:
logger.warning(f"LLM 优化失败 (尝试 {attempt+1}/2): {e}") logger.warning(f"LLM 优化失败 (尝试 {attempt+1}/2): {e}")
if attempt == 0: if attempt == 0:
@@ -229,7 +228,8 @@ def _load_platform_configs() -> Dict[str, Dict]:
def main(topic_ids: List[str] = None, today_only: bool = False): def main(topic_ids: List[str] = None, today_only: bool = False):
logger.info("=== 合规审查与优化开始 ===") logger.info("=== 合规审查与优化开始 ===")
logger.info("LLM 配置: opencode-go (model=deepseek-v4-flash) — 固定用于合规审查") llm_provider = os.getenv("LLM_TASK_PROVIDER", "sensenova")
logger.info(f"LLM 配置: {llm_provider} — 合规审查")
platform_configs = _load_platform_configs() platform_configs = _load_platform_configs()
logger.info(f"已加载 {len(platform_configs)} 个平台配置") logger.info(f"已加载 {len(platform_configs)} 个平台配置")
-300
View File
@@ -1,300 +0,0 @@
#!/usr/bin/env python3
"""
MCP Search Server — provides web search via opencode infrastructure.
Two search methods (automatic fallback):
1. npx opencode run (rate-limited but returns real web results)
2. opencode-go API + model training data (no rate limit, less fresh)
Usage:
python3 scripts/mcp_search_server.py # MCP server (stdio)
python3 scripts/mcp_search_server.py --query Q # one-shot search
python3 scripts/mcp_search_server.py --url U # one-shot webfetch
"""
import json, os, subprocess, sys, time
from pathlib import Path
from typing import Any, Dict, List, Optional
PROJECT_ROOT = Path(__file__).resolve().parent.parent
CACHE_FILE = PROJECT_ROOT / "automation" / "data" / "mcp_search_cache.json"
SESSION_FILE = PROJECT_ROOT / "automation" / "data" / "mcp_session.txt"
CACHE_TTL = 3600
SESSION_TITLE = "opencode搜索"
API_BASE = "https://opencode.ai/zen/go/v1"
API_KEY = os.environ.get("OPENCODE_API_KEY", "")
if not API_KEY:
try:
from dotenv import load_dotenv
env_path = PROJECT_ROOT / "platform" / "backend" / ".env"
load_dotenv(env_path)
API_KEY = os.environ.get("OPENCODE_API_KEY", "")
except Exception:
pass
# ── session (reuse same session for all MCP searches) ─────────────
def _load_session() -> Optional[str]:
if SESSION_FILE.exists():
try:
return SESSION_FILE.read_text().strip() or None
except Exception:
pass
return None
def _save_session_from_output(stdout: str):
for line in stdout.strip().split("\n"):
try:
ev = json.loads(line)
sid = ev.get("sessionID") or ev.get("part", {}).get("sessionID")
if sid:
SESSION_FILE.parent.mkdir(parents=True, exist_ok=True)
SESSION_FILE.write_text(sid)
return
except Exception:
pass
# ── cache ─────────────────────────────────────────────────────────
def _check_cache(query: str) -> Optional[List[Dict]]:
if not CACHE_FILE.exists():
return None
try:
data = json.loads(CACHE_FILE.read_text())
entry = data.get(query)
if entry and time.time() - entry.get("ts", 0) < CACHE_TTL:
return entry.get("results")
except Exception:
pass
return None
def _write_cache(query: str, results: List[Dict]):
CACHE_FILE.parent.mkdir(parents=True, exist_ok=True)
data = {}
if CACHE_FILE.exists():
try:
data = json.loads(CACHE_FILE.read_text())
except Exception:
pass
data[query] = {"ts": time.time(), "results": results}
keys = sorted(data.keys(), key=lambda k: data[k].get("ts", 0), reverse=True)[:200]
CACHE_FILE.write_text(json.dumps({k: data[k] for k in keys}, ensure_ascii=False))
# ── method 1: npx opencode run ────────────────────────────────────
def _search_via_opencode_cli(query: str, max_results: int) -> Optional[List[Dict]]:
"""Use npx opencode run to execute websearch tool (short timeout)."""
sid = _load_session()
args = ["npx", "opencode", "run", f"websearch {query}", "--format", "json", "--title", SESSION_TITLE]
if sid:
args.extend(["--session", sid, "--continue"])
try:
r = subprocess.run(
args, capture_output=True, text=True, timeout=15,
env={**os.environ, "OPENCODE_DISABLE_AUTOUPDATE": "1"}
)
except subprocess.TimeoutExpired:
return None
except Exception:
return None
if r.returncode != 0:
return None
# Save session ID for reuse
_save_session_from_output(r.stdout)
for line in r.stdout.strip().split("\n"):
try:
ev = json.loads(line)
if ev.get("type") == "tool_use":
part = ev.get("part", {})
state = part.get("state", {})
if part.get("tool") == "websearch" and state.get("status") == "completed":
data = json.loads(state["output"])
results = []
for item in (data.get("results") or [])[:max_results]:
url = (item.get("url") or "").strip()
title = (item.get("title") or "").strip()
excerpts = item.get("excerpts") or []
content = (excerpts[0] if excerpts else "")[:500]
if url and title:
results.append({"title": title, "url": url, "content": content, "source": "opencode_cli"})
return results
except Exception:
pass
return None
# ── method 2: opencode-go API + training data ─────────────────────
def _search_via_api(query: str, max_results: int) -> Optional[List[Dict]]:
"""Use opencode-go API to answer query from training data (no rate limit)."""
if not API_KEY:
return None
import requests
try:
resp = requests.post(
f"{API_BASE}/chat/completions",
headers={"Authorization": f"Bearer {API_KEY}", "Content-Type": "application/json"},
json={
"model": "deepseek-v4-flash",
"messages": [{"role": "user", "content": (
f"你现在是一个网络搜索工具。用户查询: {query[:100]}\n\n"
f"请根据你的训练数据,提供{max_results}条最相关的网页结果,包含标题、URL和摘要。"
f"以JSON格式输出: [{{\"title\":\"...\",\"url\":\"...\",\"content\":\"...\"}}]"
f"仅输出JSON数组,不要其他文字。如果URL不确定,用合理占位。"
)}],
"temperature": 0.3,
"max_tokens": 2000,
},
timeout=30
)
data = resp.json()
content = data.get("choices", [{}])[0].get("message", {}).get("content", "")
# Extract JSON array
import re as _re
m = _re.search(r'\[.*?\]', content, _re.DOTALL)
if m:
items = json.loads(m.group())
if isinstance(items, list):
for item in items:
item["source"] = "opencode_api"
return items[:max_results]
except Exception:
pass
return None
# ── search ─────────────────────────────────────────────────────────
def web_search(query: str, max_results: int = 8) -> List[Dict]:
max_results = min(max_results, 10)
cached = _check_cache(query)
if cached:
return cached[:max_results]
results = _search_via_opencode_cli(query, max_results)
if results:
_write_cache(query, results)
return results
results = _search_via_api(query, max_results)
if results:
_write_cache(query, results)
return results
return []
def webfetch(url: str) -> Optional[str]:
sid = _load_session()
args = ["npx", "opencode", "run", f"webfetch {url}", "--format", "json", "--title", SESSION_TITLE]
if sid:
args.extend(["--session", sid, "--continue"])
try:
r = subprocess.run(
args, capture_output=True, text=True, timeout=60,
env={**os.environ, "OPENCODE_DISABLE_AUTOUPDATE": "1"}
)
if r.returncode == 0:
_save_session_from_output(r.stdout)
for line in r.stdout.strip().split("\n"):
try:
ev = json.loads(line)
if ev.get("type") == "tool_use":
p = ev.get("part", {})
s = p.get("state", {})
if p.get("tool") == "webfetch" and s.get("status") == "completed":
return s.get("output", "")[:10000]
except Exception:
pass
except Exception:
pass
return None
# ── MCP protocol (JSON-RPC 2.0 over stdio) ────────────────────────
def _read_msg() -> Optional[Dict]:
line = sys.stdin.readline()
if not line:
return None
try:
return json.loads(line)
except json.JSONDecodeError:
return None
def _send_msg(msg: Dict):
sys.stdout.write(json.dumps(msg, ensure_ascii=False) + "\n")
sys.stdout.flush()
def _send_error(req_id: Any, code: int, message: str):
_send_msg({"jsonrpc": "2.0", "id": req_id, "error": {"code": code, "message": message}})
def _send_result(req_id: Any, result: Any):
_send_msg({"jsonrpc": "2.0", "id": req_id, "result": result})
def serve():
sys.stdin.reconfigure(encoding="utf-8")
sys.stdout.reconfigure(encoding="utf-8")
while True:
msg = _read_msg()
if msg is None:
break
req_id = msg.get("id")
method = msg.get("method", "")
params = msg.get("params", {})
if method == "initialize":
_send_result(req_id, {
"protocolVersion": "2024-11-05",
"capabilities": {"tools": {"listChanged": False}},
"serverInfo": {"name": "opencode-search-mcp", "version": "1.0.0"}
})
elif method == "notifications/initialized":
pass
elif method == "tools/list":
_send_result(req_id, {"tools": [
{"name": "web_search", "description": "Search the web. Returns up to 10 results with title, url, content.", "inputSchema": {
"type": "object", "properties": {
"query": {"type": "string", "description": "Search query"},
"max_results": {"type": "number", "description": "Max results (1-10)", "default": 8}
}, "required": ["query"]
}},
{"name": "webfetch", "description": "Fetch and extract content from a URL.", "inputSchema": {
"type": "object", "properties": {"url": {"type": "string", "description": "URL to fetch"}},
"required": ["url"]
}}
]})
elif method == "tools/call":
name = params.get("name", "")
args = params.get("arguments", {})
try:
if name == "web_search":
results = web_search(args.get("query", ""), int(args.get("max_results", 8)))
_send_result(req_id, {"content": [{"type": "text", "text": json.dumps(results, ensure_ascii=False)}]})
elif name == "webfetch":
content = webfetch(args.get("url", ""))
_send_result(req_id, {"content": [{"type": "text", "text": content or "Failed to fetch URL"}]})
else:
_send_error(req_id, -32601, f"Unknown tool: {name}")
except Exception as e:
_send_error(req_id, -32603, str(e))
elif method == "shutdown":
_send_result(req_id, {})
break
else:
_send_error(req_id, -32601, f"Unknown method: {method}")
def main():
if "--query" in sys.argv:
idx = sys.argv.index("--query")
q = sys.argv[idx + 1] if idx + 1 < len(sys.argv) else ""
print(json.dumps(web_search(q), ensure_ascii=False, indent=2))
return
if "--url" in sys.argv:
idx = sys.argv.index("--url")
u = sys.argv[idx + 1] if idx + 1 < len(sys.argv) else ""
print(webfetch(u) or "Failed")
return
serve()
if __name__ == "__main__":
main()
-196
View File
@@ -1,196 +0,0 @@
#!/usr/bin/env python3
"""
通过 opencode CLI 执行联网搜索
利用 opencode 的 webfetch 能力(当前 AI 环境可无障碍访问互联网)
用法:
python3 scripts/opencode_search.py --query "可持续生活 趋势 2026"
python3 scripts/opencode_search.py --refresh-cache # 刷新所有分类的缓存
"""
import argparse, datetime, json, logging, os, re, subprocess, sys, time
from pathlib import Path
from typing import Dict, List, Optional
PROJECT_ROOT = Path(__file__).parent.parent
sys.path.insert(0, str(PROJECT_ROOT))
LOGS_DIR = PROJECT_ROOT / "automation" / "logs"
TODAY = datetime.datetime.now().strftime("%Y-%m-%d")
LOG_FILE = LOGS_DIR / f"opencode_search_{TODAY}.log"
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
handlers=[logging.FileHandler(LOG_FILE, encoding='utf-8'), logging.StreamHandler()])
logger = logging.getLogger(__name__)
SEARCH_CACHE_FILE = PROJECT_ROOT / "automation" / "data" / "search_cache.json"
SESSION_FILE = PROJECT_ROOT / "automation" / "data" / "opencode_session.txt"
def _get_or_create_session() -> Optional[str]:
"""获取或创建持久 session ID"""
if SESSION_FILE.exists():
try:
sid = SESSION_FILE.read_text().strip()
if sid:
result = subprocess.run(
["npx", "opencode", "run", "ping", "--session", sid, "--format", "json"],
capture_output=True, text=True, timeout=10,
cwd=str(PROJECT_ROOT),
env={**os.environ, "OPENCODE_DISABLE_AUTOUPDATE": "1"}
)
if result.returncode == 0:
return sid
except Exception:
pass
result = subprocess.run(
["npx", "opencode", "run", "init", "--format", "json"],
capture_output=True, text=True, timeout=30,
cwd=str(PROJECT_ROOT),
env={**os.environ, "OPENCODE_DISABLE_AUTOUPDATE": "1"}
)
for line in result.stdout.strip().split("\n"):
try:
event = json.loads(line)
sid = event.get("sessionID") or event.get("part", {}).get("sessionID")
if sid:
SESSION_FILE.write_text(sid)
return sid
except Exception:
pass
return None
_session_id = None
def _run_opencode(prompt: str, timeout: int = 60) -> Optional[str]:
"""调用 opencode run 执行任务,返回文本输出"""
global _session_id
if _session_id is None:
_session_id = _get_or_create_session()
args = ["npx", "opencode", "run", prompt, "--format", "json"]
if _session_id:
args.extend(["--session", _session_id, "--continue"])
try:
result = subprocess.run(
args,
capture_output=True, text=True, timeout=timeout,
cwd=str(PROJECT_ROOT),
env={**os.environ, "OPENCODE_DISABLE_AUTOUPDATE": "1"}
)
if result.returncode != 0:
logger.warning(f"opencode run 返回非零: {result.stderr[:200]}")
return None
for line in result.stdout.strip().split("\n"):
try:
event = json.loads(line)
if event.get("type") == "error":
logger.warning(f"opencode 错误: {event}")
return None
except json.JSONDecodeError:
pass
lines = []
for line in result.stdout.strip().split("\n"):
try:
event = json.loads(line)
if event.get("type") == "text":
text = event.get("part", {}).get("text", "")
if text:
lines.append(text)
except json.JSONDecodeError:
pass
output = "\n".join(lines).strip()
return output if output else None
except subprocess.TimeoutExpired:
logger.warning(f"opencode run 超时 ({timeout}s)")
return None
except Exception as e:
logger.warning(f"opencode run 失败: {e}")
return None
def search_via_opencode(query: str, max_results: int = 5) -> List[Dict]:
"""通过 MCP 搜索工具联网搜索(替代脆弱的 npx prompt 方式)"""
try:
from search_utils import search
return search(query, max_results)
except Exception as e:
logger.warning("search_utils 不可用,回退子进程: %s", e)
result = subprocess.run(
[sys.executable, str(PROJECT_ROOT / "scripts" / "mcp_search_server.py"),
"--query", query],
capture_output=True, text=True, timeout=90,
)
if result.returncode == 0:
try:
return json.loads(result.stdout)[:max_results]
except Exception:
pass
return []
def refresh_cache():
"""刷新所有搜索分类的缓存"""
try:
with open(PROJECT_ROOT / "config" / "sources.yaml") as f:
import yaml
cfg = yaml.safe_load(f)
queries = [s["query"] for s in cfg["sustainability_sources"]["web_search"]]
except Exception:
logger.warning("无法读取 sources.yaml,使用默认查询")
queries = [
"以旧换新 二手交易 循环 2026",
"新能源车 绿色通勤 低碳 2026",
"干净饮食 有机食品 2026",
"零浪费 极简生活 可持续时尚 2026",
"绿色家电 一级能效 节能 2026",
"碳账户 碳普惠 个人碳减排 2026",
"环保科技 绿色产品 可持续材料 2026",
"AI工具 人工智能 效率提升 2026",
]
cache = {"_metadata": {"updated_at": datetime.datetime.now().isoformat()}}
if SEARCH_CACHE_FILE.exists():
try:
old = json.loads(SEARCH_CACHE_FILE.read_text(encoding="utf-8"))
for k, v in old.items():
if not k.startswith("_"):
cache.setdefault(k, v)
except Exception:
pass
for i, q in enumerate(queries):
logger.info(f"[{i+1}/{len(queries)}] 搜索: {q}")
results = search_via_opencode(q, max_results=4)
if results:
cache[q] = results
else:
logger.warning(f" {q} 搜索无结果,保留旧缓存")
time.sleep(2)
SEARCH_CACHE_FILE.parent.mkdir(parents=True, exist_ok=True)
SEARCH_CACHE_FILE.write_text(json.dumps(cache, ensure_ascii=False, indent=2), encoding="utf-8")
logger.info(f"缓存已刷新: {sum(len(v) for v in cache.values())}")
def main():
parser = argparse.ArgumentParser(description="通过 opencode 联网搜索")
parser.add_argument("--query", help="搜索词")
parser.add_argument("--refresh-cache", action="store_true", help="刷新所有分类缓存")
parser.add_argument("--max-results", type=int, default=5)
args = parser.parse_args()
if args.refresh_cache:
refresh_cache()
return
if args.query:
results = search_via_opencode(args.query, args.max_results)
print(json.dumps(results, ensure_ascii=False, indent=2))
return
parser.print_help()
if __name__ == "__main__":
main()
+1 -28
View File
@@ -179,32 +179,6 @@ def _call_bing(api_key: str, api_url: str, query: str, max_results: int) -> List
} for r in items[:max_results]] } for r in items[:max_results]]
def _call_mcp(api_key: str, api_url: str, query: str, max_results: int) -> List[Dict]:
"""Call the MCP search server directly (no API key needed)."""
import subprocess
try:
r = subprocess.run(
[sys.executable, str(PROJECT_ROOT / "scripts" / "mcp_search_server.py"),
"--query", query],
capture_output=True, text=True, timeout=90,
)
if r.returncode != 0:
logger.warning("MCP搜索子进程返回非零: %s", r.stderr[:100])
return []
results = json.loads(r.stdout)
if isinstance(results, list):
for res in results:
res["source"] = "opencode"
return results[:max_results]
except json.JSONDecodeError as e:
logger.warning("MCP搜索JSON解析失败: %s", e)
except subprocess.TimeoutExpired:
logger.warning("MCP搜索超时 (90s)")
except Exception as e:
logger.warning("MCP搜索失败: %s", e)
return []
def _call_360(api_key: str, api_url: str, query: str, max_results: int) -> List[Dict]: def _call_360(api_key: str, api_url: str, query: str, max_results: int) -> List[Dict]:
"""360搜索(HTML爬取,无需 API Key""" """360搜索(HTML爬取,无需 API Key"""
from bs4 import BeautifulSoup from bs4 import BeautifulSoup
@@ -311,7 +285,6 @@ _PROVIDER_CALLS = {
"qiniu": _call_qiniu, "qiniu": _call_qiniu,
"tinyfish": _call_tinyfish, "tinyfish": _call_tinyfish,
"bing": _call_bing, "bing": _call_bing,
"mcp": _call_mcp,
"360": _call_360, "360": _call_360,
"sogou": _call_sogou, "sogou": _call_sogou,
"wechat": _call_wechat, "wechat": _call_wechat,
@@ -325,7 +298,7 @@ def search(query: str, max_results: int = 5) -> List[Dict]:
if (p.get("usage_today") or 0) >= (p.get("daily_limit") or 99999): if (p.get("usage_today") or 0) >= (p.get("daily_limit") or 99999):
logger.info("提供商 %s 已达日限 %s,跳过", p.get("name"), p.get("daily_limit")) logger.info("提供商 %s 已达日限 %s,跳过", p.get("name"), p.get("daily_limit"))
continue continue
no_key_types = {"mcp", "360", "sogou", "wechat"} no_key_types = {"360", "sogou", "wechat"}
if not p.get("api_key") and p.get("provider_type") not in no_key_types: if not p.get("api_key") and p.get("provider_type") not in no_key_types:
logger.info("提供商 %s 未配置 API Key,跳过", p.get("name")) logger.info("提供商 %s 未配置 API Key,跳过", p.get("name"))
continue continue
+3 -3
View File
@@ -3,7 +3,7 @@
网络搜索模块 网络搜索模块
三种模式(优先级从高到低): 三种模式(优先级从高到低):
1. 本地缓存(opencode webfetch 预填充 1. 本地缓存(search_cache.json
2. Bing Web Search API(设 BING_API_KEY 2. Bing Web Search API(设 BING_API_KEY
3. Bing 网页抓取(服务器环境常反爬拦截) 3. Bing 网页抓取(服务器环境常反爬拦截)
""" """
@@ -92,7 +92,7 @@ def search_scrape(query: str, max_results: int = 5) -> List[Dict]:
def search_from_cache(query: str, max_results: int = 5) -> List[Dict]: def search_from_cache(query: str, max_results: int = 5) -> List[Dict]:
""" opencode webfetch 预填充的缓存中读取(跳过超过36小时的缓存)""" """本地搜索缓存中读取(跳过超过36小时的缓存)"""
if not SEARCH_CACHE_FILE.exists(): if not SEARCH_CACHE_FILE.exists():
return [] return []
try: try:
@@ -114,7 +114,7 @@ def search_from_cache(query: str, max_results: int = 5) -> List[Dict]:
def save_to_cache(query: str, results: List[Dict]): def save_to_cache(query: str, results: List[Dict]):
"""保存搜索结果到缓存(供 opencode webfetch 填充时使用)""" """保存搜索结果到缓存(供填充时使用)"""
cache = {} cache = {}
if SEARCH_CACHE_FILE.exists(): if SEARCH_CACHE_FILE.exists():
try: try:
+2 -2
View File
@@ -116,12 +116,12 @@ test("DELETE 删除源", r.status_code == 200)
print("\n=== 7. LLM多供应商 ===") print("\n=== 7. LLM多供应商 ===")
sys.path.insert(0, str(root / "platform" / "backend")) sys.path.insert(0, str(root / "platform" / "backend"))
from app.core.nvidia_client import _get_active_provider, _get_provider_config from app.core.nvidia_client import _get_active_provider, _get_provider_config
test("默认供应商", _get_active_provider() == "opencode-go") test("默认供应商存在", _get_active_provider() in ["opencode-go", "nvidia", "sensenova"])
cfg = _get_provider_config("opencode-go") cfg = _get_provider_config("opencode-go")
test("opencode-go已配置", cfg is not None) test("opencode-go已配置", cfg is not None)
cfg_nv = _get_provider_config("nvidia") cfg_nv = _get_provider_config("nvidia")
test("nvidia备用存在", cfg_nv is not None) test("nvidia备用存在", cfg_nv is not None)
test("opencode-go模型", cfg and cfg.get("model") == "deepseek-v4-flash") test("opencode-go模型非空", bool(cfg and cfg.get("model")))
test("opencode-go URL非空", bool(cfg and cfg.get("base_url"))) test("opencode-go URL非空", bool(cfg and cfg.get("base_url")))
test("opencode-go Key非空", bool(cfg and cfg.get("api_key"))) test("opencode-go Key非空", bool(cfg and cfg.get("api_key")))
@@ -0,0 +1,175 @@
# 2026年,中国人正在重新定义"好生活"
---
## 一场静悄悄的生活方式革命
你发现了吗?身边越来越多的朋友开始自带杯买咖啡、在阳台种番茄、把旧衣服挂上闲鱼、换掉用了十年的老空调……
这不是零散的个人选择,而是一场正在中国发生的、系统性的生活方式变革。
我们梳理了2026年可持续生活方式的7个关键领域——每一个背后都有数据、有政策、有实实在在的"真金白银"。
---
## 一、你的每一次出行,都在"赚钱"
2026年五一,新能源汽车占出行车辆比例达到 **24%**——每四辆车里就有一辆是绿牌。
这不是偶然。全国日均已有 **2 亿人次**选择绿色出行,试点城市目标将绿色出行比例推至 **70% 以上**。新能源公交占比已达 **82.7%**
更重要的是真金白银的激励:
- **报废换新**:新能源车补贴车价 **12%**(最高 2 万元),燃油车 10%(最高 1.5 万元)
- **置换更新**:新能源车补贴 **8%**(最高 1.5 万),燃油车 6%(最高 1.3 万)
> 骑行 1 公里,减碳约 0.24kg——积少成多,你的每一次踩踏都值得被记录。碳普惠平台正在把这种"小行为"变成"大价值"(见下文)。
---
## 二、阳台种菜:算不过来的账,却停不下来的热爱
阳台种菜市场已达**百亿级**。
但有意思的是——成本根本算不过来。种子、土、肥料、花盆、工具,一年投入 500-2000 元,种出来的菜市价可能不到 100 元。年省 3000 元?"行不通"。
那为什么还有这么多人乐此不疲?
答案是**情绪价值 + 食品安全焦虑**。看着一粒种子发芽、长大、结果的过程,本身就是城市人稀缺的"慢体验"。而亲手种出的菜,吃得放心。
都市农业的象征意义远大于经济意义——它是一扇窗,让人在钢筋水泥中重新连接自然。
---
## 三、碳普惠:你的低碳行为,正在变成"钱"
这是2026年最值得关注的制度创新之一。
**碳普惠**Carbon Inclusion)将个人的低碳行为——骑行、自带杯、地铁通勤——量化积分为碳积分,然后积分可以:
1. **商城兑换**实物/优惠券/话费
2. **进入碳交易市场**(广东 PHCER、山西等已打通)
3. **银行信贷优惠**——碳积分越高,贷款利率越低
4. **企业认购**——企业购买你的减排量用于碳中和
5. **个人碳账本**——记录+社交+激励,积累到一定量可交易
上海崇明已完成首笔碳普惠减排量交易,广东 PHCER 已进入区域碳市场。
**瓶颈仍然存在**:仅少数省市打通了碳市场变现通道,多数地区仍以积分兑换为主。但这个方向已经明确——你的低碳行为,正在从"道德选择"变为"经济理性"。
---
## 四、以旧换新 + 闲置经济:625 亿的真金白银
2026年,国家第一批 **625 亿元**超长期特别国债已下达用于以旧换新。
关键规则变化:
| 品类 | 补贴 | 上限 |
|------|------|------|
| 6 类家电(冰箱/洗衣机/电视/空调/热水器/电脑)**仅限 1 级能效** | 售价 15% | 1,500 元/件 |
| 4 类数码(手机/平板/智能手表/智能眼镜) | 售价 15% | 500 元/件 |
智能眼镜首次纳入国补——2026 年的"新物种"值得关注。
与此同时,**二手市场规模**正在爆发式增长:
- 2024 年:**1.69 万亿元**
- 2026 年预计:**3.1 万亿元**
- 用户规模:**6.6 亿人**,Z 世代是主力
闲鱼、红布林、多抓鱼——"买二手"正在从"省钱"变成"一种生活方式标签"。
---
## 五、干净饮食:从"吃饱"到"吃对"
### 植物基:中国市场增速全球第一
全球植物基食品市场 2024 年 173.7 亿美元,预计 2035 年达 1702.8 亿美元(**CAGR 23.06%**)。
中国市场的增速更为惊人——**2025 年增速 48.6%**,全球最快。
品类结构:植物肉占 41%,但植物基乳制品增速最快(26.7%)。
关键趋势:**弹性素食(Flexitarian)**崛起——不是完全不吃肉,而是有意识地减少。这正在成为主流。
### 有机食品:从"小众"到"大众"
2025 年全球有机食品市场 1693.37 亿美元,预计 2032 年达 2610.79 亿美元(CAGR 6.38%)。
更值得关注的是消费群体的变化:
- **Z 世代** 73% 过去 12 个月内购买过植物基产品,关注"清洁标签"和蛋白质含量
- **中老年群体**关注饱和脂肪酸和膳食纤维
- **渠道变革**:传统零售从 61%→44%,社区生鲜/便利店/会员店升至 35%
"本真植选"成为 2026 年的关键词——少加工、更天然、更透明。
---
## 六、零浪费:一杯咖啡引发的连锁反应
### 自带杯:从"小众"到"8.7%"
2025 年全国现制饮品出杯量 **500 亿杯以上**——这是什么概念?每个中国人平均一年喝 35 杯。
自带杯订单占比已提升至 **8.7%**。瑞幸、Manner、星巴克自带杯立减 2-5 元。"益杯行动"目标 2026 年前万家门店响应。
上海/深圳试点自带杯积分纳入个人碳账户(每次减碳 50-80g CO₂)。
更直观的趋势:天猫自带杯销量**年增 120%**——不锈钢/硅胶折叠杯是主流。
### 极简生活:从"断舍离"到"清醒的极简"
- "断舍离"小红书年搜索量同比增 **47%**
- 趋势转向:"少买精买、长久使用"
- **胶囊衣橱**(30 件过一季)、**一物一件**成为新关键词
- 2026 年新增:**数字极简**——从物品断舍离延伸至数字生活整理
### 可持续时尚:1280 亿的市场
2026 年二手服装市场规模预计 **1280 亿元**(同比 +35%)。
优衣库 RE.UNIQLO、H&M、Patagonia 的可持续计划渗透率在一二线城市达 22%。
但挑战也在增加:"漂绿"指控越来越多,消费者信任度正在下降——品牌需要更透明的行动,而非口号。
---
## 七、绿色家电:AI 让节能不再是"牺牲"
2026 年国补政策的核心变化:
- 品类从 12 类**缩至 6 类**(灶具/烟机/净水器/洗碗机等退出)
- **仅限 1 级能效**——2 级不再享受
- 补贴比例 20%→15%,上限 2000 元→1500 元
但更值得关注的是技术突破:
- 新 1 级能效空调 vs 10 年老空调:降温 30→26℃ 耗电 **1 度 vs 5 度**
- 换新空调一年全国可节省电费约 **67 亿元**
- 格力 **AI 动态节能**空调全年能效提升 15.8%
- TCL 新风空调至高省电 40%
节能不再意味着"忍受"——AI 正在让高效和舒适同时实现。
回收体系也在完善:21 亿台家电保有量,规范处置可减碳 1100-3000 万吨。"送新收旧"一站式服务和"互联网+回收"正在普及。
---
## 结语:好生活,正在被重新定义
2026 年的中国,一个有意思的现象正在发生——
**政策**(以旧换新、碳普惠)、**市场**(二手经济、植物基)、**技术**(AI 节能、绿色制造)和**个人选择**(自带杯、极简、阳台种菜)四个力量正在汇合。
它们指向同一个方向:
> 好生活,不是拥有更多,而是用得更好、活得更清醒、和自然相处得更聪明。
这不是苦行僧式的"牺牲",而是有数据、有政策、有市场支撑的 **"理性愉悦"**。
你已经在路上了吗?
---
*数据来源:公开政策文件、行业研究报告、Websearch 检索 | 整理时间:2026 年 6 月*
+139
View File
@@ -0,0 +1,139 @@
# 可持续生活方式研究摘要(2026)
> 基于 websearch 检索结果整理,涵盖 7 个主题。
---
## 1. 绿色通勤 / 新能源车 / 骑行(2026)
### 新能源车
- 五一期间新能源汽车占出行车辆比例达 24%
- 试点城市绿色出行比例目标 70%+,全国日均 2 亿人次绿色出行
- 新能源公交占比 82.7%
- 骑行 1km 约减碳 0.24kg
### 政策
- 2026 年汽车报废更新:新能源车补贴车价 12%(最高 2 万元),燃油车 10%(最高 1.5 万元)
- 汽车置换更新:新能源车补贴 8%(最高 1.5 万),燃油车 6%(最高 1.3 万)
---
## 2. 阳台种菜 / 都市农业
- 阳台种菜市场已达百亿级
- 账面算不过账:年省 3000 元"行不通"(种子/土/肥/工具成本倒挂)
- 核心驱动力是**情绪价值 + 食品安全焦虑**,而非省钱
- 都市农业作为可持续生活方式的象征意义远大于经济意义
---
## 3. 碳普惠(Carbon Inclusion
### 机制
- 个人的低碳行为(骑行、自带杯、地铁通勤等)量化积分为碳积分
- 积分可兑换商品/优惠券/碳信用
### 5 条变现路径
1. **碳积分商城兑换**(实物/优惠券/话费)
2. **进入碳交易市场**——广东 PHCER、山西等已打通
3. **碳账户银行信贷**——银行根据碳积分给予利率优惠
4. **企业认购**——企业购买碳普惠减排量用于碳中和
5. **个人碳账本**——记录+社交+激励,积累到一定量可交易
### 最新进展
- 上海崇明完成首笔碳普惠减排量交易
- 广东 PHCER 已进入区域碳市场
- 主要平台:支付宝"蚂蚁森林"、各地碳普惠平台
- 瓶颈:仅少数省市打通碳市场变现,多数仍以积分兑换为主
---
## 4. 以旧换新 / 闲置经济(2026)
### 国家补贴
- 2026 年第一批 625 亿元超长期特别国债已下达
- 2025 年全年以旧换新惠及 3.6 亿人次,带动消费 2.6 万亿元
- 6 类家电(冰箱/洗衣机/电视/空调/热水器/电脑)仅限 **1 级能效**,售价 15% 补贴,上限 1500 元/件
- 4 类数码产品(手机/平板/智能手表/智能眼镜)15% 补贴,上限 500 元/件
- 智能眼镜首次纳入国补
### 二手市场
- 2024 年二手交易市场规模 1.69 万亿元,预计 2026 年达 3.1 万亿元
- 用户规模 6.6 亿人,Z 世代为二手交易主力
- 闲鱼/红布林/多抓鱼为主要平台
---
## 5. 干净饮食 / 有机 / 植物基 / 本地食材
### 植物基食品(2026
- 全球市场 2024 年 173.7 亿美元,预计 2035 年达 1702.8 亿美元(CAGR 23.06%
- 中国市场 2025 年增速 48.6%(全球最快)
- 品类结构:植物肉占 41%,植物基乳制品增速最快(26.7%)
- 驱动因素:健康 + 环保 + 可持续
- 弹性素食(Flexitarian)崛起——非完全素食,而是减少肉类摄入
### 有机食品
- 2025 年全球有机食品市场 1693.37 亿美元,预计 2032 年达 2610.79 亿美元(CAGR 6.38%
- Z 世代 73% 过去 12 个月内购买过植物基产品
- 消费动机:年轻群体关注"清洁标签"和蛋白质含量;中老年关注饱和脂肪酸和膳食纤维
- 渠道变革:传统零售从 61%→44%,社区生鲜/便利店/会员店升至 35%
- 品牌趋势:头部品牌重品牌信任,中小品牌靠区域资源差异化
### 本地食材
- "食本地鲜"运动兴起,缩短食物里程
- 城市农场/社区支持农业(CSA)模式持续增长
---
## 6. 零浪费 / 自带杯 / 极简生活 / 可持续时尚
### 自带杯
- 2025 年全国现制饮品出杯量 500 亿杯以上,一次性杯具消耗巨大
- 瑞幸/Manner/星巴克自带杯立减 2-5 元
- "益杯行动"目标 2026 年前万家门店响应
- 美团推"自带杯立减"首年预计万家门店参与
- 自带杯订单占比提升至 8.7%
- 上海/深圳试点自带杯积分纳入个人碳账户(每次减碳 50-80g CO₂)
- 天猫自带杯销量年增 120%(不锈钢/硅胶折叠杯为主流)
### 极简生活
- "断舍离"小红书年搜索量同比增 47%
- 趋势转向:"少买精买、长久使用"——胶囊衣橱(30 件过一季)、一物一件
- B 站相关视频播放量年增 45%,豆瓣小组 85 万人
- 2026 年关键词:数字极简(从物品断舍离延伸至数字生活整理)
- 零浪费 5R 原则(Refuse/Reduce/Reuse/Recycle/Rot)普及
### 可持续时尚
- 2026 年二手服装市场规模预计 1280 亿元(同比 +35%)
- 品牌计划渗透率:优衣库 RE.UNIQLO、H&M、Patagonia(一二线 22%
- 再生面料(rPET/天丝/麻纤维)在小众和快时尚品牌中普及
- Z 世代 44% 购买过二手或可持续面料服装;68% 愿为"环保认证"支付溢价
- "漂绿"指控增加,消费者信任度下降
---
## 7. 绿色家电 / 一级能效 / 以旧换新(2026)
### 国补政策核心变化
- 品类从 12 类缩至 6 类(灶具/烟机/净水器/洗碗机等退出)
- **仅限 1 级能效或水效**——2 级不再享受
- 补贴比例 20%→15%,上限 2000 元→1500 元
- 首批资金 625 亿元已下达
- 数码新增智能眼镜品类
### 节能数据
- 新 1 级能效空调 vs 10 年老空调:降温 30→26℃ 耗电 1 度 vs 5 度
- 换新空调一年全国可节省电费约 67 亿元(英国恩伯数据)
- 海尔冰箱 90%+ 为 1 级能效产品
- 格力 AI 动态节能空调全年能效提升 15.8%,降低耗电 13.6%
- TCL 新风空调至高省电 40%
### 回收体系
- 21 亿台家电保有量,规范处置可减碳 1100-3000 万吨
- 送新收旧一站式服务
- "互联网+回收"、"以车代库"等模式推广
---
*整理时间:2026年6月* | *来源:公开 websearch 检索*