Commit Graph

184 Commits

Author SHA1 Message Date
Yuzhiran Dev f6e5e90736 feat: Wave 3 - Dashboard redesign + RBAC + AI assistant updates
- Redesign index.html from system-monitor to user-centric workspace
  (welcome banner, summary cards, 今日待办, AI 智能建议, collapsible admin section)
- Add fetchTodos/fetchSuggestions/getPriorityClass/isOverdue methods
- Update uni-nav current-page from 'dashboard' to 'workspace'
- Add require_role to 10 admin router imports (RBAC foundation)
- Update ai-assistant.js: add factory/insights pages, org_id in chat requests
2026-06-17 18:24:57 +08:00
Yuzhiran Dev 5caf7bc52e feat: Wave 2 - API org isolation + factory.html + insights.html
- Add org_id auto-population to articles.py, publishing.py, metrics.py
- Add org_id filtering to search_rankings.py geo endpoints
- Add org_id to calendar.py create/update/delete endpoints
- Add org_id to tasks.py create endpoints
- Create factory.html content pipeline page (3-tab: 待创作/进行中/待审查)
- Create insights.html analytics page (4-tab: 概览/搜索排名/GEO/平台对比)
2026-06-17 16:13:48 +08:00
Yuzhiran Dev dcfeccc97b feat(models): add org_id to Article, PublishRecord, ContentMetrics, SearchRanking, GeoReadinessScore
- Add org_id column to 5 models with default='default'
- Update to_dict() methods for ContentMetrics, SearchRanking, GeoReadinessScore
- Add ALTER TABLE migrations in database.py init_db()
- Add org_id to 6 Pydantic response schemas
- Add require_role() helper factory in auth.py
2026-06-17 14:27:27 +08:00
Yuzhiran Dev 00431c827b feat: add org_id to ContentCalendar and ContentTask models 2026-06-17 09:02:54 +08:00
Yuzhiran Dev c8a5e8fd87 docs: Update PROGRESS.md for v21 GEO and v22 landing page
- v21: GEO tracking (geo_tracker.py, FAQ/HowTo schema, GeoReadinessScore, scheduler, admin tab)
- v22: Landing page (landing.html, routing, responsive design, auth flow changes)
- Commit count: 165 -> 170

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-06-16 08:28:31 +08:00
Yuzhiran Dev c069a140bb fix: Auth redirects and navigation for landing page routing
- login.html: Post-login redirect / -> /index.html; already-logged-in redirect / -> /index.html
- articles.html, topics.html: Auth failure redirect -> /login.html
- uni-nav.js: Dashboard link / -> /index.html; navigate function fix for prefix handling
- sw.js: Remove / pre-cache; HTML documents use Network First strategy for fresh content

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-06-16 08:28:06 +08:00
Yuzhiran Dev 24e46eced5 feat: Product landing page at / with responsive design
- landing.html: Full product website (hero, features, how-it-works, platform coverage, pricing, FAQ, CTA, footer)
- main.py: Route / to landing.html (overrides StaticFiles default index.html)
- Workspace dashboard remains at /index.html
- Responsive: desktop nav, hamburger mobile menu, single-column on small screens
- No Vue/Element Plus dependency; standalone HTML+CSS+JS (~30KB)

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-06-16 08:27:38 +08:00
Yuzhiran Dev f15e0f720a feat: Admin GEO readiness dashboard tab
- New 'GEO 就绪度' tab button in filter bar
- Overview stat cards: total checks, AI citations, citation rate, avg readiness
- AI engine breakdown display (DeepSeek/ChatGPT/Perplexity)
- Readiness scores table with 6-dimension color-coded tags
- Manual GEO tracking trigger button

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-06-16 08:27:11 +08:00
Yuzhiran Dev 7c9a8b88b4 feat: FAQ/HowTo schema injection and SEO pipeline enhancement
- writer.py: inject_geo_metadata now detects content type (article/listicle/howto/faq/review) and injects appropriate JSON-LD (FAQPage, HowTo, ItemList)
- New helpers: _detect_content_type, _extract_faq_pairs, _extract_howto_steps
- outline.py: _extract_seo_keywords regex support for CJK
- prompt_loader.py: Minor update

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-06-16 08:26:44 +08:00
Yuzhiran Dev 93d1d511e5 feat: Register scheduled GEO tracker (07:30 daily)
- New MODULE entry: scheduled_geo_tracker
- New LOG_FILE_MAP entry
- New MODULE_JOBS entry
- _run_geo_tracker method: subprocess invokes geo_tracker.py

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-06-16 08:26:13 +08:00
Yuzhiran Dev 2aedb69efd feat: AI search citation tracking script
scripts/geo_tracker.py: Simulates DeepSeek/ChatGPT/Perplexity queries via LLM to detect article citations. Calculates GEO readiness score (6 dimensions: schema/faq/howto/citations/headings/word_count). Writes results to SearchRanking(search_engine='geo') and GeoReadinessScore tables.

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-06-16 08:25:44 +08:00
Yuzhiran Dev 773080cf4f feat: GEO data model and API endpoints
- SearchRanking: +5 fields (ai_search_engine, citation_snippet, citation_url, geo_score, content_type)
- New GeoReadinessScore model (6-dimension scoring: schema/faq/howto/citations/word_count/readability/headings)
- GET /api/seo/geo/overview, GET /api/seo/geo/readiness, POST /api/seo/geo/track

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-06-16 08:25:18 +08:00
Yuzhiran Dev 63a6fabc00 fix: Script bug fixes (bare except, timezone, regex, test)
- collector.py, compliance_checker.py, trends.py: bare except -> specific
- db_helper.py: datetime.now() -> timezone.utc (8 occurrences)
- compliance_checker.py: regex \x08 -> \b word boundary + import json
- search_providers.py: minor fixes
- test_new_features.py: service reachability check retry

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-06-16 08:24:50 +08:00
Yuzhiran Dev d601a26850 fix: API route security fixes (path traversal, auth, bare except)
- articles.py: Path traversal sanitization
- optimizer_logs.py: Admin auth guard
- platform_config.py: Admin auth guard
- system.py: Path traversal whitelist
- topic_config.py: Admin auth guard
- topics.py: Minor fix

Ultraworked with [Sisyphus](https://github.com/code-yeongyu/oh-my-openagent)

Co-authored-by: Sisyphus <clio-agent@sisyphuslabs.ai>
2026-06-16 08:23:50 +08:00
Yuzhiran Dev 8595bbc521 feat: GEO/SEO structured data + search ranking tracker 2026-06-09 17:18:09 +08:00
Yuzhiran Dev 23ff63baa9 fix: 三平台内容差异化 + admin敏感词管理表格化
- writer.py: _expand_section() 去除 <100字阈值,始终调用 LLM 平台专属扩写
- prompt_loader.py: 新增 section_expansion_zhihu/wechat/xiaohongshu 三个独立 prompt
- admin.html: 配置管理标签页 + 敏感词/清理规则子标签 + 敏感词表格化管理(编辑/删除)
- config_items.py: PUT /sensitive-words/{id} 支持更新 word/category
- compliance_checker.py: AI 套话从 DB 加载 + 人称规则修正
- initial_data.py: PlatformConfig 字数迁移 + 新种子
- 各前端页面: LLM 配置 rate_limit 字段 + 供应商列表排序
2026-06-08 13:51:35 +08:00
Yuzhiran Dev d0896ef10e fix: _wrap_chinese crash on mixed-width punctuation
_image_generator.py _wrap_chinese 当标题中只有 ASCII 标点
(如 ( ) ?) 而无全角标点时,max() 收到空序列崩溃。

修复:try-except 兜底 + 补充 ASCII 标点到断点列表
2026-06-03 13:13:19 +08:00
Yuzhiran Dev c6628b22f6 fix: 任务抽屉产出结果优化 + server 启动修复
- 任务抽屉产出结果 tab 优先显示 last_message(日志消息),
  过滤 result_data 中的 None/空值
- 修复 tasks.py 因 opencode 块删除导致的 SyntaxError
- 修复 scheduled_generate 存储 None 值问题
- 清理 DB 中旧 scheduled_refresh_search_cache 的 56 条
  TaskLog + 1 条 TaskConfig
- 更新 PROGRESS.md
2026-06-03 11:06:25 +08:00
Yuzhiran Dev 499c511140 chore: opencode冗余清理 + LLM任务级模型选择 + systemd服务化
- 删除 opencode_search.py / mcp_search_server.py 及所有 MCP 引用
- 移除搜索缓存定时任务(scheduled_refresh_search_cache)
- 清理前后端所有 opencode/MCP 代码和注释
- LLM 提供商量换:opencode-go→nvidia(默认)+sensenova(合规审查)
- llm_configs 新增 is_default 字段,API 层互斥逻辑
- 所有定时任务支持独立 LLM 模型选择(LLM_TASK_PROVIDER env)
- compliance_optimizer.py 修复:import os / 解硬编码 / 关键词过滤
- Scheduler 日志修复:始终 INSERT,避免僵尸 running 行
- Systemd 服务化:Restart=always / 单 worker / Type=exec
- 搜索提供商:替换 opencode→360/搜狗/微信(免 Key)
- 更新 AGENTS.md / PROGRESS.md
2026-06-02 15:38:16 +08:00
Yuzhiran Dev abbf1468bc fix(admin): LLM 路由标签动态化
- 默认提供商按 id 排序取第一个活跃的(nvidia step-3.7-flash)
- 合规审查取活跃且 model=deepseek-v4-flash 的(sensenova)
- 不再硬编码
2026-06-01 14:46:22 +08:00
Yuzhiran Dev 53f105305f fix(admin): LLM 路由显示不准确
- 默认路由改为动态取第一个活跃提供商(当前为 nvidia step-3.7-flash)
- 合规审查标签改为固定 sensenova (deepseek-v4-flash)
- 移除硬编码的 opencode-go 标签
2026-06-01 14:42:32 +08:00
Yuzhiran Dev f1dc29f439 fix(llm): 分场景使用不同模型
- nvidia 激活、模型改为 step-3.7-flash → 用于日常生成/采集/优化等
- compliance_optimizer.py 显式指定 provider=sensenova → 合规审查走 deepseek-v4-flash
- _FALLBACK 同步更新为 step-3.7-flash
- 管理后台标签同步更新
2026-06-01 14:35:41 +08:00
Yuzhiran Dev 475945418e feat(llm): LLM 配置完全通过管理后台维护
- _get_provider_config 改从 DB 读取 api_key(.env 为 fallback)
- 前端 admin.html 新增 sensenova provider 选项,默认值更新为 sensenova
- 新增提供商不再需要改代码,后台直接配即可
2026-06-01 14:23:24 +08:00
Yuzhiran Dev 2c17f006fc feat(llm): 添加 sensenova 提供商,替换 opencode-go
_API_KEYS/_FALLBACK 添加 sensenova,DB 中设 is_active=true
与 opencode-go 同模型(deepseek-v4-flash),但走 sensenova 通道
2026-06-01 14:17:36 +08:00
Yuzhiran Dev 31d8434010 fix(llm): call_llm 429 自动 fallback + 移除 compliance 硬编码 provider
- nvidia_client.py 新增 _get_provider_fallback_list() 从 DB 读提供商顺序
- call_llm 循环尝试提供商列表,HTTP 429 自动跳过到下一个
- 移除 compliance_optimizer.py 中硬编码 provider="opencode-go"
- 新增 logging 模块引用
2026-06-01 13:23:35 +08:00
Yuzhiran Dev 09c467e56d ops(systemd): 添加 systemd 服务,自动恢复崩溃进程
- 创建 yzr-platform.service(Restart=always,崩溃后10秒恢复)
- 更新 AGENTS.md 命令文档
- 替换原来的 setsid/nohup 启动方式
2026-06-01 12:56:02 +08:00
Yuzhiran Dev 1084388fec fix(topics): 外键约束导致 delete 400
选题有关联 content_tasks/article/publish_record/metrics 表,
外键约束阻止删除。改为先级联删除关联数据再删选题。

前端同时显示服务端返回的具体错误信息,而不是笼统的 删除失败。
2026-06-01 12:08:51 +08:00
Yuzhiran Dev 468c9066cb fix(scheduler): _log_task 重复 INSERT 导致 zombie running 行
_log_task 在任务开始(running)和结束(success/failed)时分别
INSERT,导致 running 行永不更新,被 task monitor 标记为失败。

修复:新增 log_id 参数,传入时 UPDATE 已有行。所有 _run_* 方法
save running log_id 并在结束时传入。早期 return 处也标记失败。
2026-06-01 10:01:50 +08:00
Yuzhiran Dev 9b9ee4964b feat(assistant): 可执行操作改造,支持实时查询与任务触发
- 新增 assistant_actions.py 动作注册表(list_topics/get_system_stats/get_task_status/trigger_task/search_web 等)
- assistant.py 增加规则意图识别优先执行,失败则走 LLM ACTION 格式
- 格式化返回结果,前端无感知
- 支持查看系统统计、选题、任务日志、手动触发定时任务
2026-05-31 13:25:07 +08:00
Yuzhiran Dev 10ad549050 chore: gitignore 新增自动生成文件过滤
- 新增: automation/backups/ research/ automation/images/
- 新增: search_cache.json trends.json sustainability_cases.json
- untrack: 上述文件从 git 追踪中移除
2026-05-29 09:41:03 +08:00
Yuzhiran Dev 3fab87ee11 feat: 新增360/搜狗/微信搜索提供商 + PG15→16升级 & 项目文档更新
- search_utils.py: 新增 _call_360/_call_sogou/_call_wechat HTML爬取函数
- initial_data.py: 种子数据新增三个搜索提供商(priority 3/4/5)
- models.py: provider_type 注释补充新类型
- admin.html: 搜索提供商类型下拉框新增三个选项
- AGENTS.md/PROGRESS.md/README.md: PostgreSQL 15→16
- README.md: 移除硬编码数据库密码
2026-05-29 09:35:15 +08:00
Yuzhiran Dev bd3228806d fix: 合规审查卡死修复 + 小红书复制格式 + today-only过滤 2026-05-27 18:25:30 +08:00
Yuzhiran Dev 6585909ffc feat: optimize all 13 prompts by function+scenario; fix topic_selector hardcode
Prompt optimizations:
- topics_trends: 10→8 topics, simplified output, anti-hallucination
- topic_generate: stronger structure, user-pain focus
- topic_selector_gaps: add SEO+social instruction, clean JSON output
- section_expansion: shorter (removed over-detailed AI套话 list), cleaner
- title_optimize_*: 3 titles per line (no numbering), cleaned up
- outline_generation: simplified structure, strong anti-generic-titles
- compliance_fix/polish: expanded fix types, stronger output requirements
- sources_optimization: cleaner JSON schema
- tags_generation: added 'no thinking output' instruction
- research_summary: simplified output structure
- clean_ai_verbosity/clean_thinking_patterns: unchanged (rule-based)

topic_selector.py: hardcoded prompt → get_prompt('topic_selector_gaps')
_PROMPT_DEFAULTS seeds updated to match DB
2026-05-27 11:07:56 +08:00
Yuzhiran Dev 2c6adf1f43 fix: system_prompt now read from DB LLMConfig
- _get_db_defaults returns system_prompt from DB
- call_llm(system_prompt=None) falls back to DB value
- DB already has system_prompt set for both providers
2026-05-27 10:47:02 +08:00
Yuzhiran Dev b105eee9bb fix: add LLM config status display + routing info
- 状态列: icon → el-tag 文字标签(当前使用/未启用)
- 新增路由状态提示条, 显示默认 vs 审查固定路由
- 移动端卡片同样改为文字标签
2026-05-27 10:09:23 +08:00
Yuzhiran Dev 878daebfe0 fix: route compliance to opencode-go, others to nvidia; skip status on force-pass
1. DB active provider switched to nvidia (stepfun-ai/step-3.5-flash)
   → trends/collector/writer 默认用 nvidia
2. compliance_optimizer.py 固定 provider='opencode-go'
   → 合规审查用 deepseek-v4-flash(更好的模型)
3. force_passed 的选题不再更新为 'ready'(待发布)
   → 只有真正修复通过才更新状态
2026-05-27 09:36:34 +08:00
Yuzhiran Dev 004698cf98 fix: compliance optimizer now requires score improvement, retries LLM on failure
Bug: 'passed=True' check for soft quality issues (AI套话, 缺互动引导)
always exits immediately even when LLM failed to make changes.
Score 85 stays 85, content unchanged.

Fix:
1. polish_with_llm: retry once on exception (was: single attempt)
2. main loop: only accept fix when score actually improves AND
   LLM produced output (opt_logs non-empty). If LLM returns same html
   unchanged, treat as failed attempt and retry.
2026-05-27 09:23:49 +08:00
Yuzhiran Dev 29d97b5f1a fix: expand short sections (<100 chars) with LLM even if not bullet-only 2026-05-26 18:31:43 +08:00
Yuzhiran Dev 80023bbd6c fix: hybrid SVG+PIL PNG image rendering with CJK fonts, prompt URL reference 2026-05-26 17:38:28 +08:00
Yuzhiran Dev f94e6b0161 fix: correct subprocess script path and add TaskLog for trends trigger 2026-05-26 11:32:34 +08:00
Yuzhiran Dev ac8644d752 fix: content quality, image format, task monitor, calendar data source, search UI & sort 2026-05-26 11:26:10 +08:00
Yuzhiran Dev b2d043b231 新增用户管理/角色管理/菜单管理功能,修复创作流水线研究脚本
- 用户管理:新增编辑弹窗(修改用户名/角色/密码),增加组织/创建时间/最后登录列
- 角色管理:新增 Role 模型 + CRUD API,admin.html 新增角色管理 tab
- 菜单管理:新增 Menu 模型 + CRUD API,导航栏从 API 动态加载菜单项
- 个人中心:右上角下拉菜单(个人信息/修改密码/退出),新增修改密码 API
- 种子数据:initial_data.py 自动创建默认角色(admin/editor)和默认菜单(7项)
- 修复 research.py 缺少 enrich_topic_research 函数导致导入失败
- 修复 db_helper.py 中 generated_at 条件导致重创作不更新时间戳
- admin.html 操作列加宽防止按钮换行,平台配置增加删除按钮
- articles.html 预览弹窗加 lock-scroll=false 防止页面尺寸跳动
2026-05-22 18:34:27 +08:00
Yuzhiran Dev 1855f190f5 配置全面迁移数据库:PromptConfig、TaskConfig动态调度、敏感词/清洗规则/趋势映射/平台标签/痛点模板全部可编辑
- 新增 PromptConfig 模型 + API,支持提示词在线编辑(16条默认)
- 调度器动态读取 TaskConfig.schedule,admin 可调执行时间
- 新增 KeywordDomainMap、SensitiveWord、ContentCleanRule、TrendFieldMapping 表
- DOMAINS、TREND_DOMAIN_MAP、PLATFORM_TAGS、china_pains、RSS关键词、priority_weights 全部迁移到 DB
- tasks.html 重构:卡片网格+配置/产出/历史/提示词四个Tab,折叠显示
- 清理冗余代码:DEFAULT_PROMPTS死代码、collector.py unreachable代码、compliance_checker bug
- strip_thinking_html 改用 DB 规则优先
2026-05-22 11:18:23 +08:00
Yuzhiran Dev a8e0a76e07 任务页输入参数旁加'前往编辑'链接
- 采集类别参数 → admin.html#tab=categories
- 信息源参数 → admin.html#tab=sources
- admin.html 已支持 URL hash 自动切tab
- 新窗口打开,不打断当前任务浏览
2026-05-21 10:38:30 +08:00
Yuzhiran Dev 4e10cc95d3 清理 tasks.py 死代码 2026-05-21 10:21:45 +08:00
Yuzhiran Dev 9178f436b2 任务页面全面改版: 模块输入/输出/历史详情抽屉
后端新增:
- GET /api/tasks/modules/{module_id}/detail
  每个模块返回: inputs(自动填充的参数)、outputs(格式化产出结果)、
  history(今日运行记录)、log_excerpt(日志原文)

前端重构 tasks.html:
- 上半: 定时任务模块卡片网格(与仪表盘风格统一)
- 点击卡片→右侧抽屉展示3个tab:
  📥 输入参数: 搜索词/类别/待处理数量等(自动填充)
  📤 产出结果: 表格/标签/评分/建议等(按类型格式化)
  📋 运行记录: 日志原文+时间轴
- 底部保留创作任务列表(分页/筛选/详情)
- 抽屉底部立即运行按钮
2026-05-21 10:18:21 +08:00
Yuzhiran Dev 0dcfda0c80 数据新鲜度校验: 所有模块加上日期/时效检查
1. search_cache.json → 添加 _metadata.updated_at 时间戳
   web_search.search_from_cache() 跳过超过36h的旧缓存
   防止某查询失败时残留旧数据

2. metrics_feedback.json → collector 检查mtime,超过24h不采用

3. trends.json → 已有 date==TODAY 校验(load_trends)

4. collector_ai_advice → DB每日覆盖,时序安全
   creator→topic状态位避免重复生成
   optimizer→文章状态位避免重复审查
2026-05-21 10:09:51 +08:00
Yuzhiran Dev 5037d5d0ed 流水线整体前移1.5h,凌晨5点前全部就绪
01:00 search → 01:10 trends → 01:30 collect
→ 02:00 create → 03:00 review
→ 05:00 optimize_sources → 06:00 metrics

各间隔: 10min/20min/30min/60min/2h/1h
采集到创作30min缓冲足够(采集≈5-15min)
2026-05-21 10:04:41 +08:00
Yuzhiran Dev 56293a52a2 打通流水线闭环: trends+metrics+optimize_sources → collector
三大数据流精准注入采集器:

1. 热点趋势(trends) -> collector
   _get_trend_context() 读取 trends.json
   热搜注入 LLM 选题 prompt

2. 历史表现(metrics) -> collector
   scheduler 同步指标后按field聚合写入 feedback.json
   collector 读取后高互动领域获优先级提升

3. AI策略(optimize_sources) -> collector
   从 DB SystemConfig 读取 AI 建议注入 prompt
2026-05-21 09:48:38 +08:00
Yuzhiran Dev 6b790e77f6 全线异步化: 所有手动触发任务不再阻塞uvicorn worker
每个模块提供两个版本:
- run_xxxx() → Popen非阻塞(给API用)
- run_xxxx_blocking() → subprocess.run阻塞(给定时任务用)

API端点全部改为立即返回XX已后台启动,不再等子进程完成

受影响端点:
  /generate/run (原30min→12ms)
  /review/run (原10min→12ms)
  /collect/run (原5min→12ms,已修复)
  /optimize-sources/run (原2min→12ms)
  /trends/run (原2min→12ms)
  /refresh-search-cache/run (原10min→12ms)
  /metrics-sync/run (原N×10s→12ms)

同时增加对应GET状态端点:/collect/status,/generate/status,/review/status
2026-05-21 09:20:13 +08:00