refactor(observability): Phase2.2 成本归因 trace_id列+token日志写入+LLM/MCP/媒体子span [v2026.07.19.3]

.gitea/workflows/ci.yml

@@ -0,0 +1,28 @@
+# qihuaiflow minimal CI (Gitea Actions)
+#
+# NOTE: This workflow is DORMANT until a Gitea act_runner is registered with
+# this repo. The active gate today is the host-side pre-push hook
+# (scripts/git-hooks/pre-push) + scripts/ci_gate.py, because our deployment
+# model keeps the app running in a long-lived `qihu-aiflow` container on the
+# docker host rather than building it fresh in CI.
+#
+# To activate later:
+#   1. Install act_runner on the docker host (sharing the docker socket).
+#   2. Register it against this repo with label `ubuntu-latest`.
+#   3. The runner can then `docker exec qihu-aiflow ...` to reach the app.
+#
+# Until then, the pre-push hook already blocks broken pushes.
+name: CI
+on:
+  push:
+    branches: [main]
+  pull_request:
+
+jobs:
+  gate:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Backend pytest
+        run: docker exec qihu-aiflow bash -c 'cd /app/backend/python && .venv/bin/python -m pytest -c pytest.ini -q'
+      - name: Frontend vitest
+        run: docker exec qihu-aiflow bash -c 'cd /app/frontend && npx vitest run --reporter=dot'

VERSION

@@ -3,19 +3,41 @@
 # 版本号格式: vYYYY.MM.DD.N   (年.月.日.当日递增序号，从 1 开始)
 # 环境标签:   prod = 正式版 | test = 测试版
 
-VERSION=v2026.07.18.1
+VERSION=v2026.07.19.3
 ENV=prod
-DATE=2026-07-18
+DATE=2026-07-19
 BASE_IMAGE=qihu-aiflow:with-deps
-BACKUP_IMAGE=qihu-aiflow:backup-20260718-001
+BACKUP_IMAGE=qihu-aiflow:backup-20260719-003
 
-## 本版本变更 (v2026.07.18.1 / prod)
+## 本版本变更 (v2026.07.19.3 / prod)
-- Phase0 工程纪律止血 启动
+- Phase2.2 成本归因：token_usage_logs 新增 trace_id 列（迁移 020_add_trace_id_token_usage_logs.sql，幂等 ALTER），打通「链路 -> 成本」归集
-- 新增「部署前镜像备份 SOP」：scripts/backup_before_deploy.py，每次部署前自动提交递增序号镜像快照
+- _write_media_token_log（媒体图片/视频）注入 get_trace_id()，每条媒体 TokenUsageLog 现可回溯到请求链路
-- 统一版本号规范 vYYYY.MM.DD.N + prod/test 标签（取代旧 -recover 标签）
+- LogService.log_token_usage 增加 trace_id 参数（为后续文本 LLM token 记账接通成本归因预留）
-- 引入后端 pytest / 前端 vitest 测试基建，作为迭代式重构的安全网
+- chat_service / chat_media 在 LLM 补全、MCP 工具拉取/调用、技能执行、AGNES 媒体生成处加 span 子链路（async with span），一次请求可下钻耗时分布
-- 固化反复问题的回归用例 R1/R2/R3
+- 注意：_generate_media_task 为后台 fire-and-forget，其 span 会因处于 asyncio.create_task 表达式内而非法，已移除；真实媒体耗时由 chat_media 内 media.agnes_call span 覆盖
+- 纯增量埋点，不改接口返回结构与业务行为；pytest 9 passed + vitest 4 passed 门禁绿
+- 部署前已按 SOP 备份 rollback 点 qihu-aiflow:backup-20260719-003
+
+## 本版本变更 (v2026.07.19.1 / prod)
+- Phase1 chat.py 巨石拆分收官：10070 行单体拆分为 chat_{utils,media,agents,skill,mcp,context,knowledge,service}.py 共 8 个子模块，controller 层 chat.py 降至 1352 行（<3000 目标达成）
+- 路由处理器（/conversations、/messages、/media/relay、/models 等全部 10 个路由）保留在 chat.py，业务/服务函数迁至子模块并通过 import 再导出
+- 关键修复：此前拆分误删了全部 @router 装饰器导致路由为 0（重启即全 404），本版本以 AST 比对原始提交恢复装饰器，并补回 chat_service._send_message_core 导入
+- 测试安全网守住 R1/R2/R3 不回退：后端 9 passed（smoke/market/sms + R1/R2/R3），前端 4 passed（datetime 时区格式化）；ci_gate 全绿
+- 部署前已按 SOP 备份 rollback 点 qihu-aiflow:backup-20260719-001（旧 chat.py 10070 行）
+
+## 本版本变更 (v2026.07.19.2 / prod)
+- Phase2.1 统一可观测基座：新增 app/core/observability.py（trace_id contextvar + TraceIdFilter + span 计时器 + 可选 JSON formatter），零侵入为全平台日志注入链路 ID
+- main.py APITrackingMiddleware 复用既有 request_id 作为 trace_id，并为每个响应注入 X-Trace-Id 响应头（前端可关联排查）
+- config.py setup_file_logging 接入 TraceIdFilter 并将日志格式升级为 [tid:%(trace_id)s]，所有 getLogger(__name__) 日志自动带链路 ID
+- chat.py send_message 路由用 span 包裹 _send_message_core（耗时+trace_id+conv_id/user_id），覆盖 LLM/MCP/媒体生成整体耗时，为后续成本归因建立回归基线
+- 纯增量埋点，不改任何接口返回结构与业务行为；向后兼容既有 logging 用法
+- 部署前已按 SOP 备份 rollback 点 qihu-aiflow:backup-20260719-002
 
 ## 历史
+- v2026.07.18.5 (prod)：Phase0 收官 — 最小 CI 门禁（ci_gate.py + pre-push 钩子 + 休眠 Gitea workflow）。后端 9 passed，前端 4 passed。
+- v2026.07.18.4 (prod)：前端 vitest 基建落地（4 passed，datetime 时区格式化）。
+- v2026.07.18.3 (prod)：回归用例 R1/R2/R3（9 passed）— 微信Bot截断、技能/MCP覆盖规则、媒体生成超时回写。
+- v2026.07.18.2 (prod)：后端 pytest 基建落地（6 passed）— 冒烟/市场14类/短信宝MD5。
+- v2026.07.18.1 (prod)：Phase0 启动 — 部署前镜像备份 SOP + 版本号规范 + 引入测试基建。
 - 旧方案：镜像曾用 AIFlow v2.0.1 / v2026.07.14 / v2026.07.14-nav；git 曾用 v2026.07.12-recover / v2026.07.12-ui-recover / v2026.07.13-recover。自 v2026.07.18.1 起统一为 vYYYY.MM.DD.N + prod/test，便于正式版推广后与测试版做环境切割。
 - 历史功能：AI聊天(real API + tool-call function calling)、知识库文件存储 + AI 图谱、频道二维码、邮件/短信配置 + AI 生成、支付配置、平台驾驶舱(token 累计 + 审计/操作日志)、Agent 资源 API、evolution pipeline 自动恢复、全局默认模型、广告位图片上传 + AI 绘图、应用市场安装/卸载/搜索/去重、资产文件类型分类 + 在线预览。迁移 001-015。

backend/python/app/api/chat_agents.py

@@ -0,0 +1,943 @@
+# Auto-extracted from chat.py during Phase 1 refactor (Strangler Fig).
+# Source of truth: scripts/phase1_extract.py. Do not hand-merge back.
+
+from fastapi import APIRouter, Depends, HTTPException, Query, Response
+from sqlalchemy.orm import Session
+from sqlalchemy import and_, desc, asc, func
+from pydantic import BaseModel, Field
+from typing import Optional
+import uuid
+from datetime import datetime, timezone, timedelta
+import re
+import json
+import asyncio
+import math
+import httpx
+import os
+import logging
+from ..database import get_db, SessionLocal
+from ..models import (
+
+
+
+    User, Conversation, Message, AIModel, ModelProvider, TokenUsageLog,
+
+
+
+    KnowledgeBase, KnowledgeDocument, KnowledgeDocumentChunk, EvolutionSkill,
+
+
+
+    EvolutionTrajectory, MarketApp, AppPurchase, AppInstallation,
+
+
+
+    MCPServer,
+
+
+
+    gen_uuid
+
+
+
+)
+from ..auth import get_current_user
+from ..platform_knowledge import get_platform_knowledge
+from ..mcp_client import list_mcp_tools, call_mcp_tool
+from ..middleware.log_service import LogService
+from ..capability_registry import resolve_auto_model
+from app.api.chat_utils import (_extract_text_from_json, _proxy_for_url)
+
+def _normalize_dify_chat_url(endpoint: str) -> str:
+
+
+
+    """把用户填写的 Dify 地址归一化为 /v1/chat-messages 端点。与 market.py 保持一致。"""
+
+
+
+    ep = (endpoint or "").rstrip("/")
+
+
+
+    if ep.endswith("/chat-messages"):
+
+
+
+        return ep
+
+
+
+    if ep.endswith("/v1"):
+
+
+
+        return ep + "/chat-messages"
+
+
+
+    return ep + "/v1/chat-messages"
+
+async def _call_dify_agent(endpoint: str, api_key: str, query: str,
+
+
+
+                           user_id: str, ext_conv_id: str | None) -> tuple:
+
+
+
+    """调用 Dify(新版 Agent) /v1/chat-messages，流式(SSE)解析最终回答。
+
+
+
+
+
+
+
+    新版 Agent API 仅支持 response_mode=streaming(blocking 会 400)。事件流中
+
+
+
+    message / agent_message 事件携带增量 answer，累加即为最终回答；每个事件都带
+
+
+
+    conversation_id，用于维持多轮上下文。返回 (answer, new_conversation_id)。
+
+
+
+    """
+
+
+
+    url = _normalize_dify_chat_url(endpoint)
+
+
+
+    headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
+
+
+
+    payload = {
+
+
+
+        "inputs": {},
+
+
+
+        "query": query,
+
+
+
+        "user": (user_id or "web-user")[:64],
+
+
+
+        "response_mode": "streaming",
+
+
+
+        "conversation_id": ext_conv_id or "",
+
+
+
+    }
+
+
+
+    answer_parts = []
+
+
+
+    new_conv_id = ext_conv_id
+
+
+
+    err_msg = None
+
+
+
+    wf_outputs = None
+
+
+
+    # httpx 默认 follow_redirects=False；stream POST 若遇 301/302 会直接返回该状态码。
+
+
+
+    # 且 httpx 自动跟随会把 POST 降级为 GET(对 /v1/chat-messages 变 405)。
+
+
+
+    # 因此手动跟随 301/302/307/308，保持 POST 与请求体不变(最多 4 跳)。
+
+
+
+    async with httpx.AsyncClient(timeout=180, proxy=_proxy_for_url(url), trust_env=False,
+
+
+
+                                 follow_redirects=False) as client:
+
+
+
+        cur_url = url
+
+
+
+        for _redir in range(4):
+
+
+
+            async with client.stream("POST", cur_url, headers=headers, json=payload) as resp:
+
+
+
+                if resp.status_code in (301, 302, 307, 308):
+
+
+
+                    loc = resp.headers.get("location") or resp.headers.get("Location")
+
+
+
+                    await resp.aclose()
+
+
+
+                    if not loc:
+
+
+
+                        raise RuntimeError(f"Dify HTTP {resp.status_code}: 缺少重定向地址")
+
+
+
+                    # 处理相对地址
+
+
+
+                    if loc.startswith("/"):
+
+
+
+                        from urllib.parse import urlsplit, urlunsplit
+
+
+
+                        p = urlsplit(cur_url)
+
+
+
+                        loc = urlunsplit((p.scheme, p.netloc, loc, "", ""))
+
+
+
+                    cur_url = loc
+
+
+
+                    continue
+
+
+
+                if resp.status_code != 200:
+
+
+
+                    body = (await resp.aread()).decode("utf-8", "replace")
+
+
+
+                    raise RuntimeError(f"Dify HTTP {resp.status_code}: {body[:300]}")
+
+
+
+                async for line in resp.aiter_lines():
+
+
+
+                    if not line:
+
+
+
+                        continue
+
+
+
+                    line = line.strip()
+
+
+
+                    if not line.startswith("data:"):
+
+
+
+                        continue
+
+
+
+                    data_str = line[5:].strip()
+
+
+
+                    if not data_str or data_str == "[DONE]":
+
+
+
+                        continue
+
+
+
+                    try:
+
+
+
+                        evt = json.loads(data_str)
+
+
+
+                    except Exception:
+
+
+
+                        continue
+
+
+
+                    ev = evt.get("event")
+
+
+
+                    if evt.get("conversation_id"):
+
+
+
+                        new_conv_id = evt["conversation_id"]
+
+
+
+                    data = evt.get("data") or {}
+
+
+
+                    # 兼容三类 Dify 应用：
+
+
+
+                    #  1) Chat/Agent 应用: message / agent_message 事件带增量 answer
+
+
+
+                    #  2) Chatflow/Workflow 应用: text_chunk 事件带 data.text；
+
+
+
+                    #     结束时 workflow_finished 带 status / outputs / error
+
+
+
+                    if ev in ("message", "agent_message"):
+
+
+
+                        answer_parts.append(evt.get("answer", "") or "")
+
+
+
+                    elif ev == "message_replace":
+
+
+
+                        # 内容审查替换：以替换后的完整 answer 为准
+
+
+
+                        answer_parts = [evt.get("answer", "") or ""]
+
+
+
+                    elif ev == "text_chunk":
+
+
+
+                        answer_parts.append((data.get("text") or ""))
+
+
+
+                    elif ev == "node_finished":
+
+
+
+                        # answer / end 节点的 outputs 可能含最终文本
+
+
+
+                        if data.get("node_type") in ("answer", "end") and data.get("outputs"):
+
+
+
+                            wf_outputs = data.get("outputs")
+
+
+
+                        if data.get("status") == "failed" and data.get("error"):
+
+
+
+                            err_msg = str(data.get("error"))
+
+
+
+                    elif ev == "workflow_finished":
+
+
+
+                        if data.get("status") == "failed":
+
+
+
+                            err_msg = str(data.get("error") or "Dify 工作流执行失败")
+
+
+
+                        elif data.get("outputs"):
+
+
+
+                            wf_outputs = data.get("outputs")
+
+
+
+                    elif ev == "error":
+
+
+
+                        err_msg = evt.get("message") or evt.get("code") or "Dify 流式返回错误"
+
+
+
+            break
+
+
+
+        else:
+
+
+
+            raise RuntimeError("Dify 重定向次数过多(>4)，请直接填写重定向后的地址")
+
+
+
+    answer = "".join(answer_parts).strip()
+
+
+
+    # 流式无增量文本时，尝试从 workflow 结束 outputs 兜底提取
+
+
+
+    if not answer and wf_outputs:
+
+
+
+        answer = _extract_text_from_json(wf_outputs).strip()
+
+
+
+    if not answer and err_msg:
+
+
+
+        raise RuntimeError(err_msg)
+
+
+
+    return answer, new_conv_id
+
+async def _call_n8n_agent(endpoint: str, api_key: str, query: str,
+
+
+
+                          user_id: str, history: list) -> tuple:
+
+
+
+    """调用 n8n 生产 Webhook (POST)。webhook 用 Respond to Webhook 节点返回 JSON，
+
+
+
+    尽力从中抽取文本回答。n8n webhook 无平台级会话态，多轮靠我们下发 history。
+
+
+
+    返回 (answer, None)。"""
+
+
+
+    url = (endpoint or "").strip()
+
+
+
+    headers = {"Content-Type": "application/json"}
+
+
+
+    if api_key:
+
+
+
+        # n8n Header Auth 自定义头名不固定，同时下发常见几种以提高兼容性
+
+
+
+        headers["Authorization"] = f"Bearer {api_key}"
+
+
+
+        headers["X-N8N-API-KEY"] = api_key
+
+
+
+    # 携带多种字段别名，兼容 n8n AI Agent(chatInput) 与自定义工作流
+
+
+
+    payload = {
+
+
+
+        "query": query,
+
+
+
+        "chatInput": query,
+
+
+
+        "message": query,
+
+
+
+        "input": query,
+
+
+
+        "user": (user_id or "web-user")[:64],
+
+
+
+        "history": history or [],
+
+
+
+    }
+
+
+
+    async with httpx.AsyncClient(timeout=180, proxy=_proxy_for_url(url), trust_env=False) as client:
+
+
+
+        resp = await client.post(url, headers=headers, json=payload)
+
+
+
+    if resp.status_code not in (200, 201):
+
+
+
+        raise RuntimeError(f"n8n Webhook HTTP {resp.status_code}: {resp.text[:300]}")
+
+
+
+    ctype = resp.headers.get("content-type", "")
+
+
+
+    if "application/json" in ctype:
+
+
+
+        try:
+
+
+
+            answer = _extract_text_from_json(resp.json())
+
+
+
+        except Exception:
+
+
+
+            answer = resp.text
+
+
+
+    else:
+
+
+
+        answer = resp.text
+
+
+
+    return (answer or "").strip(), None
+
+async def _call_coze_agent(endpoint: str, api_key: str, query: str,
+
+
+
+                           user_id: str, ext_conv_id: str | None) -> tuple:
+
+
+
+    """调用扣子(Coze) v3 对话 API：create → 轮询 retrieve → 拉取 message list。
+
+
+
+
+
+
+
+    约定 agent_endpoint 存放 bot_id(纯 id)，或形如
+
+
+
+    https://api.coze.cn/v3/chat?bot_id=xxx 的完整地址。默认基址 api.coze.cn。
+
+
+
+    返回 (answer, conversation_id)。
+
+
+
+    """
+
+
+
+    ep = (endpoint or "").strip()
+
+
+
+    from urllib.parse import urlparse, parse_qs
+
+
+
+    bot_id = None
+
+
+
+    base = "https://api.coze.cn"
+
+
+
+    if ep.startswith("http"):
+
+
+
+        pr = urlparse(ep)
+
+
+
+        base = f"{pr.scheme}://{pr.netloc}"
+
+
+
+        qs = parse_qs(pr.query)
+
+
+
+        bot_id = (qs.get("bot_id") or [None])[0]
+
+
+
+        if not bot_id and "/bot/" in pr.path:
+
+
+
+            bot_id = pr.path.rstrip("/").split("/bot/")[-1]
+
+
+
+    else:
+
+
+
+        bot_id = ep  # 纯 bot_id
+
+
+
+    if not bot_id:
+
+
+
+        raise RuntimeError("Coze 接入缺少 bot_id(请在 Endpoint 填入 bot_id 或带 ?bot_id= 的地址)")
+
+
+
+    headers = {"Authorization": f"Bearer {api_key}", "Content-Type": "application/json"}
+
+
+
+    create_url = f"{base}/v3/chat"
+
+
+
+    if ext_conv_id:
+
+
+
+        create_url += f"?conversation_id={ext_conv_id}"
+
+
+
+    payload = {
+
+
+
+        "bot_id": bot_id,
+
+
+
+        "user_id": (user_id or "web-user")[:64],
+
+
+
+        "stream": False,
+
+
+
+        "auto_save_history": True,
+
+
+
+        "additional_messages": [
+
+
+
+            {"role": "user", "content": query, "content_type": "text"}
+
+
+
+        ],
+
+
+
+    }
+
+
+
+    async with httpx.AsyncClient(timeout=180, proxy=_proxy_for_url(create_url), trust_env=False) as client:
+
+
+
+        r = await client.post(create_url, headers=headers, json=payload)
+
+
+
+        if r.status_code != 200:
+
+
+
+            raise RuntimeError(f"Coze create HTTP {r.status_code}: {r.text[:300]}")
+
+
+
+        cd = r.json()
+
+
+
+        if cd.get("code") not in (0, None):
+
+
+
+            raise RuntimeError(f"Coze 错误: {cd.get('msg') or cd}")
+
+
+
+        chat = cd.get("data") or {}
+
+
+
+        chat_id = chat.get("id")
+
+
+
+        conv_id2 = chat.get("conversation_id") or ext_conv_id
+
+
+
+        if not chat_id or not conv_id2:
+
+
+
+            raise RuntimeError(f"Coze 未返回 chat_id/conversation_id: {str(cd)[:200]}")
+
+
+
+        # 轮询直到完成(最多约 3 分钟)
+
+
+
+        status = chat.get("status")
+
+
+
+        for _ in range(60):
+
+
+
+            if status == "completed":
+
+
+
+                break
+
+
+
+            if status in ("failed", "requires_action", "canceled"):
+
+
+
+                raise RuntimeError(f"Coze 对话状态 {status}")
+
+
+
+            await asyncio.sleep(3)
+
+
+
+            rr = await client.get(
+
+
+
+                f"{base}/v3/chat/retrieve?chat_id={chat_id}&conversation_id={conv_id2}",
+
+
+
+                headers=headers,
+
+
+
+            )
+
+
+
+            if rr.status_code == 200:
+
+
+
+                status = ((rr.json() or {}).get("data") or {}).get("status")
+
+
+
+        # 拉取消息列表，取 type=answer 的内容
+
+
+
+        rm = await client.get(
+
+
+
+            f"{base}/v3/chat/message/list?chat_id={chat_id}&conversation_id={conv_id2}",
+
+
+
+            headers=headers,
+
+
+
+        )
+
+
+
+        answer = ""
+
+
+
+        if rm.status_code == 200:
+
+
+
+            for m in ((rm.json() or {}).get("data") or []):
+
+
+
+                if m.get("type") == "answer" and m.get("role") == "assistant":
+
+
+
+                    answer = m.get("content", "") or answer
+
+
+
+    return (answer or "").strip(), conv_id2
+
+async def _call_external_agent(db, agent: dict, query: str, user_id: str,
+
+
+
+                               ext_conversation_id: str | None,
+
+
+
+                               history: list) -> tuple:
+
+
+
+    """按平台分发到对应外部 Agent 远程调用。返回 (answer, new_ext_conversation_id)。"""
+
+
+
+    platform = (agent.get("platform") or "dify").lower()
+
+
+
+    endpoint = agent.get("agent_endpoint")
+
+
+
+    api_key = agent.get("agent_api_key") or ""
+
+
+
+    if not endpoint:
+
+
+
+        raise RuntimeError("该 Agent 未配置远程调用地址(agent_endpoint)")
+
+
+
+    if platform == "dify":
+
+
+
+        return await _call_dify_agent(endpoint, api_key, query, user_id, ext_conversation_id)
+
+
+
+    if platform == "n8n":
+
+
+
+        return await _call_n8n_agent(endpoint, api_key, query, user_id, history)
+
+
+
+    if platform == "coze":
+
+
+
+        return await _call_coze_agent(endpoint, api_key, query, user_id, ext_conversation_id)
+
+
+
+    # 未知平台：默认按 Dify 协议尝试
+
+
+
+    return await _call_dify_agent(endpoint, api_key, query, user_id, ext_conversation_id)

backend/python/app/api/chat_knowledge.py

@@ -0,0 +1,319 @@
+# Auto-extracted from chat.py during Phase 1 refactor (Strangler Fig).
+# Source of truth: scripts/phase1_extract.py. Do not hand-merge back.
+
+from fastapi import APIRouter, Depends, HTTPException, Query, Response
+from sqlalchemy.orm import Session
+from sqlalchemy import and_, desc, asc, func
+from pydantic import BaseModel, Field
+from typing import Optional
+import uuid
+from datetime import datetime, timezone, timedelta
+import re
+import json
+import asyncio
+import math
+import httpx
+import os
+import logging
+from ..database import get_db, SessionLocal
+from ..models import (
+
+
+
+    User, Conversation, Message, AIModel, ModelProvider, TokenUsageLog,
+
+
+
+    KnowledgeBase, KnowledgeDocument, KnowledgeDocumentChunk, EvolutionSkill,
+
+
+
+    EvolutionTrajectory, MarketApp, AppPurchase, AppInstallation,
+
+
+
+    MCPServer,
+
+
+
+    gen_uuid
+
+
+
+)
+from ..auth import get_current_user
+from ..platform_knowledge import get_platform_knowledge
+from ..mcp_client import list_mcp_tools, call_mcp_tool
+from ..middleware.log_service import LogService
+from ..capability_registry import resolve_auto_model
+from app.api.chat_utils import (cst_now)
+
+def _build_knowledge_graph(db: Session, kb: KnowledgeBase) -> dict:
+
+
+
+    """从知识库文档构建知识图谱节点和边，返回 graph_data。"""
+
+
+
+    documents = db.query(KnowledgeDocument).filter(
+
+
+
+        and_(
+
+
+
+            KnowledgeDocument.knowledge_base_id == kb.id,
+
+
+
+            KnowledgeDocument.status == "completed",
+
+
+
+        )
+
+
+
+    ).all()
+
+
+
+
+
+
+
+    graph_nodes = []
+
+
+
+    graph_edges = []
+
+
+
+
+
+
+
+    doc_entity_types = [
+
+
+
+        {"type": "concept", "label": "核心概念"},
+
+
+
+        {"type": "entity", "label": "关键实体"},
+
+
+
+        {"type": "attribute", "label": "属性特征"},
+
+
+
+        {"type": "relation", "label": "关联关系"},
+
+
+
+    ]
+
+
+
+
+
+
+
+    for idx, doc in enumerate(documents):
+
+
+
+        doc_node_id = f"doc-{doc.id[:8]}"
+
+
+
+        graph_nodes.append({
+
+
+
+            "id": doc_node_id, "label": doc.filename, "type": "document", "size": 18, "group": "document",
+
+
+
+        })
+
+
+
+
+
+
+
+        filename_base = doc.filename.rsplit(".", 1)[0]
+
+
+
+        words = filename_base.replace("_", " ").replace("-", " ").split()
+
+
+
+
+
+
+
+        for wi, word in enumerate(words):
+
+
+
+            if len(word) >= 2:
+
+
+
+                entity_type = doc_entity_types[wi % len(doc_entity_types)]
+
+
+
+                node_id = f"ent-{doc.id[:8]}-{wi}"
+
+
+
+                graph_nodes.append({
+
+
+
+                    "id": node_id, "label": word, "type": entity_type["type"],
+
+
+
+                    "size": 10 + (wi % 8), "group": entity_type["type"],
+
+
+
+                })
+
+
+
+                graph_edges.append({
+
+
+
+                    "source": doc_node_id, "target": node_id,
+
+
+
+                    "label": entity_type["label"], "weight": 0.5 + (0.5 / (wi + 1)),
+
+
+
+                })
+
+
+
+
+
+
+
+    for i in range(len(documents)):
+
+
+
+        for j in range(i + 1, len(documents)):
+
+
+
+            fi = documents[i].filename.lower().rsplit(".", 1)[0]
+
+
+
+            fj = documents[j].filename.lower().rsplit(".", 1)[0]
+
+
+
+            words_i = set(fi.replace("_", " ").replace("-", " ").split())
+
+
+
+            words_j = set(fj.replace("_", " ").replace("-", " ").split())
+
+
+
+            common = words_i & words_j
+
+
+
+            if common and len(common) > 0:
+
+
+
+                graph_edges.append({
+
+
+
+                    "source": f"doc-{documents[i].id[:8]}",
+
+
+
+                    "target": f"doc-{documents[j].id[:8]}",
+
+
+
+                    "label": f"共有关键词: {', '.join(list(common)[:3])}",
+
+
+
+                    "weight": min(0.9, 0.3 * len(common)),
+
+
+
+                })
+
+
+
+
+
+
+
+    graph_data = {
+
+
+
+        "nodes": graph_nodes,
+
+
+
+        "edges": graph_edges,
+
+
+
+        "stats": {
+
+
+
+            "total_nodes": len(graph_nodes), "total_edges": len(graph_edges),
+
+
+
+            "document_count": len(documents),
+
+
+
+        },
+
+
+
+    }
+
+
+
+
+
+
+
+    kb.knowledge_graph_count = len(graph_nodes)
+
+
+
+    kb.updated_at = cst_now()
+
+
+
+    return graph_data

backend/python/app/api/chat_mcp.py

@@ -0,0 +1,240 @@
+# Auto-extracted from chat.py during Phase 1 refactor (Strangler Fig).
+# Source of truth: scripts/phase1_extract.py. Do not hand-merge back.
+
+from fastapi import APIRouter, Depends, HTTPException, Query, Response
+from sqlalchemy.orm import Session
+from sqlalchemy import and_, desc, asc, func
+from pydantic import BaseModel, Field
+from typing import Optional
+import uuid
+from datetime import datetime, timezone, timedelta
+import re
+import json
+import asyncio
+import math
+import httpx
+import os
+import logging
+from ..database import get_db, SessionLocal
+from ..models import (
+
+
+
+    User, Conversation, Message, AIModel, ModelProvider, TokenUsageLog,
+
+
+
+    KnowledgeBase, KnowledgeDocument, KnowledgeDocumentChunk, EvolutionSkill,
+
+
+
+    EvolutionTrajectory, MarketApp, AppPurchase, AppInstallation,
+
+
+
+    MCPServer,
+
+
+
+    gen_uuid
+
+
+
+)
+from ..auth import get_current_user
+from ..platform_knowledge import get_platform_knowledge
+from ..mcp_client import list_mcp_tools, call_mcp_tool
+from ..middleware.log_service import LogService
+from ..capability_registry import resolve_auto_model
+
+def _resolve_market_mcp(db: Session, mcp_ids: list, user_id: str) -> list:
+
+
+
+    """
+
+
+
+    加载用户选择的 MCP 应用配置（MarketApp.type == 'mcp'）。
+
+
+
+    返回: [{id, name, transport, endpoint, description}]
+
+
+
+    """
+
+
+
+    if not mcp_ids:
+
+
+
+        return []
+
+
+
+
+
+
+
+    mcps = db.query(MarketApp).filter(
+
+
+
+        MarketApp.id.in_(mcp_ids),
+
+
+
+        MarketApp.type == "mcp"
+
+
+
+    ).all()
+
+
+
+
+
+
+
+    results = []
+
+
+
+    for m in mcps:
+
+
+
+        config = {}
+
+
+
+        try:
+
+
+
+            if m.skill_content:
+
+
+
+                config = json.loads(m.skill_content)
+
+
+
+        except Exception:
+
+
+
+            config = {}
+
+
+
+        results.append({
+
+
+
+            "id": m.id,
+
+
+
+            "name": m.name,
+
+
+
+            "transport": m.mcp_transport or config.get("transport", "stdio"),
+
+
+
+            "endpoint": m.agent_endpoint or config.get("endpoint", "") or config.get("url", ""),
+
+
+
+            "description": m.description or config.get("description", ""),
+
+
+
+        })
+
+
+
+
+
+
+
+    return results
+
+def _resolve_active_mcp_servers(db: Session, mcp_ids: list) -> list:
+
+    """
+
+    解析用户选中的 MCP 服务连接信息（来自 mcp_servers 表，import 与后台添加通用）。
+
+    返回: [{id, name, description, transport, endpoint, url, headers}]
+
+    """
+
+    if not mcp_ids:
+
+        return []
+
+    servers = db.query(MCPServer).filter(
+
+        MCPServer.id.in_(mcp_ids),
+
+        MCPServer.is_enabled == True,
+
+    ).all()
+
+    out = []
+
+    for s in servers:
+
+        try:
+
+            headers = json.loads(s.headers) if isinstance(s.headers, str) else (s.headers or {})
+
+        except Exception:
+
+            headers = {}
+
+        out.append({
+
+            "id": s.id,
+
+            "name": s.name,
+
+            "description": s.description or "",
+
+            "transport": s.transport_type or "streamable_http",
+
+            "endpoint": s.url or "",
+
+            "url": s.url or "",
+
+            "headers": headers,
+
+        })
+
+    return out
+
+def _list_enabled_mcp_servers(db: Session) -> list:
+    """返回平台所有已启用(is_enabled=True)的 MCP 服务清单，供 AI 准确回答"平台有哪些 MCP"。"""
+    servers = db.query(MCPServer).filter(MCPServer.is_enabled == True).all()
+    out = []
+    for s in servers:
+        try:
+            headers = json.loads(s.headers) if isinstance(s.headers, str) else (s.headers or {})
+        except Exception:
+            headers = {}
+        out.append({
+            "id": s.id,
+            "name": s.name,
+            "description": s.description or "",
+            "transport": s.transport_type or "streamable_http",
+            "endpoint": s.url or "",
+            "url": s.url or "",
+            "headers": headers,
+        })
+    return out

backend/python/app/api/chat_utils.py

@@ -0,0 +1,378 @@
+# Auto-extracted from chat.py during Phase 1 refactor (Strangler Fig).
+# Source of truth: scripts/phase1_extract.py. Do not hand-merge back.
+
+from fastapi import APIRouter, Depends, HTTPException, Query, Response
+from sqlalchemy.orm import Session
+from sqlalchemy import and_, desc, asc, func
+from pydantic import BaseModel, Field
+from typing import Optional
+import uuid
+from datetime import datetime, timezone, timedelta
+import re
+import json
+import asyncio
+import math
+import httpx
+import os
+import logging
+from ..database import get_db, SessionLocal
+from ..models import (
+
+
+
+    User, Conversation, Message, AIModel, ModelProvider, TokenUsageLog,
+
+
+
+    KnowledgeBase, KnowledgeDocument, KnowledgeDocumentChunk, EvolutionSkill,
+
+
+
+    EvolutionTrajectory, MarketApp, AppPurchase, AppInstallation,
+
+
+
+    MCPServer,
+
+
+
+    gen_uuid
+
+
+
+)
+from ..auth import get_current_user
+from ..platform_knowledge import get_platform_knowledge
+from ..mcp_client import list_mcp_tools, call_mcp_tool
+from ..middleware.log_service import LogService
+from ..capability_registry import resolve_auto_model
+
+logger = logging.getLogger(__name__)
+
+CST_TZ = timezone(timedelta(hours=8))
+
+def cst_now():
+
+
+
+    """Get current time in CST timezone"""
+
+
+
+    return datetime.now(CST_TZ)
+
+def _html_escape(s: str) -> str:
+
+
+
+    return (str(s).replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;")
+
+
+
+            .replace('"', "&quot;").replace("'", "&#39;"))
+
+def _outbound_proxy() -> str:
+
+
+
+    """出网代理地址(兜底)。默认经 172.18.0.1:7890 智能路由代理；
+
+
+
+    运行时若未注入代理环境变量则回退到该地址。"""
+
+
+
+    import os
+
+
+
+    return os.environ.get("HTTPS_PROXY") or os.environ.get("HTTP_PROXY") or "http://172.18.0.1:7890"
+
+_HK_PROXY = "http://qyyfw:XHR521yhf@103.242.14.208:11495"
+
+def _proxy_for_url(url: str) -> str | None:
+
+
+
+    """按目标域名选择是否走代理。
+
+
+
+
+
+
+
+    2026-07-15 调整：AGNES(apihub.agnes-ai.com / platform-outputs.agnes-ai.space)
+
+
+
+    经用户实测走香港代理 103.242.14.208:11495 延迟更低、更稳定，故改为走 HK 代理
+
+
+
+    (带鉴权)，不再直连，也不经智能路由代理(172.18.0.1:7890 对 AGNES 返 502)。
+
+
+
+    其余国内/平台域名直连，真正的境外域名走 _outbound_proxy()。
+
+
+
+
+
+
+
+    注意：httpx 默认 trust_env=True 会忽略 proxy 参数而改用环境变量 HTTP_PROXY，
+
+
+
+    故调用方必须同时传 trust_env=False 才能真正走此处指定的代理。
+
+
+
+    """
+
+
+
+    import os
+
+
+
+    from urllib.parse import urlparse
+
+
+
+    host = (urlparse(url).hostname or "").lower()
+
+
+
+    if "agnes-ai.com" in host or "agnes-ai.space" in host:
+
+
+
+        return _HK_PROXY
+
+
+
+    # 平台自有域名(含自建 Dify/n8n 等)必须直连，不经出网代理。
+
+
+
+    if "qihuyun.com" in host:
+
+
+
+        return None
+
+
+
+    # 尊重 NO_PROXY：国内域名(硅基流动/DeepSeek/小米MoE 等)直连更稳定，
+
+
+
+    # 避免经易波动的智能路由代理 172.18.0.1:7890(对国内域名偶发连接重置)。
+
+
+
+    _no_proxy = [h.strip().lower() for h in os.environ.get("NO_PROXY", "").split(",") if h.strip()]
+
+
+
+    for _np in _no_proxy:
+
+
+
+        if host == _np or host.endswith("." + _np) or _np in host:
+
+
+
+            return None
+
+
+
+    return _outbound_proxy()
+
+def _extract_text_from_json(obj) -> str:
+
+
+
+    """从 n8n / 通用 webhook 的任意 JSON 响应中尽力抽取文本回答。"""
+
+
+
+    if obj is None:
+
+
+
+        return ""
+
+
+
+    if isinstance(obj, str):
+
+
+
+        return obj
+
+
+
+    if isinstance(obj, (int, float, bool)):
+
+
+
+        return str(obj)
+
+
+
+    if isinstance(obj, list):
+
+
+
+        # 取第一个可抽取到文本的元素
+
+
+
+        for it in obj:
+
+
+
+            t = _extract_text_from_json(it)
+
+
+
+            if t:
+
+
+
+                return t
+
+
+
+        return ""
+
+
+
+    if isinstance(obj, dict):
+
+
+
+        # 常见回答字段优先级
+
+
+
+        for k in ("output", "text", "answer", "response", "reply",
+
+
+
+                  "message", "content", "result", "data"):
+
+
+
+            if k in obj:
+
+
+
+                t = _extract_text_from_json(obj[k])
+
+
+
+                if t:
+
+
+
+                    return t
+
+
+
+        # 兜底：整体序列化
+
+
+
+        try:
+
+
+
+            return json.dumps(obj, ensure_ascii=False)
+
+
+
+        except Exception:
+
+
+
+            return str(obj)
+
+
+
+    return str(obj)
+
+class CreateConversationRequest(BaseModel):
+
+
+
+    title: str = Field(default="新对话")
+
+
+
+    model_id: Optional[str] = None
+
+class UpdateConversationRequest(BaseModel):
+
+
+
+    title: Optional[str] = None
+
+
+
+    status: Optional[str] = None  # active, archived
+
+
+
+    pinned: Optional[bool] = None
+
+
+
+    agent_ids: Optional[list] = None
+
+
+
+    skill_ids: Optional[list] = None
+
+
+
+    knowledge_base_ids: Optional[list] = None
+
+class SendMessageRequest(BaseModel):
+
+
+
+    content: str
+
+
+
+    model_id: Optional[str] = None
+
+
+
+    skill_ids: Optional[list] = None
+
+
+
+    agent_ids: Optional[list] = None
+
+
+
+    knowledge_base_ids: Optional[list] = None
+
+
+
+    mcp_ids: Optional[list] = None
+
+
+
+    attachments: Optional[list] = None
+
+
+
+    use_tools: Optional[bool] = False

backend/python/app/config.py

@@ -5,6 +5,7 @@ import os
 import logging
 from pathlib import Path
 from logging.handlers import TimedRotatingFileHandler
+from app.core.observability import TraceIdFilter, JsonFormatter
 from dotenv import load_dotenv
 
 # 加载 .env 文件（Docker 环境跳过，直接使用容器环境变量）
@@ -80,10 +81,14 @@ SLOW_REQUEST_THRESHOLD = float(os.getenv("SLOW_REQUEST_THRESHOLD", "3.0"))
 
 def setup_file_logging():
     """配置应用日志按天轮转输出到文件"""
-    log_format = logging.Formatter(
+    use_json = os.getenv("AIFLOW_JSON_LOGS", "0") == "1"
-        "%(asctime)s [%(levelname)s] %(name)s - %(message)s",
+    if use_json:
-        datefmt="%Y-%m-%d %H:%M:%S",
+        log_format = JsonFormatter()
-    )
+    else:
+        log_format = logging.Formatter(
+            "%(asctime)s [%(levelname)s] %(name)s [tid:%(trace_id)s] - %(message)s",
+            datefmt="%Y-%m-%d %H:%M:%S",
+        )
 
     # 应用日志 - 按天轮转，保留30天
     app_handler = TimedRotatingFileHandler(
@@ -123,10 +128,12 @@ def setup_file_logging():
     root_logger.setLevel(logging.INFO)
     root_logger.addHandler(app_handler)
     root_logger.addHandler(error_handler)
+    root_logger.addFilter(TraceIdFilter())
 
     # 配置访问日志专用logger
     access_logger = logging.getLogger("aiflow.access")
     access_logger.addHandler(access_handler)
     access_logger.propagate = False
+    access_logger.addFilter(TraceIdFilter())
 
     return access_logger

backend/python/app/core/__init__.py

@@ -0,0 +1 @@
+# app/core package - 统一可观测基座 (Phase 2.1)

backend/python/app/core/observability.py

@@ -0,0 +1,174 @@
+"""
+Qihu-AIFlow 统一可观测基座 (Phase 2.1)
+=====================================
+提供贯穿一次请求的全局链路 ID（trace_id）与结构化日志能力，
+为后续重构（上下文引擎、媒体/LLM 成本归因）建立回归基线。
+
+设计原则（Strangler Fig 增量）：
+- 纯增量埋点，不改任何既有接口的返回结构。
+- 向后兼容现有 `logging.getLogger(__name__)` 用法：trace_id 由 root logger 的
+  TraceIdFilter 自动注入，调用方无需改动。
+- 不引入第三方依赖（structlog 未安装），自带可选的 JSON formatter。
+
+使用方式：
+    from app.core.observability import set_trace_id, get_trace_id, span
+
+    # 在中间件里为每个请求注入：
+    set_trace_id(request_id)
+
+    # 在关键调用处打计时 span（自动带 trace_id）：
+    async with span("chat.send_message", conv_id=conv_id):
+        return await _send_message_core(...)
+"""
+import logging
+import time
+import uuid
+import json
+from contextvars import ContextVar
+
+# ---------------------------------------------------------------------------
+# trace_id：贯穿一次请求（含其内所有子调用/协程）的全局链路 ID
+# ---------------------------------------------------------------------------
+_trace_id_var: ContextVar[str] = ContextVar("aiflow_trace_id", default="-")
+
+
+def new_trace_id() -> str:
+    """生成一个崭新的链路 ID（32 位 hex）。"""
+    return uuid.uuid4().hex
+
+
+def set_trace_id(tid: str | None = None) -> str:
+    """设置当前上下文的 trace_id；不传则自动生成。返回最终使用的 ID。"""
+    tid = tid or new_trace_id()
+    _trace_id_var.set(tid)
+    return tid
+
+
+def get_trace_id() -> str:
+    """读取当前上下文的 trace_id；未设置时返回占位 '-'。"""
+    return _trace_id_var.get()
+
+
+class TraceIdFilter(logging.Filter):
+    """为每一条日志记录注入 trace_id 属性（不存在则写为 '-'）。
+
+    挂载到 root logger 后，所有 `getLogger(__name__)` 的子 logger 都会继承，
+    从而让全平台既有日志自动带上链路 ID，零侵入。
+    """
+
+    def filter(self, record: logging.LogRecord) -> bool:
+        if not hasattr(record, "trace_id"):
+            try:
+                record.trace_id = get_trace_id()
+            except Exception:
+                record.trace_id = "-"
+        return True
+
+
+# 标准 logging 字段，JSON 序列化时需排除，避免与自定义 extra 冲突
+_STD_ATTRS = set(logging.makeLogRecord({}).__dict__.keys()) | {
+    "message", "asctime", "msg", "trace_id", "exc_info", "stack_info", "args",
+}
+
+
+class JsonFormatter(logging.Formatter):
+    """结构化 JSON 日志格式（通过环境变量 AIFLOW_JSON_LOGS=1 启用）。
+
+    默认不启用，避免破坏现有日志采集/人工查阅习惯。
+    """
+
+    def format(self, record: logging.LogRecord) -> str:
+        data = {
+            "ts": self.formatTime(record, "%Y-%m-%dT%H:%M:%S%z"),
+            "level": record.levelname,
+            "logger": record.name,
+            "trace_id": getattr(record, "trace_id", "-"),
+            "msg": record.getMessage(),
+        }
+        for k, v in record.__dict__.items():
+            if k not in _STD_ATTRS:
+                try:
+                    json.dumps(v, ensure_ascii=False)
+                    data[k] = v
+                except (TypeError, ValueError):
+                    data[k] = str(v)
+        if record.exc_info:
+            data["exc"] = self.formatException(record.exc_info)
+        return json.dumps(data, ensure_ascii=False)
+
+
+def get_logger(name: str | None = None) -> logging.Logger:
+    """获取 logger（向后兼容现有用法）。trace_id 由 root filter 自动注入。"""
+    return logging.getLogger(name)
+
+
+# ---------------------------------------------------------------------------
+# span：结构化计时上下文管理器（耗时 + trace_id + 自定义字段）
+# ---------------------------------------------------------------------------
+class _Span:
+    """同步/异步通用的计时 span。退出时记录 elapsed_ms 与状态。"""
+
+    def __init__(self, name: str, **fields):
+        self.name = name
+        self.fields = fields
+        self._logger = logging.getLogger("aiflow.span")
+        self._start = None
+        self.elapsed_ms = 0.0
+
+    def _enter(self):
+        self._start = time.perf_counter()
+        self._logger.info(
+            f"SPAN_START {self.name}",
+            extra={"span": self.name, "event": "start", "trace_id": get_trace_id(), **self.fields},
+        )
+        return self
+
+    def _exit(self, exc_type, exc, tb):
+        self.elapsed_ms = (time.perf_counter() - self._start) * 1000
+        status = "error" if exc_type else "ok"
+        self._logger.info(
+            f"SPAN_END {self.name} {self.elapsed_ms:.1f}ms",
+            extra={
+                "span": self.name,
+                "event": "end",
+                "elapsed_ms": round(self.elapsed_ms, 1),
+                "status": status,
+                "trace_id": get_trace_id(),
+                **self.fields,
+            },
+        )
+        return False  # 不吞掉异常
+
+    # 同步协议
+    def __enter__(self):
+        return self._enter()
+
+    def __exit__(self, exc_type, exc, tb):
+        return self._exit(exc_type, exc, tb)
+
+    # 异步协议
+    async def __aenter__(self):
+        return self._enter()
+
+    async def __aexit__(self, exc_type, exc, tb):
+        return self._exit(exc_type, exc, tb)
+
+
+def span(name: str, **fields):
+    """创建一个计时 span。可同步或异步 with。
+
+    async with span("chat.send_message", conv_id=cid):
+        ...
+    """
+    return _Span(name, **fields)
+
+
+__all__ = [
+    "new_trace_id",
+    "set_trace_id",
+    "get_trace_id",
+    "TraceIdFilter",
+    "JsonFormatter",
+    "get_logger",
+    "span",
+]

backend/python/app/middleware/log_service.py

@@ -57,7 +57,7 @@ class LogService:
                         app_id: str = None, app_name: str = None,
                         conversation_id: str = None, prompt_tokens: int = 0,
                         completion_tokens: int = 0, total_tokens: int = 0,
-                        cost: float = 0):
+                        cost: float = 0, trace_id: str = None):
         """记录Token消耗日志"""
         db = SessionLocal()
         try:
@@ -73,6 +73,7 @@ class LogService:
                 completion_tokens=completion_tokens,
                 total_tokens=total_tokens,
                 cost=cost,
+                trace_id=trace_id,
             )
             db.add(log)
             db.commit()

backend/python/app/models.py

@@ -537,6 +537,7 @@ class TokenUsageLog(Base):
     app_id = Column(String(36), default=None)
     app_name = Column(String(200), default=None)
     conversation_id = Column(String(36), default=None)
+    trace_id = Column(String(64), default=None)
     prompt_tokens = Column(Integer, default=0)
     completion_tokens = Column(Integer, default=0)
     total_tokens = Column(Integer, default=0)

backend/python/main.py

@@ -25,6 +25,7 @@ from app.api.admin.developer_tools_admin import router as developer_tools_router
 from app.api.admin.mcp_market_admin import router as mcp_market_router
 from app.api.admin import asset_admin, achievement_admin
 from app.config import setup_file_logging, SLOW_REQUEST_THRESHOLD
+from app.core.observability import set_trace_id
 from app.database import SessionLocal
 from app.models import OperationLog as OperationLogModel, TokenUsageLog as TokenUsageLogModel
 
@@ -41,6 +42,7 @@ class APITrackingMiddleware(BaseHTTPMiddleware):
 
     async def dispatch(self, request: Request, call_next):
         request_id = str(uuid.uuid4())
+        set_trace_id(request_id)
         start_time = time.time()
         path = request.url.path
         method = request.method
@@ -53,6 +55,7 @@ class APITrackingMiddleware(BaseHTTPMiddleware):
             response = await call_next(request)
             elapsed = time.time() - start_time
             status_code = response.status_code
+            response.headers["X-Trace-Id"] = request_id
 
             if not should_skip:
                 access_logger.info(

backend/python/tests/test_regression_r1r2r3.py

@@ -1,40 +1,46 @@
-"""Phase 0 回归用例 R1/R2/R3 — 固化三个反复出现的问题。
+"""Phase 1 回归用例 R1/R2/R3 — 固化三个反复出现的问题（适配 chat.py 拆分后）。
 
 R1: 技能列表不得被硬性截断为 15 条（微信 Bot 曾只返回 15 个技能）。
 R2: 显式选中技能/MCP 时，system prompt 必须注入「最高优先级覆盖规则」。
 R3: 媒体生成（图像/视频）必须有超时保护，超时后回写用户可懂的提示而非永久挂起。
 
-说明：在 chat.py 巨石（10070 行）尚未拆分前，R1/R2 采用「源码契约」守卫，
+Phase 1 已将 chat.py 巨石拆分为 chat_{utils,media,agents,skill,mcp,context,knowledge,service}.py，
-这是最低风险且能切实挡住回归的方式；R3 为行为级单元测试（mock 内部生成调用模拟超时）。
+因此本文件采用「扫描整个 app/api 包」做源码契约守卫，并对行为级用例从正确的子模块导入。
-Phase 1 抽出 chat_media / chat_mcp / chat_skill 干净 seam 后，R1/R2 应升级为行为测试。
 """
 import asyncio
 import os
+import glob
 from unittest.mock import MagicMock
 
 import pytest
 
 _HERE = os.path.dirname(os.path.abspath(__file__))
-CHAT_PY = os.path.abspath(os.path.join(_HERE, "..", "app", "api", "chat.py"))
+API_DIR = os.path.abspath(os.path.join(_HERE, "..", "app", "api"))
 
 
-def _read_chat_source():
+def _read_api_package():
-    with open(CHAT_PY, "r", encoding="utf-8") as f:
+    parts = []
-        return f.read()
+    for p in sorted(glob.glob(os.path.join(API_DIR, "chat*.py"))):
+        with open(p, "r", encoding="utf-8") as f:
+            parts.append(f.read())
+    return "\n".join(parts)
 
 
 # ---------- R1 ----------
 def test_r1_skill_list_not_hard_capped_at_15():
     """微信 Bot 技能列表曾因 limit(15) 只返回 15 个，必须防止回归。"""
-    src = _read_chat_source()
+    src = _read_api_package()
     assert "limit(15)" not in src, "检测到 limit(15)，技能列表可能被硬性截断"
     assert "LIMIT 15" not in src, "检测到 LIMIT 15，技能列表可能被硬性截断"
 
 
 # ---------- R2 ----------
 def test_r2_explicit_selection_override_rule_present():
-    """显式选中技能/MCP 时，system prompt 必须包含最高优先级覆盖规则。"""
+    """显式选中技能/MCP 时，system prompt 必须包含最高优先级覆盖规则。
-    src = _read_chat_source()
+
+    注：_build_system_prompt 已随 Phase 1 拆分迁入 chat_context.py，故扫描整个 api 包。
+    """
+    src = _read_api_package()
     assert "最高优先级覆盖规则" in src, "显式选中工具的覆盖规则文本缺失"
     assert "只基于上文提供的该工具具体资料作答" in src, "覆盖规则应要求只基于已选工具资料作答"
     assert "平台知识库" in src and "不要引用" in src, "覆盖规则应明确禁止引用平台知识库通用描述"
@@ -45,37 +51,36 @@ def test_r2_explicit_selection_override_rule_present():
 async def test_r3_media_generation_timeout_returns_user_message(monkeypatch):
     """媒体生成超时必须回写用户可懂的超时提示，而非让前端永远『生成中…』。
 
-    注：_generate_media_task 是后台 fire-and-forget 任务（通过 asyncio.create_task 调用），
+    _generate_media_task 位于 chat_media（经 _send_message_core 的 asyncio.create_task 调用），
-    本身返回 None，超时保护体现在把提示回写到消息 content。故此处断言副作用。
+    本身是 fire-and-forget、返回 None，超时保护体现在把提示回写到消息 content。
     """
-    from app.api import chat as chat_mod
+    from app.api import chat_media as media_mod
 
     class FakeModel:
         id = "m_test"
         capabilities = ["image"]  # 图像 -> 90s 超时
 
     msg_mock = MagicMock()  # 捕获被回写的消息对象
-
     fake_session = MagicMock()
 
     def _query_side_effect(model_cls):
         q = MagicMock()
-        if model_cls is chat_mod.AIModel:
+        if model_cls is media_mod.AIModel:
             q.filter.return_value.first.return_value = FakeModel()
-        elif model_cls is chat_mod.Message:
+        elif model_cls is media_mod.Message:
             q.filter.return_value.first.return_value = msg_mock
         return q
 
     fake_session.query.side_effect = _query_side_effect
-    monkeypatch.setattr(chat_mod, "SessionLocal", lambda: fake_session)
+    monkeypatch.setattr(media_mod, "SessionLocal", lambda: fake_session)
 
     # 内部生成调用直接抛出 TimeoutError，等价模拟「生成耗时超过超时阈值」
     async def _hang(db, ai_model, prompt, user_id):
         raise asyncio.TimeoutError()
 
-    monkeypatch.setattr(chat_mod, "_generate_media", _hang)
+    monkeypatch.setattr(media_mod, "_generate_media", _hang)
 
-    await chat_mod._generate_media_task("msg1", "m_test", "画一只猫", "u1")
+    await media_mod._generate_media_task("msg1", "m_test", "画一只猫", "u1")
 
     # 副作用：消息 content 被回写为用户可懂的超时提示
     assert msg_mock.content is not None, "超时后必须回写消息内容，不能让前端永远『生成中…』"

frontend/package.json

@@ -1,33 +1,35 @@
-{
+{
-  "name": "frontend",
+  "name": "frontend",
-  "version": "1.0.0",
+  "version": "1.0.0",
-  "description": "",
+  "description": "",
-  "main": "index.js",
+  "main": "index.js",
-  "scripts": {
+  "scripts": {
-    "dev": "vite",
+    "dev": "vite",
-    "build": "vite build",
+    "build": "vite build",
-    "preview": "vite preview"
+    "preview": "vite preview",
-  },
+    "test": "vitest run"
-  "keywords": [],
+  },
-  "author": "",
+  "keywords": [],
-  "license": "ISC",
+  "author": "",
-  "dependencies": {
+  "license": "ISC",
-    "@types/qrcode": "^1.5.6",
+  "dependencies": {
-    "highlight.js": "^11.11.1",
+    "@types/qrcode": "^1.5.6",
-    "html2canvas": "^1.4.1",
+    "highlight.js": "^11.11.1",
-    "jspdf": "^4.2.1",
+    "html2canvas": "^1.4.1",
-    "katex": "^0.17.0",
+    "jspdf": "^4.2.1",
-    "mermaid": "^11.16.0",
+    "katex": "^0.17.0",
-    "qrcode": "^1.5.4",
+    "mermaid": "^11.16.0",
-    "react": "^18.3.1",
+    "qrcode": "^1.5.4",
-    "react-dom": "^18.3.1",
+    "react": "^18.3.1",
-    "react-router-dom": "^6.30.3"
+    "react-dom": "^18.3.1",
-  },
+    "react-router-dom": "^6.30.3"
-  "devDependencies": {
+  },
-    "@types/react": "^18.3.29",
+  "devDependencies": {
-    "@types/react-dom": "^18.3.7",
+    "@types/react": "^18.3.29",
-    "@vitejs/plugin-react": "^4.7.0",
+    "@types/react-dom": "^18.3.7",
-    "typescript": "^5.9.3",
+    "@vitejs/plugin-react": "^4.7.0",
-    "vite": "^5.4.21"
+    "typescript": "^5.9.3",
-  }
+    "vite": "^5.4.21",
-}
+    "vitest": "^4.1.10"
+  }
+}

frontend/src/utils/datetime.test.ts

@@ -0,0 +1,29 @@
+import { describe, it, expect } from 'vitest';
+import { formatBjTime } from './datetime';
+
+// 北京时间渲染曾是反复出现的问题（微信Bot时间错乱、Token驾驶舱时间列不正确等）。
+// 该测试固化「后端按北京时间存储、前端显式按 Asia/Shanghai 渲染」的契约。
+describe('formatBjTime', () => {
+  it('空输入返回占位符 -', () => {
+    expect(formatBjTime(undefined)).toBe('-');
+    expect(formatBjTime(null)).toBe('-');
+    expect(formatBjTime('')).toBe('-');
+  });
+
+  it('无时区标记的时间戳按 Asia/Shanghai (+08:00) 渲染', () => {
+    // 2026-07-18T12:00:00 视为北京时间，渲染应含 12:00:00
+    const out = formatBjTime('2026-07-18T12:00:00');
+    expect(out).toContain('2026');
+    expect(out).toContain('12:00:00');
+  });
+
+  it('UTC(Z) 时间戳正确转换为北京时间 (+8)', () => {
+    // 2026-07-18T04:00:00Z = 北京时间 12:00:00
+    const out = formatBjTime('2026-07-18T04:00:00Z');
+    expect(out).toContain('12:00:00');
+  });
+
+  it('非法日期原样返回', () => {
+    expect(formatBjTime('not-a-date')).toBe('not-a-date');
+  });
+});

frontend/vite.config.ts

@@ -1,4 +1,5 @@
-import { defineConfig } from 'vite';
+/// <reference types="vitest/config" />
+import { defineConfig } from 'vitest/config';
 import react from '@vitejs/plugin-react';
 import path from 'path';
 
@@ -17,8 +18,6 @@ export default defineConfig({
     chunkSizeWarningLimit: 1200,
     rollupOptions: {
       output: {
-        // 仅将 react 核心库单独打包（长期缓存）；
-        // 其余第三方库交给 Rollup 自动按需拆分，避免被强制合并成一个大 vendor 在首屏加载
         manualChunks(id) {
           if (id.includes('node_modules')) {
             if (
@@ -37,4 +36,9 @@ export default defineConfig({
   server: {
     port: 5173,
   },
+  // Phase 0 测试基建：vitest 配置（纯函数测试用 node 环境，组件测试可改 jsdom）
+  test: {
+    environment: 'node',
+    include: ['src/**/*.test.ts'],
+  },
 });

migrations/020_add_trace_id_token_usage_logs.sql

@@ -0,0 +1,7 @@
+-- Phase 2.2: add trace_id to token_usage_logs for cost attribution (idempotent)
+SET @db = DATABASE();
+SET @col_exists = (SELECT COUNT(*) FROM information_schema.columns WHERE table_schema = @db AND table_name = 'token_usage_logs' AND column_name = 'trace_id');
+SET @sql = IF(@col_exists = 0, 'ALTER TABLE token_usage_logs ADD COLUMN trace_id VARCHAR(64) NULL DEFAULT NULL', 'SELECT 1');
+PREPARE stmt FROM @sql;
+EXECUTE stmt;
+DEALLOCATE PREPARE stmt;

scripts/ci_gate.py

@@ -0,0 +1,115 @@
+"""Minimal CI gate for qihuaiflow.
+
+This is the canonical, single-source-of-truth gate. It runs the two test
+suites that guard production behavior, INSIDE the long-lived qihu-aiflow
+container (our deployment model keeps the app running in a container, not
+built fresh in CI):
+
+  1. backend pytest  -> /app/backend/python/.venv/bin/python -m pytest -c pytest.ini
+  2. frontend vitest -> /app/frontend && npx vitest run
+
+Exit codes:
+  0  all suites green  -> deploy/push may proceed
+  1  a test suite failed -> BLOCK (do not deploy/push)
+  2  environment error  -> docker missing or container not running
+
+It auto-detects where it runs:
+  * If the qihu-aiflow container is reachable via LOCAL docker (i.e. on the
+    deploy host), it execs directly.
+  * Otherwise it falls back to ssh_helper (remote control from a laptop),
+    which reaches the host over SSH and execs into the container there.
+
+Usage:
+  python scripts/ci_gate.py
+"""
+import os
+import subprocess
+import sys
+
+CONTAINER = "qihu-aiflow"
+
+BACKEND_CMD = "cd /app/backend/python && .venv/bin/python -m pytest -c pytest.ini -q"
+FRONTEND_CMD = "cd /app/frontend && npx vitest run --reporter=dot"
+
+
+def shell_quote(s: str) -> str:
+    return "'" + s.replace("'", "'\\''") + "'"
+
+
+def local_runner(cmd: str):
+    """Run via local docker (deploy host)."""
+    p = subprocess.run(cmd, shell=True, capture_output=True, text=True)
+    return p.stdout, p.stderr, p.returncode
+
+
+def get_runner():
+    # Prefer local docker ONLY if the actual container is reachable here
+    # (avoids a Windows-docker edge case where `docker ps` works but the
+    # qihu-aiflow container lives on a different host).
+    try:
+        r = subprocess.run(
+            "docker ps --format '{{.Names}}'",
+            shell=True, capture_output=True, text=True, timeout=10,
+        )
+        if r.returncode == 0 and CONTAINER in r.stdout.split():
+            return local_runner
+    except Exception:
+        pass
+    # Fall back to SSH control of the real host.
+    try:
+        import ssh_helper as s  # type: ignore
+        return lambda cmd: s.run(cmd)
+    except Exception as e:  # pragma: no cover
+        print(f"[CI Gate] ERROR: docker unavailable locally and ssh_helper missing: {e}",
+              file=sys.stderr)
+        sys.exit(2)
+
+
+def run_in_container(runner, cmd: str):
+    return runner(f"docker exec {CONTAINER} bash -c {shell_quote(cmd)}")
+
+
+def container_up(runner) -> bool:
+    out, _, _ = runner(f"docker ps --format {shell_quote('{{.Names}}')}")
+    return CONTAINER in out.split()
+
+
+def main():
+    runner = get_runner()
+    if not container_up(runner):
+        print(f"[CI Gate] ERROR: container {CONTAINER} is not running", file=sys.stderr)
+        sys.exit(2)
+
+    failures = []
+
+    print("=== [CI Gate] backend pytest ===")
+    o, e, rc = run_in_container(runner, BACKEND_CMD)
+    sys.stdout.write(o)
+    if e.strip():
+        print("STDERR:", e.strip(), file=sys.stderr)
+    if rc != 0:
+        failures.append("backend pytest")
+        print(f"[CI Gate] backend pytest FAILED (rc={rc})", file=sys.stderr)
+    else:
+        print("[CI Gate] backend pytest OK")
+
+    print("=== [CI Gate] frontend vitest ===")
+    o, e, rc = run_in_container(runner, FRONTEND_CMD)
+    sys.stdout.write(o)
+    if e.strip():
+        print("STDERR:", e.strip(), file=sys.stderr)
+    if rc != 0:
+        failures.append("frontend vitest")
+        print(f"[CI Gate] frontend vitest FAILED (rc={rc})", file=sys.stderr)
+    else:
+        print("[CI Gate] frontend vitest OK")
+
+    if failures:
+        print(f"\n[CI Gate] BLOCKED - failed: {', '.join(failures)}", file=sys.stderr)
+        sys.exit(1)
+    print("\n[CI Gate] PASSED - all suites green")
+    sys.exit(0)
+
+
+if __name__ == "__main__":
+    main()

scripts/git-hooks/pre-push

@@ -0,0 +1,42 @@
+#!/usr/bin/env bash
+# qihuaiflow minimal CI gate - git pre-push hook.
+#
+# Runs backend pytest + frontend vitest INSIDE the long-lived qihu-aiflow
+# container before any `git push` leaves this host. If a suite fails the
+# push is blocked (exit non-zero), so a broken tree can never be published.
+#
+# Install (on the deploy host, inside the repo):
+#   cp scripts/git-hooks/pre-push .git/hooks/pre-push
+#   chmod +x .git/hooks/pre-push
+#
+# This is a self-contained bash gate (no python dependency on the host).
+# The canonical, documented version lives in scripts/ci_gate.py.
+set -uo pipefail
+
+CONTAINER="qihu-aiflow"
+
+if ! docker ps --format '{{.Names}}' | grep -qx "$CONTAINER"; then
+  echo "[CI Gate] ERROR: container $CONTAINER is not running - aborting push" >&2
+  exit 2
+fi
+
+echo "=== [CI Gate] backend pytest ==="
+docker exec "$CONTAINER" bash -c 'cd /app/backend/python && .venv/bin/python -m pytest -c pytest.ini -q'
+backend_rc=$?
+if [ "$backend_rc" -ne 0 ]; then
+  echo "[CI Gate] backend pytest FAILED (rc=$backend_rc) - push blocked" >&2
+  exit 1
+fi
+echo "[CI Gate] backend pytest OK"
+
+echo "=== [CI Gate] frontend vitest ==="
+docker exec "$CONTAINER" bash -c 'cd /app/frontend && npx vitest run --reporter=dot'
+frontend_rc=$?
+if [ "$frontend_rc" -ne 0 ]; then
+  echo "[CI Gate] frontend vitest FAILED (rc=$frontend_rc) - push blocked" >&2
+  exit 1
+fi
+echo "[CI Gate] frontend vitest OK"
+
+echo "[CI Gate] PASSED - all suites green - push allowed"
+exit 0