qyyfw/qihuaiflow
1
0 Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
247 Commits 1 Branch 24 Tags
33d0f9c32e562a68a12ebb63f209c9478668b21d
Commit Graph
17 Commits
Author SHA1 Message Date
QihuAI Design 33d0f9c32e feat(workflow #874): NodeRegistry 统一工作流节点抽象 + 双 Claw 事件对齐 
#874 P3 工作流引擎 NodeRegistry 统一（加速带「工具即节点」思想落地，零源码拷贝重写）：

- app/core/node_registry.py（新增）：把 skill / MCP / Agent / DAG 四套割裂表达收敛为
  单一 Node 抽象。BaseNode(Protocol) 定义 node_type/name/input_schema/output_schema/
  risk_level + async run(ctx)；NodeContext 携带 db/tenant_id/claw_type/trace_id/黑板/emit；
  NodeResult.to_blackboard() 与既有 orchestrator 黑板结构无缝互转。内建八型节点
  llm/reduce/local/external/agent/skill/mcp/loop，均继承 NodeBase 公共生命周期
  （发 start 步骤 -> 执行 -> ok/fail 覆盖同 index），异常统一归一为 NodeError（带结构化
  diagnosis）。SkillNode 用 asyncio.to_thread 包裹同步执行器避免阻塞事件循环；
  LoopNode 硬限流 20 次迭代 / 并发上限 5，子节点经 ctx.child() 复用同一 StepCounter 与
  emit，保证步骤序号全图唯一。CapabilitySnapshot 统一承载 skills/mcp_servers/agents/
  function_definitions/mcp_tool_map，供 _build_system_prompt、run_graph、yaml 校验三处
  共用同一份能力真相。normalize_claw() 将 im_wechat/im_platform 归一为 "im"，避免
  Grafana 维度碎片化。

- app/core/capability_discovery.py：新增 discover_agents()，与 discover_skills 同构的
  三级降级策略（显式安装 -> published 兜底 -> rank_by_query 裁剪），补齐 Agent 维度的
  统一能力发现，杜绝各处内联第二份发现逻辑（#866 回退事故教训）。

- app/core/orchestrator.py：VALID_NODE_TYPES 与 validate_graph 改为运行期读
  registry.node_types()/requires_agent()，注册表成为节点类型的单一真相源；_run_node 改为
  拼装 NodeContext 后委托 registry.get(ntype).run(ctx)，本体只保留调度与拓扑；run_graph
  新增 on_event/claw_type/tenant_id/snapshot 参数，编排全过程以统一 EngineEvent 下发。

- app/core/yaml_workflow.py：valid_node_types() 实时读注册表；validate_yaml 补 skill/mcp/
  loop 必填校验；yaml_to_orchestration / orchestration_to_yaml 双向透传节点扩展字段
  （label/skill_id/func_name/tool/server_id/args/body/items/split/max_iterations/
  concurrency/join），修复配置往返丢字段。

- app/core/engine_event.py：make_step 新增 node 参数，以 step["node"]={id,type,label}
  承载节点身份。设计书原文的 EngineEvent(kind="node") 与前端仅识别 thinking/skill/mcp/
  agent/kb/media 的现状冲突（未知 kind 会被静默丢帧），故映射为已知 kind + 附加字段，
  线格式对旧前端字节兼容。

- app/api/chat_service.py：Web Claw 编排调用点透传 on_event/claw_type/tenant_id，
  租户与通道标签复用 #875 的 ContextVar（get_route_ctx），不再新增一层参数透传。
  IM Claw 经 EngineCore 自动继承同一事件流（单引擎原则，im_llm.py 零改动）。

- tests/test_node_registry.py（新增）：R-NODE-1~11 确定性哨兵，无网络无外部依赖。守卫
  内建八型齐全、注册表为单一真相源（orchestrator 与 yaml 校验实时同源）、describe 契约、
  双 Claw 事件对齐（SseAdapter 与 BufferAdapter 逐键相等且均带 node 身份）、claw 归一与
  tenant 标签、新节点必填校验、YAML 往返不吞字段、Loop 限流聚合、失败结构化诊断两端可见、
  禁止第二份能力发现、register 拒绝非法节点。

验证：容器内 py_compile 全通过；pytest tests/ 退出码 0（85 项 = 83 passed + 2 skipped，
0 FAILED/ERROR），含 R-NODE 16 项与 R-DUAL 双 Claw 哨兵。

safe-align: 仅复制 7 个 git-tracked 文件，未触碰 secrets/.venv/node_modules/dist/caches。
本地提交不 push。
 2026-08-04 15:16:00 +08:00
QihuAI Design 04a0641202 feat(otel #876): 双 Claw 强制标签 + 入口接线 + Grafana 双 Claw 对比看板 
#876 OpenTelemetry 可观测（双 Claw 标签）：
- app/core/otel.py: 新增 set_otel_ctx/get_otel_ctx(ContextVar 三元组 tenant_id+claw_type+
  trace_id)；current_span_cm 自动注入双 Claw 强制标签，全部 13 个 span 调用点零签名侵入带维度；
  record_span_metrics 指标维度补 tenant_id+claw_type（trace_id 因高基数不计入指标标签）。
- app/core/llm_router.py: set_route_ctx 桥接转调 set_otel_ctx（单次入口注入双端共享：
  路由上下文 + OTel 标签），失败安全。
- app/api/chat.py: send_message 与 send_message_stream 入口注入 set_route_ctx(web)，
  stream 经 asyncio.create_task 子协程自动传播 contextvars，使 _send_message_core 内
  所有 span 均带 tenant_id+claw_type。
- app/api/im_llm.py: _call_real_llm 入口最早注入 set_route_ctx(im_wechat)，并显式开启
  im.send_message span（此前 IM Claw 完全无 span，Grafana 无 IM 数据点），finally 关闭。
- tests/test_dual_claw.py: R-DUAL-6 由 skip 转为确定性断言（Web/IM 双 Claw span 必带
  tenant_id+claw_type），作为持续哨兵。
- 交付 Grafana 双 Claw 对比看板 JSON（aiflow_span_* / aiflow_llm_route_total 按 claw_type
  分组：P50/P95 延迟、成功率、QPS、租户健康 Top10）。Alloy OTLP 仅桥接 metrics→Prometheus，
  trace 无 pipeline（#800 已压制 UNIMPLEMENTED 噪声），看板基于 metrics 出图。

safe-align: 仅复制 5 个 git-tracked 文件，未触碰 secrets/venv/node_modules/caches。本地提交不 push。
 2026-08-04 13:49:34 +08:00
QihuAI Design 180409d317 feat(#875): LLM Router shadow mode + consolidate litellm master key 2026-08-04 13:16:30 +08:00
QihuAI Design 037c965c26 feat(engine): #873 dual-Claw event contract + SseAdapter/BufferAdapter + chat.py SSE delegate + im_llm EngineCore thin-shell (gray-gated) 
- app/core/engine_event.py (#880): EngineEvent dataclass + make_* factories, single construction entry, byte-compatible wire with legacy ChatPage.tsx
- app/core/claw_adapters.py (#884): SseAdapter (hot frames zero-metadata, terminal keeps trace_id) + BufferAdapter (assemble() preserves __media__)
- app/api/chat.py (#881): /messages/stream now delegates to SseAdapter; wire bytes unchanged (46B delta / done+message_id)
- app/api/im_llm.py (#883): IM_USE_ENGINE_CORE gray gate; _call_engine_core reuses EngineCore via BufferAdapter (default OFF, preserves media fast-path)
- app/api/chat_service.py: capture 3 container-ahead fixes (stray import asyncio removed, logger.info format, LiteLLM master key)
 2026-08-04 12:25:12 +08:00
QihuAI Design 29a1ce2e54 fix: 双Claw能力对齐 + MCP/LLM代理绕过（P0止血） 
- 新增 app/core/capability_discovery.py：统一能力发现层，Web/IM 双Claw共用单一实现
  （三级降级：用户安装 → 平台published → 相关性裁剪；MCP 自动挂载已启用服务）
- im_llm.py：IM Bot Claw 改用统一发现层，恢复 skills(12) + MCP(26工具) 能力，
  修复 #866 部署回退导致的 agents=[]/mcp=[] 回归；mcp__ 前缀分流到 call_mcp_tool
- mcp_client.py：注入 httpx_client_factory(trust_env=False) 绕过容器 HTTP_PROXY，
  修复外部 MCP(xiaobenyang.com / anysearch.com) tools/list 全部静默失败（mcp_tools 0→26）
- llm_client.py / im_bot_user.py：2026-07-31 代理劫持内部请求 + 媒体断连修复（随镜像已上线）
 2026-08-04 10:21:19 +08:00
QihuAI Design c624d8f37c mimo-token-plan-adapter
CI / gate (push) Has been cancelled
Details
2026-07-26 21:47:48 +08:00
QihuAI Design a2504bce34 fix(chat): resolve Auto-route failover event-loop freeze (#812) 
Root cause: ensure_structured_column() executed a synchronous blocking
engine.connect().execute(SELECT information_schema.columns ...) inside the
async request path (_build_chat_history / _maybe_compress_history), freezing
the uvicorn event loop and hanging all endpoints (~200s, recovered only by
docker restart).

Fix:
- chat_context.py: remove per-request ensure_structured_column() calls; run
  once idempotently at app startup (main.py lifespan). Use CURRENT_DATABASE()
  for PostgreSQL compatibility.
- main.py: add startup-only ensure_structured_column() hook (try/except).
- llm_client.py: trust_env=False + bounded httpx.Timeout/wait_for so broken
  endpoints fail fast instead of hanging the event loop.
- chat_service.py: run media generation in a background daemon thread with its
  own event loop (_launch_media_task_bg) so SSE end does not cancel image/video
  tasks (#811); remove TEMP faulthandler diagnostic.

Safe-align: persist accumulated live container changes (admin/*, knowledge,
market, models, config, observability, mcp_client, platform_knowledge,
im_bot_platform, chat_media, capability_registry, requirements) and add
workflow_templates.

Verified: live failover test (model-006 -> conn-refused) returns in ~1.9s with
concurrent login 0.83s/200; event-loop responsiveness restored (sub-second).
 2026-07-26 00:35:35 +08:00
QihuAI Design 771c8456b8 chore: normalize line endings to LF via .gitattributes 
Root-cause fix for mixed CRLF/LF in the repo (no .gitattributes previously).
Adds '* text=auto eol=lf' and explicit binary rules so all future checkouts
are LF. Renormalizes all tracked text files (no logical content change).
Backup tags: image qihuaiflow:backup-lineend-20260725-121632, git backup-lineend-pre-20260725-121632.
 2026-07-25 12:16:39 +08:00
QihuAI Design 256d9f33e6 chore(align): sync Git with running container as authoritative source 
Phase 0 SSE 流式聊天 landed in the running container and captured in Git so
the three sources (container := image := Git) agree.

Phase 0 — SSE 流式聊天（Web 端，IM bot 路径保持不变、共用引擎）:
- app/api/chat.py: 新增独立 SSE 端点
  POST /app/api/chat/conversations/{conv_id}/messages/stream
  基于 asyncio.Queue + create_task 将 _send_message_core 桥接为
  text/event-stream 生成器，吐出 delta / reasoning / done / error 事件
- app/api/chat_service.py: _send_message_core 增加 stream + on_event 参数；
  普通（无工具）聊天 round-0 主调用走 achat_complete(stream=True) 流式；
  done 事件携带 message_id / content / process_steps / used_skills /
  used_mcp / generating
- app/core/llm_client.py: 修复 _create_kwargs 硬编码 stream=False 的根因 bug
  （流式调用传 stream=True 导致 TypeError: multiple values for keyword），
  现 _create_kwargs(request_body, stream=True) 字段随参数变化
- frontend/src/pages/front/ChatPage.tsx + ChatPage.css + types/index.ts:
  发消息改为 fetch SSE 消费，实时增量渲染 AI 回复文本与思考链(reasoning)，
  done 事件用 message_id 定稿并渲染过程步骤；网络/流式失败自动回退非流式

Alignment performed 2026-07-24 18:25:11. No deletions; secrets/venv/node_modules/
caches were not touched (tracked-source-only copy).
 2026-07-24 18:25:15 +08:00
QihuAI Design 3788a93425 chore(align): sync Git with running container as authoritative source 
Phase 0 SSE 流式聊天 landed in the running container and captured in Git so
the three sources (container := image := Git) agree.

Phase 0 — SSE 流式聊天（Web 端，IM bot 路径保持不变、共用引擎）:
- app/api/chat.py: 新增独立 SSE 端点
  POST /app/api/chat/conversations/{conv_id}/messages/stream
  基于 asyncio.Queue + create_task 将 _send_message_core 桥接为
  text/event-stream 生成器，吐出 delta / reasoning / done / error 事件
- app/api/chat_service.py: _send_message_core 增加 stream + on_event 参数；
  普通（无工具）聊天 round-0 主调用走 achat_complete(stream=True) 流式；
  done 事件携带 message_id / content / process_steps / used_skills /
  used_mcp / generating
- app/core/llm_client.py: 修复 _create_kwargs 硬编码 stream=False 的根因 bug
  （流式调用传 stream=True 导致 TypeError: multiple values for keyword），
  现 _create_kwargs(request_body, stream=True) 字段随参数变化
- frontend/src/pages/front/ChatPage.tsx + ChatPage.css + types/index.ts:
  发消息改为 fetch SSE 消费，实时增量渲染 AI 回复文本与思考链(reasoning)，
  done 事件用 message_id 定稿并渲染过程步骤；网络/流式失败自动回退非流式

Alignment performed 2026-07-24 16:28:19. No deletions; secrets/venv/node_modules/
caches were not touched (tracked-source-only copy).
 2026-07-24 16:28:19 +08:00
QihuAI Design e8a1791c9f chore(align): sync Git with running container as authoritative source 
Standards alignment (STANDARDS_AUDIT_2026-07-24): Phase 1 标准化核心 +
Phase 2 质量层 landed in the running container and captured in Git so the
three sources (container := image := Git) agree.

Backend (Phase 1 标准化核心):
- mcp_client.py: replaced hand-rolled JSON-RPC with official `mcp` SDK
  (ClientSession, streamable_http/sse transports, protocol auto-negotiation)
- core/otel.py + core/observability.py: standard OpenTelemetry SDK bridge
  (OTLP gRPC), wired to Grafana Alloy (qihu-aiflow-alloy:4317) which forwards
  metrics to Prometheus; emits aiflow_span_duration_ms / aiflow_span_count
- deployments: Grafana Alloy container + Prometheus remote-write receiver

Backend (Phase 2 质量层):
- core/rag_pipeline.py: standardized RAG (recursive overlap chunking +
  rerank hook) reusing pgvector_search
- core/llm_client.py: SDK-ified LLM + embeddings (aembed/embed) collapsing
  hand-rolled httpx calls; trust_env=False to bypass container proxy

Migrations:
- 018_normalize_marketapp_scene.sql: fixed capability allowlist
  (prevents capability wipe on restart)

Alignment performed 2026-07-24 12:18:51. No deletions; secrets/venv/node_modules/
caches were not touched (tracked-source-only copy).
 2026-07-24 12:18:52 +08:00
QihuAI Design a08ec87b22 chore(align): sync Git with running container as authoritative source 
One-time alignment to close drift between the running container
(qihu-aiflow / image qihuaiflow:with-deps) and this Git repository.
Container hotfixes previously persisted only via `docker commit` are now
captured in Git so Git becomes the single source of truth.

Backend:
- chat_service.py: accumulated fixes incl. EigenFlux README guidance
- chat_context.py / chat_media.py: URL/web-repo context injection
- api/url_reader.py: NEW SSRF-safe web/README reader (Issue #725)
- skills_admin.py & Skills.tsx: 14-scene capability unification
- pgvector_search.py, orchestrator.py, platform_knowledge.py,
  market.py, admin/models.py: drift sync

Frontend:
- ChatPage.tsx: collapsible AI think/act block (Issue #724)
- MarketPage / EvolutionDashboard / TokenDashboard styling sync

Migrations:
- 018_normalize_marketapp_scene.sql: fixed capability allowlist
  (prevents capability wipe on restart)

Alignment performed 2026-07-23 05:42:47. No deletions; secrets/venv/node_modules/
caches were not touched (tracked-source-only copy).
 2026-07-23 05:42:48 +08:00
QihuAI Design 79c27795a8 Phase 4.2+4.3: YAML workflow spec + ACA protocol abstraction (v2026.07.19.10)
CI / gate (push) Has been cancelled
Details
2026-07-19 20:17:49 +08:00
QihuAI Design 669c16b0a0 Phase 4.1: 技能市场安检引擎 + 版本化 + 绿标 (v2026.07.19.9)
CI / gate (push) Has been cancelled
Details
2026-07-19 19:15:35 +08:00
QihuAI Design 96b3f0af37 Phase 3.3 多 Agent 编排图引擎: orchestration JSON 列 + 拓扑并行引擎 + local/external/reduce 节点 + 工具沙箱继承 + 后台编排 API
CI / gate (push) Has been cancelled
Details
2026-07-19 16:41:27 +08:00
QihuAI Design 3b34ba78b8 feat(agents): Phase3.2 Agent凭据Fernet加密存储 写入加密+调用点解密+返回脱敏+主密钥持久化防竞态 [v2026.07.19.7]
CI / gate (push) Has been cancelled
Details
2026-07-19 15:27:15 +08:00
QihuAI Design 6aefdfa5c2 refactor(observability): Phase2.1 统一可观测基座 trace_id+结构化日志+span计时 [v2026.07.19.2]
CI / gate (push) Has been cancelled
Details
2026-07-19 07:47:25 +08:00