mirror of
https://github.com/luckyyzh/pi-agent-integrated.git
synced 2026-10-03 02:59:35 +00:00
feat: 识图提速(模型常驻/精简模板/磁盘缓存)+ WebUI 上传压缩 + 缓存清理脚本
- vision.ts: keep_alive:-1 常驻显存(治冷加载 7.7GB);转录缓存持久化 data/agent/vision-cache.json(重启后历史图秒回);hook 转录用精简模板 - ChatInput/image-attachments: 上传 JPEG 长边>1600px 自动压缩 q0.85 - scripts/cleanup-cache.mjs: npm/opengrep 缓存、过期会话、旧图瘦身清理 - README 双语同步
This commit is contained in:
@@ -156,7 +156,7 @@ DeepSeek 等纯文本模型不能接收图片。仓库内置 `vision` 子代理
|
|||||||
**后端一:本地 Ollama(默认,免费私密)**
|
**后端一:本地 Ollama(默认,免费私密)**
|
||||||
|
|
||||||
- 前置:本机安装 [Ollama](https://ollama.com) 并 `ollama pull qwen3-vl:8b`。
|
- 前置:本机安装 [Ollama](https://ollama.com) 并 `ollama pull qwen3-vl:8b`。
|
||||||
- 环境变量:`OLLAMA_HOST`(默认 `http://localhost:11434`)、`OLLAMA_VISION_MODEL`(默认 `qwen3-vl:8b`)。
|
- 环境变量:`OLLAMA_HOST`(默认 `http://localhost:11434`)、`OLLAMA_VISION_MODEL`(默认 `qwen3-vl:8b`)、`OLLAMA_VISION_KEEP_ALIVE`(默认 `-1` 常驻显存,避免每次识图冷加载大模型;也可设 `30m` 等时长)。
|
||||||
|
|
||||||
**后端二:OpenAI 兼容视觉 API**
|
**后端二:OpenAI 兼容视觉 API**
|
||||||
|
|
||||||
@@ -166,7 +166,9 @@ DeepSeek 等纯文本模型不能接收图片。仓库内置 `vision` 子代理
|
|||||||
|
|
||||||
纯文本主模型(如 DeepSeek)无法接收图片,直接在 WebUI 上传会让请求失败(DeepSeek API 返回 HTTP 400)。`vision` 扩展注册了 `before_provider_request` 钩子:请求发出前检测到图片附件时,自动调用配置的视觉后端生成文本描述并替换进消息,主模型直接基于描述继续推理——上传即用,无需手动操作。支持图片的主模型则原样透传,不受影响。
|
纯文本主模型(如 DeepSeek)无法接收图片,直接在 WebUI 上传会让请求失败(DeepSeek API 返回 HTTP 400)。`vision` 扩展注册了 `before_provider_request` 钩子:请求发出前检测到图片附件时,自动调用配置的视觉后端生成文本描述并替换进消息,主模型直接基于描述继续推理——上传即用,无需手动操作。支持图片的主模型则原样透传,不受影响。
|
||||||
|
|
||||||
描述按**单张图片**做会话级缓存:只有新上传的图片会调用视觉模型,历史图片秒回缓存。每轮请求会把历史图片的描述文本一并注入上下文以保持主模型的记忆——上下文体积会随历史图片数增长,属已知取舍(Ollama 端已显式提升 `num_ctx`,DeepSeek 前缀缓存可摊薄费用)。
|
描述按**单张图片**缓存并**持久化到磁盘**(`data/agent/vision-cache.json`,上限 64 条):只有新上传的图片会调用视觉模型,历史图片(含重启后)秒回缓存。自动转录使用**精简模板**(约百字摘要;`vision` 工具仍返回完整 OCR),并显式 `keep_alive: -1` 让模型常驻显存。每轮请求会把历史图片的描述文本一并注入上下文以保持主模型的记忆——上下文体积会随历史图片数增长,属已知取舍(Ollama 端已显式提升 `num_ctx`,DeepSeek 前缀缓存可摊薄费用)。
|
||||||
|
|
||||||
|
- WebUI 上传 JPEG 会自动压缩(长边 >1600px 时缩放至 1600px、质量 0.85):相机照片从数 MB 降到几百 KB,会话文件不膨胀、加载与转录更快;PNG/WebP/GIF 原样保留(无损/动画)。
|
||||||
|
|
||||||
- 用法:对主模型说“用 vision 子代理看 <图片路径>”即可;也可 `/run vision`(子代理用于主动深度分析多图;上传自动转录已覆盖日常识图)。
|
- 用法:对主模型说“用 vision 子代理看 <图片路径>”即可;也可 `/run vision`(子代理用于主动深度分析多图;上传自动转录已覆盖日常识图)。
|
||||||
- 主会话直用:重启 pi 后 `vision` 工具在主会话也可用,可对磁盘上的图片主动调用。
|
- 主会话直用:重启 pi 后 `vision` 工具在主会话也可用,可对磁盘上的图片主动调用。
|
||||||
@@ -245,6 +247,7 @@ npm run smoke:search -- "关键词" # 使用真实 SearXNG;需要配置
|
|||||||
- Rewind 检查点包含大量松散 Git 对象时自动执行压缩,但保留所有有效检查点;
|
- Rewind 检查点包含大量松散 Git 对象时自动执行压缩,但保留所有有效检查点;
|
||||||
- 已找不到对应会话的孤儿 Rewind 检查点保留 7 天后自动删除;
|
- 已找不到对应会话的孤儿 Rewind 检查点保留 7 天后自动删除;
|
||||||
- 会话、记忆、凭据、模型配置、已安装插件和仍有关联的检查点不会被自动删除。
|
- 会话、记忆、凭据、模型配置、已安装插件和仍有关联的检查点不会被自动删除。
|
||||||
|
- 手动清理:`node scripts/cleanup-cache.mjs`(默认预览,加 `--apply` 执行)——清 npm/opengrep 缓存、按天数移走过期会话、把旧会话图片替换为占位符瘦身;删除先进回收目录。
|
||||||
|
|
||||||
```powershell
|
```powershell
|
||||||
npm run storage:status # 只查看受管缓存大小
|
npm run storage:status # 只查看受管缓存大小
|
||||||
@@ -395,7 +398,7 @@ Configuration: the “Vision” tab inside the “Models” panel in the lower-l
|
|||||||
**Backend 1: local Ollama (default, free and private)**
|
**Backend 1: local Ollama (default, free and private)**
|
||||||
|
|
||||||
- Prerequisite: install [Ollama](https://ollama.com) and run `ollama pull qwen3-vl:8b`.
|
- Prerequisite: install [Ollama](https://ollama.com) and run `ollama pull qwen3-vl:8b`.
|
||||||
- Env: `OLLAMA_HOST` (default `http://localhost:11434`), `OLLAMA_VISION_MODEL` (default `qwen3-vl:8b`).
|
- Env: `OLLAMA_HOST` (default `http://localhost:11434`), `OLLAMA_VISION_MODEL` (default `qwen3-vl:8b`), `OLLAMA_VISION_KEEP_ALIVE` (default `-1` — keep the model resident in VRAM to avoid cold-loading it on every transcription; can be set to e.g. `30m`).
|
||||||
|
|
||||||
**Backend 2: OpenAI-compatible vision API**
|
**Backend 2: OpenAI-compatible vision API**
|
||||||
|
|
||||||
@@ -405,7 +408,9 @@ Configuration: the “Vision” tab inside the “Models” panel in the lower-l
|
|||||||
|
|
||||||
A text-only main model such as DeepSeek cannot receive images — uploading one in the Web UI fails the request (DeepSeek API returns HTTP 400). The `vision` extension registers a `before_provider_request` hook: when it detects image attachments, it transcribes them through the configured vision backend and replaces them with text before the request is sent, so the main model keeps reasoning seamlessly. Vision-capable main models pass through untouched.
|
A text-only main model such as DeepSeek cannot receive images — uploading one in the Web UI fails the request (DeepSeek API returns HTTP 400). The `vision` extension registers a `before_provider_request` hook: when it detects image attachments, it transcribes them through the configured vision backend and replaces them with text before the request is sent, so the main model keeps reasoning seamlessly. Vision-capable main models pass through untouched.
|
||||||
|
|
||||||
Descriptions are cached **per image** for the session: only genuinely new uploads call the vision model, while previously seen images resolve from cache instantly. Every request also re-injects the accumulated image descriptions so the main model keeps its memory of them — a known trade-off where context grows with the number of images (the Ollama backend raises `num_ctx` explicitly, and DeepSeek prefix caching keeps the cost modest).
|
Descriptions are cached **per image** and **persisted to disk** (`data/agent/vision-cache.json`, capped at 64 entries): only genuinely new uploads call the vision model, while previously seen images — including after a restart — resolve from cache instantly. The automatic transcription pipeline uses a **concise prompt** (~100-character summary; the `vision` tool still returns full OCR) and sends `keep_alive: -1` so the model stays resident in VRAM. Every request also re-injects the accumulated image descriptions so the main model keeps its memory of them — a known trade-off where context grows with the number of images (the Ollama backend raises `num_ctx` explicitly, and DeepSeek prefix caching keeps the cost modest).
|
||||||
|
|
||||||
|
- Web UI uploads auto-compress JPEGs (downscaled to 1600px long edge at quality 0.85 when larger): camera photos drop from several MB to a few hundred KB, so session files stop growing and load/transcribe faster; PNG/WebP/GIF pass through untouched (lossless/animated).
|
||||||
|
|
||||||
- Usage: ask the main model to “use the vision subagent to look at <path>”, or run `/run vision` (the subagent is for proactive deep analysis of many images; everyday image reading is covered by automatic transcription).
|
- Usage: ask the main model to “use the vision subagent to look at <path>”, or run `/run vision` (the subagent is for proactive deep analysis of many images; everyday image reading is covered by automatic transcription).
|
||||||
- Main-session use: after restarting pi, the `vision` tool is also available in the main session for images on disk.
|
- Main-session use: after restarting pi, the `vision` tool is also available in the main session for images on disk.
|
||||||
@@ -477,6 +482,7 @@ Before `dev`, `restart`, `build`, or `start` launches Pi Web, the integrated lau
|
|||||||
- Rewind repositories with many loose Git objects are compacted without dropping valid checkpoints;
|
- Rewind repositories with many loose Git objects are compacted without dropping valid checkpoints;
|
||||||
- orphan Rewind checkpoints whose session no longer exists are removed after a seven-day grace period;
|
- orphan Rewind checkpoints whose session no longer exists are removed after a seven-day grace period;
|
||||||
- sessions, memory, credentials, model configuration, installed plugins, and linked checkpoints are never automatically deleted.
|
- sessions, memory, credentials, model configuration, installed plugins, and linked checkpoints are never automatically deleted.
|
||||||
|
- Manual cleanup: `node scripts/cleanup-cache.mjs` (dry-run by default; add `--apply` to execute) — clears npm/opengrep caches, moves expired sessions aside by age, and can shrink old sessions by replacing images with placeholders; deletions go to a trash directory first.
|
||||||
|
|
||||||
```powershell
|
```powershell
|
||||||
npm run storage:status # Report managed cache sizes without changing data
|
npm run storage:status # Report managed cache sizes without changing data
|
||||||
|
|||||||
@@ -6,6 +6,7 @@ import { clearDraft, getDraft, setDraft, type ChatDraftImage } from "@/lib/draft
|
|||||||
import {
|
import {
|
||||||
MAX_ATTACHED_IMAGE_BYTES,
|
MAX_ATTACHED_IMAGE_BYTES,
|
||||||
MAX_ATTACHED_IMAGES,
|
MAX_ATTACHED_IMAGES,
|
||||||
|
compressImageFile,
|
||||||
isBase64ImageWithinLimits,
|
isBase64ImageWithinLimits,
|
||||||
} from "@/lib/image-attachments";
|
} from "@/lib/image-attachments";
|
||||||
import {
|
import {
|
||||||
@@ -390,20 +391,21 @@ export const ChatInput = forwardRef<ChatInputHandle, Props>(function ChatInput({
|
|||||||
pendingImageCountRef.current += imageFiles.length;
|
pendingImageCountRef.current += imageFiles.length;
|
||||||
try {
|
try {
|
||||||
const newImages = await Promise.all(
|
const newImages = await Promise.all(
|
||||||
imageFiles.map(
|
imageFiles.map(async (file) => {
|
||||||
(file) =>
|
// 大 JPEG 先压缩(长边 1600/q0.85),会话不再内嵌数 MB base64,加载/传输/转录都更快
|
||||||
new Promise<AttachedImage>((resolve, reject) => {
|
const compressed = await compressImageFile(file);
|
||||||
const reader = new FileReader();
|
const base64 = await new Promise<string>((resolve, reject) => {
|
||||||
reader.onload = () => {
|
const reader = new FileReader();
|
||||||
const result = reader.result as string;
|
reader.onload = () => resolve((reader.result as string).split(",")[1]);
|
||||||
// result is "data:<mime>;base64,<data>"
|
reader.onerror = reject;
|
||||||
const base64 = result.split(",")[1];
|
reader.readAsDataURL(compressed);
|
||||||
resolve({ data: base64, mimeType: file.type, previewUrl: URL.createObjectURL(file) });
|
});
|
||||||
};
|
return {
|
||||||
reader.onerror = reject;
|
data: base64,
|
||||||
reader.readAsDataURL(file);
|
mimeType: compressed.type,
|
||||||
})
|
previewUrl: URL.createObjectURL(compressed),
|
||||||
)
|
};
|
||||||
|
})
|
||||||
);
|
);
|
||||||
setAttachedImages((prev) => {
|
setAttachedImages((prev) => {
|
||||||
const accepted = newImages.slice(0, Math.max(0, MAX_ATTACHED_IMAGES - prev.length));
|
const accepted = newImages.slice(0, Math.max(0, MAX_ATTACHED_IMAGES - prev.length));
|
||||||
|
|||||||
@@ -1,6 +1,11 @@
|
|||||||
export const MAX_ATTACHED_IMAGE_BYTES = 10 * 1024 * 1024;
|
export const MAX_ATTACHED_IMAGE_BYTES = 10 * 1024 * 1024;
|
||||||
export const MAX_ATTACHED_IMAGES = 10;
|
export const MAX_ATTACHED_IMAGES = 10;
|
||||||
|
|
||||||
|
/** 上传时自动压缩:长边超过此像素的 JPEG 缩到此边长(相机照片主场景,视觉模型输入上限 ~1344px,1600 无损)。 */
|
||||||
|
export const MAX_IMAGE_EDGE_PX = 1600;
|
||||||
|
/** JPEG 重编码质量:避免伪影糊掉边缘/小字。 */
|
||||||
|
export const IMAGE_JPEG_QUALITY = 0.85;
|
||||||
|
|
||||||
export interface Base64ImageAttachment {
|
export interface Base64ImageAttachment {
|
||||||
data: string;
|
data: string;
|
||||||
mimeType: string;
|
mimeType: string;
|
||||||
@@ -37,6 +42,41 @@ export function isBase64ImageWithinLimits(value: unknown): value is Base64ImageA
|
|||||||
return bytes !== null && bytes <= MAX_ATTACHED_IMAGE_BYTES;
|
return bytes !== null && bytes <= MAX_ATTACHED_IMAGE_BYTES;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/**
|
||||||
|
* 上传前压缩:仅 JPEG 且长边超过 MAX_IMAGE_EDGE_PX 时缩放重编码(EXIF 方向由
|
||||||
|
* createImageBitmap 默认 from-image 纠正)。PNG/WebP/GIF 原样返回(无损/动画场景)。
|
||||||
|
* 重编码无收益(更小或失败)时退回原文件。
|
||||||
|
*/
|
||||||
|
export async function compressImageFile(file: File): Promise<File> {
|
||||||
|
if (file.type !== "image/jpeg") return file;
|
||||||
|
let bitmap: ImageBitmap;
|
||||||
|
try {
|
||||||
|
bitmap = await createImageBitmap(file);
|
||||||
|
} catch {
|
||||||
|
return file; // 解码失败退化为原文件
|
||||||
|
}
|
||||||
|
try {
|
||||||
|
const edge = Math.max(bitmap.width, bitmap.height);
|
||||||
|
if (edge <= MAX_IMAGE_EDGE_PX) return file; // 本就不大,避免无谓重编码
|
||||||
|
const scale = MAX_IMAGE_EDGE_PX / edge;
|
||||||
|
const width = Math.max(1, Math.round(bitmap.width * scale));
|
||||||
|
const height = Math.max(1, Math.round(bitmap.height * scale));
|
||||||
|
const canvas = document.createElement("canvas");
|
||||||
|
canvas.width = width;
|
||||||
|
canvas.height = height;
|
||||||
|
const ctx = canvas.getContext("2d");
|
||||||
|
if (!ctx) return file;
|
||||||
|
ctx.drawImage(bitmap, 0, 0, width, height);
|
||||||
|
const blob = await new Promise<Blob | null>((resolve) =>
|
||||||
|
canvas.toBlob(resolve, "image/jpeg", IMAGE_JPEG_QUALITY)
|
||||||
|
);
|
||||||
|
if (!blob || blob.size >= file.size) return file; // 重编码无收益则用原文件
|
||||||
|
return new File([blob], file.name.replace(/\.\w+$/, ".jpg"), { type: "image/jpeg" });
|
||||||
|
} finally {
|
||||||
|
bitmap.close();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/** Return an API-safe error for prompt, steering, and follow-up image arrays. */
|
/** Return an API-safe error for prompt, steering, and follow-up image arrays. */
|
||||||
export function validateAgentImages(value: unknown): string | null {
|
export function validateAgentImages(value: unknown): string | null {
|
||||||
if (value === undefined) return null;
|
if (value === undefined) return null;
|
||||||
|
|||||||
@@ -18,7 +18,7 @@
|
|||||||
|
|
||||||
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
import type { ExtensionAPI } from "@earendil-works/pi-coding-agent";
|
||||||
import { createHash } from "node:crypto";
|
import { createHash } from "node:crypto";
|
||||||
import { readFile, stat } from "node:fs/promises";
|
import { readFile, stat, writeFile } from "node:fs/promises";
|
||||||
import { homedir } from "node:os";
|
import { homedir } from "node:os";
|
||||||
import { join } from "node:path";
|
import { join } from "node:path";
|
||||||
import { Type } from "typebox";
|
import { Type } from "typebox";
|
||||||
@@ -37,6 +37,17 @@ const DEFAULT_PROMPT = [
|
|||||||
"主模型看不到图,完全依赖你的转录,文字务必穷尽。",
|
"主模型看不到图,完全依赖你的转录,文字务必穷尽。",
|
||||||
].join("\n");
|
].join("\n");
|
||||||
|
|
||||||
|
/** Hook transcription prompt: the automatic Web-UI-image pipeline. Kept short
|
||||||
|
* on purpose — each run is ~15s of Ollama time and the text is injected into
|
||||||
|
* the main model's context every turn, so verbosity costs both latency and
|
||||||
|
* tokens. The `vision` tool keeps the detailed DEFAULT_PROMPT above. */
|
||||||
|
const HOOK_PROMPT = [
|
||||||
|
"用中文简要描述这张图片(主模型依赖此转录理解图片,需准确但精简):",
|
||||||
|
"1. 所有可见文字:按阅读顺序转录(含标签、数字、按钮、代码),无文字则写“无”。",
|
||||||
|
"2. 图片内容:主体、场景、布局,2-3 句。",
|
||||||
|
"总长约 100 字,不要分节模板,直接输出。",
|
||||||
|
].join("\n");
|
||||||
|
|
||||||
const visionParams = Type.Object({
|
const visionParams = Type.Object({
|
||||||
image_paths: Type.Array(
|
image_paths: Type.Array(
|
||||||
Type.String({
|
Type.String({
|
||||||
@@ -130,6 +141,11 @@ async function describeWithOllama(
|
|||||||
// Ollama defaults to a small num_ctx (4096 on this setup); multi-image
|
// Ollama defaults to a small num_ctx (4096 on this setup); multi-image
|
||||||
// requests blow past it. Raise explicitly — the model supports 262k.
|
// requests blow past it. Raise explicitly — the model supports 262k.
|
||||||
const numCtx = parseInt(envOr("OLLAMA_NUM_CTX", "16384"), 10) || 16384;
|
const numCtx = parseInt(envOr("OLLAMA_NUM_CTX", "16384"), 10) || 16384;
|
||||||
|
// Keep the vision model resident in VRAM: system OLLAMA_KEEP_ALIVE is 30s on
|
||||||
|
// this machine, so every transcribe would cold-load 7.7GB otherwise. -1 =
|
||||||
|
// stay loaded until memory pressure evicts it (numeric -1, not "-1").
|
||||||
|
const keepAliveRaw = envOr("OLLAMA_VISION_KEEP_ALIVE", "-1");
|
||||||
|
const keepAlive: number | string = keepAliveRaw === "-1" ? -1 : keepAliveRaw;
|
||||||
const body = {
|
const body = {
|
||||||
model,
|
model,
|
||||||
messages: [
|
messages: [
|
||||||
@@ -141,6 +157,7 @@ async function describeWithOllama(
|
|||||||
],
|
],
|
||||||
stream: false,
|
stream: false,
|
||||||
think: false,
|
think: false,
|
||||||
|
keep_alive: keepAlive,
|
||||||
options: { temperature: 0, num_ctx: numCtx },
|
options: { temperature: 0, num_ctx: numCtx },
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -358,10 +375,46 @@ async function describeImages(
|
|||||||
}
|
}
|
||||||
|
|
||||||
/**
|
/**
|
||||||
* Session-scoped description cache: compaction, session restore, or repeated
|
* Description cache, persisted to disk so restarts don't force re-transcribing
|
||||||
* turns replay the same image parts; avoid re-running the vision model each time.
|
* the whole conversation's history. Each entry is ~300B; capped at 64.
|
||||||
*/
|
*/
|
||||||
const IMAGE_DESCRIPTION_CACHE = new Map<string, string>();
|
const IMAGE_DESCRIPTION_CACHE = new Map<string, string>();
|
||||||
|
const CACHE_MAX_ENTRIES = 64;
|
||||||
|
let cacheLoaded = false;
|
||||||
|
|
||||||
|
function visionCachePath(): string {
|
||||||
|
const agentDir = process.env.PI_CODING_AGENT_DIR?.trim();
|
||||||
|
return (
|
||||||
|
(agentDir && agentDir.length > 0
|
||||||
|
? agentDir
|
||||||
|
: join(homedir(), ".pi", "agent")) + "/vision-cache.json"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function loadDescriptionCache(): Promise<void> {
|
||||||
|
if (cacheLoaded) return;
|
||||||
|
cacheLoaded = true;
|
||||||
|
try {
|
||||||
|
const parsed = JSON.parse(
|
||||||
|
await readFile(visionCachePath(), "utf8"),
|
||||||
|
) as Record<string, string>;
|
||||||
|
for (const [key, value] of Object.entries(parsed)) {
|
||||||
|
if (typeof value === "string") IMAGE_DESCRIPTION_CACHE.set(key, value);
|
||||||
|
}
|
||||||
|
} catch {
|
||||||
|
/* no cache file yet, or corrupt — start empty */
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function persistDescriptionCache(): void {
|
||||||
|
void writeFile(
|
||||||
|
visionCachePath(),
|
||||||
|
JSON.stringify(Object.fromEntries(IMAGE_DESCRIPTION_CACHE)),
|
||||||
|
"utf8",
|
||||||
|
).catch(() => {
|
||||||
|
/* disk cache is best-effort; failures must never break transcription */
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
function dataUrlToLoadedImage(url: string): LoadedImage | null {
|
function dataUrlToLoadedImage(url: string): LoadedImage | null {
|
||||||
const match = /^data:(image\/[a-z0-9.+-]+);base64,(.+)$/i.exec(url);
|
const match = /^data:(image\/[a-z0-9.+-]+);base64,(.+)$/i.exec(url);
|
||||||
@@ -388,14 +441,16 @@ async function getImageDescription(
|
|||||||
signal: AbortSignal | undefined,
|
signal: AbortSignal | undefined,
|
||||||
): Promise<string> {
|
): Promise<string> {
|
||||||
const key = imageCacheKey(image);
|
const key = imageCacheKey(image);
|
||||||
|
await loadDescriptionCache();
|
||||||
const cached = IMAGE_DESCRIPTION_CACHE.get(key);
|
const cached = IMAGE_DESCRIPTION_CACHE.get(key);
|
||||||
if (cached) return cached;
|
if (cached) return cached;
|
||||||
const description = await describeImages([image], { prompt, signal });
|
const description = await describeImages([image], { prompt, signal });
|
||||||
IMAGE_DESCRIPTION_CACHE.set(key, description);
|
IMAGE_DESCRIPTION_CACHE.set(key, description);
|
||||||
if (IMAGE_DESCRIPTION_CACHE.size > 64) {
|
if (IMAGE_DESCRIPTION_CACHE.size > CACHE_MAX_ENTRIES) {
|
||||||
const oldest = IMAGE_DESCRIPTION_CACHE.keys().next().value;
|
const oldest = IMAGE_DESCRIPTION_CACHE.keys().next().value;
|
||||||
if (oldest !== undefined) IMAGE_DESCRIPTION_CACHE.delete(oldest);
|
if (oldest !== undefined) IMAGE_DESCRIPTION_CACHE.delete(oldest);
|
||||||
}
|
}
|
||||||
|
persistDescriptionCache();
|
||||||
return description;
|
return description;
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -497,7 +552,7 @@ export default function visionExtension(pi: ExtensionAPI) {
|
|||||||
try {
|
try {
|
||||||
const desc = await getImageDescription(
|
const desc = await getImageDescription(
|
||||||
images[i],
|
images[i],
|
||||||
DEFAULT_PROMPT,
|
HOOK_PROMPT,
|
||||||
ctx.signal,
|
ctx.signal,
|
||||||
);
|
);
|
||||||
transcribed.push(`【图片${i + 1}】\n${desc}`);
|
transcribed.push(`【图片${i + 1}】\n${desc}`);
|
||||||
|
|||||||
@@ -0,0 +1,299 @@
|
|||||||
|
#!/usr/bin/env node
|
||||||
|
/**
|
||||||
|
* pi-agent-integrated 缓存/会话清理脚本
|
||||||
|
*
|
||||||
|
* 背景:会话 jsonl 内嵌上传图片的 base64(单张相机照片可达 5.7MB),
|
||||||
|
* 每次 WebUI 加载会话要整体读出+序列化+前端解码渲染 → 加载变慢;
|
||||||
|
* opengrep/npm 缓存占磁盘数 GB。
|
||||||
|
*
|
||||||
|
* 用法:node scripts/cleanup-cache.mjs [--apply] [--target all|sessions|strip-images|npm|opengrep|artifacts] [--days N] [--keep N] [--purge] [--yes]
|
||||||
|
*
|
||||||
|
* --target all (默认) | sessions(移走旧会话) | strip-images(旧会话图片替换为占位,保留对话文本) |
|
||||||
|
* npm(清 cache/npm) | opengrep(清 AppData/Local/opengrep) | artifacts(清旧子代理产物)
|
||||||
|
* --days N 会话保留 N 天内的 (默认 7);配合 --target sessions/strip-images
|
||||||
|
* --keep N 额外保留最近 N 个会话文件 (默认 0)
|
||||||
|
* --apply 真正执行;缺省为 dry-run 只预览
|
||||||
|
* --purge 同时删除回收目录中 30 天前的备份(默认只清理,不删回收站)
|
||||||
|
* --yes 跳过确认(--apply 时)
|
||||||
|
*
|
||||||
|
* 安全边界:永不触碰 agent/npm(扩展安装)、.pi-lens/tools、.pi-lens/bin、memory(记忆仓库)。
|
||||||
|
* 所有删除 = 先移入回收目录 data/agent/tmp/session-trash/<时间戳>/,--purge 才清旧备份。
|
||||||
|
*/
|
||||||
|
import fs from "node:fs";
|
||||||
|
import path from "node:path";
|
||||||
|
|
||||||
|
const ROOT = path.resolve(import.meta.dirname, "..");
|
||||||
|
const DATA = path.join(ROOT, "data");
|
||||||
|
const SESSIONS_DIR = path.join(DATA, "agent", "sessions");
|
||||||
|
const TRASH_ROOT = path.join(DATA, "agent", "tmp", "session-trash");
|
||||||
|
const NPM_CACHE = path.join(DATA, "cache", "npm");
|
||||||
|
const OPENGREP = path.join(DATA, "home", "AppData", "Local", "opengrep");
|
||||||
|
const CUTOFF_DAYS = () => Date.now() - opt.days * 86400_000;
|
||||||
|
|
||||||
|
const args = process.argv.slice(2);
|
||||||
|
const opt = {
|
||||||
|
apply: args.includes("--apply"),
|
||||||
|
purge: args.includes("--purge"),
|
||||||
|
yes: args.includes("--yes"),
|
||||||
|
days: 7,
|
||||||
|
keep: 0,
|
||||||
|
target: "all",
|
||||||
|
};
|
||||||
|
for (let i = 0; i < args.length; i++) {
|
||||||
|
if (args[i] === "--days") opt.days = parseInt(args[++i], 10);
|
||||||
|
if (args[i] === "--keep") opt.keep = parseInt(args[++i], 10);
|
||||||
|
if (args[i] === "--target") opt.target = args[++i];
|
||||||
|
}
|
||||||
|
if (isNaN(opt.days) || opt.days < 0) opt.days = 7;
|
||||||
|
const TARGETS = ["all", "sessions", "strip-images", "npm", "opengrep", "artifacts"];
|
||||||
|
if (!TARGETS.includes(opt.target)) {
|
||||||
|
console.error(`未知 --target: ${opt.target}(可选 ${TARGETS.join(" / ")})`);
|
||||||
|
process.exit(1);
|
||||||
|
}
|
||||||
|
const want = (t) => opt.target === "all" || opt.target === t;
|
||||||
|
|
||||||
|
const mode = opt.apply ? "执行" : "预览(dry-run)";
|
||||||
|
console.log(`[cleanup-cache] ${mode} | target=${opt.target} | 会话保留 ${opt.days} 天${opt.keep ? ` + 最近 ${opt.keep} 个` : ""}\n`);
|
||||||
|
|
||||||
|
let freed = 0;
|
||||||
|
const actions = [];
|
||||||
|
const note = (s) => console.log(" " + s);
|
||||||
|
const planned = (file, size, why) => {
|
||||||
|
freed += size;
|
||||||
|
actions.push({ file, why });
|
||||||
|
note(`- ${why}: ${file} (${fmtSize(size)})`);
|
||||||
|
};
|
||||||
|
function fmtSize(n) {
|
||||||
|
if (n >= 1 << 30) return (n / (1 << 30)).toFixed(2) + " GB";
|
||||||
|
if (n >= 1 << 20) return (n / (1 << 20)).toFixed(1) + " MB";
|
||||||
|
if (n >= 1 << 10) return (n / (1 << 10)).toFixed(0) + " KB";
|
||||||
|
return n + " B";
|
||||||
|
}
|
||||||
|
function dirSize(p) {
|
||||||
|
if (!fs.existsSync(p)) return 0;
|
||||||
|
let total = 0;
|
||||||
|
const st = fs.statSync(p);
|
||||||
|
if (st.isFile()) return st.size;
|
||||||
|
for (const e of fs.readdirSync(p, { withFileTypes: true })) {
|
||||||
|
total += dirSize(path.join(p, e.name));
|
||||||
|
}
|
||||||
|
return total;
|
||||||
|
}
|
||||||
|
function moveToTrash(p) {
|
||||||
|
const stamp = new Date().toISOString().replace(/[:.]/g, "-");
|
||||||
|
const dest = path.join(TRASH_ROOT, stamp, path.basename(p));
|
||||||
|
fs.mkdirSync(path.dirname(dest), { recursive: true });
|
||||||
|
fs.renameSync(p, dest);
|
||||||
|
return dest;
|
||||||
|
}
|
||||||
|
function countImages(file) {
|
||||||
|
let n = 0;
|
||||||
|
let bytes = 0;
|
||||||
|
try {
|
||||||
|
const lines = fs.readFileSync(file, "utf8").split("\n");
|
||||||
|
for (const line of lines) {
|
||||||
|
if (!line || !line.includes('"image"')) continue;
|
||||||
|
let e;
|
||||||
|
try { e = JSON.parse(line); } catch { continue; }
|
||||||
|
if (e?.type !== "message" || !Array.isArray(e.message?.content)) continue;
|
||||||
|
for (const b of e.message.content) {
|
||||||
|
if (b?.type !== "image") continue;
|
||||||
|
const data = typeof b.data === "string" ? b.data
|
||||||
|
: b.source?.type === "base64" ? b.source.data : null;
|
||||||
|
n++;
|
||||||
|
if (data) {
|
||||||
|
const pad = data.endsWith("==") ? 2 : data.endsWith("=") ? 1 : 0;
|
||||||
|
bytes += Math.floor((data.length * 3) / 4) - pad;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
} catch { /* skip */ }
|
||||||
|
return { n, bytes };
|
||||||
|
}
|
||||||
|
function stripImagesInPlace(file) {
|
||||||
|
const lines = fs.readFileSync(file, "utf8").split("\n");
|
||||||
|
let stripped = 0;
|
||||||
|
let bytes = 0;
|
||||||
|
const out = [];
|
||||||
|
for (const line of lines) {
|
||||||
|
if (!line || !line.includes('"image"')) { out.push(line); continue; }
|
||||||
|
try {
|
||||||
|
const e = JSON.parse(line);
|
||||||
|
if (e?.type === "message" && Array.isArray(e.message?.content)
|
||||||
|
&& (e.message.role === "user" || e.message.role === "toolResult")) {
|
||||||
|
const content = [];
|
||||||
|
for (const b of e.message.content) {
|
||||||
|
if (b?.type === "image") {
|
||||||
|
const data = typeof b.data === "string" ? b.data
|
||||||
|
: b.source?.type === "base64" ? b.source.data : null;
|
||||||
|
const pad = data?.endsWith("==") ? 2 : data?.endsWith("=") ? 1 : 0;
|
||||||
|
bytes += data ? Math.floor((data.length * 3) / 4) - pad : 0;
|
||||||
|
stripped++;
|
||||||
|
content.push({ type: "text", text: "[图片已由 cleanup-cache 替换为占位,原始数据已移入回收目录]" });
|
||||||
|
} else content.push(b);
|
||||||
|
}
|
||||||
|
e.message.content = content;
|
||||||
|
out.push(JSON.stringify(e));
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
} catch { /* keep as-is */ }
|
||||||
|
out.push(line);
|
||||||
|
}
|
||||||
|
if (stripped > 0) {
|
||||||
|
fs.writeFileSync(file, out.join("\n"));
|
||||||
|
}
|
||||||
|
return { stripped, bytes };
|
||||||
|
}
|
||||||
|
function listSessionFiles() {
|
||||||
|
const files = [];
|
||||||
|
const walk = (dir) => {
|
||||||
|
if (!fs.existsSync(dir)) return;
|
||||||
|
for (const e of fs.readdirSync(dir, { withFileTypes: true })) {
|
||||||
|
if (e.name === "subagent-artifacts") continue;
|
||||||
|
const p = path.join(dir, e.name);
|
||||||
|
if (e.isDirectory()) walk(p);
|
||||||
|
else if (e.name.endsWith(".jsonl")) files.push(p);
|
||||||
|
}
|
||||||
|
};
|
||||||
|
walk(SESSIONS_DIR);
|
||||||
|
return files;
|
||||||
|
}
|
||||||
|
function sessionInfo(file) {
|
||||||
|
const st = fs.statSync(file);
|
||||||
|
const { n, bytes } = countImages(file);
|
||||||
|
return { file, size: st.size, mtime: st.mtimeMs, images: n, imgBytes: bytes };
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------- 1. sessions / strip-images ----------
|
||||||
|
if (want("sessions") || want("strip-images")) {
|
||||||
|
console.log(`[会话] 目录 ${SESSIONS_DIR}`);
|
||||||
|
const infos = listSessionFiles().map(sessionInfo).sort((a, b) => b.mtime - a.mtime);
|
||||||
|
if (infos.length === 0) { console.log(" (无会话文件)"); }
|
||||||
|
const cutoff = CUTOFF_DAYS();
|
||||||
|
const keepIdx = Math.min(opt.keep, infos.length);
|
||||||
|
const doomed = infos.filter((s, i) => s.mtime < cutoff && i >= keepIdx);
|
||||||
|
if (want("sessions")) {
|
||||||
|
for (const s of doomed) {
|
||||||
|
planned(s.file, s.size, s.images > 0 ? `旧会话(含 ${s.images} 图)` : "旧会话");
|
||||||
|
}
|
||||||
|
if (doomed.length === 0) console.log(` (最近 ${opt.days} 天内无会话需要移走)`);
|
||||||
|
}
|
||||||
|
if (want("strip-images")) {
|
||||||
|
for (const s of doomed) {
|
||||||
|
if (s.images === 0) continue;
|
||||||
|
note(`- 图片瘦身: ${path.basename(s.file)} (${s.images} 图, ~${fmtSize(s.imgBytes)} 将替换为占位)`);
|
||||||
|
freed += s.imgBytes;
|
||||||
|
}
|
||||||
|
const recent = infos.filter((s, i) => !(s.mtime < cutoff && i >= keepIdx));
|
||||||
|
for (const s of recent) {
|
||||||
|
if (s.images === 0) continue;
|
||||||
|
note(`- 图片瘦身(最近窗口, 可选): ${path.basename(s.file)} (${s.images} 图, ~${fmtSize(s.imgBytes)})`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
console.log();
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------- 2. npm cache ----------
|
||||||
|
if (want("npm")) {
|
||||||
|
const sz = dirSize(NPM_CACHE);
|
||||||
|
if (sz > 0) planned(NPM_CACHE, sz, "npm 缓存(_cacache/_npx/_logs)");
|
||||||
|
else console.log("[npm] 缓存已空");
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------- 3. opengrep ----------
|
||||||
|
if (want("opengrep")) {
|
||||||
|
const sz = dirSize(OPENGREP);
|
||||||
|
if (sz > 0) planned(OPENGREP, sz, "opengrep 语义扫描缓存");
|
||||||
|
else console.log("[opengrep] 缓存已空");
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------- 4. artifacts ----------
|
||||||
|
if (want("artifacts")) {
|
||||||
|
const dirs = [];
|
||||||
|
const findArtifactDirs = (dir) => {
|
||||||
|
if (!fs.existsSync(dir)) return;
|
||||||
|
for (const e of fs.readdirSync(dir, { withFileTypes: true })) {
|
||||||
|
const p = path.join(dir, e.name);
|
||||||
|
if (e.isDirectory()) {
|
||||||
|
if (e.name === "subagent-artifacts") dirs.push(p);
|
||||||
|
else findArtifactDirs(p);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
findArtifactDirs(SESSIONS_DIR);
|
||||||
|
let cnt = 0;
|
||||||
|
let sz = 0;
|
||||||
|
const files = [];
|
||||||
|
for (const d of dirs) {
|
||||||
|
for (const e of fs.readdirSync(d, { withFileTypes: true })) {
|
||||||
|
const p = path.join(d, e.name);
|
||||||
|
const st = fs.statSync(p);
|
||||||
|
if (st.mtimeMs < CUTOFF_DAYS()) { cnt++; sz += st.size; files.push(p); }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (cnt > 0) { note(`- ${cnt} 个旧子代理产物文件 (~${fmtSize(sz)})`); for (const f of files) planned(f, fs.statSync(f).size, "旧子代理产物"); }
|
||||||
|
else console.log("[artifacts] 无旧产物");
|
||||||
|
}
|
||||||
|
|
||||||
|
// ---------- 汇总 ----------
|
||||||
|
console.log(`\n合计可释放: ${fmtSize(freed)}`);
|
||||||
|
if (!opt.apply) {
|
||||||
|
console.log("\n以上为预览。加 --apply 真正执行(删除前先移入回收目录)。");
|
||||||
|
process.exit(0);
|
||||||
|
}
|
||||||
|
if (!opt.yes) {
|
||||||
|
console.log(`\n确认执行 ${actions.length} 项清理?输入 yes 继续:`);
|
||||||
|
const readline = (await import("node:readline")).default;
|
||||||
|
const rl = readline.createInterface({ input: process.stdin, output: process.stdout });
|
||||||
|
const ans = await new Promise((r) => rl.question("> ", r));
|
||||||
|
rl.close();
|
||||||
|
if (ans.trim().toLowerCase() !== "yes") { console.log("已取消。"); process.exit(1); }
|
||||||
|
}
|
||||||
|
|
||||||
|
for (const a of actions) {
|
||||||
|
try {
|
||||||
|
if (a.file.includes("opengrep") || a.file === NPM_CACHE) {
|
||||||
|
fs.rmSync(a.file, { recursive: true, force: true });
|
||||||
|
console.log(`已删除 ${a.file}`);
|
||||||
|
} else {
|
||||||
|
const dest = moveToTrash(a.file);
|
||||||
|
console.log(`已移入回收目录 ${dest}`);
|
||||||
|
}
|
||||||
|
} catch (err) {
|
||||||
|
console.error(`失败: ${a.file}: ${err.message}`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (want("strip-images")) {
|
||||||
|
const infos = listSessionFiles().map(sessionInfo).sort((a, b) => b.mtime - a.mtime);
|
||||||
|
const keepIdx = Math.min(opt.keep, infos.length);
|
||||||
|
const cutoff = CUTOFF_DAYS();
|
||||||
|
for (const s of infos) {
|
||||||
|
if (s.images === 0) continue;
|
||||||
|
const i = infos.indexOf(s);
|
||||||
|
if (!(s.mtime < cutoff && i >= keepIdx)) continue;
|
||||||
|
const backup = moveToTrash(s.file);
|
||||||
|
const { stripped, bytes } = stripImagesInPlace(s.file);
|
||||||
|
console.log(`图片瘦身完成: ${path.basename(s.file)} (${stripped} 图, ~${fmtSize(bytes)}, 原备份 ${backup})`);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if (opt.purge) {
|
||||||
|
const purged = [];
|
||||||
|
const old = Date.now() - 30 * 86400_000;
|
||||||
|
const walk = (dir) => {
|
||||||
|
if (!fs.existsSync(dir)) return;
|
||||||
|
for (const e of fs.readdirSync(dir, { withFileTypes: true })) {
|
||||||
|
const p = path.join(dir, e.name);
|
||||||
|
if (e.isDirectory()) {
|
||||||
|
const st = fs.statSync(p);
|
||||||
|
if (st.mtimeMs < old) { fs.rmSync(p, { recursive: true, force: true }); purged.push(p); }
|
||||||
|
else walk(p);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
};
|
||||||
|
walk(TRASH_ROOT);
|
||||||
|
console.log(purged.length ? `回收目录已清旧备份: ${purged.join(", ")}` : "回收目录无 30 天前备份");
|
||||||
|
}
|
||||||
|
|
||||||
|
console.log(`\n完成。释放 ${fmtSize(freed)}。注意:当前正在使用的会话若被移走会丢失历史——建议先退出 pi 再执行。`);
|
||||||
Reference in New Issue
Block a user