gitadmin/pi-agent-integrated
1
0
Fork 0
mirror of https://github.com/luckyyzh/pi-agent-integrated.git synced 2026-10-03 02:59:35 +00:00
Code Issues Packages Projects Releases Wiki Activity

fix: vendor repaired smart fetch package

Browse Source
This commit is contained in:
luckyyzh
2026-08-05 03:41:42 +08:00
parent 836042b97c
commit 19ae25040d
10 changed files with 13156 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+11 -5
View File
@@ -67,6 +67,8 @@ npm run dev
5. Windows 预缓存 Playwright MCP 包并确认系统 Edge 可用；macOS 跳过 Playwright 安装，浏览器自动化按需配置； 5. Windows 预缓存 Playwright MCP 包并确认系统 Edge 可用；macOS 跳过 Playwright 安装，浏览器自动化按需配置；
6. 检查前后端版本和构建产物。 6. 检查前后端版本和构建产物。
`pi-smart-fetch` 使用仓库内的修复版，不再直接加载上游 `0.3.17` 发布包。setup 会在 `resources/packages/pi-smart-fetch/` 安装运行时依赖，并自动迁移旧的 `npm:pi-smart-fetch` 配置；该修复为 ESM 构建注入 `createRequire(import.meta.url)`，避免 Node/Next.js 加载 `mime-types` 时触发 `Dynamic require of "path" is not supported`。
安装脚本默认不继承 `HTTP_PROXY` / `HTTPS_PROXY`，避免失效代理阻塞 npm。如果安装必须使用代理，Windows PowerShell 设置 `$env:PI_SETUP_USE_PROXY = "1"`，macOS/Linux 使用 `PI_SETUP_USE_PROXY=1 npm run setup`。 安装脚本默认不继承 `HTTP_PROXY` / `HTTPS_PROXY`，避免失效代理阻塞 npm。如果安装必须使用代理，Windows PowerShell 设置 `$env:PI_SETUP_USE_PROXY = "1"`，macOS/Linux 使用 `PI_SETUP_USE_PROXY=1 npm run setup`。
### 首次模型配置 ### 首次模型配置
@@ -122,7 +124,8 @@ pi-web/ Next.js Web 前端与 HTTP/SSE 服务
config/ 可提交的缺省设置、MCP 和子代理策略 config/ 可提交的缺省设置、MCP 和子代理策略
resources/skills/ 应用级 Skill resources/skills/ 应用级 Skill
resources/extensions/ 应用级 Pi 扩展 resources/extensions/ 应用级 Pi 扩展
resources/prompts/ 提示词模板 resources/packages/ 项目内置包（含 pi-smart-fetch 修复版）
resources/prompts/ 提示词模板
resources/themes/ 主题 resources/themes/ 主题
scripts/ 安装、启动、迁移与验证脚本 scripts/ 安装、启动、迁移与验证脚本
data/agent/ 会话、认证、模型、插件、记忆与工具状态 data/agent/ 会话、认证、模型、插件、记忆与工具状态
@@ -144,7 +147,7 @@ data/workspaces/default/ 默认工作目录
| `pi-lens@3.8.73` | LSP、AST、符号检索和项目诊断 | 模型按需激活代码智能工具；可用 `/lens-health`、`/lens-tools`、`/lens-map` 检查 | | `pi-lens@3.8.73` | LSP、AST、符号检索和项目诊断 | 模型按需激活代码智能工具；可用 `/lens-health`、`/lens-tools`、`/lens-map` 检查 |
| `pi-memory@0.4.0` | Markdown 长期记忆、日志、临时工作区和恢复记录 | 默认轻量模式保留读写、状态和恢复工具，但不注册依赖 qmd 的 `memory_search`；文件位于 `data/agent/memory/` | | `pi-memory@0.4.0` | Markdown 长期记忆、日志、临时工作区和恢复记录 | 默认轻量模式保留读写、状态和恢复工具，但不注册依赖 qmd 的 `memory_search`；文件位于 `data/agent/memory/` |
| `pi-subagents@0.37.2` | 创建研究、规划或执行子代理 | 简单任务不委派，跨模块或可并行复杂任务自动判断；也可使用 `/run`、`/parallel` | | `pi-subagents@0.37.2` | 创建研究、规划或执行子代理 | 简单任务不委派，跨模块或可并行复杂任务自动判断；也可使用 `/run`、`/parallel` |
| `pi-smart-fetch@0.3.17` | 抓取单个或批量 URL 内容 | 模型按需使用 `web_fetch` / `batch_web_fetch`，也提供给研究子代理 | | `pi-smart-fetch`（项目内修复版，基于 `0.3.17`） | 抓取单个或批量 URL 内容 | 模型按需使用 `web_fetch` / `batch_web_fetch`，也提供给研究子代理；修复 Node/Next.js ESM 加载兼容性 |
| `@ayulab/pi-rewind@0.4.6` | 每轮前后创建代码检查点 | `/rewind` 恢复代码、会话或两者；`/checkpoint` 管理存储。自动恢复文件默认关闭 | | `@ayulab/pi-rewind@0.4.6` | 每轮前后创建代码检查点 | `/rewind` 恢复代码、会话或两者；`/checkpoint` 管理存储。自动恢复文件默认关闭 |
| `@upstash/context7-pi@0.1.2` | 查询当前库、框架、SDK 和 API 文档 | 模型先解析库 ID，再按需查询文档；无 Key 可使用公共限额 | | `@upstash/context7-pi@0.1.2` | 查询当前库、框架、SDK 和 API 文档 | 模型先解析库 ID，再按需查询文档；无 Key 可使用公共限额 |
| `@narumitw/pi-retry@0.31.0` | 识别瞬时供应商错误和卡住的流 | 复用 Pi 内置重试；默认 180 秒无事件视为停滞，不增加正常请求的模型调用 | | `@narumitw/pi-retry@0.31.0` | 识别瞬时供应商错误和卡住的流 | 复用 Pi 内置重试；默认 180 秒无事件视为停滞，不增加正常请求的模型调用 |
@@ -319,7 +322,9 @@ npm run dev
Open <http://127.0.0.1:30141>. Open <http://127.0.0.1:30141>.
Setup initializes the repository-local profile, builds Pi, links Pi Web to the local packages, installs and loads the pinned plugins, caches the Playwright MCP Node package and verifies system Edge on Windows, skips Playwright on macOS, and checks integration artifacts. Setup initializes the repository-local profile, builds Pi, links Pi Web to the local packages, installs and loads the pinned plugins, installs the repository-local repaired `pi-smart-fetch` dependencies, caches the Playwright MCP Node package and verifies system Edge on Windows, skips Playwright on macOS, and checks integration artifacts.
The repaired `pi-smart-fetch` build is based on upstream `0.3.17`; it adds `createRequire(import.meta.url)` for the bundled CommonJS dependencies so Node/Next.js does not fail with `Dynamic require of "path" is not supported`. Existing managed profiles are migrated from `npm:pi-smart-fetch` to `resources/packages/pi-smart-fetch` automatically.
Child npm operations ignore `HTTP_PROXY` and `HTTPS_PROXY` by default to avoid stale proxy configuration. Set `$env:PI_SETUP_USE_PROXY = "1"` on Windows, or run `PI_SETUP_USE_PROXY=1 npm run setup` on macOS/Linux, when registry access requires the proxy. Child npm operations ignore `HTTP_PROXY` and `HTTPS_PROXY` by default to avoid stale proxy configuration. Set `$env:PI_SETUP_USE_PROXY = "1"` on Windows, or run `PI_SETUP_USE_PROXY=1 npm run setup` on macOS/Linux, when registry access requires the proxy.
@@ -374,7 +379,8 @@ pi-web/ Next.js frontend and HTTP/SSE service
config/ Versioned settings, MCP, and subagent policy config/ Versioned settings, MCP, and subagent policy
resources/skills/ Application skills resources/skills/ Application skills
resources/extensions/ Application Pi extensions resources/extensions/ Application Pi extensions
resources/prompts/ Prompt templates resources/packages/ Repository-local packages (including repaired pi-smart-fetch)
resources/prompts/ Prompt templates
resources/themes/ Themes resources/themes/ Themes
scripts/ Setup, launch, migration, and verification scripts/ Setup, launch, migration, and verification
data/agent/ Sessions, auth, models, packages, memory, tools data/agent/ Sessions, auth, models, packages, memory, tools
@@ -396,7 +402,7 @@ Versions are pinned in the platform defaults under `config/`: Windows uses `mcp.
| `pi-lens@3.8.73` | LSP, AST, symbols, project diagnostics | The model activates code intelligence on demand; inspect with `/lens-health`, `/lens-tools`, `/lens-map` | | `pi-lens@3.8.73` | LSP, AST, symbols, project diagnostics | The model activates code intelligence on demand; inspect with `/lens-health`, `/lens-tools`, `/lens-map` |
| `pi-memory@0.4.0` | Markdown durable facts, logs, scratchpad, recovery | Lightweight mode retains read/write, status, and recovery tools but does not register qmd-dependent `memory_search`; files live under `data/agent/memory/` | | `pi-memory@0.4.0` | Markdown durable facts, logs, scratchpad, recovery | Lightweight mode retains read/write, status, and recovery tools but does not register qmd-dependent `memory_search`; files live under `data/agent/memory/` |
| `pi-subagents@0.37.2` | Research, planning, execution subagents | Simple tasks stay local; complex or parallel work may delegate automatically; `/run` and `/parallel` remain available | | `pi-subagents@0.37.2` | Research, planning, execution subagents | Simple tasks stay local; complex or parallel work may delegate automatically; `/run` and `/parallel` remain available |
| `pi-smart-fetch@0.3.17` | Single and batched URL retrieval | Provides `web_fetch` and `batch_web_fetch` to the main and research agents | | `pi-smart-fetch` (project-local repair, based on `0.3.17`) | Single and batched URL retrieval | Provides `web_fetch` and `batch_web_fetch` to the main and research agents; fixes Node/Next.js ESM loading compatibility |
| `@ayulab/pi-rewind@0.4.6` | Per-turn code checkpoints | `/rewind` restores code, conversation, or both; `/checkpoint` manages storage. Automatic file restore is off | | `@ayulab/pi-rewind@0.4.6` | Per-turn code checkpoints | `/rewind` restores code, conversation, or both; `/checkpoint` manages storage. Automatic file restore is off |
| `@upstash/context7-pi@0.1.2` | Current library, framework, SDK, API docs | Resolves a library ID and queries docs when needed; public quota works without a key | | `@upstash/context7-pi@0.1.2` | Current library, framework, SDK, API docs | Resolves a library ID and queries docs when needed; public quota works without a key |
| `@narumitw/pi-retry@0.31.0` | Transient provider and stalled-stream classification | Uses Pi's built-in retry path; 180 seconds without events is a stall; no extra normal model calls | | `@narumitw/pi-retry@0.31.0` | Transient provider and stalled-stream classification | Uses Pi's built-in retry path; 180 seconds without events is a stall; no extra normal model calls |
config/settings.default.json
+1 -1
View File
@@ -3,8 +3,8 @@
"npm:pi-mcp-adapter@2.15.0", "npm:pi-mcp-adapter@2.15.0",
"npm:pi-lens@3.8.73", "npm:pi-lens@3.8.73",
"npm:pi-memory@0.4.0", "npm:pi-memory@0.4.0",
"../../resources/packages/pi-smart-fetch",
"npm:pi-subagents@0.37.2", "npm:pi-subagents@0.37.2",
"npm:pi-smart-fetch@0.3.17",
"npm:@ayulab/pi-rewind@0.4.6", "npm:@ayulab/pi-rewind@0.4.6",
"npm:@upstash/context7-pi@0.1.2", "npm:@upstash/context7-pi@0.1.2",
"npm:@narumitw/pi-retry@0.31.0" "npm:@narumitw/pi-retry@0.31.0"
resources/README.md
+5
View File
@@ -4,12 +4,17 @@ This directory is the version-controlled resource layer for the integrated app.
- `skills/` contains authored or reviewed skills. - `skills/` contains authored or reviewed skills.
- `extensions/` contains Pi extensions. - `extensions/` contains Pi extensions.
- `packages/` contains repository-local Pi packages, including the repaired `pi-smart-fetch` build.
- `prompts/` contains prompt templates. - `prompts/` contains prompt templates.
- `themes/` contains custom Pi themes. - `themes/` contains custom Pi themes.
The managed launcher loads these directories for every workspace. Project-local The managed launcher loads these directories for every workspace. Project-local
`.pi` and `.agents` resources from an opened workspace are not loaded. `.pi` and `.agents` resources from an opened workspace are not loaded.
The local `resources/packages/pi-smart-fetch` package is configured through
`config/settings.default.json`; setup installs its runtime dependencies and
migrates older `npm:pi-smart-fetch` profile entries automatically.
## Included extensions ## Included extensions
- `extensions/package.json` is the explicit Pi extension manifest. Add future - `extensions/package.json` is the explicit Pi extension manifest. Add future
resources/packages/pi-smart-fetch/.gitignore
+1
View File
@@ -0,0 +1 @@
node_modules/
resources/packages/pi-smart-fetch/LICENSE
+21
View File
@@ -0,0 +1,21 @@
MIT License
Copyright (c) 2026 Thinkscape
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
resources/packages/pi-smart-fetch/README.md
+114
View File
@@ -0,0 +1,114 @@
# pi-smart-fetch (project-local repair)
This vendored build is based on `pi-smart-fetch@0.3.17` and adds the ESM/CommonJS compatibility shim required by this project's Node/Next.js runtime. The bundled CommonJS dependencies receive a real `require` from `createRequire(import.meta.url)`, so loading the extension no longer fails with `Dynamic require of "path" is not supported`.
`pi-smart-fetch` adds smarter web fetching tools to pi.dev.
![pi Smart Fetch](https://raw.githubusercontent.com/Thinkscape/agent-smart-fetch/main/packages/pi-smart-fetch/demo.gif)
## Features
- 🔐 **Browser-like TLS/SSL + HTTP fingerprints** — better success on bot-defended pages
- 🧹 **Defuddle extraction** — clean readable content instead of noisy HTML
- 🧠 **Useful metadata** — title, author, site, language, published date when available
- 📦 **Downloads + large file support** — stream attachments and binaries to temp files
- 🔁 **Client-side `<meta>` redirects** — follows sane meta refresh redirects with loop limits
- 🔗 **Alternate content fallback** — when extraction produces no/thin content, follows qualified `<link rel="alternate" type="...">` entries in `<head>` that match the requested output format
- ⚡ **Batch fetch** — fetch many URLs with bounded concurrency
- 📝 **Multiple output formats** — `markdown`, `html`, `text`, `json`, `raw`
## Site optimisations
This package works on general web pages, but some site types benefit especially from Defuddle's extractors and cleanup:
- YouTube pages and transcripts
- Reddit posts and comment threads
- X / Twitter posts
- GitHub pages, issues, PRs, and discussions
- Hacker News threads
- Substack posts
- Pages with code blocks, footnotes, math, and callouts
Notes:
- Defuddle is the cleanup layer: it strips common page chrome like nav, sidebars, related links, share widgets, and footers
- It does **not** execute JavaScript or solve interactive anti-bot/login flows
- If an HTML shell advertises alternate content in `<head>`, smart-fetch can follow matching alternates such as `text/markdown`, `text/plain`, `text/html`, or JSON media types according to the requested `format`
## Install
This repository loads the repaired package automatically from
`resources/packages/pi-smart-fetch`. The setup script installs its runtime
dependencies into that package directory; `config/settings.default.json`
points Pi at the local package instead of the broken npm build.
The original upstream package can still be installed separately when needed:
```bash
pi install npm:pi-smart-fetch
```
## Pi tools
Registers:
- `web_fetch`
- `batch_web_fetch`
Synopsis:
```text
web_fetch(url, browser?, os?, headers?, maxChars?, timeoutMs?, format?, removeImages?, includeReplies?, proxy?, verbose?)
batch_web_fetch(requests, verbose?)
```
For `batch_web_fetch`, each item in `requests` accepts the same parameters as `web_fetch` except `verbose`.
## Output formats
| Format | What you get |
| --- | --- |
| `markdown` | Best default for readable page content |
| `html` | Cleaned HTML output |
| `text` | Plain text with markdown stripped |
| `json` | Structured JSON for metadata-heavy workflows |
| `raw` | Full raw server response without extraction or truncation — for further parsing |
## Global defaults
Optional settings in `~/.pi/agent/settings.json` or `.pi/settings.json`:
```json
{
"smartFetchVerboseByDefault": false,
"smartFetchDefaultMaxChars": 50000,
"smartFetchDefaultTimeoutMs": 15000,
"smartFetchDefaultBrowser": "chrome_145",
"smartFetchDefaultOs": "windows",
"smartFetchDefaultRemoveImages": false,
"smartFetchDefaultIncludeReplies": "extractors",
"smartFetchDefaultBatchConcurrency": 8,
"smartFetchTempDir": "/tmp/smart-fetch-pi"
}
```
| Setting | Default | Description |
| --- | ---: | --- |
| `smartFetchVerboseByDefault` | `false` | Stored default for the compatibility `verbose` flag |
| `smartFetchDefaultMaxChars` | `50000` | Default `maxChars` limit |
| `smartFetchDefaultTimeoutMs` | `15000` | Default request timeout in milliseconds |
| `smartFetchDefaultBrowser` | `chrome_145` | Default browser fingerprint profile |
| `smartFetchDefaultOs` | `windows` | Default OS fingerprint profile |
| `smartFetchDefaultRemoveImages` | `false` | Strip image references by default |
| `smartFetchDefaultIncludeReplies` | `extractors` | Include replies/comments only when site extractors support them |
| `smartFetchDefaultBatchConcurrency` | `8` | Default bounded concurrency for `batch_web_fetch` |
| `smartFetchTempDir` | OS temp dir | Base directory for attachment and binary downloads |
Notes:
- Project `.pi/settings.json` overrides global `~/.pi/agent/settings.json`
- Legacy `webFetch*` aliases are still supported
## Dev and publishing note
This repo uses Bun for local development, tests, and workspace scripts. Package publishing still goes through `npm publish` in CI so npm Trusted Publishing can be used.
resources/packages/pi-smart-fetch/dist/index.js
Vendored
+12412
View File
File diff suppressed because it is too large Load Diff
resources/packages/pi-smart-fetch/package-lock.json
Generated
+469
View File
@@ -0,0 +1,469 @@
{
"name": "pi-smart-fetch-project-fix",
"version": "0.3.17-project.1",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "pi-smart-fetch-project-fix",
"version": "0.3.17-project.1",
"license": "MIT",
"dependencies": {
"@earendil-works/pi-tui": "^0.82.1",
"@sinclair/typebox": "^0.34.52",
"defuddle": "^0.19.2",
"linkedom": "^0.18.13",
"mime-types": "^3.0.2",
"wreq-js": "^2.3.1"
},
"engines": {
"node": ">=22.19.0"
},
"peerDependencies": {
"@earendil-works/pi-coding-agent": "*"
}
},
"node_modules/@earendil-works/pi-tui": {
"version": "0.82.1",
"resolved": "https://registry.npmjs.org/@earendil-works/pi-tui/-/pi-tui-0.82.1.tgz",
"integrity": "sha512-9yN8hALfKaxZq7n54EMxqhFCWnMi6LHkraMJ/1YjHiATq75XrI6XDMVppn9EDtiK7Fks8hUe1SDXUTrIvwRWfQ==",
"license": "MIT",
"dependencies": {
"get-east-asian-width": "1.6.0",
"marked": "18.0.5"
},
"engines": {
"node": ">=22.19.0"
}
},
"node_modules/@mixmark-io/domino": {
"version": "2.2.0",
"resolved": "https://registry.npmjs.org/@mixmark-io/domino/-/domino-2.2.0.tgz",
"integrity": "sha512-Y28PR25bHXUg88kCV7nivXrP2Nj2RueZ3/l/jdx6J9f8J4nsEGcgX0Qe6lt7Pa+J79+kPiJU3LguR6O/6zrLOw==",
"license": "BSD-2-Clause",
"optional": true
},
"node_modules/@sinclair/typebox": {
"version": "0.34.52",
"resolved": "https://registry.npmjs.org/@sinclair/typebox/-/typebox-0.34.52.tgz",
"integrity": "sha512-XiMQh7qqVlxZzcVD+kkGMNGMzcTrDMLWI7S4x7z1MkCkbDPrekpZXEUK0eZqZFMuHQg2a2DZOcDIh9o5v3Gonw==",
"license": "MIT"
},
"node_modules/@xmldom/xmldom": {
"version": "0.9.10",
"resolved": "https://registry.npmjs.org/@xmldom/xmldom/-/xmldom-0.9.10.tgz",
"integrity": "sha512-A9gOqLdi6cV4ibazAjcQufGj0B1y/vDqYrcuP6d/6x8P27gRS8643Dj9o1dEKtB6O7fwxb2FgBmJS2mX7gpvdw==",
"license": "MIT",
"optional": true,
"engines": {
"node": ">=14.6"
}
},
"node_modules/boolbase": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/boolbase/-/boolbase-2.0.0.tgz",
"integrity": "sha512-DkVaaQHymRhpYEYo9x1oo7Q7B0Y6KJUsjm3c9eTyFDby4MHLBTwZ6ZDWBel5zrYxj1WsZgC5oLpiz+93MluXeA==",
"license": "ISC",
"engines": {
"node": ">=20.19.0"
},
"funding": {
"type": "github",
"url": "https://github.com/sponsors/fb55"
}
},
"node_modules/commander": {
"version": "12.1.0",
"resolved": "https://registry.npmjs.org/commander/-/commander-12.1.0.tgz",
"integrity": "sha512-Vw8qHK3bZM9y/P10u3Vib8o/DdkvA2OtPtZvD871QKjy74Wj1WSKFILMPRPSdUSx5RFK1arlJzEtA4PkFgnbuA==",
"license": "MIT",
"engines": {
"node": ">=18"
}
},
"node_modules/css-select": {
"version": "7.0.0",
"resolved": "https://registry.npmjs.org/css-select/-/css-select-7.0.0.tgz",
"integrity": "sha512-snmjEVXy+1LnwXdxhYvTMj1d9tOh4HxkA1YmoayVBeeyR2C14Pum7fcxJIm4SswYspVy866eYNwlH6xC3/VH5g==",
"license": "BSD-2-Clause",
"dependencies": {
"boolbase": "^2.0.0",
"css-what": "^8.0.0",
"domhandler": "^6.0.1",
"domutils": "^4.0.2",
"nth-check": "^3.0.1"
},
"engines": {
"node": ">=20.19.0"
},
"funding": {
"type": "github",
"url": "https://github.com/sponsors/fb55"
}
},
"node_modules/css-what": {
"version": "8.0.0",
"resolved": "https://registry.npmjs.org/css-what/-/css-what-8.0.0.tgz",
"integrity": "sha512-DH0Bqq3DNp5tdOReuNyAA+Ev4Y2GS5FMbZpeTLP6C4CDi0h5nL0BmUPChXw3o/qbHLDWHl49sbNqQVY7bMSDdw==",
"license": "BSD-2-Clause",
"engines": {
"node": ">=20.19.0"
},
"funding": {
"type": "github",
"url": "https://github.com/sponsors/fb55"
}
},
"node_modules/cssom": {
"version": "0.5.0",
"resolved": "https://registry.npmjs.org/cssom/-/cssom-0.5.0.tgz",
"integrity": "sha512-iKuQcq+NdHqlAcwUY0o/HL69XQrUaQdMjmStJ8JFmUaiiQErlhrmuigkg/CU4E2J0IyUKUrMAgl36TvN67MqTw==",
"license": "MIT"
},
"node_modules/defuddle": {
"version": "0.19.2",
"resolved": "https://registry.npmjs.org/defuddle/-/defuddle-0.19.2.tgz",
"integrity": "sha512-FY+a9Byo3uS04SQnQjLaguNoEA0Y58vXTB1XvvnOkO6M0zo897ipekv2Unci42QEY70emZ6OLe7C0vPojr1JEA==",
"license": "MIT",
"dependencies": {
"commander": "^12.1.0"
},
"bin": {
"defuddle": "dist/cli.js"
},
"optionalDependencies": {
"linkedom": "^0.18.12",
"mathml-to-latex": "^1.8.0",
"temml": "^0.13.3",
"turndown": "^7.2.0"
}
},
"node_modules/dom-serializer": {
"version": "3.1.1",
"resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-3.1.1.tgz",
"integrity": "sha512-4MEa38/QexBob6gFNwu+EGdWvhJ1OKuNwdYY3Y3NyeWDQfnGeDYQUDfIRzWu5B5gsv03so2Uxd28YC6zrsx3Lw==",
"license": "MIT",
"dependencies": {
"domelementtype": "^3.0.0",
"domhandler": "^6.0.0",
"entities": "^8.0.0"
},
"engines": {
"node": ">=20.19.0"
},
"funding": {
"type": "github",
"url": "https://github.com/cheeriojs/dom-serializer?sponsor=1"
}
},
"node_modules/domelementtype": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-3.0.0.tgz",
"integrity": "sha512-umCQid3jKbDmVjx8jGaW7uUykm4DEUeyV21hPxNMo2nV955DhUThwqyOIDtreepP31hl84X7G5U9ZfsWvIB3Pg==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/fb55"
}
],
"license": "BSD-2-Clause",
"engines": {
"node": ">=20.19.0"
}
},
"node_modules/domhandler": {
"version": "6.0.1",
"resolved": "https://registry.npmjs.org/domhandler/-/domhandler-6.0.1.tgz",
"integrity": "sha512-gYzvtM72ZtxQO0T048kd6HWSbbGCNOUwcnfQ01cqIJ4X2IYKFFHZ5mKvrQETcFXxsRObZulDaKmy//R7TPtsBg==",
"license": "BSD-2-Clause",
"dependencies": {
"domelementtype": "^3.0.0"
},
"engines": {
"node": ">=20.19.0"
},
"funding": {
"type": "github",
"url": "https://github.com/fb55/domhandler?sponsor=1"
}
},
"node_modules/domutils": {
"version": "4.0.2",
"resolved": "https://registry.npmjs.org/domutils/-/domutils-4.0.2.tgz",
"integrity": "sha512-qI4JLRKnSzqFqr7hAlS5xQDusBCjKSEG4t4+7aNrIQMHBcsC2TGEhuyABJdYkgSewL57PNLYEiibY2iPKhKpaA==",
"license": "BSD-2-Clause",
"dependencies": {
"dom-serializer": "^3.0.0",
"domelementtype": "^3.0.0",
"domhandler": "^6.0.0"
},
"engines": {
"node": ">=20.19.0"
},
"funding": {
"type": "github",
"url": "https://github.com/fb55/domutils?sponsor=1"
}
},
"node_modules/entities": {
"version": "8.0.0",
"resolved": "https://registry.npmjs.org/entities/-/entities-8.0.0.tgz",
"integrity": "sha512-zwfzJecQ/Uej6tusMqwAqU/6KL2XaB2VZ2Jg54Je6ahNBGNH6Ek6g3jjNCF0fG9EWQKGZNddNjU5F1ZQn/sBnA==",
"license": "BSD-2-Clause",
"engines": {
"node": ">=20.19.0"
},
"funding": {
"url": "https://github.com/fb55/entities?sponsor=1"
}
},
"node_modules/get-east-asian-width": {
"version": "1.6.0",
"resolved": "https://registry.npmjs.org/get-east-asian-width/-/get-east-asian-width-1.6.0.tgz",
"integrity": "sha512-QRbvDIbx6YklUe6RxeTeleMR0yv3cYH6PsPZHcnVn7xv7zO1BHN8r0XETu8n6Ye3Q+ahtSarc3WgtNWmehIBfA==",
"license": "MIT",
"engines": {
"node": ">=18"
},
"funding": {
"url": "https://github.com/sponsors/sindresorhus"
}
},
"node_modules/html-escaper": {
"version": "3.0.3",
"resolved": "https://registry.npmjs.org/html-escaper/-/html-escaper-3.0.3.tgz",
"integrity": "sha512-RuMffC89BOWQoY0WKGpIhn5gX3iI54O6nRA0yC124NYVtzjmFWBIiFd8M0x+ZdX0P9R4lADg1mgP8C7PxGOWuQ==",
"license": "MIT"
},
"node_modules/htmlparser2": {
"version": "10.1.0",
"resolved": "https://registry.npmjs.org/htmlparser2/-/htmlparser2-10.1.0.tgz",
"integrity": "sha512-VTZkM9GWRAtEpveh7MSF6SjjrpNVNNVJfFup7xTY3UpFtm67foy9HDVXneLtFVt4pMz5kZtgNcvCniNFb1hlEQ==",
"funding": [
"https://github.com/fb55/htmlparser2?sponsor=1",
{
"type": "github",
"url": "https://github.com/sponsors/fb55"
}
],
"license": "MIT",
"dependencies": {
"domelementtype": "^2.3.0",
"domhandler": "^5.0.3",
"domutils": "^3.2.2",
"entities": "^7.0.1"
}
},
"node_modules/htmlparser2/node_modules/dom-serializer": {
"version": "2.0.0",
"resolved": "https://registry.npmjs.org/dom-serializer/-/dom-serializer-2.0.0.tgz",
"integrity": "sha512-wIkAryiqt/nV5EQKqQpo3SToSOV9J0DnbJqwK7Wv/Trc92zIAYZ4FlMu+JPFW1DfGFt81ZTCGgDEabffXeLyJg==",
"license": "MIT",
"dependencies": {
"domelementtype": "^2.3.0",
"domhandler": "^5.0.2",
"entities": "^4.2.0"
},
"funding": {
"url": "https://github.com/cheeriojs/dom-serializer?sponsor=1"
}
},
"node_modules/htmlparser2/node_modules/dom-serializer/node_modules/entities": {
"version": "4.5.0",
"resolved": "https://registry.npmjs.org/entities/-/entities-4.5.0.tgz",
"integrity": "sha512-V0hjH4dGPh9Ao5p0MoRY6BVqtwCjhz6vI5LT8AJ55H+4g9/4vbHx1I54fS0XuclLhDHArPQCiMjDxjaL8fPxhw==",
"license": "BSD-2-Clause",
"engines": {
"node": ">=0.12"
},
"funding": {
"url": "https://github.com/fb55/entities?sponsor=1"
}
},
"node_modules/htmlparser2/node_modules/domelementtype": {
"version": "2.3.0",
"resolved": "https://registry.npmjs.org/domelementtype/-/domelementtype-2.3.0.tgz",
"integrity": "sha512-OLETBj6w0OsagBwdXnPdN0cnMfF9opN69co+7ZrbfPGrdpPVNBUj02spi6B1N7wChLQiPn4CSH/zJvXw56gmHw==",
"funding": [
{
"type": "github",
"url": "https://github.com/sponsors/fb55"
}
],
"license": "BSD-2-Clause"
},
"node_modules/htmlparser2/node_modules/domhandler": {
"version": "5.0.3",
"resolved": "https://registry.npmjs.org/domhandler/-/domhandler-5.0.3.tgz",
"integrity": "sha512-cgwlv/1iFQiFnU96XXgROh8xTeetsnJiDsTc7TYCLFd9+/WNkIqPTxiM/8pSd8VIrhXGTf1Ny1q1hquVqDJB5w==",
"license": "BSD-2-Clause",
"dependencies": {
"domelementtype": "^2.3.0"
},
"engines": {
"node": ">= 4"
},
"funding": {
"url": "https://github.com/fb55/domhandler?sponsor=1"
}
},
"node_modules/htmlparser2/node_modules/domutils": {
"version": "3.2.2",
"resolved": "https://registry.npmjs.org/domutils/-/domutils-3.2.2.tgz",
"integrity": "sha512-6kZKyUajlDuqlHKVX1w7gyslj9MPIXzIFiz/rGu35uC1wMi+kMhQwGhl4lt9unC9Vb9INnY9Z3/ZA3+FhASLaw==",
"license": "BSD-2-Clause",
"dependencies": {
"dom-serializer": "^2.0.0",
"domelementtype": "^2.3.0",
"domhandler": "^5.0.3"
},
"funding": {
"url": "https://github.com/fb55/domutils?sponsor=1"
}
},
"node_modules/htmlparser2/node_modules/entities": {
"version": "7.0.1",
"resolved": "https://registry.npmjs.org/entities/-/entities-7.0.1.tgz",
"integrity": "sha512-TWrgLOFUQTH994YUyl1yT4uyavY5nNB5muff+RtWaqNVCAK408b5ZnnbNAUEWLTCpum9w6arT70i1XdQ4UeOPA==",
"license": "BSD-2-Clause",
"engines": {
"node": ">=0.12"
},
"funding": {
"url": "https://github.com/fb55/entities?sponsor=1"
}
},
"node_modules/linkedom": {
"version": "0.18.13",
"resolved": "https://registry.npmjs.org/linkedom/-/linkedom-0.18.13.tgz",
"integrity": "sha512-ES/o9qotMpzpN2MHs+Iq/JcVoOj8Fa5wiQYrTdFpvAnwXL0g66XHHUc9WUMk6nAlBtGsFQ24ne+SYnvnaQ2FSw==",
"license": "ISC",
"dependencies": {
"css-select": "^7.0.0",
"cssom": "^0.5.0",
"html-escaper": "^3.0.3",
"htmlparser2": "^10.1.0",
"uhyphen": "^0.2.0"
},
"engines": {
"node": ">=16"
},
"peerDependencies": {
"canvas": ">= 2"
},
"peerDependenciesMeta": {
"canvas": {
"optional": true
}
}
},
"node_modules/marked": {
"version": "18.0.5",
"resolved": "https://registry.npmjs.org/marked/-/marked-18.0.5.tgz",
"integrity": "sha512-S6GcvALHg6K4ohtu4E7x0a1AqhAjp6cV8KhLSyN9qVapnzJkusVBxZRcIU9AeYsbe6P1hKDusSbEOzGyyuce6w==",
"license": "MIT",
"bin": {
"marked": "bin/marked.js"
},
"engines": {
"node": ">= 20"
}
},
"node_modules/mathml-to-latex": {
"version": "1.8.0",
"resolved": "https://registry.npmjs.org/mathml-to-latex/-/mathml-to-latex-1.8.0.tgz",
"integrity": "sha512-gQ0uK3zqB8HwlfaXJkEL5rgaZNbKUiBMmBP/B/W+b+t6KcseLSuYb1b0BjLgS9ZiQa24ePkqTX8/6FaQuDL7wQ==",
"license": "MIT",
"optional": true,
"dependencies": {
"@xmldom/xmldom": "^0.9.10"
}
},
"node_modules/mime-db": {
"version": "1.54.0",
"resolved": "https://registry.npmjs.org/mime-db/-/mime-db-1.54.0.tgz",
"integrity": "sha512-aU5EJuIN2WDemCcAp2vFBfp/m4EAhWJnUNSSw0ixs7/kXbd6Pg64EmwJkNdFhB8aWt1sH2CTXrLxo/iAGV3oPQ==",
"license": "MIT",
"engines": {
"node": ">= 0.6"
}
},
"node_modules/mime-types": {
"version": "3.0.2",
"resolved": "https://registry.npmjs.org/mime-types/-/mime-types-3.0.2.tgz",
"integrity": "sha512-Lbgzdk0h4juoQ9fCKXW4by0UJqj+nOOrI9MJ1sSj4nI8aI2eo1qmvQEie4VD1glsS250n15LsWsYtCugiStS5A==",
"license": "MIT",
"dependencies": {
"mime-db": "^1.54.0"
},
"engines": {
"node": ">=18"
},
"funding": {
"type": "opencollective",
"url": "https://opencollective.com/express"
}
},
"node_modules/nth-check": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/nth-check/-/nth-check-3.0.1.tgz",
"integrity": "sha512-GX0gsdbGVCgnRgbeGaubfjpBXyYRWOOCVeYh08bSQvDZqxz5ndXs1OTfAt/h36G1xvI94YIspsI0sVFqAV9+RQ==",
"license": "BSD-2-Clause",
"dependencies": {
"boolbase": "^2.0.0"
},
"engines": {
"node": ">=20.19.0"
},
"funding": {
"type": "github",
"url": "https://github.com/fb55/nth-check?sponsor=1"
}
},
"node_modules/temml": {
"version": "0.13.4",
"resolved": "https://registry.npmjs.org/temml/-/temml-0.13.4.tgz",
"integrity": "sha512-k1yolMBswx34Jw9hZn5Xh2/GBlwqlW+wiN7QaUYUMhSa1ypfti6rlCfSmCxWyooxH1G32C/Z+qVvgAsBOELZBQ==",
"license": "MIT",
"optional": true,
"engines": {
"node": ">=18.13.0"
}
},
"node_modules/turndown": {
"version": "7.2.4",
"resolved": "https://registry.npmjs.org/turndown/-/turndown-7.2.4.tgz",
"integrity": "sha512-I8yFsfRzmzK0WV1pNNOA4A7y4RDfFxPRxb3t+e3ui14qSGOxGtiSP6GjeX+Y6CHb7HYaFj7ECUD7VE5kQMZWGQ==",
"license": "MIT",
"optional": true,
"dependencies": {
"@mixmark-io/domino": "^2.2.0"
},
"engines": {
"node": ">=18",
"npm": ">=9"
}
},
"node_modules/uhyphen": {
"version": "0.2.0",
"resolved": "https://registry.npmjs.org/uhyphen/-/uhyphen-0.2.0.tgz",
"integrity": "sha512-qz3o9CHXmJJPGBdqzab7qAYuW8kQGKNEuoHFYrBwV6hWIMcpAmxDLXojcHfFr9US1Pe6zUswEIJIbLI610fuqA==",
"license": "ISC"
},
"node_modules/wreq-js": {
"version": "2.3.1",
"resolved": "https://registry.npmjs.org/wreq-js/-/wreq-js-2.3.1.tgz",
"integrity": "sha512-vaKasaKeskrDKEuuO5Q5uamEG9a6FrF5ZSicH7TCvYS4RxF7/gzaU/vYqwJzcs+uydyJPVWY1KCvfVCgp0tiGA==",
"cpu": [
"x64",
"arm64"
],
"license": "MIT",
"os": [
"darwin",
"linux",
"win32"
]
}
}
}
resources/packages/pi-smart-fetch/package.json
+32
View File
@@ -0,0 +1,32 @@
{
"name": "pi-smart-fetch-project-fix",
"version": "0.3.17-project.1",
"private": true,
"type": "module",
"description": "Project-local repaired build of pi-smart-fetch with ESM/CommonJS compatibility.",
"license": "MIT",
"repository": {
"type": "git",
"url": "https://github.com/Thinkscape/agent-smart-fetch"
},
"main": "./dist/index.js",
"pi": {
"extensions": [
"./dist/index.js"
]
},
"engines": {
"node": ">=22.19.0"
},
"dependencies": {
"@earendil-works/pi-tui": "^0.82.1",
"@sinclair/typebox": "^0.34.52",
"defuddle": "^0.19.2",
"linkedom": "^0.18.13",
"mime-types": "^3.0.2",
"wreq-js": "^2.3.1"
},
"peerDependencies": {
"@earendil-works/pi-coding-agent": "*"
}
}
scripts/install-managed-packages.mjs
+90 -5
View File
@@ -1,16 +1,98 @@
import { spawnSync } from "node:child_process"; import { spawnSync } from "node:child_process";
import { existsSync, readFileSync } from "node:fs"; import { existsSync, readFileSync, writeFileSync } from "node:fs";
import { join } from "node:path"; import { join, resolve } from "node:path";
import { configurePiMemory } from "./configure-pi-memory.mjs"; import { configurePiMemory } from "./configure-pi-memory.mjs";
import { configurePiAiVision } from "./configure-pi-ai-vision.mjs"; import { configurePiAiVision } from "./configure-pi-ai-vision.mjs";
import { agentDir, managedEnvironment, rootDir } from "./profile.mjs"; import { agentDir, managedEnvironment, rootDir } from "./profile.mjs";
const settings = JSON.parse( let settings;
readFileSync(join(rootDir, "config", "settings.default.json"), "utf8"), try {
); settings = JSON.parse(readFileSync(join(rootDir, "config", "settings.default.json"), "utf8"));
} catch (error) {
const message = error instanceof Error ? error.message : String(error);
throw new Error(`Invalid config/settings.default.json: ${message}`);
}
const packages = Array.isArray(settings.packages) ? settings.packages : []; const packages = Array.isArray(settings.packages) ? settings.packages : [];
const cliPath = join(rootDir, "pi", "packages", "coding-agent", "dist", "cli.js"); const cliPath = join(rootDir, "pi", "packages", "coding-agent", "dist", "cli.js");
const env = managedEnvironment(); const env = managedEnvironment();
const npmCliPath = process.env.npm_execpath;
const localSmartFetchSource = "../../resources/packages/pi-smart-fetch";
function parseLocalSource(source) {
if (
source.startsWith("npm:") ||
source.startsWith("git:") ||
/^[a-z][a-z0-9+.-]*:\/\//i.test(source)
) {
return undefined;
}
return resolve(agentDir, source);
}
function packageDependenciesReady(packageDir) {
const packageJsonPath = join(packageDir, "package.json");
if (!existsSync(packageJsonPath)) return false;
try {
const packageJson = JSON.parse(readFileSync(packageJsonPath, "utf8"));
const dependencies = packageJson.dependencies ?? {};
return Object.keys(dependencies).every((name) =>
existsSync(join(packageDir, "node_modules", ...name.split("/"), "package.json")),
);
} catch {
return false;
}
}
function ensureLocalPackage(source) {
const packageDir = parseLocalSource(source);
if (!packageDir) return false;
if (!existsSync(join(packageDir, "package.json"))) {
throw new Error(`Local managed package is missing: ${packageDir}`);
}
if (packageDependenciesReady(packageDir)) {
console.log(`Local managed package ready: ${source}`);
return true;
}
console.log(`Installing local managed package dependencies: ${source}`);
const args = ["install", "--prefix", packageDir, "--omit=dev", "--legacy-peer-deps"];
const result = npmCliPath
? spawnSync(process.execPath, [npmCliPath, ...args], { cwd: rootDir, env, stdio: "inherit" })
: spawnSync("npm", args, { cwd: rootDir, env, stdio: "inherit" });
if (result.error) throw result.error;
if (result.status !== 0) process.exit(result.status ?? 1);
return true;
}
function ensureSmartFetchConfigured() {
const settingsPath = join(agentDir, "settings.json");
if (!existsSync(settingsPath)) return;
let settings;
try {
settings = JSON.parse(readFileSync(settingsPath, "utf8"));
} catch {
return;
}
if (!settings || typeof settings !== "object" || Array.isArray(settings)) return;
const packages = Array.isArray(settings.packages) ? settings.packages : [];
const migrated = packages.map((entry) =>
entry === "npm:pi-smart-fetch" ||
(typeof entry === "string" && entry.startsWith("npm:pi-smart-fetch@"))
? localSmartFetchSource
: entry,
);
if (!migrated.some((entry) => entry === localSmartFetchSource)) {
migrated.push(localSmartFetchSource);
}
if (JSON.stringify(packages) !== JSON.stringify(migrated)) {
writeFileSync(settingsPath, `${JSON.stringify({ ...settings, packages: migrated }, null, 2)}\n`, "utf8");
console.log(`Configured project-local smart fetch package: ${localSmartFetchSource}`);
}
}
function parseExactNpmSource(source) { function parseExactNpmSource(source) {
if (!source.startsWith("npm:")) return undefined; if (!source.startsWith("npm:")) return undefined;
@@ -32,8 +114,11 @@ function installedAtExpectedVersion(source) {
} }
} }
ensureSmartFetchConfigured();
for (const source of packages) { for (const source of packages) {
if (typeof source !== "string" || !source.trim()) continue; if (typeof source !== "string" || !source.trim()) continue;
if (ensureLocalPackage(source)) continue;
if (installedAtExpectedVersion(source)) { if (installedAtExpectedVersion(source)) {
console.log(`Managed package ready: ${source}`); console.log(`Managed package ready: ${source}`);
continue; continue;