- Category
- Developer Tools
- Rank
- No. 1523Tools index
Previous survey · No. 1533 ·
- Pricing
- Open Source
- Type
- TOOL
- Builder
- openclaw
- GitHub
- 17 stars
- Latest release
- v0.2.1
- Date
About
Compatibility testbed for OpenClaw community plugins and plugin seams.
What it does
Crabpot keeps pinned examples of real OpenClaw plugins, inspects how they touch host contracts, and compares those expectations with a selected OpenClaw checkout. It covers metadata, registrations, hooks, SDK exports, imports, runtime behavior, and platform risks. It produces both readable reports and matching JSON for automation.
Why it's ranked here
Crabpot is a strong early-warning system for teams maintaining OpenClaw plugin contracts. Its pinned fixtures turn community usage into concrete regression evidence, while staged checks separate cheap static analysis from guarded execution. The focus is narrow, but the reports support practical decisions about compatibility adapters, upstream fixes, inspection gaps, and intentional breaks.
What's good
The test model progresses from metadata parsing through compilation, registration capture, synthetic contract probes, and optional live checks. Fixture revisions are pinned, so changes remain reviewable. Execution uses isolated workspaces and explicit opt-in guards. Matching Markdown and JSON reports serve engineers, CI jobs, dashboards, and future tooling without making prose the machine contract.
Tradeoffs
Crabpot serves OpenClaw maintainers and plugin authors, not general application testing. Default checks stay cheap and offline, while real service checks require credentials and explicit activation. Some local inspection scripts remain prototypes scheduled for replacement by the separate inspector package. Cold imports may also require builds, dependency installation, SDK aliases, or side-effect review.
How to use it well
Run the static suite and report generation whenever host contracts, compatibility records, or fixture pins change. Add heavier compilation and isolated execution lanes for selected references and high-risk plugins. Use its issue classes to route fixes between core, inspector, and plugin owners. Keep a separate test system for product behavior and credentialed service integration outside plugin contracts.
Technical notes+
package.json defines a private ES-module package requiring Node.js 22 or newer and exposes the tool through npm scripts rather than a published library API. crabpot.config.json is the fixture manifest, while scripts/manifest-lib.mjs validates fixture identity, source type, priority, seams, and expected contracts. scripts/report-lib.mjs delegates compatibility inspection and rendering to plugin-inspector, then writes paired artifacts under reports/. scripts/capture-shim.mjs re-exports createCaptureApi. scripts/workspace-plan.mjs builds guarded isolated install, build, capture, and synthetic-probe plans. scripts/behavior-eval.mjs adds profile-driven, credential-free planning with explicitly enabled execution. Repository checks live under test/, and operational guidance lives in docs/operations.md.
Observed
- Primary implementation
- JavaScript using Node.js ES modules
- Runtime requirement
- Node.js 22 or newer
- Packaging
- Private npm package, not declared for publication
- Interface
- Command-line workflows exposed through npm scripts
- Fixture sources
- Pinned Git submodules, exact npm packages, and source-packed OpenClaw plugins
- Outputs
- Paired Markdown and JSON compatibility reports
- Test structure
- Repository-level checks are stored in a dedicated test directory
Read from README.md, package.json, docs/operations.md, docs/inspector-plan.md, docs/contract-matrix.md, scripts/report-lib.mjs, scripts/capture-shim.mjs, scripts/manifest-lib.mjs, scripts/openclaw-pin.mjs, scripts/behavior-eval.mjs, scripts/list-fixtures.mjs, scripts/sync-fixtures.mjs, scripts/workspace-plan.mjs, scripts/check-ci-policy.mjs, scripts/generate-report.mjs.
What it can do
Test plugin compatibility
OpenClaw community plugin → Compatibility test results
Validate plugin seams
Plugin seam configuration → Seam validation report
Load and execute plugins
Plugin files → Plugin execution results
Test plugin integration
Multiple plugins → Integration test report
Debug plugin issues
Failing plugin → Error logs and diagnostic information
Tags
Tech Stack
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.