- Category
- Developer Tools
- Rank
- No. 1065Tools index
- Pricing
- Open Source
- Type
- TOOL
- Builder
- wevm
- GitHub
- 232 stars
- Latest release
- curl.md@0.1.3
- Date
About
Fetch any URL and get clean markdown back — built for feeding pages to coding agents like Claude Code, Codex, and OpenCode.
What it does
It turns a page into compact Markdown through a layered conversion pipeline. It accepts existing Markdown directly, checks pages for alternate Markdown sources, applies site-specific rules and profiles, then extracts useful content from HTML. Thin client-rendered pages can trigger a browser-rendered retry when a compatible transport is available.
Why it's ranked here
This is a strong choice for agent workflows because it combines unusually broad access options with thoughtful extraction behavior. A URL shortcut, command-line aliases, browser access, an application library, agent plugins and skills all lead into the same focused job. The conversion path also avoids expensive rendering unless simpler extraction produces little useful content.
What's good
The converter preserves native Markdown when available, reads frontmatter, follows redirects and records source-token metadata. It can discover alternate Markdown links, rewrite known sites, apply custom extraction rules and normalize the final output. The repository also includes worker-level tests covering authentication redirects, validation, account creation, encrypted provider tokens and command-line session credentials.
Tradeoffs
Browser rendering is conditional on an optional transport, so the core converter alone may return thin results for difficult client-only sites. Requests have a fixed fifteen-second timeout unless the caller aborts sooner. The product includes accounts, billing, usage tracking and hosted infrastructure, which adds operational scope beyond simple conversion. It handles individual page requests, not a documented site crawler or search index.
How to use it well
Use it when coding agents repeatedly need readable page content with lower context cost. Start with the URL shortcut or short command-line alias for interactive work, then adopt the library when you need custom headers, fetch behavior, site rules or profiles. Add an agent plugin or skill for habitual use. Choose another system for multi-page crawling, indexing, discovery or archival storage.
Technical notes+
The conversion core in src/md/mod.ts exposes create(), rule factories, profiles, transports, native Markdown handling, HTML extraction, a 15-second timeout, and conditional rendered-page fallback. src/md/index.ts re-exports conversion, chunking, rules, sites and transports. src/entry-server.ts runs the Cloudflare Worker entry point, routes dotted shortcut paths into the API, negotiates Markdown documentation responses, processes queues and runs scheduled cleanup. src/api.ts builds the Hono API with PostgreSQL access, API keys, signed sessions, GitHub OAuth, Stripe and Sentry. package.json defines pnpm workspace scripts for builds, Vitest worker tests and Playwright end-to-end tests.
Observed
- License
- MIT
- Primary language
- TypeScript and TSX
- Install surface
- Shell installer plus global npm and Bun packages
- Interfaces
- URL shortcut, CLI with md alias, browser, JavaScript SDK, agent plugins and skills
- Runtime architecture
- Cloudflare Workers application with Hono, TanStack Start, queues and KV
- Testing
- Vitest worker tests and Playwright end-to-end test scripts are present
Read from package.json, src/api.ts, src/og.tsx, src/env.d.ts, src/router.tsx, src/vite-env.d.ts, src/entry-server.ts, src/routeTree.gen.ts, src/auto-imports.d.ts, src/api.workers.test.ts, src/md/mod.ts, src/md/index.ts, src/routes/index.tsx, src/routes/docs/index.tsx, src/routes/_dash.$login/index.tsx.
What it can do
Fetch content from any URL
URL/web address → Raw web content
Convert web pages to clean markdown
Web page content → Clean markdown text
Prepare web content for AI coding agents
Web page URL → AI-ready markdown format
Extract and clean HTML content
HTML web page → Structured markdown without HTML tags
Tags
Tech Stack
Media
Comments (0)
No comments yet
Editorially curated, with community endorsements as a secondary signal. Corrections welcome.