clipper.js

Why we included this project

Web pages rarely arrive as clean text. Clipper is a Node.js command line tool that extracts the main content from a URL or a folder of HTML files and converts it to Markdown or JSONL, using Mozilla's Readability and Turndown for the parsing. Its crawl command walks a site, clips pages that match a glob pattern, and saves them into a single dataset file, which saves you from writing scraper code when you need a corpus for retrieval or training. The JSONL output feeds directly into downstream NLP or RAG pipelines. It's a small, focused utility rather than a platform, but for repeatable HTML-to-Markdown conversion it does exactly what it promises.