nsfw_data_scraper

Why we included this project

Anyone who has tried to train a content moderation filter learns fast that the bottleneck is the data, not the model. This project is a set of Docker-wrapped shell scripts that crawl subreddits and other Ripme-supported sources to pull tens of thousands of images into five loose classes spanning pornography, hentai, and non-pornographic explicit material, plus safe-for-work photos and drawings. The maintainer already ran the URL-gathering step, so the repo ships with pre-scraped URL lists, and the later scripts split the downloads into train and test folders while removing corrupted files. He is upfront that the dataset is noisy and expects you to clean it before training, which is a fair trade for skipping most of the tedious scraping. If you want a working classifier end to end, there is also a fastai notebook in the repo.