#133 · Primary category: Knowledge Base & RAG

chatWeb

ai chatgpt crawler docx embedding faiss gpt gpt-35-turbo news-extractor newspaper openai pdf pgvector postgresql vector-database

ChatWeb can crawl web pages, read PDF, DOCX, TXT, and extract the main content, then answer your questions based on the content, or summarize the key points.

Project last updated:05/25/26

GitHub Stars

916

Forks

137

Contributors

4

License

MIT

Why we included this project

ChatWeb turns a folder of documents into something you can ask questions of, without sending the source files off to a third party. Point it at a web page, a PDF, a Word file, or plain text, and it strips the boilerplate, embeds the meaningful paragraphs into a local vector store (FAISS or Postgres with pgvector), then answers queries grounded in only that material or produces a summary of the key points. That makes it a lightweight fit for research reading, contract or policy documents, or any pile of files a team keeps asking the same things about. One nice detail is that it builds the search vector from extracted keywords rather than the raw user question, which improves the relevance of the retrieved passages. You can reach it through a console, an API, or a web UI, and there are Colab and Docker entry points, so it is easy to try on a small corpus before scaling up to a bigger knowledge base.

Articles for this project

No articles for this project yet.

To suggest a topic or contribute an article, contact us.

Related projects in this category