PrivateGPT – Running “ChatGPT” offline on local documents

You can download the GPT model and interrogate local files using Python without having to send any data to the cloud. PrivateGPT is a python script to interrogate local files using GPT4ALL, an open source large language model. It is pretty straight forward to set up:

Clone the repo
Download the LLM – about 10GB – and place it in a new folder called models.
Place the documents you want to interrogate into the source_documents folder – by default, there’s a text of the last US state of the union in there.
Run the ingest.py script – this can a long time. On this MacBook M1 it was 2 minutes though
Run the privateGPT.py script and you get a prompt in your Terminal:

I asked it what the US president has to say about the situation in the Ukraine and it gave me a synopsis and where in the document the information is from. Screenshot of the terminal with a question providing an analysis of the situation in the Ukraine and the answer Neat!