Day: May 31, 2024

 
LLMs

How to block GPTBot (OpenAI’s web crawler) from accessing your website

If you don’t want your content to be used by Open AI’s GPT models (for training purposes e.g.) you can disallow the GPT Bot in your robots.txt. What is the GPTBot? GPTBot is the web crawler from OpenAI. It is used to train their models as well as for providing real-time information in ChatGPT (and […]

Read More