OpenAI launches webcrawler GPTBot, and instructions on how to block it
OpenAI has launched a web crawler to improve artificial intelligence models like GPT-4. Called GPTBot, the system combs through the Internet to train and enhance AI's capabilities. Using GPTBot has the potential to improve existing AI models when it comes to aspects like accuracy and safety, according to a blog post by OpenAI. "Web pages crawled with the GPTBot user agent may potentially be used to improve future models and are filtered to remove sources that require paywall access, are known to gather personally identifiable information (PII), or have text that violates our policies," reads the post. Websites can choose to restrict access to the web crawler, however, and prevent GPTBot from accessing their sites, either partially or by opting out entirely. OpenAI said that website operators can disallow the crawler by blocking its IP address or on a site's Robots.txt file. Previously, OpenAI has landed in hot water for how it collects data and for things like copyright infringement and privacy breaches. This past June, the AI platform was sued for "stealing" personal data to train ChatGPT. Its opt-out functions were only recently implemented, with features like disabling chat history allowing users more control over what personal data can be accessed. ChatGPT 3.5 and 4 were trained on online data and text dating up to Sept. 2021. There is currently no way to remove content from that dataset. According to OpenAI, you can disallow GPTBot by adding it to your site's Robots.txt, which is essentially a text file that instructs web crawlers on what they can or cannot access from a website. You can also customize what parts a web crawler can use, allowing certain pages and disallowing others.
Related StoriesHow to prevent GPTBot from using your website's content
相关推荐
-
Abrar Ahmed returns as Pakistan names squad for second Test against Bangladesh
-
People are sharing their best spill videos after one woman spilled 22 quarts of ranch dressing
-
Critics be damned, Apple's AirPods are dominating wireless headphone sales
-
The dyed Chicago River doubles as a perfect green screen in this hilarious video
-
Haider all set for Paralympics Ceremony
-
Mural pays homage to Australian teen who egged Islamophobic politician
- 最近发表
-
- PCB official under probe for conflict of interest
- Watch Hillary Clinton answer a cute FaceTime call from her grandchildren during an event
- US will hold North Korea accountable for 'egregious' human rights violations: official
- Read an exclusive excerpt of Jeff Zentner's upcoming 'Goodbye Days'
- It's Unnecessary But, AMD Is Basically Lying About CPU Performance
- Snapchat is quietly making updates that Facebook advertisers like
- Victims dismiss expanded Oxy RB compensation plan
- Europe's 'biggest ever' LSD bust nets €4.5 million in cryptocurrency
- Best CPU Deals, AMD vs Intel: Holiday CPU Buying Guide
- Barca looking for 'patience' despite spending spree
- 随机阅读
-
- Elon Musk's AI facility is reportedly operating gas turbines without a permit
- How to download the macOS 10.14 Mojave public beta
- 'How to be a good father': Ministry publishes parenting guide for men
- Nadal missing from Spain's Davis Cup squad
- Malan retires from international cricket
- Coric crashes Nadal's welcome back party
- Vlahovic double sinks Sassuolo in Serie A
- Steam detected at North Korea's plutonium reprocessing plant: US think tank
- The Future of Tech: The Desktop PC
- Read an exclusive excerpt of Jeff Zentner's upcoming 'Goodbye Days'
- North Korean leader's wife makes 1st public appearance in over a year
- 'Legal loophole behind rise in underage prostitution’
- What to expect from Apple's September event: iPhone 16, Apple Watch 10, and more
- John Mayer really wants to join Fifth Harmony
- Female breadwinners rise in stature in South Korea
- 抓好基层党建深化阵地规范化建设
- 16 of the Most Epic Sandwiches Around the Planet
- This meme highlights some of the absolutely wild news stories that come out of Florida
- The dyed Chicago River doubles as a perfect green screen in this hilarious video
- 与时间赛跑 汉源老县城进入大搬迁阶段
- 搜索
-
- 友情链接
-