Claude Instant with 100,000 tokens beats leading AI chatbots

anthropic released a new version of Claude that accepts 100,000 tokens, or approximately 75,000 words of input.

This allows users to analyze and perform tasks on long excerpts from books, code, documents, transcripts, and more.

Poe screenshot, June 2023

Read on to learn more about the latest version of the anthropomorphic AI chatbot, Claude-instant-100k, and see how it performs against its competitors: ChatGPT, Bing AI, and Google Bard.

AI Chatbot Entry Limitations

Want to ask questions about a piece of content? Here’s the length of input you can send to generative AI chatbots from OpenAI, Microsoft, and Google.

OpenAI’s standard GPT-4 model provides 8,000 context tokens, while the extended context length model provides 32,000 tokens / 25,000 words of entry The Bing AI chat request input box shows 0/4000, suggesting that it only accepts up to 4000 characters. However, the chatbot accepted 40,400 characters of lorem ipsum and accurately counted that it was a total of 6,000 words. Google Bard accepts up to 10,000 characters.

Claude-instant-100k accepts: three times more input than the best GPT-4 model.

But how far can they follow instructions?

I experimented with three tasks with Claude-instant-100k, ChatGPT Plus, Bing AI and Google Bard. Here are the results.

Task no. 1: Edit a list of 1580 words with Claude, ChatGPT, Bing and Bard

The entry: a list of 126 items (some of the first ChatGPT plugins) with 1,850 words / 12,500 characters.

The Message: You can reword and edit the descriptions listed below:

[the list]

Claude completed the task

Claude finished editing my listing in item #126 without any hallucinations and followed the instructions to reword the item descriptions. Compared to the other AI chatbots, Claude-instant-100k finished the task faster.

ChatGPT GPT-4 by default hallucinated (Chat link)

ChatGPT stops halfway through item #37. I clicked Continue generating – ChatGPT finished #37 and stopped again. I asked him to continue. ChatGPT starts listing items that weren’t in my original entry. After a few more clicks to continue generating, ChatGPT finished editing my list at item #142.

Bing AI tested

Bing accepted the entire request and listing. He asked me if this was what I was looking for; I told him yes and please continue. After further prompting it to continue, Bing finished editing my listing to item #126 without any hallucinations. Unfortunately, it stopped rewording the descriptions after item #18 and the list came out as I entered it.

Bard could not create images

Due to the 10,000 character input limitation, I was only able to submit 99 list items in the request. Bard replied, “I can’t create images yet, so I can’t help you with that.” or “That’s not something I can do yet.”

Finally, Claude-instant-100k completed the task with maximum accuracy and speed.

You can compare the output of Claude, ChatGPT, and Bing with the original list of items from this Google sheet.

Task no. 2: Summarize a 23,500-word PDF with Claude, ChatGPT, Bing, and Bard

The entry: 23,508 words / 125,104 characters copied from a PDF documenting TikTok’s responses to a hearing on how Congress can safeguard US citizens from the social platform.

The message: Please summarize the following:

[the text]

Claude succinctly summarized the entire text

Poe screenshot, June 2023