OpenAI upgrades GPT-4 Turbo and cuts API prices with new embeddings

OpenAI has updated GPT-4 Turbo and launched two new embedding models, an essential technology that helps AI assistants and search engines find relevant information in proprietary documents. The company has also cut the price of GPT-3.5 Turbo and upgraded its text moderation model.

The announcement, made on Thursday, does not introduce a new chatbot for the general public. Its main impact will be felt by companies and developers using OpenAI’s API to build internal search engines, customer service assistants, document analysis tools and text-generation products.

GPT-4 Turbo aims to fix the “laziness” problem

The new preview version of the model is called gpt-4-0125-preview. OpenAI says it improves GPT-4 Turbo’s ability to complete complex tasks and follow instructions more reliably.

The change addresses a recurring complaint from users of the most advanced models: when handling some lengthy prompts, GPT-4 Turbo tended to over-summarize, return code snippets instead of a complete solution or ask users to continue on their own. OpenAI has acknowledged this behavior as a priority for improvement in the update.

GPT-4 Turbo retains its 128,000-token context window. A token is a unit of text that models use to process language; as a rough guide, it is equivalent to part of a word. This capacity makes it possible to work with books, lengthy contracts or large documentation repositories in a single conversation, although the cost and final quality depend on how the information is prepared.

The updated version remains a preview, meaning it is intended for testing before becoming a stable model. Companies that need consistent production results should bear in mind that OpenAI may change these versions and that they should evaluate their applications before replacing an earlier model.

Two models that turn text into mathematical meaning

The most significant new development for many applications is not text generation but the new text-embedding-3-small and text-embedding-3-large models.

Embeddings convert text into lists of numbers that represent its meaning. This allows a system to recognize that a question about vacation time is related to an internal leave policy even when the wording is different. They are the usual foundation for semantic search and retrieval-augmented generation systems, known as RAG: applications that retrieve documents before asking a model to draft a response.

According to OpenAI’s published results on the MTEB benchmark, a common reference for measuring information retrieval, text-embedding-3-small scores 62.3 points, compared with 61 points for text-embedding-ada-002, the comparable model in the lower-cost range. The new small model costs $0.02 per million tokens, five times less than its predecessor.

The large model raises the score to 64.6 on the same benchmark and costs $0.13 per million tokens. It is intended for cases where finding the right document is especially valuable: large knowledge bases, legal search engines or platforms with extensive catalogs.

OpenAI also allows users to reduce the size of the vectors generated by these models. This matters because shorter vectors take up less space and lower the cost of the specialized databases used to search them. The trade-off is that compressing them too much can reduce matching accuracy.

Lower GPT-3.5 Turbo costs and new moderation

The company has cut the price of GPT-3.5 Turbo. Input text processing now costs $0.50 per million tokens, a 50% reduction, while output falls to $1.50 per million tokens, a 25% cut.

GPT-3.5 Turbo does not match GPT-4 in capability, but it remains an important option for frequent, less demanding tasks such as classifying messages, extracting fields from documents, drafting simple content or answering highly structured queries. For these uses, the cost per million processed tokens can matter more than producing the most sophisticated response.

OpenAI has also launched text-moderation-007, a new free model in its moderation API. Its role is to identify potentially harmful text or content that violates a platform’s policies. Automated moderation does not eliminate the need for human review in sensitive contexts, but it can filter large volumes of content before they reach users or support teams.

The announcement combines a GPT-4 Turbo update with pricing changes and new tools for developers. For those using semantic search, the new embeddings are cheaper than text-embedding-ada-002, although the actual savings will depend on the migration, processing volume and infrastructure configuration.