OpenAI’s DevDay: Upgrades and New Features

Well, that was a whole lot to take in! I just finished watching the Open AI DevDay event and boy, did they announce a lot of new features and upgrades. From more tokens and speed, to increased control, and custom GPTs, there is a lot to cover! So, let’s dive in.

GPT-4-Turbo with 128k Context

Open AI increased the context length from 8k to 32k and increased ChatGPT’s tokens to 128,000 with GPT-4 Turbo. This 128k context window so it can fit the equivalent of more than 300 pages of text in a single prompt! The company says they also optimized its performance to offer GPT-4 Turbo at a 3x cheaper price for input tokens and a 2x cheaper price for output tokens compared to GPT-4.

Function Calling Updates

Function calling. This feature lets you integrate external APIs into conversational flows. Now, one chat message can trigger multiple functions. For instance, previously “open the car window and turn off the A/C” required two separate requests. But with multi-function calls, ChatGPT handles chained actions in a single prompt.

No more multi-step conversations to accomplish related tasks. ChatGPT can now condense workflows and build more efficient assistants. Making APIs conversational unlocks so many possibilities for efficient, modular architecture.

And for non-devs, this means assistants that understand multi-step requests and execute them flawlessly.

Improved Instruction Following & JSON Mode

OpenAI said GPT-4 Turbo performs better than previous models on tasks that require the careful following of instructions, such as generating specific formats (e.g., “always respond in XML”).

Turbo also introduces JSON mode using the new response_format parameter. This ensures 100% valid JSON objects for devs generating programmatic responses. The efficiency gains are massive. Turbo allows conversational development workflows that previously required extensive input scrubbing.

Assistants API, Retrieval, and Code Interpreter

OpenAI also launched the Assistants API, designed to help developers build agent-like experiences within their own applications. Assistants let you build purpose-driven AI with special instructions, external data, and modular function calls. The API handles the heavy lifting so you can focus on high-quality experiences.

Flexibility is key – use cases range from natural language data analytics to voice-controlled DJs and more. If you can dream it, this API can help build it!

Under the hood, Assistants leverages the same capabilities powering GPT-3.5 – custom prompts, Codex programming, and function integration.

But the game changer is infinite threading. The API manages context so you don’t have to! Simply add new messages to an existing thread for persistent, unbroken conversations.

This solves one of the biggest pain points around context limits. With handoff to OpenAI, we can finally create flows as deep as real-world interactions demand.

Assistants also have access to call new tools as needed, including:

• Code Interpreter: writes and runs Python code in a sandboxed execution environment, and can generate graphs and charts, and process files with diverse data and formatting. It allows your assistants to run code iteratively to solve challenging code and math problems, and more.
• Retrieval: augments the assistant with knowledge from outside our models, such as proprietary domain data, product information or documents provided by your users. This means you don’t need to compute and store embeddings for your documents, or implement chunking and search algorithms. The Assistants API optimizes what retrieval technique to use based on our experience building knowledge retrieval in ChatGPT.
• Function calling: enables assistants to invoke functions you define and incorporate the function response in their messages.

GPT-4 Turbo With Vision

Vision has come to AI conversation! This game-changing upgrade allows generative models to see and respond to images. The implications are enormous!

Now GPT-4 Turbo can generate captions from photos, analyze real world visuals, and read text-heavy images. For people with visual impairments, it’s a total game-changer. BeMyEyes already uses Turbo to identify products and navigate for blind users.

Accessing this is easy – just enable the gpt-4-vision-preview model and you’re ready for visual chat. Anthropic plans to roll vision into the stable Turbo release soon.

Pricing is based on image size, starting at a fraction of a cent for smaller visuals. Passing a 1080×1080 pixel photo costs less than a penny! Considering the value unlocked, that’s an unbelievable deal.

DALL·E 3

Artists and developers, get ready to unlock DALL-E 3’s creative potential! OpenAI just launched API access to its ultra-realistic image generator. We can now integrate DALL-E magic into apps and products.

By specifying the dall-e-3 model, your programs can automatically generate stunning visuals and designs. OpenAI partners like Snap, Coca-Cola, and Shutterstock are already leveraging it for campaigns and experiences.

And good news – built-in moderation helps developers use this responsibly. The last thing we’d want is misuse ruining it for everyone.

OpenAI offers various quality levels and formats at just $0.04 per image. Considering DALL-E 3’s photorealism, that kind of democratized art is an unbelievable bargain! This API opens the floodgates for developers to integrate leading-edge generative art into their workflow. Combined with textual AI like GPT-3, the creative possibilities are endless!

Text-to-speech (TTS)

Developers, say hello to extraordinarily lifelike text-to-speech from OpenAI! The new TTS API generates human-quality voiceovers directly from text input.

You can choose from 6 expressive preset voice types, then deploy either the real-time optimized tts-1 or premium tts-1-hd models. Pricing starts at an incredible $0.015 per 1,000 characters.

The vocal outputs sound wonderfully natural and human. I can’t wait to see if they start adding more voices from which to choose. In the meantime, the potential uses are everywhere. Whether you’re making voice assistants more relatable, powering audiobook narration, or boosting accessibility, this API unlocks it all.

Custom Models + GPT Stores

Recently, I introduced you all to Latimer, what folks are calling the “Black-GPT” because it was trained on content form the HBCUs. And, I predicted that this was a sign of things to come and that we would see more audience-specific, custom GPTs. Well, OpenAI is giving us exactly that.

This is what I am most excited about from the entire DevDay:

Organizations with massive proprietary datasets can now create their own custom GPTs!

At the moment, OpenAI is only offering this to select enterprises that want to develop GPT-4 variants tailored to their niche domain. We’re talking full customization – specialized pre-training, bespoke RL fine-tuning, the works!

Per the company’s policies, your custom models and data stay completely private. No sharing across customers or reuse for other training. Total peace of mind.

Considering the intensive resources required, this will clearly be an extremely limited program at first. But for organizations with billions of proprietary tokens, it unlocks game-changing potential!

Having researchers optimize GPT-4 for your niche unlocks precision like never before. The use cases for sectors like healthcare and engineering are incredible. Companies can now wield AI truly built around their domain. But crucially, OpenAI ensures ethics and privacy steer these tools toward uplifting outcomes. Exciting and responsible!

What custom AI capabilities will transform your business? While availability is scarce, it’s still thrilling to imagine tailored AI’s potential! OpenAI says this will be a very limited (and expensive) program to start—interested orgs can apply here.

These custom GPTs can be private, or for enterprise users, they can be listed available only within your company. And, (drumroll please) OpenAI is launching a GPT Store where you can make your custom GPT available to others, earning money with each licensed use.

Lower Prices and Higher Rate Limits

OpenAI is decreasing several prices across the platform to pass on savings to developers (all prices below are expressed per 1,000 tokens):

• GPT-4 Turbo input tokens are 3x cheaper than GPT-4 at $0.01 and output tokens are 2x cheaper at $0.03.
• GPT-3.5 Turbo input tokens are 3x cheaper than the previous 16K model at $0.001 and output tokens are 2x cheaper at $0.002. Developers previously using GPT-3.5 Turbo 4K benefit from a 33% reduction on input tokens at $0.001. Those lower prices only apply to the new GPT-3.5 Turbo introduced today.
• Fine-tuned GPT-3.5 Turbo 4K model input tokens are reduced by 4x at $0.003 and output tokens are 2.7x cheaper at $0.006. Fine-tuning also supports 16K context at the same price as 4K with the new GPT-3.5 Turbo model. These new prices also apply to fine-tuned gpt-3.5-turbo-0613 models.

	Older models	New models
GPT-4 Turbo	GPT-4 8K Input: $0.03 Output: $0.06 GPT-4 32K Input: $0.06 Output: $0.012	GPT-4 Turbo 128K Input: $0.01 Output: $0.03
GPT-3.5 Turbo	GPT-3.5 Turbo 4K Input: $0.0015 Output: $0.002 GPT-3.5 Turbo 16K Input: $0.003 Output: $0.004	GPT-3.5 Turbo 16K Input: $0.001 Output: $0.002
GPT-3.5 Turbo fine-tuning	GPT-3.5 Turbo 4K fine-tuning Training: $0.008 Input: $0.012 Output: $0.016	GPT-3.5 Turbo 4K and 16K fine-tuning Training: $0.008 Input: $0.003 Output: $0.006

Higher Rate Limits

To help you scale your applications, the company is doubling the tokens per minute limit for all paying GPT-4 customers.

You can view your new rate limits in your rate limit page. They’ve also published usage tiers that determine automatic rate limits increases, so you know what to expect in how your usage limits will automatically scale. You can now request increases to usage limits from your account settings.

Copyright Shield

This is an important one for all marketers who seek to utilize the output from ChatGPT commercially.

Following in the footsteps of IBM, who announced back in October that they would protect their customers from copyright infringement claims, OpenAI says the company is committed to protecting its customers with built-in copyright safeguards in our systems.

They introduced Copyright Shield—which states the company will now step in and defend our customers, and pay the costs incurred, if they face legal claims around copyright infringement. This applies to generally available features of ChatGPT Enterprise and our developer platform.

Whisper v3 and Consistency Decoder

OpenAI is releasing Whisper large-v3, the next version of their open source automatic speech recognition model (ASR) which features improved performance across languages.

The company is also open sourcing the Consistency Decoder, a drop in replacement for the Stable Diffusion VAE decoder. This decoder improves all images compatible with the by Stable Diffusion 1.0+ VAE, with significant improvements in text, faces and straight lines.

Whew! That was a lot to take in. I will start testing these new upgrades and I will share my thoughts as I use them. Let me know if there is anything I missed. And, drop a note in the comments telling me what you’re most excited about from OpenAI’s first DevDay. As for me, I can’t wait to start playing!

---

If you need assistance understanding how to leverage Generative AI in your marketing, advertising, or public relations campaigns, contact us today. In-person and virtual training workshops are available. Or, schedule a session for a comprehensive AI Transformation strategic roadmap to ensure your marketing team utilizes the right GAI tech stack for your needs.

Read more: OpenAI’s DevDay: Upgrades and New Features