OpenAI has introduced Structured Outputs in the API, a feature that would guarantee the radical conformity of the model’s outputs to JSON Schemas given by the developers. The extraction of formatted and processed data from raw inputs is one of the most important application of AI in today’s world. Businessmen use the OpenAI API to design prompt-intelligent assistants that can perform analyses using function invocation(opens in a new window), filling in templates to accomplish data entry in another language, and designing sequential three-partite tasks to allow the LLM to execute actions
How do Structured Outputs work?
It completely functions with the help of constrained decoding that encourages the model to produce only valid tokens about the given schema. It realizes the translation of JSON schemas into Context-Free Grammar and offers the capability of more complicated and recursive data structures compared with previous works.
How to make use of the Structured Outputs?
Structuring the Outputs isn’t a hard process. OpenAI has introduced two forms for the API:
1. Function Call:
It should be stated that the use of tools allows for structured outputs by setting strict: true in the function definition. This feature is available for all models that support tools: all versions of gpt-4-0613, and gpt-3. 5-turbo-0613 and later. When structured outputs are enabled, the model’s outputs will be consistent with the defined tools.
2. response_format new option:
Previous to this release, developers could specify JSON schemas in the request for options that included JSON through the new parameter, json_schema incorporated in the option for the response_format parameter. This is useful when the model engages with users in a reply to questions rather than triggering tool commands. This feature is available for our latest GPT-4o models: the recently launched gpt-4o-2024-08-06 and gpt-4o-mini-2024-07-18.. When response_format is provided along with strict: which is true, the model’s outputs will correspond with the specified schema.
Secure with Structured output API
The new Structured Outputs functionality will adhere to all our safety policies and the model will still be able to decline an unsafe request.
SDK Support
To support structure outputs, OpenAI adds and updates more efficient Python and Node SDKs. In essence, developers have to present Pydantic or Zod objects, The SDKs by OpenAI will then parse the JSON response and perform type conversion if needed.
Major restrictions and limitations of the model
- Structured Outputs can only use the selection of JSON Schema that is described in OpenAI docs. This in a way helps us guarantee the highest level of performance possible.
- The model can deviate from the schema as a result of the journey if the model feels like rejecting a dangerous request. If it decides to refuse, then the return message that will be sent will include the parameter refusal as true.
- The model can fail to adhere to the schema if the employing generation hits on max_tokens or another end signal before it has been completed.
- As for the types of model errors, let it be noted that Structured Outputs are designed to prevent only some of them.
- Structured Outputs are not compatible with parallel function calls. If a parallel function call is created, then it is possible for it to not match the supplied schemas.