Sources indicate that OpenAI has fallen behind schedule for the creation of GPT-5 due to data shortage issues. According to the report, the AI firm has faced several challenges in getting the successor of GPT-4 to the level of capability desired, due to data scarcity issues and the need for substantial funding. As it concerns the development of GPT-5, internally known as ‘Orion,’ OpenAI has faced considerable difficulties pushing the timeline to mid-2024.
What Caused OpenAI’s GPT-5 Development Setback?
As per the source, Wall Street Journal, the primary challenge is that little high-quality training data is available for use in the models. Even though the public internet contains an almost unimaginable quantity of data, they have been unable to provide Next with the data that would improve the capabilities of GPT-5 over GPT-4.
To this end, OpenAI has brought in professional assistance in crafting new training materials from such things as software codes and mathematical problems, among other things, and is also considering the usage of synthetic data collected from other AI pre-existing models. However, such efforts have not resulted in such advancements as anticipated.
There are at least two large training sessions held at OpenAI, and each of them has taken months for the company to process tremendous amounts of data to make the Orion better. Every time something was tested, more issues were revealed, and the software could not deliver the outcome researchers expected.
The sources said, in the best case, Orion surpasses OpenAI’s existing services, but it has not evolved significantly enough to justify the high expense of operating the new model. According to WSJ, the six-month training run can cost affordably $500 million; the paper said there are public and private estimates of various aspects of the training.
This is the case given that financial constraints continue to pose a major challenge; each training run is estimated to be valued at about half a billion US dollars. Even after the board has devoted over 18 months of development and significant investment into the enhancements, it has not achieved the level of improvement needed. Project hiccups have also been amplified internally through issues such as the removal of Sam Altman from office and other top managers.
To these misfortunes, OpenAI is moving towards a new promising reasoning model like the just released “o1,” focusing on improved reasoning against language forecast. This shift in focus is planned to address the problems seen when increasing the size of language models such as GPT-5.
These developments accentuate the industry’s overall problems with AI technology, notably data restraints and the constant increase in the expense of generating more complex versions of AI models. The current AI endeavors are now seeking new ways in reasoning models and data-efficient architectures to bring the next paradigm shift in artificial intelligence.