How to Control the Cost of AI Training

When OpenAI launched its AI platform ChatGPT in November 2022, it also launched an API to GPT, enticing developers to create solutions on top of its large language model (LLM). Not to be outdone, Google also opened an API key for its ChatGPT rival, Bard.

The result? AI cloud serviceshave sprung up faster than weeds in a garden.

Many AI cloud services are being offered in the standard tiers of free, midlevel, and premium, which usually has a cost. IT managers must be ready to weed out the AI solutions that seem to be providing productivity distractions rather than meaningful value.

Tier Pricing for AI Tools

Tier pricing for cloud services is not new — it is a pricing model that has existed for a while, an evolution from the web's early days when people mainly downloaded their software. However, as software as a service (SaaS), cloud services, and internet infrastructure became more robust and widely available to customers, providers started using tiered pricing models to cover a wide variety of customer use cases. A customer needing occasional access to software could choose a free or low-cost subscription, while more frequent users paid a higher subscription price to gain access to more features.

The pricing for some AI-based tools that offer straightforward features seem in line with pay-as-you-go and tier pricing in the past. But there are many AI-based solutions with a higher step-up in monthly tiers than what software providers once offered. One tool, for example, has a $19/month tier, then $149/month, then a customized quoted tier. Most tiers are smaller increments, from say $19 to $39, for example.

Related:Can AI Help Automate IT Operations?

Training AI Models Isn't Cheap

The rapidly growing cost to train AI models is a factor for the higher prices. Large language models require increasingly larger databases to be trained, but the cost to actual train models has skyrocketed even faster. According to the 2023 State of AI in 14 Charts, published by the Institute for Human-Centered AI at Stanford University, the training cost for GPT-2 was $50,000 when it was released in 2019, while in 2022 the training costs of PaLM was $8 million. This cost difference is a significantly greater rate of change than the increase in training parameters (over 15,000% versus over 3,000%).

AIIndex_2023_StateofAI_Blog_2

Solution providers seem to incorporate these costs into the pricing of their software as they update the software with AI-based features. Even well-established tech companies are not immune. The most notable price increase came from Salesforce. In July, the software producer announced a 9% price increase across its product offerings — its first in seven years. Part of the reason for the increase was marketplace economics, but the steady implementation of AI in its tools was clearly a factor as well.

Related:Employees Tap ChatGPT, but Keep It Quiet

The price hike fever is hitting cloud-based services of all scales. One product I have used over the years, Evernote, announced earlier this year a subscription price increase in anticipation of new features — including a pair of AI-enhanced features for note cleanup and search within the app. The email announcing the price change acknowledged that the increase would "risk upsetting valued Evernote customers."

I admit I was among the upset customers when I read the email — unlike other solutions that have gone up in price, I am not even able to use the new features behind the price hike. In other words, I am paying for AI to come, instead of an AI feature available for me to use now. 

Many of the AI tools seem to deliver on the promise of enhanced productivity. They consolidate multiple tasks so that an end product or service can be evaluated more quickly than by conventional means. Many of the writing and editing AI tools have more affordable pay-as-you-go tiers, in which users are charged per word, but the more useful tiers cost $1,000 to $1,500 or more to cover 15,000 to 25,000 words — an amount better-suited for publishing a novel or whitepaper.

But producing that productivity runs the risk of technical debt. Technical debt is the trade-off decision between launching a simple version of a product versus a version with all the desired features. The decision avoids the future cost of incorporating those features, replaced by building an easy, limited solution.

The technical debt for users of AI-based solutions is the time and cost spent on tools that automate processes that do not always influence strategic objectives. Acquiring a must-have tool increases the chance that a budget is being spent on a solution that may bring mixed results. Over time, organizational interest in the solution diminishes due to dissatisfication over those results. The result is a hesitancy to further spend on required maintenance.

The same can also extend to marketing department spend for content creation, as marketers may be tempted to use the plethora of AI tools for enhancing their content production. Moreover, a flurry of content can create "content debt" — a flood of content that triggers extensive rework to fit different types of media. Content debt impacts the resources available to produce, manage, and optimize media meant for a business strategy.

How IT Managers Can Reduce the Cost of AI Investments

So, what can IT managers do to reduce the associated costs with AI-tool adoption and usage?

IT audits can encourage departments to lightly audit solution usage, helping to identify the volume of usage among department heads. It's not important that Renee in Accounting is using a solution with a high price tier, but having a few dozen accountants using such software does add up quickly. If interest and usage wane without ending subscriptions, those subscriptions become an unnecessary expense.

IT teams should also keep a close watch on how teams adopt new AI developments. More open source LLMs are being launched, such as Meta's notable release of Llama 2, an open source framework that allow users to create their own AI-based solutions. Llama 2 is available for Azure AI — Microsoft is a partner with Meta on the launch — as well as Amazon Web Services (AWS), Hugging Face, and other providers. New launches like this mean teams will be enticed to develop their own solutions. It also means IT will need to monitor how development of training data adds to costs.

Saying that AI fever will not diminish anytime soon is an understatement.  AI-based solutions are jockeying for customer attention, with current leaders looking to deepen customer relationships through how they iterate on service. But in doing so, professionals must take pause and not inadvertently create havoc in cost, and other serious missteps, such as the spread of misinformation. In a webinar for Sitecore, "AI and the Future of Content," marketer Ann Handley noted the need to assess usage: "In a world where we are having AI increase our efficiency, it is incumbent on all of us to take a more measured and responsible approach to AI implementation."

With AI in so many cloud services that rank among tech professionals' top IT spending categories, managing spending and usage will gain a significant focus. AI is a nascent technology in which its value is still being proven among users, especially beyond the immediate buzz of content creation.

AI can empower creativity, but IT managers will be the right agents to ensure that creativity with AI is balanced with valuable productivity.