Saturday, December 2, 2023

How A lot Does it Price to Use an LLM? by @ttunguz

How a lot does AI value? It will depend on a a couple of completely different dimensions, however for a typical use case the place a consumer enters a question & expects a few 200 phrase response, the associated fee varies from $0.03 to $3.60.


Totally different fashions value completely different quantities. Additionally, the scale of the context window is a crucial issue.

The context window is a elaborate approach of claiming the amount of background data the consumer can present to the AI.

When asking an AI to write down a two-paragraph essay within the model of Hemingway, having a couple of sentences of his writing received’t assist a lot. However maybe importing one-third of the Outdated Man & the Sea imbues Ernest’s model throughout the prose – no adverbs, easy direct sentences.

Longer tracts value extra. A 500 phrase essay totals $8.40 with the fanciest model of GPT-4 & $0.07 with Llama2, Fb’s open-source mannequin.

There’s a 120x distinction in value between the highest of the road AI & the free model.

Which is best? It will depend on the use case & the quantity of optimization one is prepared to decide to the mannequin.

These costs aren’t fastened. Earlier this week, OpenAI introduced a 3x discount in prices – a pattern that can proceed as engineers optimize the chips, the algorithms, & the techniques for AI.

Relative to different infrastructure prices, these charges will be important & understanding the gross-margin affect of this new layer of software program might be important, & will probably push startups from the costlier bigger fashions to extra capital-efficient AI over time.

Related Articles


Please enter your comment!
Please enter your name here

Latest Articles