The Cost of Doing (Copilot's) Business
One of the things that is a constant question for me around AI is realistic costs. Ed Zitron has a post on costs, but there are issues I take with some of the conclusions he takes.1 Ed's reporting is based on this post, which seems pretty solid.
But there is new evidence to support Ed's position. GitHub Copilot announced last month that they are moving to usage-based billing and there is now a way to see what your previous month's usage would look like under their new billing structure.
Well, it seems the day when an LLM becomes more expensive than a traditional developer is coming sooner than we expected.
How to check: Github Account Settings -> Billing and Licensing -> Premium Request Analysis -> Preview your billing impact
— Reddit user rostilos
The post includes a screenshot of rostilos' current billing and then a "preview" of their billing under the new billing.
The bill jumps from $39 to $1,063.
For ChristianRauchenwald, their $105 bill jumps to $2,757.
Technical_Visit8084? $10 to $766.
Savings-Tomorrow4090? $39 to $672.
Now, the worst posts got the most traction and for some people have a lot smaller jumps.
For example, one user goes from $10 to $40. Another from $0 to $10. One user even finds their costs staying the same ($10 to $10).
But Ed's premise that AI has been very heavily subsidised to date?
Well, that certainly seems to be true for programmers.
The main issue I take is that he rails against the subscription model as being impossible to make work because it lets some people burn "upwards of $8 in compute for every dollar of your subscription". This assumption seems specious because it assumes their API costs have zero profit margin and two he treats it as though people maxing out their limits are the norm—not outliers. A bit like an all-you-can-eat buffet, some people can eat way more than they pay for—they just need enough profit margin to allow for those people who consume above the norm.↩