Ask HN: Where can I track training cost trend for AI models?
I'm curious how the cost of training AI models (compute, energy, data, etc) has changed over time.
Are there any public resources or datasets tracking training costs for open-weight models (I'm guessing this data is hard to get for closed models, but happy to be proved wrong.)
I'm especially interested in understanding which architectural changes (e.g., attention variants, parameter sharing, mixture-of-experts) have led to major cost optimizations, and NOT just from the companies behind these models, but from anyone who has trained or replicated them.
Check out epoch.ai. You won't get a direct answer but it'll give you directional, and useful data points.