The biggest bottleneck of running top open models locally is memory bandwidth and sadly the ram costs are off the roof, if there's still a roof, sam altman wants your pc ram for his data centre btw.
A fascinating new research paper out of Stanford University proposed an architecture called Next Concept Modeling (NCM), and if it holds up to the test, it could turn our current reliance on hyper-expensive memory chips into a bad investment, for what its worth, i don't think current AI investment are 100% secured against these kinds of innovations, but this is just my opinion.
Comments URL: https://news.ycombinator.com/item?id=50508174
Points: 1
# Comments: 0
