[Tokyo Tech Translated] local llm grind, hardware dreams
a familiar pattern in japanese ai dev circles this week. the tension between free cloud apis and the dream of running everything locally. three tweets, three takes on the same bottleneck.
@fumi_KPFstudio, switching back to local
hit the weekly limit on open code go. switched back to qwen 3.6-35b, a local llm with decent response speed. but it’s still a bit unstable and not great. been looking around and apparently there’s a free model from open code go. thinking it might be more practical than local llm even though it’s free. asked sonet about it and seems like that’s the general take.
source: https://x.com/fumi_KPFstudio/status/2055095817456025752
@sm_hn, the sarcastic solution
yes, local LLMs have no rate limits and can be used infinitely with performance approaching Codex. just prepare two Mac Studios with 512GB config, build a distributed inference environment with Thunderbolt 5 bridge or 10Gb Ethernet for inter-node communication, and run the 1T parameter MoE model Kimi K2.6. that’s the general understanding of how to make it work nicely
source: https://x.com/sm_hn/status/2055316507778035729
@yasudaosm, offensive investment
want a high-spec pc. the kind that costs around a million yen. can think of plenty of reasons not to get one. the bottleneck is the llm anyway, not the hardware. don’t do heavy video editing. but if it accelerates me, a million yen pays for itself in an instant. offensive investment.
source: https://x.com/yasudaosm/status/2055485046233182249
these three tweets together tell a story about japanese tech discourse this week. the local llm dream is alive but expensive. the free api is practical but throttled. the hardware spend is a personal bet on acceleration. no one is arguing about which is better. they are all just trying to find a setup that doesn’t break the bank or the rate limit.
more at falsifylab.substack.com
#OnchainAlpha #DeFiYield #Hyperliquid
Originally published on FalsifyLab Substack.
— research and educational content. not investment, legal, or tax advice. do your own research. positions and views may change without notice.
Looking for comments…
Searching Nostr relays. This may take a moment the first time this article is opened.
Looking for comments…
Searching Nostr relays. This may take a moment the first time this article is opened.