jeffw@lemmy.world to Technology@lemmy.worldEnglish · 3 days agoOpenAI hits back at DeepSeek with o3-mini reasoning modelarstechnica.comexternal-linkmessage-square20fedilinkarrow-up176arrow-down120
arrow-up156arrow-down1external-linkOpenAI hits back at DeepSeek with o3-mini reasoning modelarstechnica.comjeffw@lemmy.world to Technology@lemmy.worldEnglish · 3 days agomessage-square20fedilink
minus-squarebrucethemoose@lemmy.worldlinkfedilinkEnglisharrow-up1·3 days agoDense models that would fit in 100-ish GB like mistral large would be really slow on that box, and there isn’t a SOTA MoE for that size yet. So, unless you need tons of batching/parallel requests, its… kinda neither here nor there? As someone else said, the calculus changes with cheaper Strix Halo boxes (assuming those mini PCs are under $3K).
Dense models that would fit in 100-ish GB like mistral large would be really slow on that box, and there isn’t a SOTA MoE for that size yet.
So, unless you need tons of batching/parallel requests, its… kinda neither here nor there?
As someone else said, the calculus changes with cheaper Strix Halo boxes (assuming those mini PCs are under $3K).