They’re probably referring to the 671b parameter version of deepseek. You can indeed self host it. But unless you’ve got a server rack full of data center class GPUs, you’ll probably set your house on fire before it generates a single token.
If you want a fully open source model, I recommend Qwen 2.5 or maybe deepseek v2. There’s also OLmo2, but I haven’t really tested it.
Mistral small 24b also just came out and is Apache licensed. That is something I’m testing now.
Most open/local models require a fraction of the resources of chatgpt. But they are usually not AS good in a general sense. But they often are good enough, and can sometimes surpass ChatGPT in specific domains.
It’s enough to run quantized versions of the distilled r1 model based on Qwen and Llama 3. Don’t know how fast it’ll run though.
Please bring back the overflow menu!
For stuff like that, it’s best to have an auto formatter like checkstyle or something.
Had a team lead that kept requesting nitpicky changes, going in a FULL CIRCLE about what we should change or not, to the point that changes would take weeks to get merged. Then he had the gall to say that changes were taking too long to be merged and that we couldn’t just leave code lying around in PRs.
Jesus fucking Christ.
There’s a reason that team imploded…
Yes, that’s about right. The answer is somewhere in between
It’s a systemic issue going back decades. To me, it seems the Dutch government always wants to fix it with a hammer. Repeatedly. Discrimination increases, no REAL effort for integration is made (forcing people to take totally-not-racist “civic integration exams” is not an effort), and over the years the divide increases. Tell people they are monsters long enough, and that’s what they’ll become. But no one wants to hear that fixing it would take years or even decades of sustained effort and change. They just want it fixed. And fixed now.
There is no one magic bullet solution, unfortunately. And then it all comes to a head with the events in Amsterdam. The instigators need to be arrested and tried, but society needs to take a close look at what caused this to happen to begin with. And I doubt that will happen. Just more hammers.
Don’t forget that they don’t call these people Dutch. They like a to pretend they’re “guests” in the country.
Stærsta þjóðin í heimi!
They can build a keyboard into it, sure. It’s just UI elements and a bunch of buttons. Won’t be a good keyboard, but it can be done.
Skálmöld á Sauðárkróki? Hvenær ætla þau að koma?
Well VTR is a roleplaying game. It’s similar to Vampire the Masquerade, but different setting and somewhat different mechanics. I guess it’s best explained as “nutrients.” Animal blood and blood from e.g. blood bags gives less Vitae (magic blood points resource) than blood harvested from living humans. And as the character becomes more powerful, eventually that “lesser” blood can’t actually give them Vitae.
The vampiric curse in VTR is explicitly stated to be supernatural, though, so there’s not a necessary scientific explanation for it. The curse imparts the Beast, which is the predator in all vampires.
Some fiction has it. In vampire the requiem, low level vampires can survive on animal blood. But more powerful ones need human or even vampire blood.
Where can I get a sub 400 AMD card with 26 GB of VRAM?
https://agnos.is/posts/tech-recruitment-is-out-of-control.html
This was my experience at the beginning of 2024. It was bad enough that I had to write a blog post about it.
Have you tried Matrix?
LLMs are statistical word association machines. Or tokens more accurately. So if you tell it to not make mistakes, it’ll likely weight the output towards having validation, checks, etc. It might still produce silly output saying no mistakes were made despite having bugs or logic errors. But LLMs are just a tool! So use them for what they’re good at and can actually do, not what they themselves claim they can do lol.
OpenWebUI connected tabbyUI’s OpenAI endpoint. I will try reducing temperature and seeing if that makes it more accurate.
Lol, there are smaller versions of Deepseek-r1. These aren’t the “real” Deepseek model, but they are distilled from other foundation models (Qwen2.5 and Llama3 in this case).
For the 671b parameter file, the medium-quality version weighs in at 404 GB. That means you need 404 GB of RAM/VRAM just to load the thing. Then you need preferably ALL of that in VRAM (i.e. GPU memory) to get it to generate anything fast.
For comparison, I have 16 GB of VRAM and 64 GB of RAM on my desktop. If I run the 70b parameter version of Llama3 at Q4 quant (medium quality-ish), it’s a 40 GB file. It’ll run, but mostly on the CPU. It generates ~0.85 tokens per second. So a good response will take 10-30 minutes. Which is fine if you have time to wait, but not if you want an immediate response. If I had two beefy GPUs with 24 GB VRAM each, that’d be 48 total GB and I could run the whole model in VRAM and it’d be very fast.