• 0 Posts
  • 33 Comments
Joined 22 days ago
cake
Cake day: April 7th, 2025

help-circle





  • vivendi@programming.devtoMicroblog Memes@lemmy.worldGood manners are priceless.
    link
    fedilink
    English
    arrow-up
    19
    arrow-down
    3
    ·
    edit-2
    3 days ago

    Inference costs are very, very low. You can run Mistral Small 24B finetunes that are better than GPT-4o and actually quite usable on your own local machine.

    As for training costs, Meta’s LLAMA team displace their emissions with environmental programs, which is more green than 99.9% of any company making any product you use

    TLDR; don’t use ClosedAI use Mistral or other foss projects

    EDIT: I recommend cognitivecomputations Dolphin 3.0 Mistral Small R1 fine tune in particular. I’ve only used it for mathematical workloads in truth, but it has been exceedingly good at my tasks thus far. The training set and the model are both FOSS and uncensored. You’ll need a custom system prompt to activate the Chain of Thought reasoning, and you’ll need a comparatively low temperature to keep the model from creating logic loops for itself (0.1 - 0.4 range should be OK)














  • That is a one time cost. Do you add the cost of every single watt of electricity ever generated, the cost of making factories, extracting materials and so on, to the “climate impact” of electrical rail service or EVs?

    This is just hysteria, I’m going to be absolutely deadass with you

    Also, you theoretically could make an LLM entirely on your own PC, it’d just take a bit of time


  • This is … Incredibly dumb.

    The whole climate impact of AI is overstated a million times over. You can run a perfectly capable, GPT 4o destroying LLM on your own GPU right now. What is the difference between jonkling to vidya gaems or running an LLM for doing something productive for a fucking change?

    The servers running the network you posted this shit over , the servers used to develop your favorite Gacha coom game, the servers used to deliver your shit ass Xitter feed, or your Netflix garbage, use more power than AI and LLMs are actually actively more productive than that shit