Once you’ve trained your large language model on the entire written output of humanity, where do you go? Here’s Ilya Sutskever, ex-OpenAI, admitting to Reuters that they’ve plateaued: [Reuters] The…

I’m usually the one saying “AI is already as good as it’s gonna get, for a long while.”

This article, in contrast, is quotes from folks making the next AI generation - saying the same.

  • hendrik@palaver.p3x.deEnglish
    133·
    4 days ago

    Though, I don’t think that means they won’t get any better. It just means they don’t scale by feeding in more training data. But that’s why OpenAI changed their approach and added some reasoning abilities. And we’re developing/researching things like multimodality etc… There’s still quite some room for improvements.

    • MajorHavoc@programming.devOPEnglish
      52·
      4 days ago

      Though, I don’t think that means they won’t get any better. It just means they don’t scale by feeding in more training data.

      Agreed. There’s plenty of improvement to be had, but the gravy train of “more CPU or more data == better results” sounds like it’s ending.