• TheBlackLounge@lemmy.world
      link
      fedilink
      English
      arrow-up
      26
      ·
      edit-2
      1 year ago

      Distilling has been around since forever. It’s a legitimate technique that can give you a better model depending on your needs.

      OpenAI does it too to improve its models.

    • FaceDeer@kbin.social
      link
      fedilink
      arrow-up
      8
      ·
      1 year ago

      People use compilers to compile compilers, or just generally computers to design computer hardware, all the time. It’s not so strange.

    • Zeth0s@lemmy.world
      link
      fedilink
      English
      arrow-up
      6
      ·
      1 year ago

      It’s like pupils learning from teachers, with the exception that there’s a single teacher in the whole world