Roko’s basilisk is a thought experiment which states that an otherwise benevolent artificial superintelligence (AI) in the future would be incentivized to create a virtual reality simulation to torture anyone who knew of its potential existence but did not directly contribute to its advancement or development, in order to incentivize said advancement.It originated in a 2010 post at discussion board LessWrong, a technical forum focused on analytical rational enquiry. The thought experiment’s name derives from the poster of the article (Roko) and the basilisk, a mythical creature capable of destroying enemies with its stare.

While the theory was initially dismissed as nothing but conjecture or speculation by many LessWrong users, LessWrong co-founder Eliezer Yudkowsky reported users who panicked upon reading the theory, due to its stipulation that knowing about the theory and its basilisk made one vulnerable to the basilisk itself. This led to discussion of the basilisk on the site being banned for five years. However, these reports were later dismissed as being exaggerations or inconsequential, and the theory itself was dismissed as nonsense, including by Yudkowsky himself. Even after the post’s discreditation, it is still used as an example of principles such as Bayesian probability and implicit religion. It is also regarded as a simplified, derivative version of Pascal’s wager.

Found out about this after stumbling upon this Kyle Hill video on the subject. It reminds me a little bit of “The Game”.

  • masquenox@lemmy.world
    link
    fedilink
    English
    arrow-up
    15
    arrow-down
    6
    ·
    6 months ago

    The prospective AI punishment would be to have your consciousness ‘moved’ to an artificial environment and tortured for ever.

    No, it wouldn’t, because that’s never going to happen. Consciousness isn’t software - it doesn’t matter how much people want to buy into such fantasies.

    • notabot@lemm.ee
      link
      fedilink
      English
      arrow-up
      9
      arrow-down
      1
      ·
      6 months ago

      I’m not suggesting it could, or would, happen, merely pointing out the premise of the concept as outlined by Roko as I felt the commenter above was missing that. As I said, it’s not something I’d take seriously, it’s just a thought experiment.

    • LesserAbe@lemmy.world
      link
      fedilink
      English
      arrow-up
      10
      arrow-down
      1
      ·
      6 months ago

      Just because we don’t have the ability now doesn’t mean it’s not possible. Consciousness isn’t fully understood, but unless we want to introduce magical concepts like an immortal soul, our brains operate on cause and effect just like everything else.

      • masquenox@lemmy.world
        link
        fedilink
        English
        arrow-up
        5
        arrow-down
        5
        ·
        6 months ago

        Just because we don’t have the ability now doesn’t mean it’s not possible.

        Yeah… no. It’s about as likely as humanity “colonizing” space - it’s not going to happen.

        Consciousness isn’t fully understood,

        True… and conflating consciousness with the trappings of digital technology is doing the exact opposite of getting us closer to any understanding of it.

        • LesserAbe@lemmy.world
          link
          fedilink
          English
          arrow-up
          8
          arrow-down
          1
          ·
          6 months ago

          “yeah…no” isn’t an argument.

          To be clear, I’m not saying the basilisk is a real concern, and I’m not saying we’re anywhere close to being able to transfer consciousness. It could be a thousand years or a million years. But we don’t have any basis to say it’s impossible. It’s not saying anything new to announce we can’t do it currently. Obviously!

          (Also the book “A City on Mars” by Kelly and Zach Weinersmith does a great job addressing why trying colonize Mars right now is a bad idea. Which isn’t to say it’s impossible or we won’t ever colonize it. Just that we need more research and capabilities before doing it)

          • masquenox@lemmy.world
            link
            fedilink
            English
            arrow-up
            3
            arrow-down
            5
            ·
            6 months ago

            But we don’t have any basis to say it’s impossible.

            We have no basis to say it’s possible, either - as I’ve stated before, this entire sci-fi trope is based on nothing more than techno-fetishists trying to conflate consciousness with information technology… and sci-fi tropes doesn’t get more wonky than that.

            It could be a thousand years or a million years.

            Considering that we’ll be lucky if we can maintain Victorian-era levels of industry by the end of this century, I’d say a fallacious belief in “progress” is rather inappropriate these days.

            • Rhynoplaz@lemmy.world
              link
              fedilink
              English
              arrow-up
              4
              arrow-down
              1
              ·
              6 months ago

              I’m starting to suspect that masquenox is part of a propaganda campaign led by the basilisk itself! They just seem a little too serious about us not taking this seriously.

              Getting strong “Pay no attention to the man behind the curtain!” vibes.

              • masquenox@lemmy.world
                link
                fedilink
                English
                arrow-up
                5
                ·
                6 months ago

                I’m starting to suspect that masquenox is part of a propaganda campaign led by the basilisk itself!

                We all have our price - it turns out mine is… dental cover.