On the Myth of Friendly AGI

There's been a lot of discussion on the subject of Artificial General Intelligence (AGI) and the risks involved in creating such a thing. Joshua Fox presents a good summary

Quantum Confusion

Versified argument continues over at The Digital Cuttlefish.

What it has shown me is that, as staggering as it seems, there are still working physicists who insist on a Copenhagen interpretation with a strict requirement for a "conscious" observer. *sigh* It's enough to drive one to Kuhn. At least the bulk of the physics community has given up on anthropocentrism.

I also stumbled across David Chalmers' (tongue in cheek?) Law of Minimization of Mystery: "consciousness is mysterious and quantum mechanics is mysterious, so maybe the two mysteries have a common source." It seems about as useful as Tim Minchin's
Peace Anthem For Palestine.

Quantum computing abhors decoherence;
It's there with no conscious observer to see.
Still there are mystics with stubborn adherence
To quantum descriptions that need me or thee.

Puzzles abound at the limits of science:
Mysterious cans full of worms to mislead.
Why mix up disparate cans in defiance
Of reason, experiment, logic or need?

Mind in behavior and cells and potentials
Is yielding a torrent of useful results.
Clutching at yet unexplored non-essentials
Works better for book-deals and starting new cults.

The Conversation Continues

Verse versus verse continues in the discussion over at The Digital Cuttlefish. This is another extract.

A phenomenon is "observed" when an observer becomes aware of it. This requires the observed system to affect the ultimate observer system, which is known to be localized in the brain and probably in the cerebral cortex.
Zeh, H. D. (1979). Quantum Theory and Time Assymetry. Foundations of Physics, Vol 9, pp 803-818 (1979).

This was no time for sleep;
This was no time to eat;
There were comments to write
Using metrical feet.

All that old, old, old phi -
All that phi had to die.

We shoveled the verses;
Thoughts shoveled them back,
Until out of the blue came a Quantum Attack.

It was Little Cat Zeh, a dualist dealer
And out of his hat he extracted
    A Wheeler!

Zeh implied "Let's make space for a god of the gaps:
All things that my Wheeler observes must collapse!"

You see, if we grant this wild Wheeler admission
And let his conceit go, then superposition
Descends on all universe parts unobserved
And keeps all their possible presents preserved.

"Oh no!" I said, "Cuttle, please fetch me a mop.
This anthropocentric conceit has to stop!"

I called up a friend who'd seen all this before
And told him the problem...

Copernicus swore.

Seussian Symbol Grounding

When the photons from my futon
Find the focus of my eyes
They will kick my cones and rods,
Thus causing signals to arise.

I cannot span the spectrum
But my special cells respond
To their windows on the wavelengths
From the wondrous world beyond.

My cortex then combines the cues
And cottons on to patterns;
With fancy feature filtering
The futon form unflattens.

My nervous networks notice
Both the novel and mundane,
Matching models, melding motifs,
For my memories to retain.

In my Hebbian web of me-ness
Not one neuron stands alone:
Every concept gains its context
From connections that it's grown.

Yet my net of wet connections
Are not abstract facts that lack
Any impact, since they're cinched
To visceral states from bliss to wrack.

(As you may have guessed, this was part of a continuing discussion over at The Digital Cuttlefish.)

Symbol Grounding

The Chinese Room won't save your "soul"
Old Searle has dug himself a hole.
You see, a man within his room
Need not be produce of a womb
Since OCR and lines of code
Could lift that secretarial load.
The "understanding" must have been
Performed by Searle's adept machine.

Once you see Searle's dopey drone
Can lack a brain and flesh and bone
You'll see the Chinese Room's a joke
That shouldn't baffle clever folk.
Alas, the ruse has gained esteem
And thrives - a most persistent meme -
In those who can't complete their weaning
From outmoded views of meaning.

My symbol grounding doesn't mess
With idle infinite regress -
No turtles, turtles, all the way,
No eyes that other eyes survey.
But every part of how I think -
Every symbol, every link -
Finds routes to run to states all real:
The correlates of how I feel.

This was a comment I posted on The Digital Cuttlefish's Daniel Dennett's Darwin Day Delivery in response to another commenter.


I felt driven to compose a reply to The Digital Cuttlefish, but it's time consuming for slow writers like me.

Isn't it queer?
Are we but hosts?
Robots of robot machines
Rid of our ghosts?
Send in the memes.

What is it like
Knowing at all?
Zombily Bayesing our nerves --
Store, then recall.
Are they just memes?
Send in the memes.

Easy to claim, "Freedom evolves!"
Making the term we're defining the thing that it solves,
Nailing that feeling of agency down to its seat
Inside a blind
Theatre of meat.

Learning is fun.
Shall I explore?
Using the memeplex of science
I can learn more.
But would these be memes
Not shared with my teams?
Perhaps there's a flaw.

Memes can be rich:
Memes everywhere
Driving the dualist dreams
Into despair.
But we're more than memes
Though sometimes it seems
Non-memebots are rare.

(With apologies to Stephen Sondheim, Daniel Dennett and Sue Blackmore.)

Engendered Species: The Quale

To be a quale means
Never knowing what it's like
To be a nagel

To be locke'd up
And preserved in spectral form
A misleading thought

To be a quale is
To face absurd reduction
And strangely survive

To thrive in tangles
Of robustly rooted weeds
Ancient and stubborn

To feed on undead
Inconceivably painless
As they jerk and scream

To be used as bait
By the chalmer of zombies
To capture new brains

Take away the quale
An ecosystem falters
Without its guano

The circle of mind
Loses its tenuously
Balanced stagnation

I Punctuate

I thought I remembered a previous thought.
Who had changed?
I, Thought,
I remembered a previous Thought who had changed.

Physical Connections

I stayed long in Mary's room,
too white to feel.
With an old Razor
I carved my flesh free from gnosis.
Mary gagged,
covered her eyes
too late to avert epiphany.
In an instant
a new connection
in her grey.

On death and deluded philosophers

Today I listened to The Philosopher's Zone The only good philosopher is a dead one, where philosophy professor Simon Critchley starts out interesting and finishes up telling us how he values death. It's an unconvincing argument, typical of those who have grown so accustomed to the inevitability of death that they create strange rationalizations in support of it.

Simon says:

"because we've bought certain myths propounded by, say, medical science, and belief in technology and things that our children and grandchildren will live forever, and the idea that, y'know we can experience a life without limitation, but a life without limitation would be awful"

He goes on to claim that immortality "would be the worst form of captivity" (and uses Gulliver's Travels, a book of fiction, to support his claim). I might agree with him if enforced immortality was something that anyone else was advocating, but he's presenting enforced immortality as the only option to our limited lifespans and then arguing against it. It's a straw man argument.

How can a philosophy professor, writing a book about death, ignore the bleedin' obvious alternative: immortality as a choice? What is wrong with having the choice of when to die? To me that would be infinitely better than what we have now.

Now we are subject to the capricious whim of mortality, knowing that at any time someone we love could be snatched away from us, or we from them. A kinder world gives me control of my own power switch. When I have no more dreams to follow and have nothing to offer my friends, then I still might not want to switch off, but that choice should be mine.

Simon says, "so to be free is to die."

Let's look even closer at that oft-used idea that it's only through constraints that we can express freedom. Does that mean that every extra constraint that we impose on a being adds to its freedom? Does every constraint that we remove make a person less free? Was the invention of flight something that made man less free by removing the vertical constraint?

This freedom through death idea is a weird distortion of the very concept of freedom. What freedoms do we get from the unmovable constraint of death that we wouldn't have if we were given the choice of when to die? None that I can see. Our very limited lifespan doesn't create interesting challenges that make life more meaningful, it just limits the scope of interesting challenges we can consider attempting. It limits our freedom.

Epicurus: "Death is nothing to us; for that which is dissolved, is without sensation, and that which lacks sensation is nothing to us."

Me: "Death is the ultimate barrier to my personal plans. It should be avoided at all costs."

Death is to be feared as one would the biggest obstacle to one's plans, hopes, and desires. Death is a catastrophic loss for those who rely on your experience. It is to be feared as a source of intense pain for those you love.

Since the potential death of a friend is a valid source of fear for me, why should I not fear my own death on behalf of my friends?

Death is an enemy. It causes pain and limits our freedom. Let's work towards controlling it. Philosophers who think it adds to our freedom have deluded themselves.

Syndicate content