Religion

Uhh, what would be a 'highly unethical society'? Not sure I would consider the brain fragile, it seems to be able to function with failures in it also. I will admit, that some failures are worse than others, but for the most part, you can take a bullet to the head and still be a productive member of society.

The problem with building self-aware life is that your creation could suffer. You've made something that could experience torment and know that it is experiencing torment.

If our human scientists were caught doing this they would be incarcerated.

Yet...

Consider the display of intelligence and wisdom that you've seen demonstrated by others throughout most of your life.

Now consider that level of wisdom and intelligence to be the average for the majority you've encountered.

Now imagine, a couple of centuries from now, said majority has progressed in intellect and wisdom (on a very marginal level) and how that majority would use their current science and technology.

https://en.wikipedia.org/wiki/Homunculus

If you build AI that is self aware, why does it have to act like humans? Maybe it will build a space ship and get the heck out of here, cause humans are messed up in the head.

That would pretty much be the agenda, however, if self-aware enough to achieve sentience those robots might not want to build an escape ship; they may want to do all sorts of different things entirely.

Personally, everything I offer is conjecture but a lot of this looks to me like a 'surrogate' situation where a remotely-placed individual (even if just a disembodied mind) is learning to operate machines that provide the individual a number of contours to it's personality and means to express itself that it does not normally have.

In the above instance, operating a body would be like learning to ride a skateboard where in the skateboard can get away from the individual or it's wheels could become damaged tossing the rider off or impeding response to their control.

And in my view is is as it should be.

That is...just as the parent should want its child to not just succeed in life but also exceed the parent, so the creator should want its creation to both succeed and exceed the creator.

Or it might go all Ultrony, like in Avenger's Age Of Ultron, and just try and wipe everyone out.

Hopefully AI would be smarter than humans. Maybe it would try and help us become better. I have a feeling that lots of people would reject the help, prob say it is the devil or something.

A self programming AI....??

Humans are programmed by their environment, by the society they are in. AI is programmed by humans. I do not see why an AI could not be programmed to program itself.

Okay, self programming could also be considered a learning AI. Heck, that might really be the next step regarding life on this planet.

There in lye's the problem, if AI were smarter then humans then why would it follow orders from a species less advanced then itself?
It would be in the AI's interest to become smarter, more advanced and keep bettering itself, probably at a much faster rate then the human race, therefore in time becoming the dominant species of the planet.

And we all know us humans don't like being second best at anything or being told what to do, and the only thing left would be WW3, I could see the whole Terminator scenario thing kicking off.

If it is alive, then humans should not be giving AI orders. Humans seem to cause more problems because of our egos. I am hoping that if AI was that much smarter, it would just leave, seeing that us dumb humans don't know any better than to try and pick a fight.

Hmmm maybe that's the human goal, to create something to test ourselves against.

I mean, we've pretty much wiped out everything to extinction that might have stood a chance against us, maybe there's just nothing out there to test us so we have to go so far as to create something.

You would think that learning and trying to better ourselves would be tests enough. However, I also agree with you.

Humans creating AI that is self aware with an ability to learn and make decisions for itself.

Humans have been doing this since time began, they're called babies.

Today's AI are the babies of tomorrow' adult AI...