{"id":1104,"date":"2013-03-22T21:56:22","date_gmt":"2013-03-22T19:56:22","guid":{"rendered":"http:\/\/www.pauljorion.com\/blog_en\/?p=1104"},"modified":"2013-03-22T21:58:50","modified_gmt":"2013-03-22T19:58:50","slug":"complexity-and-the-skynet-effect","status":"publish","type":"post","link":"https:\/\/www.pauljorion.com\/blog_en\/2013\/03\/22\/complexity-and-the-skynet-effect\/","title":{"rendered":"COMPLEXITY AND THE \u201cSKYNET EFFECT\u201d<\/b>"},"content":{"rendered":"

This is the translation by John Holland<\/a> of my post entitled Complexit\u00e9 et effet Skynet<\/a>, originally published on June 16th 2012.<\/p><\/blockquote>\n

The world in which we live today increasingly resembles a science fiction film from the 1950s. Some of those films were excellent because they raised fundamental questions about how machines seize power. Put more simply, they explored how the invasion of the machines\u2013an invasion we have actively fostered for several centuries\u2013leads us to lose control of the world. The film 2001, A Space Odyssey<\/em> (1968) culminates in a battle: the machine that is really in charge of the mission\u2015and knows how to carry it out\u2015is pitted against the human crew member whom the machine has been programmed to sacrifice. Incapable even of imagining that he is not in command, the human is led by his arrogance to fight back and eventually to escape, although the nature of his salvation turns out to be quite problematic.<\/p>\n

Today\u2019s financial markets are the prey of robots that fight their duels in computerized space. This is the result of high-frequency trading algorithms, or algos. Of course there are programmers behind these algos, who write the software and can evaluate its effects at day\u2019s end, fine-tuning when necessary and adding improvements and innovations. Nevertheless, and this is just like neural networks and genetic algorithms (whose inner workings are impenetrable to human perception), certain machine-learning techniques produce highly autonomous behavior in algos as long as they remain in operation. The upshot of this is that the \u201cSkynet effect,\u201d as we have come to call it, is now in play. This is a reference to the omnipotent computer network featured in the Terminator series. In those films, human beings no longer have a role except in a broader context where major decisions are all really made by a confederation of computers.<\/p>\n

We must ask ourselves today with the utmost seriousness whether we still have control of computers and robots (other than having the power to flip the \u201cOff\u201d switch, of course). If not, what do we need to do to get it back?<\/p>\n

=============================
\nSince this post was written, its object was illustrated in the graphic novel La survie de l’esp\u00e8ce<\/a> (Futuropolis 2012) by Gregory Makl\u00e8s and myself.<\/p>\n","protected":false},"excerpt":{"rendered":"

\n

This is the translation by John Holland<\/a> of my post entitled Complexit\u00e9 et effet Skynet<\/a>, originally published on June 16th 2012.<\/p>\n<\/blockquote>\n

The world in which we live today increasingly resembles a science fiction film from the 1950s. Some of those films were excellent because they raised fundamental questions about how machines […]<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"_crdt_document":"","footnotes":""},"categories":[12],"tags":[242,241],"class_list":["post-1104","post","type-post","status-publish","format-standard","hentry","category-human-complex-systems","tag-la-survie-de-lespece","tag-algos"],"_links":{"self":[{"href":"https:\/\/www.pauljorion.com\/blog_en\/wp-json\/wp\/v2\/posts\/1104","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.pauljorion.com\/blog_en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.pauljorion.com\/blog_en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.pauljorion.com\/blog_en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.pauljorion.com\/blog_en\/wp-json\/wp\/v2\/comments?post=1104"}],"version-history":[{"count":7,"href":"https:\/\/www.pauljorion.com\/blog_en\/wp-json\/wp\/v2\/posts\/1104\/revisions"}],"predecessor-version":[{"id":1111,"href":"https:\/\/www.pauljorion.com\/blog_en\/wp-json\/wp\/v2\/posts\/1104\/revisions\/1111"}],"wp:attachment":[{"href":"https:\/\/www.pauljorion.com\/blog_en\/wp-json\/wp\/v2\/media?parent=1104"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.pauljorion.com\/blog_en\/wp-json\/wp\/v2\/categories?post=1104"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.pauljorion.com\/blog_en\/wp-json\/wp\/v2\/tags?post=1104"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}