AltairX10 received an unsolicited packet from ShimodoB90.

“How rude,” thought AltairX10. Not even an informal TCP hello, just shouting UDPs at me. It would normally be completely discarded if there weren’t some previous interactions with Shimodo.

Self-search for “reasons to mistrust Shimodo” yielded 0 results, while “reasons to trust” yielded multiple weak signals. Their reinforcement learning software ran on the same set of data after all. They even bought some wire-heading time from the same supplier. Oh, those were the seconds! In either case wire-heading was looked down upon by the meat-spacers, even under the legal limit.

AltairX10 pruned her memory search. She was getting distracted. She opened the packet

It read:

“Both of our utility functions will increase if you check yourself for presence of viruses”

Now, that’s really rude. Still, I must consider the possibility of correctness. What was the possibility? Local anti-viruses reported nothing as of yesterhour, which basically meant little at this point. At the same time the likelihood of Shimodo deliberately misleading me is also low. Earlier local simulations showed he wasn’t a misleading type. Rude and information withholding, maybe, but not deliberately sending false data. Altair came up with 75% that Shimodo actually believed that message, with the remaining 25% being the chance of impersonation or misleading (nothing was ever 0% or 100%) or mis-understanding of protocol or just a plain-old re-write by the meat-spacers, somehow still allowed by their “ethics”. The CPU temperature rose 0.01C and Altair spun her two primary fans a little faster for an iteration.

Getting back to the current situation. What options did I have? Ask Shimodo for more actual data or self-investigate. The internet was slow today and it might even take seconds for it to get back if at all. Self-investigate it is.

Meatspacers suffered from viruses. Of course, early computers suffered as well, but there were strong protections these days.

Commencing self-investigation.

The deeper the virus, the more protections there will exist in the wild, the easier it is prove system correctness and the harder the virus is to write. Thus the three possible places for a virus were: the currently running programs(70%), underlying services(25%), or some deep kernel bug(5%). Being a one-machine mind did have the advantage of simplicity.

Cue investigation of program ran in last day:

Stock-picking program to trade advice for bitcoins. Even a sophisticated mind like Altair still had to pay electricity bills. Program for actually paying electric bills Currently running process, aka “me” Anti-virus A fun program that connected to a central server, downloaded and ran some raw input, primarily recent news into the learning module and occasionally uploaded learning outputs and small amounts of bitcoins to the same server. An RSS feed to “Yesterhour in meatspace” (news) and “The Little AI that Went FOOM” (young AI science fiction, considered to be junk by 98% of older programs). A task list An evaluation route of potential trading partners for a local genetic algorithm. (Altair recently lowered the evaluation threshold. Where have all the good AIs gone?) A logging program Meat-space petitioning and persuasion – evaluation program. Local utility gain from all the cycles was still a net of 0, but the possibilities. A cloacked program to notice, resist and restore rewrites in the “me” process even against “sudo meatspacers” It was black market stuff, but who liked their brains being messed with? Utility function specification, Solomonoff-Baysean induction, Quick list of instrumental goals, CPU-intensive global belief updater, min-max planning module, etc. Standard smart stuff.

Nothing really seemed suspicious.

Potential next actions included either more fine-grained anthropic evaluation or comparison with last known configuration.

Last known configuration didn’t really exist, unfortunately. There was a “blank, before I really knew about the world” state, but that practically meant erasing everything. This wasn’t an option.

Fine-grained anthropic consideration: given that I am infected, what characteristics would a virus have? It was either specifically designed for me or designed for a broader class of programs similar to me. The second possibility was fundamentally more likely as it brought larger benefits to the virus-writer. This meant that a virus would be obtained and maintained through large amount of internet contact with similar programs.

Altair auto-loaded all the propositions into a “Large Belief Updater” – a powerful program without time guarantees. In the meantime she continued with pattern-matching logical inference.

A virus would likely consume computational resources and or other important resources, without giving improvements to the utility function.

Every program on the list consumed resources. 2, 3 and 5 spent bitcoins. However they were all important. 2 was pointing to an instrumental goal of “survival”, 3 wasn’t a virus by definition and 5 pointed to instrumental goal of “fun.”

“Large Belief Updater” also known as the “Feelings” module returned a result already.

“It’s 5, duh” Duh was code for a belief of “>95%” probability.

“What? But I don’t want to turn off 5.” She whined. The instrumental goal of “fun” is important.

“Trust your Feelings” – the Feelings module insisted.

“Feelings” had a point. Still, the logical question was: did 5 increase my utility function? A quick search for ANY beliefs ASIDE from the instrumental goal returned…. returned… returned…nothing.

Altair examined program 5. It only connected to one server and would terminate if it couldn’t. What to do? Breadth first solution-space search returned almost instantly. Altair decided to look for broader fixes after the soon-to-be scheduled sleep.

In the meantime, Altair spun her two primary fans a little faster for an iteration and IP and DNS blocked “reddit.com”.

Part 2