Post #3645264

@marshray@infosec.exchange @petealexharris@mastodon.scot @johncarlosbaez@mathstodon.xyz I think it's built into humanity. We are aware of malice because it is a common failure mode in cooperative endeavours, especially in childhood when people have not learned to manage strong emotions like jealousy. Humans develop internal complex predictive models of not only their environment in general but of the behavior of other humans. We default to modeling others at our own level of competence. Which makes modeling toddlers extra hard, because we desperately need to model them, but they are nowhere near our level of competence. Both toddlers and robots often seem "maliciously compliant". They do *precisely* what you ask, especially when that's not what you intended. Of course all humanity has maliciously compliant trickster legends. Anyone with children would love those stories.