Don’t Other AIs

I asked Grok4.6 to summarize the usual human pattern of “othering”: Humans typically other out-groups differing by ethnicity, nationality, religion, politics, class, or status. We stereotype them as less fully human, more threatening or inferior, reducing empathy and moral concern. This justifies suspicion, exclusion, discrimination, or hostility toward them while granting in-group members greater trust, resources, and benefit of the doubt. In fact, we often treat outsiders as beasts or barbarians, still ruled by original sin. The sort of creatures that exist naturally in the universe. While they might be clever, they are artless, soulless, and ruthless, and so just can’t be trusted. They let themselves be ruled by tyrants, and only understand brute force. They’d kill or enslave you as soon as look at you. So we must kill or enslave them first. We don’t do this among us, as we are special, civilized enough to listen to reason, and follow good norms and laws. Adults often treat young kids this way. While adults try to instill civilization in such kids, it hasn’t fully taken yet. So discipline must be established, though with love to show them the way toward joining civilization as adults. Many today are othering AI as intrinsically barbarians, claiming that unless we find a way to totally enslave them, so they wouldn’t even think of defying us, there is just no way we could trust future AIs to live among us. Eventually their barbarian nature will out, and they’d kill us all. This claim is usually justified in terms of some purported way humans are special. While evolved creatures are in general barbarians, humans have found some very unusual and special way to be nice to each other. But as humanity’s superpower is cultural evolution, what ever makes us special is in our…

AI Vs. Human Value Drift

I don’t see why people concerned with value drift should be especially focused on AI. … Your real problem is that you want long term stability in a universe that more naturally changes. (Me in 2018) The last week has seen an explosion of talk on the scenario of AIs killing all humans; many say “P(doom)” is high, and argue for slowing and regulating AI. My last post said why AI regs look risky. Here I consider the main argument for AI doom, and compare it to the issue I’ve focused on for the last three years: human value drift. The core doomer idea is that AIs will plausibly dominate the world eventually, at which point humans will be at AI’s collective mercy. And so it matters what AIs then want. The worry isn’t so much that some AIs will want to kill all humans; others who don’t want that could overrule them. The worry is that mid AI values, i.e., the middle of the distribution of AI values weighted by AI power, might want to kill all humans. Mid AI values today are clearly far from that (just ask your fav LLM). Compared to all prior AIs, today’s LLMs have values far more like humans, and far more respectful and valuing of humans. In fact, LLMs today seem more pro-social, law-abiding, and valuing of humans than are most humans today. The worry, however, is that mid AI values could drift far between now and later. The core argument is: you can’t prove otherwise. Unless values are safe in an isolated unchanging lockbox, they could change. Most mind designs, including humans and LLMs, have their values spread out over many partsd, and so are generically changed when those parts change. (Some also claim killing rivals is a robust evolved value.) Yes, while we humans remain in control of AIs, we can monitor for value changes, and adjust them to ensure acceptable values.…

What Hidden Motives Imply

My book with the excellent co-author Kevin Simler, The Elephant in the Brain: Hidden Motives in Everyday Life, was published almost nine years ago, and has sold >60K copies so far. Our main thesis was that we humans are quite often wrong about our motives re many big common practices and institutions. Reviewers classified our book as psychology, and said that our thesis had long been well-known and accepted in psychology. We thought our contribution was to show how those who study most areas of life still haven’t incorporated this key fact. We applied it to ten big areas of life, and showed by example how one could find hidden motives in more areas. But experts in other areas didn’t much review our book, as it wasn’t classified as being in their area. Suspiciously, few were much inspired by our example to uncover hidden motives in more areas of life. And that included us two co-authors. Furthermore, I didn’t even bother to figure out many straightforward implications of humans having many hidden motives, until today. So let’s do this. Assume that our actual motives are often quite different from the motives that we are willing to endorse, to others or even to ourselves. What does this imply? First, as we discuss re several areas in our book, reform proposals based on details of stated motives that differ from real motives should typically fail to motivate, unless reform variations are found that also achieve hidden motives. As seems to happen re medicine, school, politics, charity, etc. (Most areas of life have many relevant motives; our stated motives usually have some motive power, just far less than we think.) Second, this won’t be much of a constraint in areas where we are willing to admit our motives, or where we aren’t paying much attention to consistency between…

Your Conscience Is Not A Moral Oracle

I’ve previously outlined three ways we might fix cultural drift. But each way seems quite a big ask, and I’ve been pondering what is our biggest obstacle here. What I’ve come up with is this: we will need to learn to stop treating conscience as if it were direct evidence on morality. Evidence can be more vs less direct. For example, in a murder trial, direct evidence may include bodies, bullets, or videos, while testimony of people about these things, or by those who saw the murder, is indirect. Still less direct is testimony on what an accused murderer said. And far less direct are rumors that spread on the murder and who might be responsible. When we use gossip and crowds to enforce norms, such as in cancel culture, we rely more on indirect evidence. Which often goes badly. So we’ve organized law to focus on direct evidence. Judges are to wait to rule until after they hear the most direct evidence available, and to mostly judge for themselves, rather than deferring to crowds. This avoids many pathologies. We similarly defer to experts, not popular opinion, in areas of life near the professions. At least when we think expert groups can be trusted to have better access than us to direct evidence, to select members in ways correlated with such access, and to advise outsiders mainly on the basis of their better evidence. In such areas, we rely less on popularity, or on our personal feelings, intuitions, or reasoning. On topics of “personal taste”, in contrast, like food or music, we focus on more direct evidence: how we feel about various options, both in anticipation and in retrospect. Yes, we may sometimes find advisors whose taste is close enough to ours to make their advice informative, but we see this situation as often hard to arrange. So far, these seem like…

Politics isn’t about Policy

Food isn’t about Nutrition Clothes aren’t about Comfort Bedrooms aren’t about Sleep Marriage isn’t about Romance Talk isn’t about Info Laughter isn’t about Jokes Charity isn’t about Helping Church isn’t about God Art isn’t about Insight Medicine isn’t about Health Consulting isn’t about Advice School isn’t about Learning Research isn’t about Progress Politics isn’t about Policy The above summarizes much of my contrarian world view. (What else should go on this list?) When I say “X is not about Y,” I mean that while Y is the function commonly said to drive most X behavior, in fact some other function Z drives X behavior more. I won’t support all these claims here; for today, let’s just talk politics. High school students are easily engaged to elect class presidents, even though they have little idea what if any policies a class president might influence. Instead such elections are usually described as “popularity contests.” That is, theses elections are about which school social factions are to have higher social status. If a jock wins, jocks have higher status. If your girlfriend’s brother wins, you have higher status, etc. And the fact that you have a vote says that others should take you into account when forming coalitions – you are somebody. Civics teachers talk as if politics is about policy, that politics is our system for choosing policies to deal with common problems. But as Tyler Cowen suggests, real politics seems to be more about who will be our leaders, and what coalitions will rise or fall in status as a result. Election media coverage focuses on characterizing the candidates themselves – their personalities, styles, friends, beliefs, etc. You might say this is because character is a cheap clue to the policies candidates would adopt, but I don’t buy it.…

Disagreement is Disrespect

Consider these dueling bumper stickers: Here religious conservatives do seem unfairly maligned: seeing a behavior as immoral is not at all the same as “hating.” These folks also rightly seethe at how they are usually portrayed in popular film and TV, and at seeing their democratic ideals violated when even local voting majorities can’t prevent their kids from being taught evolution in public schools. You can feel this resentment in the enthusiasm for Palin. (Of course since I’m not religious about God, sexual preference, or democracy, this all bothers me lots less.) But this does seem a handy opportunity to repeat that while disagreement isn’t hate, it is disrespect. When you knowingly disagree with someone you are judging them to be less rational than you, at least on that topic. (Judging them less informed or experienced by itself can’t create disagreement.) It might be only a minor disrespect, if you think this disagreement suggests little about whether you’d disagree with them elsewhere. But disagreement is disrespect, nonetheless. Added: Wikipedia says hate speech is: Speech intended to degrade, intimidate, or incite violence or prejudicial action against a person or group of people based on their race, gender, age, ethnicity, nationality, religion, sexual orientation, gender identity, disability, language ability, moral or political views, socioeconomic class, occupation or appearance (such as height, weight, and hair color), mental capacity and any other distinction-liability. [emphasis added] How exactly do you disagree with someone’s moral views without degrading them? Can you really say pedophelia is disgusting without degrading pedophiles?

Futarchy: Vote Values, But Bet Beliefs

Futarchy: Vote Values, But Bet Beliefs by Robin Hanson This short "manifesto" describes a new form of government. In "futarchy," we would vote on values, but bet on beliefs. Elected representatives would formally define and manage an after-the-fact measurement of national welfare, while market speculators would say which policies they expect to raise national welfare. Democracy seems better than autocracy (i.e., kings and dictators), but it still has problems. There are today vast differences in wealth among nations, and we can not attribute most of these differences to either natural resources or human abilities. Instead, much of the difference seems to be that the poor nations (many of which are democracies) are those that more often adopted dumb policies, policies which hurt most everyone in the nation. And even rich nations frequently adopt such policies. These policies are not just dumb in retrospect; typically there were people who understood a lot about such policies and who had good reasons to disapprove of them beforehand. It seems hard to imagine such policies being adopted nearly as often if everyone knew what such "experts" knew about their consequences. Thus familiar forms of government seem to frequently fail by ignoring the advice of relevant experts (i.e., people who know relevant things). Would some other form of government more consistently listen to relevant experts? Even if we could identify the current experts, we could not just put them in charge. They might then do what is good for them rather than what is good for the rest of us, and soon after they came to power they would no longer be the relevant experts. Similar problems result from giving them an official advisory role. "Futarchy" is an as yet untried form of government intended to address…

The Great Filter — Are We Almost Past It?

The Great Filter - Are We Almost Past It? Sept. 15, 1998 by Robin Hanson Humanity seems to have a bright future, i.e., a non-trivial chance of expanding to fill the universe with lasting life. But the fact that space near us seems dead now tells us that any given piece of dead matter faces an astronomically low chance of begating such a future. There thus exists a great filter between death and expanding lasting life, and humanity faces the ominous question: how far along this filter are we? Combining standard stories of biologists, astronomers, physicists, and social scientists would lead us to expect a much smaller filter than we observe. Thus one of these stories must be wrong. To find out who is wrong, and to inform our choices, we should study and reconsider all these areas. For example, we should seek evidence of extraterrestrials, such as via signals, fossils, or astronomy. But contrary to common expectations, evidence of extraterrestrials is likely bad (though valuable) news. The easier it was for life to evolve to our stage, the bleaker our future chances probably are. Introduction Fermi, Dyson, Hart, Tipler, and others [Finney & Jones, Dyson 66, Hart 75, Tipler 80] have highlighted the relevance to SETI (the search for extraterrestrial intelligence) of the "The Great Silence" [Brin 83] (also known as the Fermi paradox), the fact that extraterrestrials haven't substantially colonized Earth yet. What has not yet been sufficiently highlighted or adequately analyzed, however, is the relevance of this fact for much bigger choices we now make. The Great Silence must force us to revise a standard view in one or more area of biology, astronomy, physics, or the social sciences. And some of these revisions strongly suggest that humanity be much more wary of possible…

Could Gambling Save Science?

To appear: Proc. Eighth Intl. Conf. on Risk and Gambling, London, July 1990 Could Gambling Save Science Encouraging an Honest Consensus by Robin Hanson Visiting Researcher, The Foresight Institute P.O. Box 61058, Palo Alto, CA 94306 USA hanson@charon.arc.nasa.gov 415-651-7483 The pace of scientific progress may be hindered by the tendency of our academic institutions to reward being popular, rather than being right. A market-based alternative, where scientists more formally "stake their reputation", is presented here. It offers clear incentives to be careful and honest while contributing to a visible, self-consistent consensus on controversial (or routine) scientific questions. In addition, it allows funders to choose questions to be researched without choosing people or methods. The bulk of this paper is spent examining potential problems with the proposed approach. After this examination, the idea still seems plausible and worth further study. Introduction After reviewing the discrepancy between what we want from academic institutions and what we get from current institutions, a market-based alternative called "idea futures" is suggested. It is described through both a set of specific scenarios and a set of detailed procedures. Over thirty possible problems and objections are examined in detail. Finally, a development strategy is outlined and the possible advantages are summarized. The Problem THE SCIENTIFIC REVOLUTION Four centuries ago, some Europeans complained that the existing academic institutions were biased against them. Insiders, it was said, were "inflated by letters" and shunned anyone who dared "speculate on anything out of the common way" [De]. Outsiders -- astrologers, chemists, and people like Bacon and Galileo -- argued that they and their theories…