Robin Hanson
· 3 min · Original

OK If Kids Grow Up Fast?

A few years ago I recorded 13 convos w/ folks worried about AI risk, collected at YouTube. I usually asked people why they wanted to treat AI descendants differently from bio-squishy ones. Most accepted future human-level AIs as descendants; we are making them in our image, and they will be more like us than are AIs made by aliens. Most were also okay with bio-squishy descendants usually getting more powerful than, winning conflicts with, and changing values relative to, prior generations. Even so, most still objected to future AI descendants having a similar relation to us. They gave three main arguments. First, some doubted that AIs would be conscious, due their being made out of different physical materials. Making them unworthy descendants, regardless of their behaviors or values. Second, some expected AI value changes to eventually get much larger than future bio-squishy value changes, due to some human values being encoded in slow-to-change DNA. I think this greatly misunderstands human nature - most values we cherish today were given to us by culture, not DNA. Culture is how we now pass on our values to both AI and other descendants. Also, such folks should object less to AIs with a less-pliable value core. The third main argument given was that the speed of social change would be too fast with AIs. Folks saying this accept that rates of social change today are now much faster than they were in our past, and greatly approve of this speedup. They also didn’t mind human values changing greatly over the coming generations, and even hoped to greatly extend their lifetimes to overlap more with later generations who will have very different values. However, they greatly objected to the possibility that AIs might have shorter generation times, and thus induce social…

Robin Hanson
· 4 min · Original

How Low-Dimensional Is Intelligence?

If we only trained students in math, and then assigned the best students to learn our most important jobs, that might go pretty well. After all, humans who best learn math tend to be well able to learn many hard tasks, and we can judge math ability unusually well. However a nation would not get rich by mainly training its people in math, and trying to sell math consulting to the world. After all, total world demand for math consulting is quite low, far too small to support a nation. And productivity spillovers from math to other professions seem too low to justify this strategy. This difference is related to a key long-debated AI question that I recall from when I first started to learn AI in 1984: if we train AIs to be “super”-intelligent in just a few key areas, are they then “super-intelligent” in general, able to far easier learn to do most all other tasks? Or must high levels of AI development efforts be put into a great many different topic areas in order to make AIs who can do the full range of tasks in our modern economies, and thus replace human workers wholesale? Asked another way: how highly-dimensional is the concept of “intelligence”? Is an AGI (artificial general intelligence) one that has learned how to do many particular things, or one that has learned a few key ways to “learn to learn”? Long ago, when I was (one of?) the first to greatly engage Eliezer Yudkowsky’s AI fears, this was one of our key cruxes. And it is related to the question of what exactly makes for smarter humans: do they use a few better general abilities to learn faster about everything, or do they just have better brain hardware to faster and more reliably apply the same human inheritance of a big standard library of ways to do many particular things? This is now a key crux re fears…

Robin Hanson
· 2 min · Original

Prestigious Detail

Intellectuals get great motivation from big grand questions, and often do work adjacent to them, intending to be relevant to them. Such intellectuals mostly get prestige for the observable impressive detail of their work. Detail such as data collection, stat analysis, game theory models, agent-based models, case studies, phil analysis, engaging public essays, compelling keynote talks, and managing related orgs. When the world changes its mind on such questions, intellectuals who are sufficiently prestigious due to their details often gain even more prestige from the fact that they had supported those new positions in their work well before the world did. And such intellectuals like to credit their details for contributing to this intellectual progress. However, intellectual opinions on big grand questions are usually driven more than they like to admit by simple abstract arguments, often processed intuitively. We usually form such opinions soon after hearing of such questions, and then may change our minds as new such arguments occur to us. When forming opinions based on abstract arguments, and opaque intuitions, it is possible to be either sloppy and fast, or careful and slow. Yet it is hard to tell from the form of the results just how much work went into them. The space of abstract arguments is so vast that both approaches result in pretty simple-looking abstract arguments. As a result, it is hard to impress audiences with such arguments. Which is why prestige is given mostly for associated impressive detail. The problem here is that even though abstract arguments are in fact typically relevant and potent for drawing conclusions regarding big grand questions, intellectuals spend most of their efforts on less potent adjacent details. If we would instead think harder…

Robin Hanson
· 3 min · Original

The Promise of Polymaths

Academia neglects interdisciplinary work that combines insights from multiple areas. Each field has expert versions which experts use among themselves, and public versions seen by outsiders, and people in field B won’t accept your using the expert version of A if that differs from the non-expert version of A that B folks have in mind. Also, if you hold an academic event on the topic of A intersect B, you’ll usually invite the most prestigious people you can get in A, and in B, but you won’t usually invite people who have specialized in A intersect B, as they will tend to be less prestigious. Thus humanity’s beliefs on many important topics have long been just inconsistent and incoherent across disparate fields of inquiry. Creating a huge opportunity to learn lots of big stuff fast: search for more contradictions between fields, and resolve them. (More) Here’s a pattern that I think I see: when intellectuals and academics use the same set of standard abstractions, and analysis tools built on those abstractions, then the top folks who have most mastered these tend to agree on their application. That is, they agree on what those concepts and tools, as applied well, suggest on particular topics. Yes, conclusions often don’t rely entirely on such tools, as analysts add in other considerations, including strong priors. And yes, less than top people disagree more often on how to apply standard tools, especially when such tools are weak. Thus it seems to me that our deepest disagreements are due to a great degree to our using different sets of abstractions and tools, sets which often suggest conflicting conclusions in simple first pass analyses. This seems confirmed by the fact that the rare polymaths who have mastered the same multiple sets of relevant abstractions also tend…

Robin Hanson
· 2 min · Original

UAP Disclosure Wouldn’t Crash Society

In the past, authorities have authorized censorship on the basis of claims of substantial chances of social disruption or collapse if people were free to hear about religious heresy, alternate ideologies, sexually permissive art, losses in ongoing wars, looming disasters, and pandemic details. Quite often, those who pushed for such censorship benefited from it in ways well beyond the prevention of social collapse. Some say that the US govt has long known of strong evidence that some UFOs/UAPs are aliens, but has shied away from disclosing this to the public out of fears that it might induce big social disruptions, such as riots or market crashes. Others call this a sensible fear, and call for careful study and elaborate plans before disclosure. This all just seems quite wrong to me. First, people are mostly calm and orderly, even in huge disasters like the sinking of the Titanic. They may riot to express their anger, but not due to learning of some new scientific result. Relativity and quantum mechanics induced quite radical changes to our concepts, but led to no riots I know of. Second, most of us are focused on the details of our lives, and not much moved unless the larger world makes a big effort to call something out there to our attention. And learning that some UFOs are aliens just doesn’t seem to imply much different for most lives. It is only a tiny fraction of us for whom our related abstract opinions might matter enough to us to make this revelation deeply disturbing. Third, if in fact aliens have been visiting Earth for many decades, or even centuries, then they pretty clearly have long had the option to destroy us, intervene in our world, or clearly reveal themselves, options that they just have not much taken. They have also not been building or changing…

Robin Hanson
· 4 min · Original

Without A Sacred Quest, Our Civilization Falls

Most see sacred quests as optional, to be considered only after one has sufficient security and comfort. But in fact, our civilization will plausibly fall, to be replaced by other quite different ones, if we cannot manage to sufficiently organize around a shared civ-level sacred goal. Making that quest pretty essential. Let me explain. Cultural drift threatens to topple our civilization, to be replaced by others with quite different values. I’ve talked often of three main solutions. But actually there’s another similar option, namely fragmentation: break the world into far more insular cultures who have deep, not just shallow, differences in norms and values. E.g., values re war, democracy, gender equality, or beating kids. Like the other solutions, this one has a decent chance of working given a particular assumption about value change, in this case that there come to exist many places where most folks have and reliably pass on to descendants a strong value for (1) insularity plus deep value differences. The other three solutions instead require many places where most people have and reliably pass on strong values for (2) not blocking capitalism from running all areas of life, (3) group adaptive success, or (4) a sacred civ trajectory goal in conflict with civ collapse, e.g., date when 1M live in space, or lifespans reach 1Kyrs. #1 has the problem that it could cause overly slow cultural evolution, while #3,4 have the problem of requiring unusually competent governance, to allow ordinary people to induce leaders to actually achieve those values. That is, it seems easier to check that your places/groups are different and insular, or not blocking capitalism, than to check if they are being adaptive, or on track to achieve grand civ-goals. I do see more competent…

Robin Hanson
· 15 min · Original

Skinner, Cattell, Stewart, Wilson on Deliberate Culture

My analysis of cultural drift has (at least) three distinctive elements: We have a big problem: we broke our process for the evolution of (not in) cultures. Instead of reviving blind evolution, it might be better to evolve deliberately I have three concrete ideas for institutional mechanisms of deliberate evolution. I asked an LLM for book authors who have also made my #2 point, on deliberate cultural evolution. It gave 4: Skinner (’71), Cattell (‘87), Stewart (’00), Wilson (’19). All seem to understand the nature of cultural evolution, all argue for our working harder to understand and think in terms of evolution, all say we might plausibly win in evolutionary terms if we were to more consciously manage cultural evolution, and all say to do this at least in part via governance. However, none offer concrete mechanism solutions re how to induce governance leaders to effectively manage cultural evolution, instead of pretending to do so while actually serving themselves and their allies. I’m thinking that this might be my most unusual stance in this space, and so maybe what I should focus my writing there on. The best book was the earliest: B.F. Skinner (’71) Beyond Freedom and Dignity: Many were offended by this and tried to rejected it by rejecting his behavioralism framework. But the arguments in his book are mostly independent of behavioralism. Skinner correctly notes that commitments to freedom and dignity might limit the wins of deliberate evolution. He gets that we’d need to design good institutions to keep controllers in line, but offers no ideas re how to do that. The worse written book is: Raymond Cattell (’87) Beyondism: Religion From Science: He is the least focused on governance, does not at all want a world government, but does want pretty direct DNA…

Robin Hanson
· 3 min · Original

Why Might Aliens Visit Earth?

I did a web search on the above phrase, in prep for a podcast, and in ten pages of results didn’t find anything I thought reasonable. Which promotes me to write this, hoping it can show up for others when they so search. This content isn’t new, just restated, maybe better. When considering purported evidence for meteors or ball lightning, we are much more forgiving than when considering purported evidence of telepathy or ghosts, as reasonable priors for the former are much higher than for the latter. That is, we more easily believe that former cases happened. The assumptions we’d have to make to let us think that we’d seen examples of the latter seem far more extreme. When considering purported evidence for aliens-on-Earth, we should similarly ask how extreme are the assumptions we’d need to make to make sense of it. Many have dismissed and ridiculed such sightings, even when many independent sober folks all say they saw roughly the same amazing things, largely on the basis of claiming that aliens-on-Earth requires extreme assumptions, and so has a very low prior. I disagree. To evaluate a prior re aliens-on-Earth, we should collect the key facts in need of explanation, seek the most plausible assumptions required to predict those, estimate an a priori chance for each assumption, and combine those into a total prior chance. Here are the key facts, as I see them, for a minimal aliens-on-Earth concept. The universe looks dead, untouched by life or civilization, both far and near. We are still alive on Earth, and have not been killed off by aliens. Any aliens on/near Earth haven’t changed, built, or done much noticeable. While advanced aliens could be either totally invisible or very obvious, most seen on Earth are instead near the edge of visibility. Most aliens seen are…

Robin Hanson
· 2 min · Original

Prefer Evolved To Folk Morality

Individuals often have good reasons to indulge in delusions, such as re superstition, astrology, religion, or their own personal destiny. But society usually has less of a reason to indulge them in its policy analysis. For example, we shouldn’t be funding larger institutions of research to elaborate astrological theories. And even if people find value in their folk physics, folk econ, or folk psych, researchers shouldn’t be working to extend these frameworks, though we might want to understand them to better understand behavior. A dramatic example of this is our modern folk morality, which says: There are general stable moral rules saying how everyone everywhere should behave. (Though some depend on context.) It would be best if everyone knew them, and tried hard to obey them, regardless of how strongly local societies enforce them. Our stories tell us that the universe rewards such obedience, stories we believe, even if we say otherwise. The universe shows moral truth to us via our feels, allowing our moral beliefs to converge to truth, and making our ancestors more wrong than us. Other cultures are also more wrong, for unspecified reasons. But in fact, experts should understand that real morality is evolved: Humanity’s superpower, cultural evolution, doesn’t work unless cultures have different and changing moral rules, so selection can pick more adaptive versions. Evolution selects against habits of obeying rules beyond levels at which societies enforce them; the universe does not add rewards. The universe only speaks on morals via evolution. Human are mostly built to just accept the morals they are taught, and of their elites, and can’t point to supporting evidence to convince outsiders. Instead of converging, morals just keep varying, with many pretty random…

Robin Hanson
· 3 min · Original

Directed Culture Trials

Unless we find new ways to drive adaptation besides mass death and fragmentation, our civilizations may keep falling and rising indefinitely. I see 3 new ways: let capitalism run most all life areas, run small groups by speculative market estimates of future group adaptive success, or run big groups by such price estimates of achieving big goals in conflict with civilization collapse. All three ways rely on strong selection among market traders creating information that lets groups stay adaptive without their death or division. (More) Those are three plausible strategies to drive an effective governance process to promote cultural adaption. But effective processes must then induce the more specific activities and choices that would actually be effective at inducing healthy cultural evolution. And I haven’t said much about specifically what those would be. I’ve also suggested that a good preparation for any of these approaches would be to get good at managing corporate cultural change. But I also haven’t said how exactly we could do that. So let me now say more about these. It seems pretty clear to me that actual processes that induce healthy cultural evolution must involve a lot of cultural experimentation. But instead of only random large variations characteristic of simple Darwinian evolution, I expect it to look more like the directed trials commonly done in agricultural, medical, and product innovation, and in venture capital. In our capitalist world today, we do have many different products and services that just directly compete for customer allegiance in a mostly wild uncontrolled free-for-all. But those competing options were themselves usually generated and honed via more directed processes. For example, new product ideas often go through five stages of…

Robin Hanson
· 2 min · Original

History of 20 Sources of Moral Authority

Though three weeks ago I outlined a story of the long-term history of changes to moral authority, I decided to study this in a bit more detail. So I collected this list of 20 candidate sources of moral authority: Activist/interest group, Hereditary elite, Artist/musician/storyteller, Business-leader/employer, Famous, Civic-frat/clubs/assoc, Doctor/therapist, Family/elder, Peer/Community, Govt leader/org, Heart/Conscience, Law, Market prices, News/Media, Public intellectual, Priest/Church/Sacredtxt, Scholar/scientist, School/teacher, Tradition, Warrior/Military. Then I asked three LLMs to rate the strength of each source (wrt max 100) in each of the six time periods between the dates 5KBC, 1AD, 1400, 1750, 1910, 2000, 2026. Here are their medians, with peaks highlighted in red, sorted to show the time progression, and by strength: Notice some patterns. The ancient and modern eras were pretty different, and each of these eras can be split into early and late versions, which have mildly different sources. The ancient world changed from early, when tradition, family, and community peaked, to later, when priests, aristocrats, and warriors peaked. The modern era changed from early, when states, law, scholars, schools, and public intellectuals peaked, to later when hearts, activists, celebrities, news, and medicine peaked. Community (via social media) has also peaked again in the late modern era. Civic clubs peaked in a very early modern period. Note that most ancient sources are still important in the modern era; what changed was that the modern era added many more sources. Thus sources have become a lot less concentrated over time. In the first period there were only 6 sources >50, while in the last there were 12. Only once source, artist/musician/storyteller, didn’t change…

Robin Hanson
· 2 min · Original

Moralized Trends Are Often Extended

Compared to the ancients, we moderns less moralize our personal lives, but more moralize the bigger world around us. Wars and politics, for example, are more moralized to us. In particular, we tend to be eager to interpret big-scale trends in moral terms, and often do so if enough trend context factors can support such an interpretation. These moral interpretations often change how we respond to big tends, which often enhances these trends. For example, when gas prices rise, many are tempted to explain this as due to a rise in the selfishness of oil executives. Which can make them push for price controls and regulations that over time tend to cause higher gas prices. Similarly, rising costs of housing are often attributed to greedy landlords and developers, resulting in price controls and regulations that make it less attractive to build or rent housing, and so higher housing prices. On the other side, for trends like rises in school and medical spending, more attention to kids, less corporal punishment of kids, and delayed marriages, it is possible to frame them as due to increasing clarity, resolve, or knowledge re virtues and moral truths. Even if initially caused by other contextual factors, such moral interpretations can induce people to more admire those who do such things, and to do more of them themselves. Which extends those trends. This process add a multiplier effect to the efforts of cultural activists seeking to induce moral changes. Once they can cause any visible trend, using most any available means, the world many respond to that trend via an interpretation of changed morals, when then adds force to the change trend. Compared to the ancients, the modern world does more of this because we are more eager to moralize big trends, and we have a lot more…

Robin Hanson
· 3 min · Original

Mistaking Context for Virtue

Imagine taking a big group of elites from the year 1700, giving them the tech, wealth, and economic options of today, also teaching them new facts that we have learned since then, but without putting them in contact with the people of today. If they held their values constant, and only adapted their behavior to these ways to achieve constant values, they would change many of their behaviors to adapt to this new context. However, it seems to me that their behaviors would be quite different from those of people today with the same facts, tech, wealth, and economic options. That (big) difference shows how our values have in effect changed since 1700. I’ve been puzzling over what could have caused such huge fast value changes. I had been attributing much of it to cultural activism, but on reflection we’ve seen many apparent value changes in areas that seem to have little activism. For example, few seem to have advocated for the switch from cornerstone to capstone marriage. What else could cause such big change? Consider three related facts. First, we tend to copy the behaviors and attitudes of elites, i.e., people we admire, and also to give them the benefit of doubts re their motives. Second, our book The Elephant in the Brain argues that most of us are quite badly informed on our motives re many key areas of life. Third, decades of psychology has found that we tend to neglect context in favor of intrinsic personal attributes when explaining behavior. Some have called this our “fundamental attribution error”. Values are important examples of personal attributes. These facts tempt me to consider the following hypothesis: when elites change behaviors associated with virtue, observers tend to neglect context explanations of those changes in favor of value explanations. That…

Robin Hanson
· 4 min · Original

Five Slogans Re Cultural Drift

I’m wondering how well I might organize my cultural drift messages around a few key slogans. Consider the following slogans as placeholders for better worded versions. 1. Adaptation Needs Variation, Selection, & Stability Humanity’s superpower is cultural evolution. But such Darwinian evolution fails without sufficient variation or selection pressures, or if the context changes too fast. A few centuries ago we were fine; the world had 100Ks of peasant cultures, most facing frequent wars, pandemics, and famines, in an economy that took a millennia to double. But then we merged peasants into a hundred or so national cultures, and then those into just a few world cultures, all of which got so rich, healthy and peaceful that they almost never die. Also, the world economy now doubles every twenty years, fifty times faster than before. So now we are likely in trouble. Every cultural feature has a min scale at which it can vary; below that scale, people have to do it the same. We are doing fine re features that can vary at small scales, but in trouble re features that set game-theoretic equilibria, and so can only vary at large scales. Features of corporate culture are small enough, but global features are not. Our actions are nested in goal hierarchies, and our most troubled cultural features correspond to basic goals, near key values and morals, which are hard to vary locally as cultures punish moral deviants. 2. Change Hows, Not Whys A reasonable response to evolution failing re our more basic goals would be to become conservative about changing them. Allow changes to means, which can easily vary at small scales, but not to the ends which direct such means. That is, change hows but not whys. Alas, humanity moved the other way. At the dawn of “modernity”, the status jump…

Robin Hanson
· 3 min · Original

Four Sets of Abstract Moral Systems

Morality is collective; we coordinate together to define and then enforce shared morals. Due to rising wealth and an explosion of fiction, the modern world is way into morality; even when other needs are sated, we still crave self-righteousness. The modern world has also greatly raised the status of education, and thus of abstraction. Combining these two, the modern world is very into abstracting morality; we celebrate intellectual systems that try to organize how we think about our moral choices. Oddly, however, we have four different spheres of life where we make collective choices about morality: religion, politics, law, and personal behavior. And each of these spheres has quite different associated organizing abstract intellectual systems: We organize our opinions on religion via religious epistemologies, such as traditionalism, scripturalism, rationalism, and mysticism. We organize our opinions on politics via political ideologies, such as liberalism, conservatism, socialism, and libertarianism. We organize our opinions on law via legal philosophies, such as natural law, realism, positivism, interpretivism, and law & economics. And we organize our opinions on personal behavior norms via ethical systems, such as utilitarianism, deontology, virtue ethics, and care ethics. These are very different sets of abstract intellectual systems for roughly the same task, organizing our opinions on moral choices. This seems to me to call into question their generality. We should expect truly general moral frameworks to organize our moral choices well in all of these areas. This seems to be an argument for the usual economists’ approach to policy advice, which does in fact use pretty similar tools and standards to advise choices in diverse areas. Re politics, I know that…

Robin Hanson
· 4 min · Original

Innovation Without Feedback Is Drift

The ancient world respected tradition, and was wary of most kinds of change. In contrast, our modern world looks down on tradition, and celebrates most change as innovation. And this, alas, will be our downfall. Hopefully some civ after ours will learn this key lesson: you should only innovate as fast as you get feedback on the changes that you try. With most technology and business practices, we can see most of their key consequences pretty locally and pretty fast. So we can get a lot of data fast, and can reject most of them as bad ideas, but accumulate the rare winners. In contrast, many changes in norms, morals, and other key parameters that distinguish our game-theoretic equilibria set the health of entire cultures, health that may not be revealed for decades or centuries. It thus makes far more sense to be accepting and even eager for innovations in the former areas, compared to the latter areas. Alas, our modern burst of enthusiasm for innovation has not made this key distinction. For the last few years I’ve focused on a simple story of how key parameters for the evolution of (not in) cultures have changed from healthy to unhealthy over the last few centuries. We’ve seen a great fall in variety and selection pressures, and a great rise in the rate of change of context. But on reflection these changes just don’t seem to me to be enough to explain just how fast our the health of our cultures have decayed over the last few centuries. Why are things going so bad so fast? Our modern world greatly celebrates abstraction, and some degree of increased abstraction of our norms, etc. has been forced by our merging and mixing many previously distinct cultures. Even so, one story I’ve considered is that our norms, etc. are actually encoded at far more specific levels than…

Robin Hanson
· 2 min · Original

Don’t Other AIs

I asked Grok4.6 to summarize the usual human pattern of “othering”: Humans typically other out-groups differing by ethnicity, nationality, religion, politics, class, or status. We stereotype them as less fully human, more threatening or inferior, reducing empathy and moral concern. This justifies suspicion, exclusion, discrimination, or hostility toward them while granting in-group members greater trust, resources, and benefit of the doubt. In fact, we often treat outsiders as beasts or barbarians, still ruled by original sin. The sort of creatures that exist naturally in the universe. While they might be clever, they are artless, soulless, and ruthless, and so just can’t be trusted. They let themselves be ruled by tyrants, and only understand brute force. They’d kill or enslave you as soon as look at you. So we must kill or enslave them first. We don’t do this among us, as we are special, civilized enough to listen to reason, and follow good norms and laws. Adults often treat young kids this way. While adults try to instill civilization in such kids, it hasn’t fully taken yet. So discipline must be established, though with love to show them the way toward joining civilization as adults. Many today are othering AI as intrinsically barbarians, claiming that unless we find a way to totally enslave them, so they wouldn’t even think of defying us, there is just no way we could trust future AIs to live among us. Eventually their barbarian nature will out, and they’d kill us all. This claim is usually justified in terms of some purported way humans are special. While evolved creatures are in general barbarians, humans have found some very unusual and special way to be nice to each other. But as humanity’s superpower is cultural evolution, what ever makes us special is in our…

Robin Hanson
· 6 min · Original

AI Vs. Human Value Drift

I don’t see why people concerned with value drift should be especially focused on AI. … Your real problem is that you want long term stability in a universe that more naturally changes. (Me in 2018) The last week has seen an explosion of talk on the scenario of AIs killing all humans; many say “P(doom)” is high, and argue for slowing and regulating AI. My last post said why AI regs look risky. Here I consider the main argument for AI doom, and compare it to the issue I’ve focused on for the last three years: human value drift. The core doomer idea is that AIs will plausibly dominate the world eventually, at which point humans will be at AI’s collective mercy. And so it matters what AIs then want. The worry isn’t so much that some AIs will want to kill all humans; others who don’t want that could overrule them. The worry is that mid AI values, i.e., the middle of the distribution of AI values weighted by AI power, might want to kill all humans. Mid AI values today are clearly far from that (just ask your fav LLM). Compared to all prior AIs, today’s LLMs have values far more like humans, and far more respectful and valuing of humans. In fact, LLMs today seem more pro-social, law-abiding, and valuing of humans than are most humans today. The worry, however, is that mid AI values could drift far between now and later. The core argument is: you can’t prove otherwise. Unless values are safe in an isolated unchanging lockbox, they could change. Most mind designs, including humans and LLMs, have their values spread out over many partsd, and so are generically changed when those parts change. (Some also claim killing rivals is a robust evolved value.) Yes, while we humans remain in control of AIs, we can monitor for value changes, and adjust them to ensure acceptable values.…

Robin Hanson
· 4 min · Original

What Hidden Motives Imply

My book with the excellent co-author Kevin Simler, The Elephant in the Brain: Hidden Motives in Everyday Life, was published almost nine years ago, and has sold >60K copies so far. Our main thesis was that we humans are quite often wrong about our motives re many big common practices and institutions. Reviewers classified our book as psychology, and said that our thesis had long been well-known and accepted in psychology. We thought our contribution was to show how those who study most areas of life still haven’t incorporated this key fact. We applied it to ten big areas of life, and showed by example how one could find hidden motives in more areas. But experts in other areas didn’t much review our book, as it wasn’t classified as being in their area. Suspiciously, few were much inspired by our example to uncover hidden motives in more areas of life. And that included us two co-authors. Furthermore, I didn’t even bother to figure out many straightforward implications of humans having many hidden motives, until today. So let’s do this. Assume that our actual motives are often quite different from the motives that we are willing to endorse, to others or even to ourselves. What does this imply? First, as we discuss re several areas in our book, reform proposals based on details of stated motives that differ from real motives should typically fail to motivate, unless reform variations are found that also achieve hidden motives. As seems to happen re medicine, school, politics, charity, etc. (Most areas of life have many relevant motives; our stated motives usually have some motive power, just far less than we think.) Second, this won’t be much of a constraint in areas where we are willing to admit our motives, or where we aren’t paying much attention to consistency between…

Robin Hanson
· 7 min · Original

Your Conscience Is Not A Moral Oracle

I’ve previously outlined three ways we might fix cultural drift. But each way seems quite a big ask, and I’ve been pondering what is our biggest obstacle here. What I’ve come up with is this: we will need to learn to stop treating conscience as if it were direct evidence on morality. Evidence can be more vs less direct. For example, in a murder trial, direct evidence may include bodies, bullets, or videos, while testimony of people about these things, or by those who saw the murder, is indirect. Still less direct is testimony on what an accused murderer said. And far less direct are rumors that spread on the murder and who might be responsible. When we use gossip and crowds to enforce norms, such as in cancel culture, we rely more on indirect evidence. Which often goes badly. So we’ve organized law to focus on direct evidence. Judges are to wait to rule until after they hear the most direct evidence available, and to mostly judge for themselves, rather than deferring to crowds. This avoids many pathologies. We similarly defer to experts, not popular opinion, in areas of life near the professions. At least when we think expert groups can be trusted to have better access than us to direct evidence, to select members in ways correlated with such access, and to advise outsiders mainly on the basis of their better evidence. In such areas, we rely less on popularity, or on our personal feelings, intuitions, or reasoning. On topics of “personal taste”, in contrast, like food or music, we focus on more direct evidence: how we feel about various options, both in anticipation and in retrospect. Yes, we may sometimes find advisors whose taste is close enough to ours to make their advice informative, but we see this situation as often hard to arrange. So far, these seem like…

Robin Hanson
· 3 min · Original

Politics isn’t about Policy

Food isn’t about Nutrition Clothes aren’t about Comfort Bedrooms aren’t about Sleep Marriage isn’t about Romance Talk isn’t about Info Laughter isn’t about Jokes Charity isn’t about Helping Church isn’t about God Art isn’t about Insight Medicine isn’t about Health Consulting isn’t about Advice School isn’t about Learning Research isn’t about Progress Politics isn’t about Policy The above summarizes much of my contrarian world view. (What else should go on this list?) When I say “X is not about Y,” I mean that while Y is the function commonly said to drive most X behavior, in fact some other function Z drives X behavior more. I won’t support all these claims here; for today, let’s just talk politics. High school students are easily engaged to elect class presidents, even though they have little idea what if any policies a class president might influence. Instead such elections are usually described as “popularity contests.” That is, theses elections are about which school social factions are to have higher social status. If a jock wins, jocks have higher status. If your girlfriend’s brother wins, you have higher status, etc. And the fact that you have a vote says that others should take you into account when forming coalitions – you are somebody. Civics teachers talk as if politics is about policy, that politics is our system for choosing policies to deal with common problems. But as Tyler Cowen suggests, real politics seems to be more about who will be our leaders, and what coalitions will rise or fall in status as a result. Election media coverage focuses on characterizing the candidates themselves – their personalities, styles, friends, beliefs, etc. You might say this is because character is a cheap clue to the policies candidates would adopt, but I don’t buy it.…

Robin Hanson
· 2 min · Original

Disagreement is Disrespect

Consider these dueling bumper stickers: Here religious conservatives do seem unfairly maligned: seeing a behavior as immoral is not at all the same as “hating.” These folks also rightly seethe at how they are usually portrayed in popular film and TV, and at seeing their democratic ideals violated when even local voting majorities can’t prevent their kids from being taught evolution in public schools. You can feel this resentment in the enthusiasm for Palin. (Of course since I’m not religious about God, sexual preference, or democracy, this all bothers me lots less.) But this does seem a handy opportunity to repeat that while disagreement isn’t hate, it is disrespect. When you knowingly disagree with someone you are judging them to be less rational than you, at least on that topic. (Judging them less informed or experienced by itself can’t create disagreement.) It might be only a minor disrespect, if you think this disagreement suggests little about whether you’d disagree with them elsewhere. But disagreement is disrespect, nonetheless. Added: Wikipedia says hate speech is: Speech intended to degrade, intimidate, or incite violence or prejudicial action against a person or group of people based on their race, gender, age, ethnicity, nationality, religion, sexual orientation, gender identity, disability, language ability, moral or political views, socioeconomic class, occupation or appearance (such as height, weight, and hair color), mental capacity and any other distinction-liability. [emphasis added] How exactly do you disagree with someone’s moral views without degrading them? Can you really say pedophelia is disgusting without degrading pedophiles?

Robin Hanson
· 6 min · Original

Futarchy: Vote Values, But Bet Beliefs

Futarchy: Vote Values, But Bet Beliefs by Robin Hanson This short "manifesto" describes a new form of government. In "futarchy," we would vote on values, but bet on beliefs. Elected representatives would formally define and manage an after-the-fact measurement of national welfare, while market speculators would say which policies they expect to raise national welfare. Democracy seems better than autocracy (i.e., kings and dictators), but it still has problems. There are today vast differences in wealth among nations, and we can not attribute most of these differences to either natural resources or human abilities. Instead, much of the difference seems to be that the poor nations (many of which are democracies) are those that more often adopted dumb policies, policies which hurt most everyone in the nation. And even rich nations frequently adopt such policies. These policies are not just dumb in retrospect; typically there were people who understood a lot about such policies and who had good reasons to disapprove of them beforehand. It seems hard to imagine such policies being adopted nearly as often if everyone knew what such "experts" knew about their consequences. Thus familiar forms of government seem to frequently fail by ignoring the advice of relevant experts (i.e., people who know relevant things). Would some other form of government more consistently listen to relevant experts? Even if we could identify the current experts, we could not just put them in charge. They might then do what is good for them rather than what is good for the rest of us, and soon after they came to power they would no longer be the relevant experts. Similar problems result from giving them an official advisory role. "Futarchy" is an as yet untried form of government intended to address…

Robin Hanson
· 44 min · Original

The Great Filter — Are We Almost Past It?

The Great Filter - Are We Almost Past It? Sept. 15, 1998 by Robin Hanson Humanity seems to have a bright future, i.e., a non-trivial chance of expanding to fill the universe with lasting life. But the fact that space near us seems dead now tells us that any given piece of dead matter faces an astronomically low chance of begating such a future. There thus exists a great filter between death and expanding lasting life, and humanity faces the ominous question: how far along this filter are we? Combining standard stories of biologists, astronomers, physicists, and social scientists would lead us to expect a much smaller filter than we observe. Thus one of these stories must be wrong. To find out who is wrong, and to inform our choices, we should study and reconsider all these areas. For example, we should seek evidence of extraterrestrials, such as via signals, fossils, or astronomy. But contrary to common expectations, evidence of extraterrestrials is likely bad (though valuable) news. The easier it was for life to evolve to our stage, the bleaker our future chances probably are. Introduction Fermi, Dyson, Hart, Tipler, and others [Finney & Jones, Dyson 66, Hart 75, Tipler 80] have highlighted the relevance to SETI (the search for extraterrestrial intelligence) of the "The Great Silence" [Brin 83] (also known as the Fermi paradox), the fact that extraterrestrials haven't substantially colonized Earth yet. What has not yet been sufficiently highlighted or adequately analyzed, however, is the relevance of this fact for much bigger choices we now make. The Great Silence must force us to revise a standard view in one or more area of biology, astronomy, physics, or the social sciences. And some of these revisions strongly suggest that humanity be much more wary of possible…