2017-10-30 RRG Notes
- Signalling is about showing off
- The whole point of signalling is to have costs
- Wearing an embarrassing T-shirt is a refusal to signal
- False: Wearing an embarrassing T-shirt is a signal that you don't care about wearing embarrassing T-shirts
- Signalling attempts to ensure honest communication:
- Signalling is meant to be costly for liars
- Driver's licenses - signal that you're qualified to drive
- Job market - potential employees have to signal that they're qualified for the position that they're interviewing for
- Job market signalling is costly for everyone, even people who are qualified
- Clothing - signals personality
- Doing nice things for friends
- The world is full of people pouring wealth into things whose only purpose is to signal wealth
- Is it really beneficial for society for people to see who is rich, who is poor, who is socially competent, who is smart, etc?
- You don't serve society by failing to signal because signalling well is part of winning
- Distinguish signalling "respectability" from working towards your cause
- My thoughts
- Katja misses an important part of signalling
- Signalling has two components - your action and others' interpretation
- Misses the fact that signalling is context dependent
- A Prada handbag in one context can be a positive signal, but can be a negative signal in another context
- Refusing to signal is itself a signal - shows that you have the resources to afford social opprobrium
- It's called "fuck-you money" for a reason
- Even effective charitable gifts have a signalling purpose - the Bill and Melinda Gates Foundation is named that for a reason
- We should try to harness signalling, not get rid of it
- The purpose of a fire alarm is to make it socially acceptable to acknowledge a fire
- People will remain in smoky conditions if there is no alarm
- People are bad at knowing what they believe, so they allow social pressure to override their better judgement
- If AGI seems far away, is it even worth doing research into AGI alignment?
- If we get 30 years' warning about aliens coming, we would start discussing what to do today - no one would advocate waiting until the aliens were six months away to start thinking about the problem
- History shows that key technological developments seem far away until they happen
- People on the cusp of powered flight were very uncertain about its feasibility
- Nobel-prize winning physicists were uncertain about whether it would be possible to get energy from nuclear fission
- Hindsight bias makes everything seem predictable
- Progress is driven by peak knowledge, not average knowledge
- If you're not in the field, you're going to be unaware or only dimly aware of how much the field has progressed
- Don't assume that your impression of a field is where the field is - it takes years for knowledge to percolate out
- Technological timelines are not easily foreseeable
- The future has different tools and can easily do things considered difficult today and can do things with difficult that are considered impossible
- We think that AGI is decades away for the following reasons
- Don't know how to get AGI with present technology
- Even getting the impressive results that we do have is really hard
- Current AI systems are still really dumb in a lot of ways
- In machine learning, once something is possible, it's only a short amount of time before it's easy
- The experience of researchers on the cutting edge of AI blinds them to how solutions, once discovered, become widely available
- Most of the discourse on AGI being far away isn't being driven by actual models
- It's easy to be skeptical of AGI, but it's hard to name things that are impossible over even short timelines
- The confidence that people have in AGI being far away isn't thought out
- The signs of imminent AGI will be subtle and debatable
- AlphaGo was a signal of AGI in a way that Deep Blue wasn't - AlphaGo used techniques that are more generalizable than Deep Blue's
- Experts will only believe that AGI is imminent if
- They personally can see how to build AGI with current tools
- They personal jobs given them a sense that AGI is easy to build
- When they are impressed with their AI being smart in a way that feels "magical"
- However, once you have these things, you already have AGI and it's too late
- We will never have the clear signal that tells us how many years out AGI is
- The choice to delay action until a future alarm is reckless enough that it invokes the law of continued failure
- Any civilization competent enough to deal with AGI once an alarm goes off is competent enough to not wait for the alarm in the first place
- If we were serious about addressing AGI later, we'd be reviewing the state of the art in AI every six months to see if we're on some kind of threshold
- My thoughts
- Eliezer is assuming that the main problem is not talking about AGI before AGI arrives
- But what if AGI doesn't show up?
- He talks about all the times that people failed to predict technological advances, but ignores the predicted advances that didn't happen
- Fusion power
- Advances resulting from genetics and proteomics
- Nanotechnology
- It seems that Eliezer is assigning zero cost to "embarrassment" - doing a bunch of AGI alignment research that turns out to be premature is fine too
- I'm not so sure that he's correct in this
- Other things for which there won't be a fire alarm
- Cascadia earthquake
- Yellowstone supervolcano
- Solar flares
- What makes AGI special? It doesn't matter if I'm dead from something that kills humanity, kills North America, kills the United States, kills the Pacific Northwest, or kills me in particular. I'm dead either way.
- Higher cognitive functions have two modes
- Bias world towards certain outcomes
- Appreciation of the structural symmetry in the Universe
- Bias and Symmetry
- Bias: active force bending things to our will
- Make territory fit map
- Too much bias causes people to either freak out or delude themselves into thinking that everything's okay
- Symmetry: fundamentally reflective tendency
- Update map to fit territory
- Too much symmetry leads to inaction - everything true is good; everything good is true
- Bias is social settings helps to persuade people to see things your way ("reality distortion field")
- Too much bias in social settings results in narcissism or borderline personality disorder
- However we need some level of bias in order to have motivation to do things
- Mindfulness meditation works by reducing our tendencies to bias and increasing our tendencies to appreciate symmetry
- The order of the soul
- The Bhagavad Gita talks about 3 tendencies that everyone has
- Sattva - wisdom, harmony, purity
- Rajas - activity, ambition
- Tamas - ignorance, chaos
- This tripartite model is there is a number of other philosophical traditions
- Everyone agrees that the bottom is sensual appetites and the middle is self-assertion
- However, Freud and his followers disagree with Plato and Bhagavad Gita on the top level
- Freud sees the superego as the internalized voice of authority figures
- Corresponds to Julian Jeynes' model of a bicameral mind
- In contrast sattva or Plato's logos can be analogized to thinking at the meta-level
- Good at school
- Benjamin had trouble teaching someone goal factoring because it resembled the pointless goal-oriented exercises that one does at school
- Schools have poisoned people's experiences with all sorts of subjects
- This is because formal education has a curriculum and does not tolerate people's desires to go and learn topics that interest them
- Marshmallow test is a test of the desire to pass tests, not of innate willpower
- The primary thing that schools teach is obedience
- Precepts and concepts
- Most students in school aren't thinking conceptually
- They're trying to figure out what's necessary to pass the test
- But life isn't a test
- My Thoughts
- What a lot of words for such little content
- Insight porn at its finest
- Why is this a single blog post, instead of two or three - sections aren't well tied to one another
- The Arendt example is just wrong - As Eichmann Before Jerusalem shows, Eichmann deliberately portrayed himself as colorless, faceless administrator because it was, in his estimation, his best chance at acquittal
- Papers from Argentina show that Eichmann was the leader of the exiled Nazis
- He, along with his peers, were actively trying to figure out a way to restore National Socialism
- As late as the 1950s, Eichmann was writing justifications for why the Final Solution was both legally and morally justifiable, using contemporary examples like the conflict between the Israelis and the Palestinians
- More generally, I don't think that there's enough rigor in schools, not that there's too much rigor
- Personally, I was very unprepared for college level math and college level writing because my high school was too eager to let kids 'do things their own way'
- It can be helpful to cluster your motivations and assign a persona to each cluster
- Recognize that each of your motivations has a role and a purpose
- Might be helpful for you to be more explicit about giving different parts of yourself a chance to be at the forefront
- My thoughts
- This is mostly an example
- Questionable generality
- Tim Harford - The Problem With Facts
- Argues that people are mostly impervious to facts and logic
- Talks about the "backfire effect" - probably not true
- Agnotology - the deliberate production of ignorance
- Solution to agnotology is unconvincing
- Both sides are equally capable of telling convincing stories
- The subtext to Harford's article is that the ingroup acknowledges facts, but the outgroup doesn't
- But there is no weird tribe of "fact immune troglodytes" out there
- The focus on transmission is part of the problem - implies that people will be immediately be convinced once they hear the fact
- The problem isn't that the outgroup is impervious to facts and logic, it's that there's no honest debate happening at all
- What constitutes "honest debate"
- Bilateral communication - two people are actually communicating with each other
- Both people have chosen to enter
- Spirit of mutual respect and truth-seeking
- Outside of a high-pressure "point-scoring" environment
- Single topic and try to stick to the topic at hand
- Even then, it takes more than a single conversation to get people to change their minds
- The closest thing Scott has seen to honest debate is cognitive psychotherapy
- There's no single moment of blinding revelation
- The SSC comments section has multiple examples of Trump supporters who said that the SSC post caused them to at least question their support of Trump
- The problem with debate in the style described above is that it doesn't scale
- Is there anything the media can do?
- Treat disagreement as a need to collaborate to investigate the question further
- Assume good faith of the other side
- Engage in adversarial collaboration - join with someone from the other side to explore questions with a mutually agreed-upon methodology
- Why should we engage in logical debate?
- Logical debate is an asymmetric weapon - stronger in the hands of those who are on the side of truth
- Rhetoric and violence are both symmetrical - whether they work is entirely dependent on who can tell the most compelling story or who can gather the most soldiers
- Unless you use asymmetric weapons, the best you can hope for is chance - sometimes you have more persuasive stories or more soldiers and sometimes they do
- Improving the quality of debate is a painful process
- But, we don't have to go very far to be effective - only need to convince ~2% of people in order to flip elections
- If you genuinely didn't believe facts didn't convince people, why are you even bothering to study facts instead of entering a state of pure Cartesian doubt?
- Ultimately, the other side isn't that different from you; the same facts and logic that worked on you will work on them, given time
- The only long term path to progress is raising the sanity waterline
- My thoughts
- People didn't reject fascism after reasoned and considered debate, they rejected it because of a little thing called World War 2
- Assuming good faith assumes that the other side will consent to deal with you
- There are powerful forces on both sides pushing for group solidarity instead of cross-group collaboration
- Scott doesn't consider the problem that your own side might sabotage your efforts - remember, Yitzhak Rabin was shot by an Israeli nationalist and Gandhi was shot by a Hindu nationalist
