Social Science++

This blog has moved to anyall.org/blog

2008-10-08T13:53:00.000-07:00

I've finished moving this blog to:

anyall.org/blog

All pages on the old blog redirect to their equivalents over there.

Update your bookmarks and feeds! New posts won't appear here any more.

MyDebates.org, online polling, and potentially the coolest question corpus ever

2008-10-07T17:47:00.000-07:00

MySpace and the Commission on the Presidential Debates put together a neat site, mydebates.org, which presents the candidates' positions through various mini-polls and such. It even has a cool data exploration tool for the poll results ... for example, here are two support maps, one for respondents over 65 and one for 18-24 year olds.

Anyway, the site also takes submissions of questions for tonight's debate. Apparently six million questions were submitted, and moderator Tom Brokaw will of course use only 10 or so. This begs a question, how were they selected? There's no Digg-like social filtering or anything. You could imagine automatic methods to help narrow down the pool: Topic clustering? Quality ranking on syntax and vocabulary?

Eric Fish suggested the obvious: probably someone picked 1000 randomly and sent them to Brokaw.

I'd love to see a corpus of 6 million questions on U.S. political subjects, directed at only two different people. Anyone know anyone who works at MySpace or CPD?

Time to watch the debate! (Alas, no PalinSpeak liveblog this time, of course.)

PalinSpeak.com

2008-09-29T21:14:00.001-07:00

With my friend Doug, I just finished making a game -- PalinSpeak.com -- where you can chat with a Sarah Palin simulator. Check it out, it's the best thing to hit the Internet since sliced bread.

I'll post more the technical details (n-gram generation and query-answer matching, hurrah!) later...

"Machine" translation/vision (Stanford AI courses online)

2008-09-18T01:42:00.000-07:00

The Stanford Engineering school has put up videos and course materials for several programming, AI, and optimization courses online. They did get some of the ones that are taught by excellent lecturers -- e.g. introductory programming (the CS dept has craploads of money, so can afford to hire specialist lecturers, which results in very good courses), and Brad Osgood on the FFT (he's just such a good lecturer).

Main link, minor link.

I was looking through the transcript of Chris Manning's introductory lecture for CS224N, Natural Language Processing, last year. (SEE link; actual website link.) I took this same course years ago as a sophomore, and this part sounded familiar:

So if you look at the early history of NLP, NLP essentially started in the 1950s. It started just after World War II in the beginning of the Cold War. And what NLP started off as is the field of machine translation, of can you use computers to translate automatically from one language to another language? Something that’s been noticed about the field of computing actually is that you can tell the really old parts of computing because the old parts of computer science are the ones that have machine in the name.

I wonder if it's in the zeitgeist and I heard it from somewhere else? Sounds right though.

Fukuyama: Authoritarianism is still against history

2008-08-25T10:28:00.001-07:00

The latest on the world ideologies front --

In the light of Russia's Georgia adventures, there's been lots of talk whether this represents a new rise of authoritarian Russia, which is presumably another nail in the coffin for U.S.-led liberal democratic hegemony in the world. Our "end of history" friend Francis Fukuyama just wrote an op-ed arguing that Russia and China are still not big threats to liberal democracy. There are some good points: Russia is behaving as an aggressive imperial power, but does not embrace a grand, exportable ideology with universal appeal. Similarly with China. They both still feel the need to pay lip service to democratic rituals and norms. Even Nicholas Kristof's hilarious column chronicling his experience with China's dubious protest registration system concludes that even a pale mockery of democracy is progress.

I still like Azar Gat's article which I wrote about last year, that Russia and China represent authoritarian capitalism, which will be an effective alternative to liberal democracy. Sure, it's not a war of ideologies, he argues, but now it looks like big successful nations can economically succeed without being very democratic. Furthermore, this should encourage others to not bother with democracy.

A better Obama vs McCain poll aggregation

2008-08-16T12:17:00.001-07:00

Update: Charles Franklin (of Pollster.com) kindly emailed me with many interesting points on this post. One important note is that my technique isn't really "no smoothing" -- rather, there is now implicit smoothing within the polling houses, by assuming that responses are evenly distributed across the time interval of the poll.

I was looking at Pollster.com's page that aggregates many opinion polls on the Presidential race. Here, they have a chart that shows the many polls plus lowess fits:

So there's a trend of Obama recently declining. But it wasn't clear to me that the fitted curve was correct. I downloaded the data and started playing around with it.

Here are several more graphs I made, with different smoothing parameters for the lowess fit. Your interpretation completely changes depending which smoothing parameter you like best!

Well, maybe this is an argument to use rolling averages over a fixed number of days or something. But it would be nice to directly look at the data with a minimum of extrapolation or smoothing, since they can destroy or mask effects.

Turns out this is possible with this data set by using single-day smoothing. Every poll is taken over a range of days and the table says how many respondents there were, so I did a day-by-day calculation: take the weighted average of all polls on that day, assuming that a poll's responses were evenly distributed over its date range given. (The weighted average across the day's polls just means, it's as if on that day there was one big poll and we're just calculating the Obama vs McCain percentages for it.) And to further clean things up, I omitted polls that had "Don't plan to vote" numbers and only included ones with undecideds. ("NV" responses wildly differ over time, so I figured if you ask it on a poll it must skew things. Not sure though. This cleanup step might be superfluous.) I know I just argued smoothing is bad and now I'm doing it, but note that the time resolution for this polling data is only at the day-by-day level, so I'm not doing any smoothing across time windows. What's significant is aggregating across polls, binning per-day.

Anyway, I can now see some trends on the direct scatterplot. I'm only plotting the Obama percentage:

The amount of noise significantly goes down. It makes clearer there's been a negative trend for Obama recently.

On the perils of smoothing or not -- below is Gallup's graph from their daily tracking poll, which uses a 3-day rolling average. But below that are two graphs from Alan Abramowitz, attacking the day-by-day interpretations out there -- the left shows daily with no smoothing, but the right is with a 10-day rolling average.

The Huffington Post article claims the 10-day one is the safest to interpret -- surely the single-day one seems too noisy -- but honestly I think it's hard to say what amount of smoothing is acceptable. (Be careful comparing those new graphs vs the official Gallup one, as they're for different dates, I believe a week and a half off or so. The 3-day vs 10-day is somewhat similar over the same date ranges, I think?) It's so much nicer if you don't have to do smoothing across time windows, which is why I like my graph :). Of course, it would be best if political polling had better overall methodology and there was less noise, but that's another (big) post...

If you're interested in my code and the data, I threw it up at gist.github.com/5754.

Finally, there are interesting tie-ins between this poll methodology stuff and Mechanical Turk statistical aggregation. You can think of polling as a similar problem to data annotation: there is a true quantity out there in the world (the percentage of vote for Obama, or whether a certain email is spam or not) and annotators/polls are noisy signals somewhat reflecting that true value. Under certain assumptions, simple averaging is the best technique to estimate the true value. A new Dolores Labs blog post on this is coming ...

East vs West cultural psychology!

2008-08-14T19:31:00.001-07:00

Great anti-pop-science article of the moment -- Mark Liberman does a take-down of David Brooks' apparently careless column on cultural psych experiments that purport to show that East Asians are collectivist while Westerns are individualist.

From Liberman:

Question to Language Log: Is it correct that if you show an American an image of a fish tank, the American will usually describe the biggest fish in the tank and what it is doing, while if you ask a Chinese person to describe a fish tank, the Chinese will usually describe the context in which the fish swim?

Answer: In principle, yes. But first of all, it wasn't a representative sample of Americans, it was undergraduates in a psychology course at the University of Michigan; and second, it wasn't Chinese, it was undergraduates in a psychology course at Kyoto University in Japan; and third, it wasn't a fish tank, it was 10 20-second animated vignettes of underwater scenes; and fourth, the Americans didn't mention the "focal fish" more often than the Japanese, they mentioned them less often.

In fairness, I have sympathy for Brooks' style of presenting interesting results in a provocative way. In a short opinion piece you have to aggressively summarize. But I also like science that's not totally intellectually irresponsible. Not yet sure where this work stands...

The MacGyver of data analysis

2008-07-11T15:21:00.001-07:00

Jeff Hammerbacher, who runs Facebook's data infrastructure and insight team, is leaving the company. That's too bad for them, considering a hilarious quote from a talk he gave:

Basic statistics is more useful than advanced machine learning.

I can't tell you how many interviews I've had where someone has a really cool project on their resume. Support vector machines, topic analysis on CiteSeer, or whatever... But what it boils down to is someone took toy data set A and plugged it in to machine learning library B and took the output and was like, “sweet.”

People with "machine learning" on their resume fall from the sky these days, it seems to be a very sexy discipline. The problem is if I ask them explain a t-test, those same people can't tell me what that is.

If I had a MacGyver of data analysis and all he had was a t-test and regression, he would probably be able to do 99.9% of the analyses that we do that are actually useful.

Amen. That, with the importance of data visualization, is one of the best lessons I learned working with real data in the last year or two. Here's the entire video, in which he also talks about Hadoop, Scribe, Hive (a data warehousing and analysis platform they've built on Hadoop), and other fun things. The above bit is around 35:00.

(Direct link, hopefully, though the website is weird)

There's lots more to say on statistics vs. machine learning and all that. For another post...

Link: Today's international organizations

2008-07-03T19:35:00.000-07:00

Fascinating -- a review of the current international system, focusing on international organizations (that is, organizations of states). Who runs the world? | Wrestling for influence | Economist.com

Bias correction sneak peek!

2008-07-01T12:50:00.000-07:00

I really don't have time to write up an explanation for what this is so I'll just post the graph instead. Each box is a scatterplot of an AMT worker's responses versus a gold standard. Drawn are attempts to fit linear models to each worker. The idea is to correct for the biases of each worker. With a linear model y ~ ax+b, the correction is correction(y) = (y-b)/a. Arrows show such corrections. Hilariously bad "corrections" happen. *But*, there is also weighting: to get the "correct" answer (maximum likelihood) from several workers, you weight by a^2/stddev^2. Despite the sometimes odd corrections, the cross-validated results from this model correlate better with the gold than the raw averaging of workers. (Raw averaging is the maximum likelihood solution for a fixed noise model: a=1, b=0, and each worker's variance is equal).

Much better explanation is coming... will be a blog.doloreslabs.com post I think.

Picture!

Turker classifiers and binary classification threshold calibration

2008-06-18T02:25:00.001-07:00

I wrote a big Dolores Labs blog post a few days ago. Click here to read it. I am most proud of the pictures I made for it:

Pairwise comparisons for relevance evaluation

2008-06-17T10:25:00.000-07:00

Not much on this blog lately, so I'll repost a comment I just wrote on whether to use pairwise vs. absolute judgments for relevance quality evaluation. (A fun one I know!)

From this post on the Dolores Labs blog.

The paper being talked about is Here or There: Preference Judgments for Relevance by Carterette et al.

I skimmed through the Carterette paper and it’s interesting. My concern with pairwise setup is, in order to get comparability among query-result pairs, you need to get annotators to do an O(N^2) amount of work. (Unless you do something horribly complicated with partial orders.) The absolute judgment task scales linearly, of course. Given the AMT environment and a fixed budget, if I stay in the smaller-volume task, instead of spending a lot on a quadratic taskload, I can simply get a higher number of workers per result and boil out more noise. Of course, if it’s true the pairwise judgment task is easier — as the paper claims — that might make my spending more efficient. But since it’s polynomial, no matter the cost/benefit ratios, there has to be a tipping point where, for a given data set size, you’d always want to switch back to absolute judgments.

Absolute judgments are just so much easier to compute with — both for analysis and to use as machine learning training data. I really don’t want to have fancy utility inference or stopping rule schemes just to know the relative ranking of my data. (And I think real-valued scores will always become a necessity. Theoretical microeconomists have made boatloads of theorems about representing preferences by pairwise comparisons. It turns out that when you add enough rationality assumptions — e.g. the sort that are demanded of search engine ranking tasks anyways — then your fancy ordering can always be mapped back to real-valued utility function.)

I’d be most interested in a paper that compares real-valued scores derived from some sort of pairwise comparison task, versus absolute judgments, and is mindful of the cost tradeoffs in service of an actual goal, like ranking algorithm training.

Clinton-Obama support visualization

2008-06-05T12:21:00.000-07:00

This interactive histogram is brilliant. The NYT data visualization folks never fail to impress.

margins.swf (application/x-shockwave-flash Object)

Sub-reddit for Systems Science and OR

2008-05-23T15:22:00.000-07:00

I've been a big fan of Reddit's Programing subsite for a while. Just this morning I found another sub-reddit:
SYSOR: Systems Science, Operations Research and Everything In Between, and I'm loving it. Lots of links on data mining, graph software, image recognition, learning theory, etc etc.

Not sure if the Operations Research part in the title is so big now -- there's some sort of complicated reorganization of a number of fields including operations research, systems science, computational learning, and more general computer science areas. I'm a fan of the great historical overview in the Introduction in Rusell and Norvig's AI book; I'm sure it's slanted in various ways, but what else are overarching narratives for? :)

conplot - a console plotter

2008-05-19T00:02:00.000-07:00

This has to be the most quick-and-dirty data visualizer out there: I wrote an ascii art plotter script that takes a column of numbers on stdin and throws out a plot on your console. I've been using it for several months to quickly look at numbers on the commandline, especially from logs and such. (Back in school I would use gnuplot for this; R is good too. But sometimes you want to move really fast, esp if you have a few hideous perl -pe one-liners on your hands and mucking around with temp files will interrupt your flow.)

Link: github.com/brendano/conplot

"Demo":


$ cat time.log | conplot                                            
14601                                                                     
                                                                         oooooooo
                                                                    oooooo
                                                            ooooooooo     
                                                 oooooooooooo             
11269                                     oooooooo                        
                                       oooo                               
                                     ooo                                  
                                  oooo                                    
                                 oo                                       
                               ooo                                        
7271                       ooooo                                          
                        oooo                                              
                     oooo                                                 
                  oooo                                                    
               oooo                                                       
            oooo                                                          
3272       oo                                                             
           o                                                              
           o                                                              
          oo                                                              
        ooo                                                               
      ooo                                                                 
-726  0                                                                   76826

I must say, it's way easier to throw up some code on GitHub than on to SourceForge, which is the only other open source code hosting service I've used. I guess Google Code is their biggest competitor in that respect; I haven't tried it.

The best natural language search commentary on the internet

2008-05-13T02:24:00.001-07:00

With Powerset's launch, there's an awful lot of hot air and crappy blog posts about natural language search being written. Instead of contributing to that mess, I prefer to direct the reader to the best writing on the topic that I've seen: Fernando Pereira's posts on search.

Are women discriminated against in graduate admissions? Simpson's paradox via R in three easy steps!

2008-04-13T02:04:00.000-07:00

R has a fun built-in package, datasets: a whole bunch of easy-to-use, interesting tables of data. I found the famous UC Berkeley admissions data set, from a 1970's study of whether sex discrimination existed in graduate admissions. It's famous for illustrating a particular statistical paradox. Thanks to R's awesome mosaic plots interface, we can see this really easily.

UCBAdmissions is a three-dimensional table (like a matrix): Admit Status x Gender x Dept, with counts for each category as the matrix's values. R's default printing shows the basics just fine. Here's the data for just the first of six departments:

> UCBAdmissions
, , Dept = A

          Gender
Admit      Male Female
  Admitted  512     89
  Rejected  313     19

...

Overall, women have a lower admittance rate than men:

> apply(UCBAdmissions,c(1,2),sum)

          Gender
Admit         M    F
  Admitted 1198  557
  Rejected 1493 1278

This is the phenomenon that prompted a lawsuit against Berkeley which prompted the study that collected this data.

R's plot function is overloaded to do a mosaic plot for this sort of categorical data. Very cool. With just

> plot(UCBAdmissions)

or, playing around after reading Quick-R's page on this:

> install.packages(”vcd”)
> library(vcd)
> mosaic(UCBAdmissions, condvars=c('Dept'))

We have a plot showing admittance and gender breakdowns per department:

In each department, women have similar admittance rates as men. This seems to be at odds with the fact that women have a lower admittance rate overall. This discrepancy is an example of Simpson's paradox.

This mosaic also shows the explanation: Selective departments have more female applicants. It's easy to see since the departments are ordered by selectiveness. Departments A and B let in many applicants, but they're mostly male. The reverse is true for the rest. This means that the overall female population takes big admittance hits in departments C through F, while lots of males get in via departments A and B.

I think these mosaic plots are impressive for visualizing categorical proportions for high dimensional data sets. Well, by “high” I think I mean, more than 2. I can't think of a better way to see several cross relationships in categorical data at once. And the only tuning I needed to do was play around a bit with the order of those three dimensions.

Sources:

R's UCBAdmissions help page. It comes with the standard download of R.
R's vcd::mosaic function. I recommend the pdf vigenette about it, which has many more pictures of cool mosaic plots.
I would post the original 1975 Science paper, but it's not freely available. I hate academic publishers.

a regression slope is a weighted average of pairs' slopes!

2008-04-05T17:04:00.001-07:00

Wow, this is pretty cool:

From an Andrew Gelman article on summaring a linear regression as a simple difference between upper and lower categories. I get the impression there are lots of weird misunderstood corners of linear models... (e.g. that "least squares regression" is a maximum likelihood estimator for a linear model with normal noise... I know so many people who didn't learn that from their stats whatever course, and therefore find it mystifying why squared error should be used... see this other post from Gelman.)

Datawocky: More data usually beats better algorithms

2008-04-02T12:46:00.000-07:00

This is a great post. I think I've seen it from several sources already...

Datawocky: More data usually beats better algorithms

Allende's cybernetic economy project

2008-03-29T12:09:00.000-07:00

Wow -- teletype machines and cybernetics to run an economy!

Before ’73 Coup, Chile Tried to Find the Right Software for Socialism - New York Times

(note they mean this version of the word "cybernetics")

And here's a better Guardian article on it.

The control room:

I suspect this happens anyway today -- using computers to help decisionmakers run the economy -- but without the cool Star Trek chairs and display screens. Economists at central banks get data and use computers to analyze it, then eventually the data is used to inform decisions.

Though this vision involves more fine-grained data collection and automated analysis. (And other more Internet-y things like two-way communication between workers and management/government.). I suspect it would be way easier to do today, with better computational infrastructure (CPU, memory, data transmission, and software are better these days).

Quick-R, the only decent R documentation on the internet

2008-03-23T18:48:00.000-07:00

For R users or wannabes...

I really love R, but it has horrid documentation and a steep learning curve. Recently I was introduced to Quick-R, a really excellent documentation site. I think it's made the system dramatically more useful for me.

Spending money on others makes you happy

2008-03-20T16:08:00.000-07:00

Yes, Money Can Buy Happiness . . . - TierneyLab - Science - New York Times Blog

color name study i did

2008-03-18T09:54:00.000-07:00

Where does “Blue” end and “Red” begin?

I'm writing some posts on blog.doloreslabs.com and this is the best one so far. Methodology-wise, along the lines of my earlier Amazon Mechanical Turk moral decisions survey...

PHD Comics: Humanities vs. Social Sciences

2008-03-09T19:11:00.000-07:00

PHD Comics: Humanities vs. Social Sciences

data data data

2008-03-05T17:08:00.000-08:00

This is a lot of data:

Inductio Ex Machina - A Meta-index of Data Sets

Food Fight

2008-01-30T23:21:00.000-08:00

Absolutely amazing -- a short film chronicling conflicts from World War II -- as food.

I think this has to have the highest amount of Wikipedia-linkable references per second of any film I've seen. Yes, it's U.S.-centric, but so is Wikipedia, which makes cataloguing it easier. At the very least:

World War II
- Persecution of Jews and The Holocaust
- Invasion of France
- The Blitz
- Pearl Harbor, Pacific Theater
- D-Day
- Liberation of France
  Invasion of Germany - Western and Eastern fronts
- Atomic bombing of Hiroshima
1948 Arab-Israeli War
Korean War
Cuban Missile Crisis
Vietnam War
- French Indochina War
- U.S. involvement
US/USSR nuclear arms race
First Gulf War
- Invasion of Kuwait
Israeli-Palestinian conflict
- Judging from its timing in the film, maybe specifically the First Intifada?
9/11 attacks
War in Afghanistan
- Taliban falls, but some escape, with Osama bin Laden
2003 Iraq war and aftermath

Many lessons learned about food violence, 20th century war, and Wikipedia meronymic article relationships...

Graphics! Atari Breakout and religious text NLP

2008-01-27T12:58:00.000-08:00

From a graphics/mod programming workshop, modifications of "Breakout" in awesome video form:

It uses Processing, a framework designed for animation and graphicky things. It was also used for the Similar Diversity visualization that maps out named entities and their common verbs across religious texts. It's pretty cool:

... so a name gets bigger with its frequency, I get that; but what governs the size of an arc connecting two names? They say "common activities" -- I'm guessing a distance measure on the terms in the contexts in which they appear? I'd be most interested in a graph of their explicit interactions; something like a count of co-occurrences in a proximity window (word radius? verse? chapter? book?) might do the trick...

This came out a while ago, I think; here are some related works posted on infosthetics.com.

Moral psychology on Amazon Mechanical Turk

2008-01-19T17:44:00.000-08:00

There's a lot of exciting work in moral psychology right now. I've been telling various poor fools who listen to me to read something from Jonathan Haidt or Joshua Greene, but of course there's a sea of too many articles and books of varying quality and intended audience. But just last week Steven Pinker wrote a great NYT magazine article, "The Moral Instinct," which summarizes current research and tries to spell out a few implications. I recommend it highly, if just for presenting so many awesome examples. (Yes, this blog has poked fun at Pinker before. But in any case, he is a brilliant expository writer. The Language Instinct is still one of my favorite popular science books.)

For a while now I've been thinking that recruiting subjects online could lend itself to collecting some really interesting behavioral science data. A few months ago I tried doing this with Amazon Mechanical Turk, a horribly misnamed web service that actually lets you create web-based tasks and pay online workers do them. Its canonical commercial applications include tedious tasks like search quality evaluation or image labeling, where you really need human data to perform well. You put up, say, several thousand images you want classified as "porn" or "not-porn", say you'll pay workers $0.01 to label ten images, then sit back and watch the data roll in.

So AMT advertises itself as a data annotation or machine learning substitute system, but I think its main innovation is finding out that there are lots and lots of people with free time willing to do online work for very, very low amounts of money. You can run any task you want, including surveys, and people happily respond for mere pennies. (Far below minimum wage, I might add -- their motivation seems to be more like casual gaming or so.) To that end, I tried out running one of the standard moral psych survey questions to see what would happen -- the so-called "trolley problem":

A runaway trolley is hurtling down a track towards five people who have been tied down in its path. If nothing happens, they will be killed. Fortunately, you have a switch which would divert the trolley to a different track. Unfortunately, the other track has one person tied down to it. Should you flip the switch?

It's supposed to be a classic dilemma of consequentialist vs. deontological moral reasoning. Is it acceptable to sacrifice for the greater good? Is it permissible to take an action that will cause a preventable death? And so on. I think it's neat just because when I pose it to people, different folks really do disagree, give different answers, and are willing to argue about it. There are some interesting recent fMRI findings (due to Greene I think?) that people who refuse to flip the switch seem to be engaged in a more emotional response, whereas those who do seem to be using deliberative reasoning systems. (Some, like Greene and Pinker, seem to go further and argue this is a substantive normative reason to favor flipping the switch; whether you feel like getting sucked into that debate, though, there's clearly something interesting happening here.)

So I ran this on AMT; the particpants (they call themselves "turkers") had to answer yes or no. Turns out 77% say they'd flip the tracks.

I also ran two variant scenarios of the same logical dilemma, to sacrifice one person to save five:

A trolley is hurtling down a track towards five people. You are on a bridge under which it will pass, and you can stop it by dropping a heavy weight in front of it. As it happens, there is a very fat man next to you - your only way to stop the trolley is to push him over the bridge and onto the track, killing him to save five. Should you proceed?

and

A brilliant transplant surgeon has five patients, each in need of a different organ, each of whom will die without that organ. Unfortunately, there are no organs available to perform any of these five transplant operations. A healthy young traveler, just passing through the city the doctor works in, comes in for a routine checkup. In the course of doing the checkup, the doctor discovers that his organs are compatible with all five of his dying patients. Suppose further that if the young man were to disappear, no-one would suspect the doctor. Should the doctor sacrifice the man to save his other patients?

These two, of course, feel a lot harder to say "Yes" to, but if you were willing to say "Yes" to the original question, it is hard to justify why. The participants' repsonses followed what you would expect: fewer said "Yes" to these scenarios. Here are the Yes/No responses to each of the questions (100 responses for each):

Question	Yes	No
surgeon	2	98
fat man	30	70
switch, save 5	77	23
switch, save 10	82	18
switch, save 15	83	17
switch, save 20	83	17

Only two people thought it was acceptable to sacrifice for organs, and only half as many would push the fat man as would flip the switch. I also ran variants of the switch version with more and more people on the tracks; the Yes response creeps upwards but never reaches 100%. The differences among the first three questions are statistically significant (unpaired t-tests, all p<.001 (this seems like the wrong test, can anyone correct me?)).

What's amazing is how fast responses happen. I started getting responses just minutes after posting the question. I actually posted each of the six questions as a separate, standalone task; but many of the turkers who did one found the rest in the task pool and did them too. (So what was supposed to be a between-subjects design fell into something else, oops!) The whole thing cost $6 and was done in a matter of hours. It's very encouraging -- AMT allows you to very quickly iterate and try out different designs and such. It's a bit of a pain to use, though; Amazon has certainly done a poor job in exploiting its full potential. (They have a form builder which was good enough to quickly write up these tasks, but to do anything moderately sophisticated, even just getting your data back out, you have to write programs against their somewhat mediocre API; you have to know how to use an XML parser, etc. Hm.)

I also tried an explicitly within-subject version, where each participant answered the three basic versions. I was interested in consistency -- presumably very few people would sacrifice for organs but refuse to divert the trolley. For 141 participants, here are the frequencies of the different answer triples:

% with this response triple	flip switch?	push fat man?	sacrifice traveler for organs?
42.6	Y	N	N
29.8	Y	Y	N
20.6	N	N	N
5.0	Y	Y	Y
0.7	Y	N	Y
0.7	N	Y	Y
0.7	N	Y	N

I personally find the most common responses coherent with my own gut reactions -- from left to right, I feel less and less good about sacrificing in each case. Perhaps all people feel the same gut reactions, and use different ad hoc reasons to draw the line in different places?

I'm sorry that this post started with neat moral psychology then degenerated into methodology, but hey it's fun. I've seen only two instances of any sort of research paper being written using AMT, both by computer scientists; here's a nice blog post on an information retrieval experiment (it's a great blog, btw); and someone mentioned to me this one on data processing accuracy also. Anyone know of any? It's clearly an interesting approach.

Will the humanities save us?

2008-01-06T22:22:00.000-08:00

Not so much, thinks Stanley Fish.

Indicators of a crackpot paper

2008-01-05T13:38:00.000-08:00

From Scott Aaronson: Ten Signs a Claimed Mathematical Breakthrough is Wrong.

I've often wondered how to decide whether a paper or book is worth my time to read. People who like a certain paper or book can always tell me that I shouldn't judge it until I read it. But I need some estimate of its quality/creativity/interestingness before I start. I've decided to be biased towards shorter things and more well-known things. I don't see any good alternative, unfortunately.

What is experimental philosophy?

2007-12-26T11:28:00.000-08:00

Experimental philosophy:

Suppose the chairman of a company has to decide whether to adopt a new program. It would increase profits and help the environment too. “I don’t care at all about helping the environment,” the chairman says. “I just want to make as much profit as I can. Let’s start the new program.” Would you say that the chairman intended to help the environment?

O.K., same circumstance. Except this time the program would harm the environment. The chairman, who still couldn’t care less about the environment, authorizes the program in order to get those profits. As expected, the bottom line goes up, the environment goes down. Would you say the chairman harmed the environment intentionally?

in one survey, only 23 percent of people said that the chairman in the first situation had intentionally helped the environment. When they had to think about the second situation, though, fully 82 percent thought that the chairman had intentionally harmed the environment. There’s plenty to be said about these interestingly asymmetrical results.
...
It’s part of a recent movement known as “experimental philosophy.”

This is pretty interesting. I'm wondering why it's "philosophy" though. Isn't this just experimental psychology, applied to topics of intention and theory of mind? And if you want to do it, wouldn't a psych program be better training for learning how to read fMRI papers and experimental design? But maybe a philosophy degree makes you smarter. (That's how I understood Richard Rorty's great review of Marc Hauser's book on moral psychology.)

Here's a good overview of a variety of work in the field; here are some more thoughts on what "x-phi" is. I suspect it's a thing special to analytic philosophy, which embroiled itself in all sorts of topics that rely heavily on appeals to intuition, but where empiricism might work a bit better. (E.g. any actual improvements in cognitive science should make philosophy of mind less important.)

Data-driven charity

2007-12-19T23:20:00.000-08:00

Some ex-hedge fund analysts recently started a non-profit devoted to evaluating the effectiveness of hundreds of charities, and apparently have been making waves (NYT). A few interesting reports have been posted on their website, givewell.net -- they make recommendations for which charities where donors' money is used most efficiently for saving lives or helping the disadvantaged.

(Does anyone else have interesting data on charity effectiveness? I've heard that evaluations are the big thing in philanthropy world now, and certainly the Gates Foundation talks a lot about it.)

Obviously this sort of evaluation is tricky, but it has to be the right approach. The NYT article makes them sound like they're a bit arrogant, which is too bad; on the other hand, any one who makes claims to have better empirical information than the established wisdom will always end up in that dynamic. (OK, so I love young smart people who come up with better results than a conservative, close-minded establishment. Or at least I'm a sucker for that story.)

This particular methodological criticism (from the article) struck me as odd:

“I think in general it’s a good thing,” said Thomas Tighe, president and chief executive of Direct Relief International, an agency that GiveWell evaluated but did not recommend. Like others in the field, however, Mr. Tighe has reservations about GiveWell’s method, saying it tends to be less a true measure of a charity’s effectiveness than simply a gauge of the charity’s ability to provide data on that effectiveness.

I think it's fine to penalize an organization for failing to provide data on its effectiveness. Isn't the burden of proof on them, to show that they're actually doing something useful? I guess it comes down to whether you believe empirical evaluation is necessary for organizational effectiveness. I believe this wholeheartedly.

The GiveWell people have an interesting argument that altruistic actions have a particularly poor feedback loop, which kills learning/optimization; therefore, you need to undertake explicit evaluative efforts. From their blog:

Now imagine an activity that consists of investing without looking at your results. In other words, you buy a stock, but you never check whether the stock makes money or loses money. You never read the news about whether the company does well or does poorly. How much would you value someone with this sort of experience - buying and selling stocks without ever checking up on how they do? Because that’s what “experience in philanthropy” (or workforce development, or education) comes down to, if unaccompanied by outcomes evaluation.

The peculiar thing about philanthropy is that because you’re trying to help someone else - not yourself - you need the big expensive study, or else you literally have no way of knowing whether what you did worked. And so, no way of learning from experience.

I really like this point -- which is easier to notice, that you're bankrupt or that someone else is? That your own business is doing well/badly, or that your beneficiaries are doing well/badly? Self-regarding actions get automatic evaluation but altruistic actions don't, presumably because, even if we care enough to give to others, we do not care enough to expend energy evaluating their outcomes down the line. But we really care about our own personal outcomes. Yet another example of human preferences being more selfish than altruistic; well, what's new?

Race and IQ debate - links

2007-12-09T14:39:00.000-08:00

William Saletan, a writer for Slate, recently wrote a loud series of articles on genetic racial differences in IQ in the wake of James Watson's controversial remarks. It prompted lots of discussion; here is an excellent response from Richard Nisbett, a leading authority in the field on the environmentalist side of the debate.

More academic articles: Rushton and Jensen's 2005 review of evidence for genetic differences; and what I've found to be the most balanced so far, the 1995 APA report Inteligence: Knowns and Unknowns which concludes for all the heated claims out there, the scientific evidence tends to be pretty weak.

Blog world: Funny title from Brad DeLong; and another Slate response to Saletan and Rushton/Jensen.

The politics of the race and intelligence question is a huge distraction from trying to find out the actual truth of the matter. But I suppose the political implications are why it attracts so much attention -- for good or bad.

The most interesting thing I learned is the Flynn Effect: IQ's as measured by standardized tests have been consistently rising, in all populations throughout the world ever since IQ tests were invented. This implies some sort of non-genetic determiners -- perhaps education and environmental factors -- can have a very large effect on intelligence. Here is a good overview from Ulric Neisser, the lead author on the APA report.

How did Freud become a respected humanist?!

2007-11-25T22:53:00.000-08:00

Freud Is Widely Taught at Universities, Except in the Psychology Department:

PSYCHOANALYSIS and its ideas about the unconscious mind have spread to every nook and cranny of the culture from Salinger to “South Park,” from Fellini to foreign policy. Yet if you want to learn about psychoanalysis at the nation’s top universities, one of the last places to look may be the psychology department.

A new report by the American Psychoanalytic Association has found that while psychoanalysis — or what purports to be psychoanalysis — is alive and well in literature, film, history and just about every other subject in the humanities, psychology departments and textbooks treat it as “desiccated and dead,” a historical artifact instead of “an ongoing movement and a living, evolving process.”

I've been wondering about this for a while, ever since I heard someone describe Freud as "one of the greatest humanists who ever lived." I'm pretty sure he didn't think of himself that way. If you're a crappy scientist but a decent writer, does that mean you get to be reincarnated as a humanist next? To my mind this doesn't bode well for the humanists, or for new potential Freuds in this regard.

The article duly notes that psychoanalysis as it lives in humanities academia is completely different than clinical psychoanalysis. Clinical psychoanalysis is now discredited because of its lack of empirical grounding. I guess outside of psych departments that's not an obstacle, thus psychoanalysis for gender studies and the like.

Some of the sentiments expressed in the article really irritate me though, like:

“Some of the most important things in human life are just not measurable,” he said, like happiness or genuine religious feeling.

Give me a break. There are great measurements of subjective happiness. It's even gone far enough to start studying its relation to welfare economics and policy implications. Sure, some of the brain work is at a pretty early stage, but measuring these things -- and pragmatically using this knowledge in the real world! -- can be done.

Freud, though, is particularly useful for gaining insights into questions of human existence. “There will be the discovery of problems that the standard ways don’t address,” he said, and then “there will be a swing back to Freud.”

I'll be waiting.

Actually that 2008 elections voter fMRI study is batshit insane (and sleazy too)

2007-11-15T02:01:00.000-08:00

A much more slashing commentary from Slate:

An op-ed from Sunday's New York Times, "This Is Your Brain on Politics," proposes to answer what must be the most vexing question of modern American politics: What's going on inside the head of a swing voter? The authors—a team of neuroscientists and political consultants—ran 20 of these undecided volunteers through a brain scanner and showed them pictures and video of the major candidates from both parties. The results, laid out both in print and an online slide show, purport to give us some insight as to how the upcoming primaries will play out: "Mitt Romney may have some potential," the researchers conclude, and Hillary Clinton seems to have an edge at winning over her opponents.

Don't believe a word of it. To liken these neurological pundits to snake-oil salesmen would be far too generous. Their imaging study has not been published in any science journal, nor has it been vetted by experts in the field; it can't rightly be called an "experiment," since the authors weren't testing any particular hypothesis; and the arbitrary conclusions they draw from the data aren't even consistent with their own previous research.

And they're funded by a sleazy neuromarketing consultant agency that convinces Fortune 500 companies they need brain scan focus groups! Their own employee writes glowing New York Times op-eds about their work without disclosing the connection. And the study itself is terrible.

The Slate article is well worth reading. It highlights all the classic mistakes in flaky cognitive neuroscience, like cooking up totally different psychological stories from the same brain data just to fit your desired hypothesis. 21st century phrenology, baby!

Conclusion: Slate 1, Times 0.

(previous nicer post on this is here)

Pop cog neuro is so sigh

2007-11-14T01:32:00.000-08:00

A good anti-pop-cognitive-neuroscience rant on Language Log:

In closing, there is a larger issue here, beyond the validity of a specific study of voter psychology. A number of different commercial ventures, from neuromarketing to brain-based lie detection, are banking on the scientific aura of brain imaging to bring them customers, in addition to whatever real information the imaging conveys. The fact that the UCLA study involved brain imaging will garner it more attention, and possibly more credibility among the general public, than if it had used only behavioral measures like questionnaires or people's facial expressions as they watched the candidates. Because brain imaging is a more high tech approach, it also seems more "scientific" and perhaps even more 'objective." Of course, these last two terms do not necessarily apply. Depending on the way the output of UCLA's multimillion dollar 3-Tesla scanner is interpreted, the result may be objective and scientific, or of no more value than tea leaves.

Fightin' the good fight. Maybe it's hopeless. Perhaps "it's hard to avoid the inexorable rise of cognitive neuroscience as the dominant dicourse of the next decade." Sigh. Doing lots of statistical analysis of human behavior just seems like a better use of time to me.

Authoritarian great power capitalism

2007-11-13T00:42:00.001-08:00

Before I forget -- a while back I read a terrific Foreign Affairs article, The Return of Authoritarian Great Powers. The argument is, just a century or so ago, states based on authoritarian capitalism were very powerful in the world; e.g. imperial Japan and Germany. They got plenty of the economic benefits of capitalism but not so much the democratic effects people like to talk about today. (And there are interesting points that the failure of fascism in the second world war was contingent and not inherent to the ideology.) The author argues this looks like the future: Russia and China are becoming economically strong world powers but keeping solidly non-democratic ways of governance. The period of liberal democracy we live in, with all its overhyped speculation about the inevitable spread democracy and free market capitalism -- say, an "end of history" -- might just be that, a moment caused by the vagaries of 20th century history.

After I read the article last June, I actually saw Mr. End of History himself, Francis Fukuyama, speak at the good ol' Long Now seminar series. He pointed out several challenges to liberal democracy, admitting:

China and Russia will be a test of his thesis, Fukuyama said. They are getting wealthier. If they democratize in the next twenty years, he’s right. If they remain authoritarian, he’s wrong.

But this only posed it as a test, not addressing this particular point -- that authoritarian capitalism could be efficient and powerful enough to beat out liberal democracy's hegemony. Maybe it's secondary to clashes of civilizations or environmental catastrophe, but it seems something's there.

neo institutional economic fun!

2007-10-31T16:15:00.000-07:00

[Like medieval Christian societies,] Islamic societies similarly found ingenious ways to circumvent the usury ban. The primary one was the double sale. In this transaction, the borrower would get, for example, both 100 dinars cash and a small piece of cloth valued at the absurdly high price of 15 dinars. In a year he would have to pay back 100 dinars for the loan of the cash and 15 for the cloth. These debts were upheld by Sharia courts.

The cognition of rules and ethics sure is complex. I'd love to read the reasoning from those courts.

From from Gregory Clark's rather intense review of Avner Greif's new-ish book on institutional economics. (Clark's the one who wrote that interesting but weird evolutionary argument about European economic development.) If you're in to the little worlds of institutional, evolutionary, and behavioral economics, Greif's work is really interesting. Along with other institutional (or is it neo-institutional?) economists, he works to understand the functioning of markets, contracts, and laws as stable equilibria of games -- that is, how individual incentives and behaviors can lead to different organizational efficiencies and therefore economic growth.

Greif's book expands the notion of institutions as systems "of rules, beliefs, norms, and organizations that together generate a regularity of (social) behavior." He wants to bring in cognition! Not be strictly bound to individualistic rational choice! And he even does nifty mathematical backflips to put an interesting spin on various models of cooperation, contract enforcement, and the like. It's all very exciting, but as Clark (convincingly) points out, highly theoretical, a bit vague and difficult to test. I read Greif a few years back when I was a lot more willing to work through pages of equations just for their own sake -- I guess you can't keep that up forever.

In any case, economists wanting to do something with cognition is a great, but failing to do so unfortunately common. (You can view the entire field of behavioral economics in that way -- they went ahead and modeled lots of interesting systemic effects (the biases of heuristics & biases), but the substantive bases aren't necessarily there. On the other hand, it's not clear if anyone has good substantive explanations for any human decision-making or social behavior.) I wonder if there's any hope to usefully understanding social behavior as the interaction of cognitive agents. Things might just be too complex.

Verificationism dinosaur comics

2007-10-13T11:06:00.001-07:00

I love dinosaur comics. What a quicker way to learn philosophy than reading all those books.

For example:

Now I have an opinion on verificationism! I like the yellow dinosaur better. Scientific empiricism is great, dithering about strict notions of meaning, not so great. See, isn't epistemology easy?

EEG for the Wii and in your basement

2007-09-26T01:37:00.000-07:00

There's a company, Emotiv, that's building an EEG interface for the game systems. Any company with a science-fiction-y vision statement sounds like a good time to me:

Communication between man and machine has always been limited to conscious interaction, with non-conscious communication -- expression, intuition, perception -- reserved solely for the human realm. At Emotiv, we believe that future communication between man and machine will not only be limited to the conscious communication that exists today, but non-conscious communication will play a significant part.

Our mission is to create the ultimate interface for the next-generation of man-machine interaction, by evolving the interaction between human beings and electronic devices beyond the limits of conscious interface. Emotiv is creating technologies that allow machines to take both conscious and non-conscious inputs directly from your mind.

They even have a cyborg-looking woman on the page.

Their claim is to detect emotion states of the player. Do you really need EEG for that? What about something less sexy like skin conductance? Facial expressions or more? I guess a big metal helmet detecting your brain must be inherently fun.

Along similar lines, the Internet has lots of advice on homemade EEG, with impressively detailed how-to instructions. Other sites seem to focus less on reading your brain's outputs, and more on modifying its state/signals. For example, apparently you can hack your brain with an iPod. Sign me up, what I've always wanted.

(To be fair, here's a more complete page on entrainment. The Emotiv link is from folks at neurodudes.)

Dollar auction

2007-09-15T16:13:00.001-07:00

I got nervous and panicky just reading about this game. I wonder if I could con some people into playing it.

Economics professors have a standard game they use to demonstrate how apparently rational decisions can create a disastrous result. They call it a "dollar auction." The rules are simple. The professor offers a dollar for sale to the highest bidder, with only one wrinkle: the second-highest bidder has to pay up on their losing bid as well. Several students almost always get sucked in. The first bids a penny, looking to make 99 cents. The second bids 2 cents, the third 3 cents, and so on, each feeling they have a chance at something good on the cheap. The early stages are fun, and the bidders wonder what possessed the professor to be willing to lose some money.

The problem surfaces when the bidders get up close to a dollar. After 99 cents the last vestige of profitability disappears, but the bidding continues between the two highest players. They now realize that they stand to lose no matter what, but that they can still buffer their losses by winning the dollar. They just have to outlast the other player. Following this strategy, the two hapless students usually run the bid up several dollars, turning the apparent shot at easy money into a ghastly battle of spiraling disaster.

Theoretically, there is no stable outcome once the dynamic gets going. The only clear limit is the exhaustion of one of the player's total funds. In the classroom, the auction generally ends with the grudging decision of one player to "irrationally" accept the larger loss and get out of the terrible spiral. Economists call the dollar auction pattern an irrational escalation of commitment. We might also call it the war in Iraq.

From here through here.

Is it ever rational to enter the game? What seems frightening is the aspect of losses -- if you're in the lead, any move by anyone else pushes you into the bad losing position of 2nd place.

ConnectU.com SQL injection vulnerability: a story of pathetic hubris (and fun with the password 'password')

2007-08-21T00:36:00.000-07:00

This is off-topic for this blog but here goes. ConnectU, a small college social networking site, has been in the news due to their apparently weak lawsuit against Facebook, in which they claim Mark Zuckerberg stole their business plan and computer code back when they all were Harvard undergraduates. (Judges involved have noted the case's flimsy evidence; some technology commentators -- as well as everyone I know -- have noted that the business idea wasn't all that brilliant or original in the first place.) Zuckerberg, of course, went on to found Facebook and bring it to incredible success.

I tried to use the ConnectU site recently, but got an error when searching for a funny name with an apostrophe, o'connor. It turns out this was symptomatic of a very grave security flaw in their code, an SQL injection vulnerability. While Facebook recently had a minor security-related glitch, ConnectU's flaw is far more serious. A malicious attacker could use this to easily break into user accounts, damage or delete internal databases, or probably much worse.

I contacted ConnectU about the flaw here and by now, they seem to have fixed the problem. (Sorry, I didn't get screenshots before the fix.) But this is hardly confidence-inspiring. This bug is one of the most elementary security bugs that can exist in a PHP website. It's a clear sign of a shoddy, amateurish effort; my coworker Dave Fayram, a web engineering expert, describes it as "shameful". Apparently ConnectU's lawsuit asks for all assets and ownership rights to Facebook under the presumption that Zuckerberg's actions were uniquely responsible for their relative lack of success. But this level of engineering incompetence belies any such claim (e.g. as assumed here). Mark Zuckerberg moved his operation to Palo Alto, hired boatloads of smart Stanford grads and built one of the most popular social networking sites around, while ConnectU piddled around with a provably pathetic, toy site.

Techincal details on their litany of errors:

The advanced search page (prominently linked directly off the front page) did not escape text field inputs. A search got submitted as a MySQL SELECT query, so if you cleverly used single quotes in any field, you could inject arbitrary SQL into the WHERE clause. (Much more malicious things may also be possible.) And to make matters worse, PHP debug error messages were on (bad!), so you saw MySQL error messages on malformed queries.

For example, issuing ' AND pw not null OR 'bla'=' yielded the error Unknown column 'pw' in 'where clause'. With a few more tries, it was trivial to discover that they're storing user passwords directly in the users table as plaintext (bad!) and you could even query for what users have various sorts of passwords. For instance, it turns out several users have length 1 passwords: ' AND password RLIKE '^.$' OR 'bla'='. And 192 users have the password 'password': ' AND password = 'password' OR 'bla'='. Amusingly, when you do this query you get back the standard results page that has every users' school listed; thus, of those 192, here are the top 10 schools represented among that group:

13 New York University
11 Harvard University
10 Cornell University
9 Louisiana State University
8 Boston University
7 Pennsylvania State University
7 Columbia University
6 University of Massachusetts - Amherst
6 University of Pennsylvania
6 University of California - Los Angeles

Sure, NYU and Harvard have some of the larger populations on ConnectU (around 1800 and 1300 respectively), but some some schools like Stanford have plenty of users (150) but no one at all with 'password'. Here is the list of the ten largest schools with zero password 'password's, sizes ranging from about 400 to 100:

Colgate University
Brandeis University
Syracuse University
Emerson College
Yale University
University of California - Davis
Rensselaer Polytechnic Institute
University of California - Berkeley
Stanford University
Rutgers University

Yes, this is atrociously poor statistical methodology. I only contend these lists are amusing, not substantive. But, it doesn't take much imagination to see that having access at all to such data is a critical security breach.

It's all in a name: "Kingdom of Norway" vs. "Democratic People's Republic of Korea"

2007-08-12T17:04:00.000-07:00

Sometimes it seems bad countries come with long names. North Korea is "People's Democratic Republic of Korea", Libya is "Great Socialist People's Libyan Arab Jamahiriya", and the like. But on the other hand, there's plenty of counter-examples -- it's the "United Kingdom of Great Britain and Northern Ireland" and "Republic of Cuba", after all. Do long names with good-sounding adjectives correspond with non-democratic governments?

Fortunately, this can be tested. First, what words are out there? From the CIA Factbook's data on long form names, here are some of the most popular words used by today's countries, listed with the number of occurrences across all 194 names. I limited to tokens that appear >= 3 times. A majority of countries are Republics, while there are some Kingdoms, and even a few Democracies.

(146 of) (127 Republic) (17 Kingdom) (8 the) (8 Democratic) (6 State) (6 People's) (5 United) (4 and) (4 Islamic) (4 Arab) (3 States) (3 Socialist) (3 Principality) (3 Islands) (3 Guinea) (3 Federal) (3 Commonwealth)

Now, we can group countries by included words and look at how democratic they are, as according to Freedom House's political rights scores. They look at a number of political freedoms -- free elections, ability to run for office, power sharing, lack of military intervention in government, etc. to formulate the rating. The following chart shows the average political rights score per group of countries with the given word (actually, substring) in its name.

The upper rows show a substring and the number of names that are matched by it, and the average PR score. (These groups occasionally overlap.) The lower rows are several example countries for reference. So Republics are ever so slightly less democratic than your average non-Republic, and also amusingly, Kingdoms edge them out too. But Democratic, People's, Socialist, Islamic and Arab countries are definitely the big-time un-democracies, while the only clear winners on the other side are Commonwealths and Principalities. Here are the members of the smaller groups:


Score   Name
-----   ----

/kingdom/
1       Kingdom of Belgium
1       Kingdom of Denmark
1       Kingdom of the Netherlands
1       Kingdom of Norway
1       Kingdom of Spain
1       Kingdom of Sweden
1       United Kingdom of Great Britain and Northern Ireland
2       Kingdom of Lesotho
5       Kingdom of Tonga
5       Hashemite Kingdom of Jordan
5       Kingdom of Morocco
5       Kingdom of Bahrain
6       Kingdom of Bhutan
6       Kingdom of Cambodia
7       Kingdom of Thailand
7       Kingdom of Swaziland
7       Kingdom of Saudi Arabia

/democ/
2       Democratic Republic of Sao Tome and Principe
3       Democratic Republic of Timor-Leste
4       Democratic Socialist Republic of Sri Lanka
5       Federal Democratic Republic of Ethiopia
6       People's Democratic Republic of Algeria
5       Democratic Republic of the Congo
7       Lao People's Democratic Republic
7       Democratic People's Republic of Korea

/state/
1       Federated States of Micronesia
1       United States of America
1       State of Israel
2       Independent State of Samoa
2       United Mexican States
3       Independent State of Papua New Guinea
4       State of Kuwait
6       State of Qatar
7       State of Eritrea

/feder/
1       Federal Republic of Germany
1       Federated States of Micronesia
1       Swiss Confederation
2       Federative Republic of Brazil
4       Federal Republic of Nigeria
5       Federal Democratic Republic of Ethiopia
6       Russian Federation

/people/
4       People's Republic of Bangladesh
6       People's Democratic Republic of Algeria
7       People's Republic of China
7       Lao People's Democratic Republic
7       Great Socialist People's Libyan Arab Jamahiriya
7       Democratic People's Republic of Korea

/arab/
6       Arab Republic of Egypt
6       United Arab Emirates
7       Kingdom of Saudi Arabia
7       Syrian Arab Republic
7       Great Socialist People's Libyan Arab Jamahiriya

/united/
1       United Kingdom of Great Britain and Northern Ireland
1       United States of America
2       United Mexican States
4       United Republic of Tanzania
6       United Arab Emirates

/islam/
5       Islamic Republic of Mauritania
5       Islamic Republic of Afghanistan
6       Islamic Republic of Pakistan
6       Islamic Republic of Iran

/commonwealth/
1       Commonwealth of Australia
1       Commonwealth of The Bahamas
1       Commonwealth of Dominica

/island/
1       Republic of the Marshall Islands
4       Solomon Islands
6       Republic of the Fiji Islands

/principality/
1       Principality of Andorra
1       Principality of Liechtenstein
2       Principality of Monaco

/social/
4       Democratic Socialist Republic of Sri Lanka
7       Socialist Republic of Vietnam
7       Great Socialist People's Libyan Arab Jamahiriya

I just think it's striking there's such a small set of words used to describe countries, and that so many use "Republic". It speaks to (some sort of) triumph of liberal politcal ideas that even the most dictatorial regimes have to at least pay lip service to them. This has certainly been going on for a while; I suppose names have been moving in this direction for a few hundred years.

I also looked at simple word lengths of country names. It's not exactly the clearest bubbleplot ever, but if you go ahead and force a linear model (least-squares regression) on it, turns out each word contributes 0.26 points of un-democraticness. And if you viciously remove those lower right outliers (UK and Sao Tome), that coefficient bumps up to 0.39.

Boring details:

For the 2006 CIA Factbook information, I used an XML version described here and located here. For every country it gives a "conventional long form" name. If there is none, I used the standard short name. I think the Wikipedia List of countries page might have the same information as this.
The ratings are Freedom House's Political Rights ("PR") scores for 2006. (They also have a highly correlated Civil Liberties score; I should've used the overall average score but am too lazy to redo it all now.) Therefore this analysis doesn't include any of the extinct but excitingly named communist countries like the German Democratic Republic. Freedom House actually has historical data going back decades, so this could definitely be looked at; presumably this would further tilt the weight of "socialist", "people", and "democratic" to being non-democratic.
Strings and more in Ruby, plots all from R, and the occasional assist by Excel. Learned some new tricks too.

When's the last time you dug through 19th century English mortuary records

2007-08-08T00:32:00.000-07:00

Standard problem: humans lived like crap for thousands and thousands of years, then suddenly some two hundred years ago dramatic industrialization and economic growth happened, though unevenly even through today. Here's an interesting proposal to explain all this. Gregory Clark found startling empirical evidence that, in the time around the Industrial Revolution in England, wealthier families had more children than poorer families, while middle-class social values -- non-violence, literacy, work ethic, high savings rates -- also became more widespread during this time. According to the article at least, he actually seems to favor the explanation that human biological evolution was at work; though he notes cultural evolution is possible too. (That is, the children of wealthier families are socialized with their values; as the children of middle-class-valued families increase in proportion in society, the prevalence of those values increases too.)

In any case, the argument is that behavioral changes, not institutional changes, drove the rise of capitalism. I know that some people define institutions to include cultural norms (and therefore human behavior, right?), so I'm presuming that for Clark and the academic debates vaguely mentioned in the article, "institutions" means something more boring like government structure or enforcement of property rights. (I'm reading Samuel Bowles's microeconomics book off and on, where he likes to mix behavioral and institutional ideas; and I seem to recall this from Avner Grief too; this all apparently is too confusing for me. (Bowles is quoted in the article.)) The article mentions Max Weber's Protestant ethic as related to Clark in its being a behavioral thesis.

I'm awfully skeptical of biological evolution claims without any actual genetic evidence (though I quite like cultural evolutionary claims), but the theory is very neat and the archival data gathered is incredible, as you can see in this shamelessly ripped off diagram/explanation from the NYT article about the Clark's book on this.

Are ideas interesting, or are they true?

2007-08-05T03:20:00.000-07:00

From an NYT Magazine article this Sunday, paraphrasing Isaiah Berlin:

The philosopher Isaiah Berlin once said that the trouble with academics and commentators is that they care more about whether ideas are interesting than whether they are true. Politicians live by ideas just as much as professional thinkers do, but they can’t afford the luxury of entertaining ideas that are merely interesting. They have to work with the small number of ideas that happen to be true and the even smaller number that happen to be applicable to real life. In academic life, false ideas are merely false and useless ones can be fun to play with. In political life, false ideas can ruin the lives of millions and useless ones can waste precious resources. An intellectual’s responsibility for his ideas is to follow their consequences wherever they may lead. A politician’s responsibility is to master those consequences and prevent them from doing harm.

I can't speak for that level of politics, but I've seen in applied technology the distinction between interesting and true ideas can be great. (True in the sense of, it is true that the idea solves the problem at hand.) I've wasted plenty of time recently at work chasing down very interesting ideas only to reassess and do something more expedient. A different example from empirical science: it sure sounds interesting when the 1950's Chomsky claims that language can't be statistically modelled, though that turns out to be embarrassingly false.

Cooperation dynamics - Martin Nowak

2007-07-30T22:21:00.000-07:00

Nice little NYT article on Martin Nowak, of evolution-of-cooperation fame. He's the directory of Harvard's Program for Evolutionary Dynamics which looks neat. I love the the Price Equation. Sweet.

China: fines for bad maps

2007-07-27T07:28:00.000-07:00

This is fascinating -- In China, you can get fined if you make a map of China without Taiwan or other disputed territories. Reminds me of being confused trying to find the primary airline of China.

Based of vague recollections of its name, I searched Google for {{ china air }}. The first hit was for China Airlines. But the second hit was Air China. The first is the state carrier of the ROC (Taiwan), the second is the PRC (mainland China). Turns out my intended concept, "Official Chinese airline", isn't a coherent concept if your political worldview includes both the ROC and PRC as entities. But maybe what I should have wanted was just airlines that fly around East Asia and various parts of China; in that case, getting both airlines is the right thing to do. At least Google got them both at the top of the list.

(p.s. anyone know how to force blogger to *not* destructively resize your images? sigh)

Cerealitivity

2007-07-24T20:27:00.000-07:00

This is pretty funny, an old cartoon reprinted on Language Log.

Game outcome graphs -- prisoner's dilemma with FUN ARROWS!!!

2007-07-08T03:52:00.000-07:00

I think game theory could benefit immensely from better presentation. Its default presentation is pretty mathematical. This is good because it treats social interactions in an abstract way, highlighting their essential properties, but is bad because it's hard to understand, especially at first.

However, I think I have a visualization that can sometimes capture the same abstract properties of the mathematics. Here's a stab at using it to explain everyone's favorite game, the prisoner's dilemma.

THE PD: Two players each choose whether to play nice, or be mean -- Cooperate or Defect. Then they simultaneously play their actions, and get payoffs depending on what both played. If both cooperated, they help each other and do well; if both defect, they do quite poorly. But if one tries to cooperate and the other defects, then the defector gets a big win, and the cooperator gets a crappy "sucker's payoff".

The formal PD definition looks like this:

where each of the four pairs represents the (row player payoff, column player payoff) for that pair of choices. This 2x2 table, along with the constraint a>b>c>d, together capture all the properties outlined in the above paragraph. (There is usually one more constraint that a+d<2b but I'm dropping it for this discussion.) Usually it takes a few more paragraphs of prose to really explain things, but you're reading a blog and don't have patience for such silliness. Therefore the fun pictures.

First, let's look at the group level. Is there an outcome that makes everyone happy? Or at least, is there an outcome that's incontrovertibly better compared to another outcome? Yes, actually. This relationship is true exactly one time. Here's a new diagram that puts in example values for the payoffs:

The (C,C) outcome gets a payoff of 2 for each person, whereas the (D,D) payoff gets 1 for each person. Compared to DD, CC is better for everyone. This is called a Pareto improvement. Therefore there is an arrow drawn between them. The notation X Y means that outcome Y is a Pareto improvement over X.

There are no other Pareto improvements among the outcomes in this game, just this one (D,D) (C,C).

Now let's examine individual incentives. If you were playing, what should you do? You don't know what your opponent will play, but you can reason about each situation in turn. If he is planning to cooperate, then you could either cooperate also, or else defect and exploit him. 2<3 so you'd best defect to exploit. If he is planning to defect, your choice is either to cooperate and be a sucker, or else defect as well. 0<1 so you'd best defect for self-defense.

Both players face the same incentives (the game is exactly symmetric). This diagram shows their preferences over outcomes they control. Remember, the row player's payoffs are the left side of each pair, and the column player's payoffs are the right side of each pair.

I'd like to have arrows connecting social outcomes, not individual outcomes, so let's rewrite the diagram like so:

So the marks a selfish preference aligned with a unilateral choice; that is, X Y means that one player could have control over whether X or Y is picked, and he prefers Y over X. 's can only appear horizontally or vertically, since they represent a relationship between outcomes that only exists between outcomes whose difference is only in the decision of one player. (The difference between the diagonal outcomes (C,C) and (D,D) requires a change by both players; it is not due to a mere unilateral choice.)

Looking at the diagram, it's clear the individual incentives are very stark: each player should defect under all circumstances. (Arrows on both left and right point down.) There is only one outcome that only has arrows flowing in and not out: D,D. If an outcome only has incoming arrows, and no outgoing ones, it is a Nash equilibrium. A way to think about it is, if both players are playing (D,D), there is no incentive for either to unilaterally switch away.

If we combine the diagrams, it's easy to see why this is a dilemma. Individual incentives work in clear opposition to Pareto improvement! (There may be be other ethical concerns, such as the unfairness of the exploitation outcomes, but let's put those aside for now. At the very least, this Pareto improvement seems to be a socially good thing.) The Pareto optimum is in a box, and the Nash equilibrium is circled.

Here's a game where cooperation is a bit easier. It's called a stag hunt, another odd name not really worth explaining. It's similar to a PD, except the cooperation payoff is better than exploitation payoff. (In the old language, the payoff ordering is now b>a>c>d.) Let's use numbers again -- the mutual cooperation payoff is now 4 -- and jump straight to the Pareto-Improvement + Unilateral-Selfish-Choice diagram:

Now that mutual cooperation beats exploitation, the (C,C) outcome is now a Nash equilibrium, in addition to being Pareto superior over (D,D). (There are also two new Pareto improvements from C,D and D,C, just for kicks.) Now, with two NE's on the table, it's not clear what you should do if you were a player. If you were absolutely certain your opponent was going to defect, you should defect too just like in the PD. But if you thought he was going to cooperate, you should cooperate as well.

If both sides can coordinate their moves, then they can positively benefit at the superior and maintainable (C,C) outcome.

Whew, I think that's it for now. I confess that I rather like diagramming out the incentive and payoff relationships between outcomes; I find it far more informative and instructive compared to staring at the arithmetic/algebraic tables and trying to figure it out in my head. Maybe I'm just not good enough at math.

To give credit where credit is due, I've seen the unilateral-selfish-choice arrows in only one place, Jim Fearon's excellent lecture notes, though he is not to blame for all this new crap I threw in. The arrows get really useful if you start working with games that have more than 4 outcomes, since as long as the game is discrete and you can lay out the outcomes in two dimensions, you usually can draw a bunch of graph edges between them. These diagrams can be completely formalized as much as the arithmetic algebra standardly used for game theory, since the visual graph over outcome nodes is just a way to writing down a set of binary relations on outcomes, and the row/column alignment stuff is just a way of showing how those relations interact with individual choices. You can easily imagine adding more arrows for different social preference functions, for elements of different solution concepts, etc.

Some people have done work with the taxonomy of 2x2 games; it might be useful to illustrate the differences (e.g. pure coordination versus pure conflict games) as outcome graph diagrams. Another post I guess...

Washington in 1774

2007-07-08T00:35:00.000-07:00

[I]t is not the wish, or the interest of the Government, or any other upon this Continent, separately, or collectively, to set up for Independence... I am well satisfyed, as I can be of my existence, that no such thing is desired by any thinking man in all North America; on the contrary, that it is the ardent wish of the warmest advocates for liberty, that peace & tranquility, upon Constitutional grounds, may be restored, & the horrors of civil discord prevented.

-- George Washington to Robert McKenzie, October 1774

Found this in an alternative history "what if?" essay on how the American Revolution could have never happened. In Virtual History: Alternatives and Counterfactuals, ed. Niall Ferguson.

Happiness incarnate on the Colbert Report

2007-07-06T22:09:00.000-07:00

A bit ago I finished Daniel Gilbert's "Stumbling on Happiness," which despite its name, is not about how to be happy. It's about why people are bad at predicting (and remembering) their happiness levels. (Pop science psychology, not pop psychology... something like that.) I liked a bit of it mainly because it has an entertaining overview of some cognitive psychology. A few very interesting happiness experiments get presented, but if your friend tells you about them before you read the book, it's all over.

Perhaps more amusing is stumbling on Gilbert's quite entertaining appearance on the Colbert Report last week:

Evangelicals vs. Aquarians

2007-06-28T23:54:00.000-07:00

Just read an interesting analysis on the the simultaneous rise of the cultural left and right ("hippies and evangelicals") through the 50's and 60's. Brink Lindsey argues here that they were both reactions to post-war material prosperity:

On the left gathered those who were most alive to the new possibilities created by the unprecedented mass affluence of the postwar years but at the same time were hostile to the social institutions — namely, the market and the middle-class work ethic — that created those possibilities. On the right rallied those who staunchly supported the institutions that created prosperity but who shrank from the social dynamism they were unleashing. One side denounced capitalism but gobbled its fruits; the other cursed the fruits while defending the system that bore them. Both causes were quixotic, and consequently neither fully realized its ambitions.

I love neat sweeping theories of history; I can't take it overly seriously but it is so fun. Lindsey argues that the eventual failures of either side of the culture wars has bequeathed us a sort of libertarian-ish working consensus for society's values. I'm not convinced based on the abbreviated treatment in the article, but he definitely knows how to write a good cultural history. And with great relevance -- all too often I suspect today's political and cultural dynamics are still rehashes of the 60's/70's, despite big changes like the end of Communism, the start of the Internet, and the like.

And of course the article has wonderful stories such as this:

The peculiar career of Arthur Blessitt illustrates evangelicalism’s debt to the cultural left. In the late ’60s, Blessitt hosted a psychedelic nightclub called His Place on Hollywood’s Sunset Strip, an establishment whose logo combined a cross and a peace sign. “Like, if you want to get high, you don’t have to drop Acid. Just pray and you go all the way to Heaven,” Blessitt advised in his tract Life’s Greatest Trip. “You don’t have to pop pills to get loaded. Just drop a little Matthew, Mark, Luke, or John.” In 1969 Blessitt began his distinctive ministry of carrying a 12-foot-tall cross around the country—and, later, around the world. On one of his countless stops along the way, at an April 1984 meeting in Midland, Texas, he received word that a local oilman, the son of a prominent politician, wanted to see him privately. The businessman told Blessitt that he was not comfortable attending a public meeting but wanted to know Jesus better and learn how to follow him. Blessitt gave his witness and prayed with him. The man, George W. Bush, subsequently converted to evangelical Christianity.

Apparently this article is excerpted from a new book; there's a decent enough NYT review by George Will, though burdened with his usual "look at me, I'm a smart conservative thinker and like to talk about Burke!" shtick.

"Time will tell, epistemology won't"

2007-06-16T21:35:00.000-07:00

Working on applied AI-related problems has really tempered my outlook away from theory. Apologies for another Rorty-related post, but I loved this little bit I just came across, from Stanley Fish (on slate.com):

When Rorty concluded one of his dramatically undramatic performances, the hands shot up like quivering spears, and the questions were hurled in outraged tones that were almost comically in contrast to the low-key withdrawn words that had provoked them.

Why outrage? Because more often than not a Rortyan sentence would, with irritatingly little fuss, take away everything his hearers believed in. Take, for example, this little Rortyan gem: "Time will tell; but epistemology won't." That is to say—and the fact that I have recourse to the ponderously academic circumlocution "that is to say" tells its own (for me) sad story—if you're putting your faith in some grandly ambitious account of the way we know things and hoping that if you get the account right, you will be that much closer to something called Truth, forget it; you may succeed in accomplishing the task at hand or reaching the goal you aim for, but if you do, it will not be because some normative philosophy has guided you and done most of the work, but because you've been lucky or alert enough to fashion the bits and pieces of ideas and materials at your disposal into something that hangs together, at least for the moment. Or, in other, and better words, "Time will tell; but epistemology won't."

A co-worker of mine said, "when you're in a meeting and someone mentions epistemology, the conversation is through." :-)

And similarly from that article:

"The world is out there, but descriptions of the world are not."

Reminds me of Clay Shirky on the semantic web:

The Semantic Web takes for granted that many important aspects of the world can be specified in an unambiguous and universally agreed-on fashion, then spends a great deal of time talking about the ideal XML formats for those descriptions. This puts the stress on the wrong part of the problem -- if the world were easy to describe, you could do it in Sanskrit.

Richard Rorty has died

2007-06-11T00:14:00.000-07:00

Richard Rorty, philosopher, dies at 75.

I've read enough of the analytic philosophers castigating Rorty -- and taken bits of classes from a few of them -- that I feel I just have to love the man.

I remember managing to see him speak twice. Once was on philosophy of mind at the good ol' Sym Sys Forum. (Video!) ("He is wrong, but wrong in such an interesting way!" I remember one comment.)

Most fascinating was when he gamely participated in a discussion at this very odd Christian thought conference some groups on campus put together. (The Veritas Forum, here's a link.) He was standing there, arguing with the Christian conservatives about the nature and legitimacy of authority, but humorously ceding ground where appropriate... "Look, it's not that all children will be active critical thinkers and discover everything for themselves. Getting a kid a secular liberal education isn't that much different than any other education -- you have to beat it in to them." (That is a paraphrase, not a quote. But I'm pretty sure he referred to "secular liberal education" having to be "beaten into them".) I think this is a really important point, but I just like the tongue-in-cheekness.

I confess I have never read any of his books. Perhaps I should.

Here's an interesting review he wrote of Marc Hauser's Moral Minds.

Here is the SEP entry on him.

Freak-Freakonomics (Ariel Rubinstein is the shit!)

2007-06-09T23:53:00.000-07:00

I don't care how lame anyone thinks this is, but economic theorist Ariel Rubinstein is the shit. He's funny, self-deprecating, and brilliant. I was just re-reading his delightful, sarcastic review of Freakonomics. (Overly dramatized visual depiction below; hey, conflict sells.)

The review consists of excerpts from his own upcoming super-worldwide-bestseller, "Freak-Freakonomics". It is full of golden quotes such as:

Chapter 2: Why do economists earn more than mathematicians?

...

The comparison between architects and prostitutes can be applied to mathematicians and economists: The former are more skilled, highly educated and intelligent.

To elaborate:

Levitt has never encountered a girl who dreams of being a prostitute and I have never met a child who dreams of being an economist. Like prostitutes, the skill required of economists is “not necessarily 'specialized'” (106). And, finally, here is a new explanation for the salary gap between mathematicians and economists: Many economists are hired to justify a viewpoint but I have never heard of mathematicians who proved a theorem to satisfy their masters.

This too:

Levitt is correct when he says: "Information asymmetries everywhere have in fact been mortally wounded by the Internet." (68) The curious reader can roam the Net and discover, for example, that there are some who harbor doubts regarding the (superfluous) story about the fellow who claimed to have defeated the Ku Klux Klan using a trivial tactic. It is also easy to find doubts raised about the validity of Levitt’s two important studies (including the famous and surprising study in which Levitt (and Donahue) argued that the legalization of abortion in the 1970s had a drastic impact on the decline in crime in the U.S. in the 1990s). The two studies were the subject of critiques published in the same academic journals in which Levitt gained recognition. In response Levitt acknowledged "insignificant" errors. There is no trace of the criticism in the book.

Finally,

Freakonomics aspires to "thinking sensibly about how people behave in the real world. All it requires is a novel way of looking, of discerning, of measuring. This isn’t necessarily a difficult task, nor does it require super-sophisticated thinking." (205) The authors believe that “The most likely result of having read this book is a simple one: you may find yourself asking a lot of questions.” (206)

I do not believe in magicians who know how to teach people to think, to feel and to invent. Levitt claims: "A long line of studies ... had already concluded that genes alone are responsible for perhaps 50 percent of a child’s personality and abilities.” (154). I dare to attribute (without research) 49% to the mother, father and kindergarten teacher. These numbers do not leave much room for Freakonomics.

Did I forget to say, Ariel Rubinstein is the shit? His website even has a giant listing of university town cafe's across the world (with pictures!), with spots I have heard of many times but never visited and now relate to only as mythical, legendary places, like Small World Cafe (left). His entry for Palo Alto, however, is missing the much less cool, though terribly wonderful, 24-hour Happy Donuts (right).

This showdown is not intended to parallel the above in any way.

"Stanford Impostor"

2007-05-28T20:28:00.001-07:00

I've gotten a zillion emails about it by now, but it was recently found that a young woman had been living for a year in Stanford dorms claiming to be a freshman, when in fact she was not a student of any sort at all. This seems to have engendered much discussion in the Stanford community, e.g. here. (The LA Times has a decent piece).

I mostly think the story is just sad, but a few thoughts:

The security scare angle is a bit ridiculous, as is the blame game being played.
I suspect the Stanford community is naive and high-strung about these things. My friends who went to state schools say things like this happen all the time. Besides, staffs in co-ops are sometimes happy to let vagrants hang out. The motivations for and mechanics of her deception are clearly the important part of this story (e.g. how did she live with herself?)
Her first two quarters were in Kimball, then then her last quarter was in Okada. I'm surprised the Kimball staff weren't enough in-touch with their residents to know a pair of them were having someone else stay in their room for months, and that this person was often sleeping in the lounge. It's pretty standard for Stanford ResEd RA's to have at least that much knowledge. (See, the bubble usually works!)
The comments on the Daily article (this one or this earlier one) are disturbing in how quickly things spin into classism and accusations of classism, e.g. Berkeley vs. Stanford, etc.

Update 5/29: This editorial is funny and great.

Rock Paper Scissors psychology

2007-05-24T00:37:00.001-07:00

Rock, Paper, Scissors is making the blog rounds with an excellent strategy guide from the World RPS Society and a fun mental floss article too. (Though the First International RoShamBo programming contest should be noted.) Having played far too much RPS last year, I can say these tips look pretty decent for the most part. I will say I am a huge fan of the elegant move Running With Scissors -- scissors three times in a row. "You think I'm crazy enough to play that AGAIN?" Un-frickin-stoppable.

Everyone thinks, "but wait, RPS is a lame game. Playing randomly is the best strategy!" This is not true. Playing randomly is the best defensive strategy, since your opponent can never do better than 50% against it. However, it is not the best offensive strategy if you can infer what your opponent will play. For example, if you are confident that your opponent is too scared to play scissors for a third time in a row, exploit that and play paper (the only move vulnerable to scissors, which you believe will not be played). You don't have to exactly predict your opponent's next move, but rather, you merely need to believe a non-uniform distribution over their next move. That is, if you can use any information or heuristics to make decent guesses, you can exploit that.

The game only works because of information imbalances in each player's beliefs about the other's psychological state. The problem of other minds, baby! And even more satisfyingly, the ONLY world state on which players want to make inferences is psychological state. RPS is shorn of silly artifices like a board with pieces or cards on the table. It is simply your mind versus your opponent's.

Incessant trash talking makes the game even more fun.

And don't forget 25-RPS or 4-RPS!

Simpson's paradox is so totally solved

2007-05-09T01:08:00.000-07:00

My friend Lukas just wrote a great formulation Simpson's Paradox as a puzzle:

Against left-handed pitchers, Player A has a higher batting average than Player B. Player A does better against right-handed pitchers also.

Is it possible that B has a better average than A?

Here's a beautiful ASCII art visualization that says Yes.

Each star represents a number of at-bats where the player hit; pluses represent misses. If you put them in a horizontal line you can see the batting averages (proportions) pretty clearly. The bar lenghts carry across rows -- so a longer bar means more at-bats.


Against left-handed pitchers:
A hits |**++| A misses    --> 50% avg.
B hits |*+++| B misses    --> 25% avg.

Against right-handed pitchers:
A hits |**|              A misses    --> 100% avg.
B hits |**************+| B misses    -->  93% avg.  (for many more at-bats!)

But, batting against *ALL* pitchers:
A hits  |****++| A misses                 --> 66% avg.
B hits  |***************++++| B misses    --> 79% avg.

They both do well against right-handed pitchers, but B sees way more of them than A. All those right-handed pitchers B sees helps his score, but A doesn't get much of a payoff from his high average there. This effect overwhelms the within-group differences of A outperforming B.

I was tempted to write, "any random pitcher tends to be a right-handed, therefore those matter more." But that's not quite the right explanation -- rather, we're interested that among B's at-bats, it's usually against a right-handed pitcher, where A usually bats against a leftie.

A better visualization might step up to two dimensions, showing cross-cutting boxes for each group. I am personally of the opinion that cramming more dimensions of data into a visualization can often help understanding, but I don't have time to do it right now so I shouldn't natter on about it.

This was inspired by the pie chart version here [1], which was about group selection in evolutationary theory. Say altruism is socially efficient: a group with altruists does better than a group without altruists. But altruism is individually a bad bet: altruists do worse than free riders in their group. If you do well you have more children, so altruists always lose out to their fellow freeriders.

Surprisingly, the level of altruists across multiple groups can increase. If there's a group with a very high proportion of altruists, the altruists there benefit greatly from each other so have lots of offspring -- even though the few freeloaders in that group are doing better. But that altruistic group beats out the low-altruism groups, so altruists increase in the entire cross-group population. (This gain can only be temporary if groups are fixed: eventually the free-riders in the big group overwhelm the altruists.) So the effect can be characterized that the altruistic group beats out the selfish groups; this is dubbed "group selection". Group selection is working for altruism, but individual selection works against altruism; in the case of unbalanced groups, group selection is stronger.

Crazy people have written stuff about possible implications of this.

[1] Sober, Elliott, and David Sloan Wilson. 1998. Unto Others: The Evolution and Psychology of Unselfish Behavior.

More fun with Gapminder / Trendalyzer

2007-04-08T13:37:00.000-07:00

Watching internet usage vs. income on the Gapminder.org visualizer is very interesting.

Several things are quite apparent. (1) Internet usage exploded in all countries in the world. (2) Richer countries have more internet usage (linear relationship on the scatterplot), but it's been increasing in all countries regardless.

The animation has a few issues, of course -- I think some of the funny, rapid movements at the very start are due to issues with when they started collecting reliable data on internet usage in the early 90's, and the data probably started being more reliable at different times in different countries, etc.

These are countervailing phenomena that require attention to variation within and across groups of data. I can't imagine pie charts or tables of numbers that could ever convey this level of nuance.

One last thing. Going to the linear scale (so you can see differences in large amounts of internet use), watch South Korea vs. United States. Korea, a country with about half of the U.S.'s income per capita, is quite behind, then suddenly rockets ahead and surpasses the U.S. in the late 90's. (This is the country where Starcraft is a national sport -- are we getting p0wned??)

(If you want to get these bookmarkable URL's, you use it at tools.google.com/gapminder)

Gapminder.org -- terrific world data visualizations

2007-04-08T00:41:00.000-07:00

This is an amazing information visualization from Gapminder.org. It displays a scatterplot of nations' life expectancy versus income per capita, and plays the trend across years. Countries are circles, sized by their population size. (So it's a weighted scatterplot... a "bubbleplot".)

Open in new window -- Fertility vs. Income.

It's very powerful -- you can actually change the variables on each axis, the coloring scheme, etc. Fertility vs. Income shows the well-known linear relationship that higher income countries have lower fertility. However, by playing over time it becomes apparent that middle- and low-income countries have been decreasing their fertility rates over the past several decades -- the entire line downshifts.

Numerous other interesting effects can be seen. It's fascinating to watch Hans Rosling's 2006 TED presentation: Myths about the developing world, in which he runs through this and another visualization, pointing out many phenomena. It's impressive how much you can communicate with good data presentation.

If you don't want to watch the video, here is the other he presents, of income distribution across different world regions:

Apparently Google has acquired the Trendalyzer software (the animated bubbleplot) developed at Gapminder. I hope that means it's in good hands; they say they want to enhance it and make it freely available. (Where I work we appreciate what Marissa Mayer has to say, so I'll believe that blog post :))

I was reminded of this site from the SSS blog, but I've been meaning to write about it for a while.

Random search engine searcher

2007-04-08T00:20:00.000-07:00

It's sweeping the internet -- I wrote a little plugin for the firefox/internet explorer search box, so when you search it randomly picks one of several search engines. You get to see what's out there (you mean there's something besides Google?) in your daily searching.

Search a Random Search Engine

Evil

2007-04-04T23:33:00.000-07:00

I must be too cynical. I thought I didn't like Philip Zimbardo's theatrics, but regardless I really appreciated this NYT interview with him on the universal capacity for evil.

(S.P.E. is the Stanford Prison Experiment, whose pictures here are from that lovely basement hallway, still there behind the 420-40 and 41 lecture rooms.)

In particular:

Q. What was your reaction when you first saw those photographs from Abu Ghraib?

A. I was shocked. But not surprised. I immediately flashed on similar pictures from the S.P.E. What particularly bothered me was that the Pentagon blamed the whole thing on a “few bad apples.” I knew from our experiment, if you put good apples into a bad situation, you’ll get bad apples.

That was why I was willing to be an expert witness for Sgt. Chip Frederick, who was ultimately sentenced to eight years for his role at Abu Ghraib. Frederick was the Army reservist who was put in charge of the night shift at Tier 1A, where detainees were abused. Frederick said, up front, “What I did was wrong, and I don’t understand why I did it.”

Q. Do you understand?

A. Yeah. The situation totally corrupted him. When his reserve unit was first assigned to guard Abu Ghraib, Frederick was exactly like one of our nice young men in the S.P.E. Three months later, he was exactly like one of our worst guards.

Q. Aren’t you absolving Sergeant Frederick of personal responsibility for his actions?

A. You had the C.I.A., civilian interrogators, military intelligence saying to the Army reservists, “Soften these detainees up for interrogation.”

Those kinds of vague orders were the equivalent of my saying to the S.P.E. guards, “It’s your prison.” At Abu Ghraib, you didn’t have higher-ups saying, “You must do these terrible things.” The authorities, I believe, created an environment that gave guards permission to become abusive — plus one that gave them plausible deniability.

Chip worked 40 days without a single break, 12-hour shifts. The place was overcrowded, filthy, dangerous, under constant bombardment. All of that will distort judgment, moral reasoning. The bottom line: If you’re going to have a secret interrogation center in the middle of a war zone, this is going to happen.

Q. You keep using this phrase “the situation” to describe the underlying cause of wrongdoing. What do you mean?

A. That human behavior is more influenced by things outside of us than inside. The “situation” is the external environment. The inner environment is genes, moral history, religious training. There are times when external circumstances can overwhelm us, and we do things we never thought. If you’re not aware that this can happen, you can be seduced by evil. We need inoculations against our own potential for evil. We have to acknowledge it. Then we can change it.

Q. So you disagree with Anne Frank, who wrote in her diary, “I still believe, in spite of everything, that people are truly good at heart?”

A. That’s not true. Some people can be made into monsters. And the people who abused, and killed her, were.

On a slightly lighter note. On his conscience:

Q. Why did you pull the plug on the experiment?

A. On the fifth night, my former graduate student Christina Maslach came by. She witnessed the guards putting bags over the prisoners’ heads, chain their legs and march them around. Chris ran out in tears. “I’m not sure I want to have anything more to do with you, if this is the sort of person you are,” she said. “It’s terrible what you’re doing to those boys.” I thought, “Oh my God, she’s right.”

If you ever see the official Stanford prison experiment video, Zimbardo continues: "She was right. And she later became my wife." Yikes.

Finally, on the Milgram experiment:

Here’s something that’s sort of funny. The first time I spoke publicly about the S.P.E., Stanley Milgram told me: “Your study is going to take all the ethical heat off of my back. People are now going to say yours is the most unethical study ever, and not mine.”

There's a lot to say about the ethical and methodological dimensions of the Milgram and Zimbardo experiments' (unfortunate?) legacy on the behavioral sciences. For another day's post, perhaps.

Seth Roberts and academic blogging

2007-03-26T23:14:00.000-07:00

I saw Seth Roberts briefly speak today (at an odd event) about self-experimentation. He tried drinking flavorless sugar water and it led him to lose lots of weight. He also did a great variety of other self-experiments over more than a decade, written up here (and IMHO the other ones are much more interesting).

I briefly spoke to him there and told him I heard about his work from Andrew Gelman's blog. He seemed surprised to (semi-)randomly meet someone who reads it. I think this is mistaken -- that particular blog seems quite popular in statistics/social science world. In fact, Gelman's blogging of Roberts' self-experimentation paper got picked up by the Freakonomics folks and it became a sensation and then a book deal. (Story.)

Also note, John Langford says of his own machine learning blog:

This blog currently receives about 3K unique visitors per day from about 13K unique sites per month. This number of visitors is large enough that it scares me somewhat—having several thousand people read a post is more attention than almost all papers published in academia get.

So true. Conclusion: blogs are a more effective medium for intellectual influence than journal articles.

(p.s. I wonder if free online journals will help fill the gap. E.g. PLOS or arXiv. Or in fields near and dear to Social Science++: theoretical economics, and judgment and decision-making ... and hopefully others? At least computer scientists are always good about posting their papers online... this seems to be the trend for younger researchers in general.)

Statistics is big-N logic?

2007-03-21T01:02:00.000-07:00

I think I believe one of these things, but I'm not quite sure.

Statistics is just like logic, except with uncertainty.

This would be true if statistics is Bayesian statistics and you buy the Bayesian inductive logic story -- add induction to propositional logic, via a conditional credibility operator, and the Cox axioms imply standard probability theory as a consequence. (That is, probability theory is logic with uncertainty. And then a good Bayesian thinks probability theory and statistics are the same.) Links: Jaynes' explanation; SEP article; also Fitelson's article. (Though there are negative results; all I can think of right now is a Halpern article on Cox; and also interesting is Halpern and Koller.)

Secondly, here is another statement.

Statistics is just like logic, except with a big N.

This is a more data-driven view -- the world is full of things and they need to be described. Logical rules can help you describe things, but you also have to deal with averages, correlations, and other things based on counting.

I don't have any fancy cites or much thought yet in to this.

Here are two other views I've seen...

Johan van Benthem: probability theory is "logic with numbers". I only saw this mentioned in passing in a subtitle of some lecture notes; this is not his official position or anything. Multi-valued and fuzzy logics can fit this description too. (Is fuzzy logic statistical? I don't know much about it, other than that the Bayesians claim a weakness of fuzzy logic is that it doesn't naturally relate to statistics.)
Manning and Schütze: statistics has to do with counting. (In one of the intro chapters of FSNLP). Statistics-as-counting seems more intriguing than statistics-as-aggregate-randomness.

Not sure how all these different possibilities combine or interact.

Feminists, anarchists, computational complexity, bounded rationality, nethack, and other things to do

2007-03-15T01:17:00.000-07:00

I was planning to write some WordNet lookup code tonight. But instead I've learned of too many intersecting things.

First, there are a zillion things to do this weekend (hooray flavorpill):

Picasso and American Art exhibit continuing at SFMOMA. I saw it very briefly last weekend but want some more. And Doug claims there's an interesting photography exhibit there too.
Reading from We Don't Need Another Wave: Dispatches from the Next Generation of Feminists, a fascinating looking book I've seen many times in the bookstores around here. By that I mean at least Modern Times (the neat Mission bookstore) and the Anarchist Collective Bookstore (out on the Haight). And the reading is at Modern Times, just down the street from my house! Amazing. Tomorrow at 7:30.
Since anarchists were just mentioned, fortuitously there also appears: the Bay Area Anarchist Bookfair this Saturday and Sunday! Speakers and books down by Golden Gate Park, oh my.

Can't say I'm a radical feminist or even an anarchist, but I *loved* reading that stuff back in high school. Went through waay too much of An Anarchist FAQ (which is amazingly comprehensive and highly recommended -- ironically, everyone who's read it falls into calling it "The anarchist faq" because, I guess, it's so good); and that, with snippets of MacKinnon, Foucault and other easy-to-misunderstand social theorists, I went around denouncing capitalism, statism, and patriarchy left and right. Temporarily believing and advocating radical views is the most educational thing I know how to do. (Guess I've come a long way since then, writing posts in praise of Hobbes... Though I will say reading the conservatives was rarely as fun, though Burke is pretty insightful.)

And THEN -- also tonight, I learned about the Kevin Kelly computability and induction work. I've been thinking about complexity and social behavior a lot -- computational complexity is a pervasive phenomenon in all sorts of human endeavors. It's impossible for people or organizations to solve certain types of problems, and many of the things we are our best responses under these constraints. We can't know everything, we can't consider all possibilities, we can't radically change our own beliefs, and we can't even enumerate what we already know.

There's *tremendous* potential for computational complexity theory to be applied to formal epistemology -- mathematical philosophy really needs to get away from being so reliant on pure logic. (Or maybe this is already happening.) To say nothing of cognitive modeling and theoretical neuroscience. And perhaps behavioral economics, if they ever get around to rediscovering Herbert Simon in the right way, not the small-tweaks-to-neoclassicalism that dominates mainstream behavioral theories (e.g. contrast Camerer vs. Rubinstein, p. 44 here).

But computational theory can solve this! It's a principled way to describe bounded rationality without resort to procedural rationality and all its messy algorithms and ad-hoc state machines. You could make bounded rationality models with substance and generalizable principles. In the same way that neoclassical microeconomics makes interesting insights about market behavior using convex optimization and other mathematical techniques that allow them to make statements about large classes of market situations, not just one particular market situation, I could imagine models that let you make statements about the behavior of large classes of, say, bounded-memory computational automata, not just some particular algorithm that happens to implement useful heuristics for a bounded-memory agent.

Anyways, all you hordes of Social Science++ readers may know that in the old days the my tag line was:

{social, political, economic} cross {cognition, behavior, systems}

That is, the cross-product with 9 combinations -- cells in a 3x3 matrix:

	social	political	economic
cognition
behavior
systems

In the spirit of insightful, serendipitous combinations of ideas, we can cross across feminism and anarchy versus computational complexity and, say, general equilibrium theory.

SOCIAL THEORY: Feminism
SYSTEM THEORY: Computational constraints (Turing, big-O, Chomsky hierarchy, etc.)
SOME RANDOM JOINT HYPOTHESIS: (1) Sexism is a result of computational constraints: A prerequisite to sexism is reasoning via gender. There's a bounded rational explanation: a person's gender is a neat little binary property of them, with which inferences can be made. It's more accurate than ignoring gender, but rather large mistakes can be made and when they are malicious or particularly suboptimal they are called sexism. (There are numerous issues with this hypothesis.)

SOCIAL THEORY: Anarchism
SYSTEM THEORY: Computational constraints
SOME RANDOM JOINT HYPOTHESES: (1) Anarchy is crappy because it's too hard to compute things in a distributed manner. Centralization makes coordination easier. (2) Anarchy could work better by examining and borrowing from distributed computation work. (3) There are certain problems for which it is really hard to find the solution, but it's easy to test if a proposed solution is right. (E.g. NP-complete problems, I think.) If there are lots of problems like this facing a society, more individual and organizational autonomy might increase the chances of solving them since there are lots of different approaches being tried. (A rather standard innovation argument, I suppose.)

SOCIAL THEORY: Feminism
SYSTEM THEORY: Neoclassical general equilibrium theory (e.g. Arrow and Debreu, greatest hits-style market behavior under optimizing rational trading agents, the sort of stuff in MWG (A-D joke here).
SOME RANDOM JOINT HYPOTHESES: (1) Economics isn't evil and patriarchal. Equilibrium theory, at least, is kinda innocuous -- just a pile of math that lightly suggests things about the world, e.g. markets could be efficient. (2) Economics is evil and patriarchal. Men in an ivory tower produced piles of math that lie about the world and convince (or rather, justify) free market policies that actually hurt the disadvantaged. [To be clear, this can turn in either direction:] (3) (The discipline of) economics is good because it encourages free markets which liberate women. (e.g. liberation from the domestic sphere) (4) (The discipline of) economics is bad because it encourages free markets which hurt women (due to the usual way capitalism hurts the disadvantaged, or maybe for more specific reasons.)

SOCIAL THEORY: Anarchism
SYSTEM THEORY: General equilibrium theory
SOME RANDOM JOINT HYPOTHESES: (1) Anarcho-capitalism (ok, so it's not anarchism proper) should work well because Arrow and Debreu say certain types of free markets achieve Pareto-optimal outcomes for their members. (2) General equilibrium theory is bunk because it doesn't take into account power relationships and the state (which are very important, according to leftist anarchist theory)

You can do this sort of thing forever and it is great. (I make no claim that these random hypotheses are actually true; they are merely interesting combinations of social and systems theory ideas.)

Finally: My roommate, Kevin, found nethack.alt.org, a telnet server for playing the most satirical, brilliant, and dramatic video game ever made, NetHack. Consider the following screenshots. He stepped on a polymorph trap and turned into a quivering blob, which forced him to drop all of his items. And he couldn't move, which was bad news when a troll started attacking. He managed to flee, but didn't have time to pick up all his dropped equipment. Naked and hungry, he made several attempts to slip past the troll -- now invisible and zapping him with magic missiles from a wand and other items it had jacked from the dropped pile of stuff. During one retrieval attempt, the troll moved to completely block the door, so Kevin went upstairs and found a pickaxe and, in a desperate final attempt, started chopping through a wall to create a shortcut on the other side. He almost made it, except for an invisible stalker -- appearing out of nowhere of course, since it's invisible -- that, with the troll, mercilessly destroyed him as he was starving from hunger.

Clearly this is the best game ever.

Finally^2: Muse in SF in a few weeks.

Computability and induction and ideal rationality and the simpsons

2007-03-14T21:41:00.000-07:00

Don't have time to read much right now, but received word about a neat-looking paper: Uncomputability: The Problem of Induction Internalized by Kevin Kelly.

Kevin Kelly's website has an awesome statement that mirrors thoughts I've been having for the last few years -- the incredible importance of computational constraints applied to reasoning and rationality:

Kuhn teaches that a single, deep success suffices to keep a competing paradign on the table. Not surprisingly, computational learning theory shows its superiority over ideal theories of rationality when we trade in our ideal agents for more realistic, computable agents. The foundation of the deep success is a strong structural analogy between the halting problem and the problem of inductive generalization, allowing for a unified treatment of both, from the ground up. One consequence of the approach is that one can often show that computable agents are forced to choose between ideal rationality and finding the right answer. I say "so much the worse for ideal rationality". Another is that there are learning problems that cannot be solved by computational means unless the Humean barrier between theorem proving and the external, empirical data is torn down.

Right on.

To make this post worthwhile, here is an insightful Simpsons clip.

(thanks to Shawn)

Iraq is the 9th deadliest civil war since WW2

2007-02-17T02:22:00.000-08:00

...9th largest by the metric of annual casualties (60,000 over three years). Funny how actual facts make current events clearer. Jim Fearon explains much more in his excellent FA article Why the U.S. Can't Win Iraq's Civil War.

Pascal's Wager

2007-02-15T00:12:00.000-08:00

Either God is tricky, or maybe probability is.

Pascal's Wager: Say there's only a small chance God exists. If you are an atheist but God does actually exist, He will send you to hell for eternity. This is infinitely bad. Therefore you should believe in God on the off-chance he does exist, since a small chance of something infinitely bad is worse than the alternative.

Believe in Pascal's Wager? Have I got a deal for you! says if you believe it, you should send Alex Tabarrok money because he will put in a good word to God for you. Hey, there's a small chance he has a direct line to God, which yields infinite utility (or avoids hell's infinite disutility).

FWIW, I'm thinking the paradoxes in this sort of arithmetic always happen when you start doing addition/multiplication distribution across those darn infinities. Like on the third page Tabarrok starts talking about p1*Inf - p2*Inf = (p1-p2)*Inf. That's shady shit.

And more about the big PW.

I don't like the SEP entry on it, because there's too much history and it talks too much about the boring stuff like the oddness of a decision to believe or disbelieve something.

When linguists appear on ironic parody talk shows

2007-02-04T22:31:00.000-08:00

The Colbert Report website is promising Steve Pinker this upcoming week. This could be great, or atrocious.

I bet it won't be as good as the somewhat-legendary Ali G interview of Noam Chomsky, reproduced below.

The Jungle Economy

2007-01-01T22:40:00.000-08:00

I wish I knew Hebrew. Well, fortunately there is also an English article version of Equilibrium in the Jungle (Piccione and Rubinstein '06):

In the typical analysis of an exchange economy, agents are involved in consumption and exchange goods voluntarily when mutually beneficial. The endowments and the preferences are the primitives of the model. The distribution of consumption in society is determined endogenously through trade.

This paper is motivated by a complementary perspective on human interaction. Throughout the history of mankind, it has been quite common (and we suspect that it will remain so in the future) that economic agents, individually or collectively, use power to seize control of assets held by others. The exercise of power is pervasive in every society and takes several forms. Often, power is purely physical. Sometimes, however, power is more gentle. In the male-female "market", for example, charm and attraction play a key role in obtaining a favourite mate. In an office parking lot, social conventions such as seniority allow control of the preferred parking places. The power of persuasion enables some to convince others to take actions against their own interest.

And I first thought this was fun from just from a modeling/boo-neoliberalism perspective because it stands the usual perfectly competitive free market model on its head -- their model, though substantively completely different, looks a whole lot like a standard Walrasian free market on the mathematical level. Indeed, they derive the same Pareto equilibrium results. That is, the Welfare Theorems, which show that a type of free market must be, in a sense, socially optimal, also hold in the Jungle. Thus if you think the Welfare Theorems prove that free markets are great (which I suspect all too many economists do, or did), then these Jungle Equilibrium results prove that a system of coercion and domination is great. I think this exercise casts doubt on the usefulness of formal models to shed light on substantive truths about the world, especially in such ideologically charged domains like social freedom and wealth.

When I first read about the jungle model some months ago (yay thesis procrastination), I thought the point was this sort of anti-neoclassical, anti-modelling irony. Indeed, near the start they mention a desire to "uncover some of the rhetoric hidden in standard economic theory". But actually, Rubinstein gives another interesting interpretation in the paper: by accepting the results, this indicates that a system of power and involuntary exchange may actually yield good social results in some cases. In any case, power, conflict and coercion are certainly not good to ignore.

Piccione's remarks are mostly only technical. Perhaps one is not allowed to say what one really thinks until you're a famous game theorist and willing to debunk your own entire discipline. :-)

Found on the wonderful website of Ariel Rubinstein, which has other fun things like a page of university town cafe's. Someone give this man a Nobel Prize on the double.

funny comic

2007-01-01T22:20:00.000-08:00

[doesn't fit well; please click.]

Thx Words and Other Things.

Anarchy vs. social order in Somalia

2007-01-01T20:23:00.000-08:00

In light of the Somali transitional government's recent military triumph over Islamist forces, the BBC did a piece of very interesting testimonials from everyday Somalis -- they all want a government for peace and order in their lives. Anarchy is extremely unpopular. (more info on this.) A musician complains that since nightclubs are shut down, his only work is at occasional weddings, and adds:

The wedding parties are big, spectacular affairs, you wouldn't think they were in a country without a government.

But it's only the people who are members of the big armed clans who can have these parties.

Other people get married but they cannot have big parties in case they are attacked and robbed. They just get married in secret.

I've been re-reading Hobbes and this sounds familiar -- the enjoyment of all sorts of goods requires a strong organization protecting you, to deter would-be thieves. That's what a good government does.

The musician concludes:

I cannot see all the different warlords working together. They are not going to fulfil their promises to disarm their militia.

They all still want to be president and we cannot have 15 presidents in Somalia.

And there's the case for central authority over multiple holders of power -- multipolarity, disunity, or whatever you want to call it. If the incentives for cooperation aren't there, which too often is the case, central authority is the only way to achieve order.

Somalia -- to say nothing of other places in the news like Iraq -- seems a textbook case illustrating why social order is the very first requirement for a society in which members can achieve happiness.

Surely order is not sufficient for people's happiness, as any totalitarian regime illustrates -- but it is absolutely necessary. Argue about tradeoffs between justice, liberty, and equality all you want; they're nothing without order to start with.

Double thesis action

2006-09-01T16:14:00.000-07:00

Earlier this year, it turned out that humans socially evolved cooperation through group competition and conflict. And now, it seems that biased evidence assimilation can happen through bounded rationality. Hooray.

A big, fun list of links I'm reading

2006-08-30T12:48:00.000-07:00

Since blogging is hard, but reading is easy, lately I've taken to bookmarking interesting articles I'm reading, with the plan of blogging about them later. This follow-through has happened a few times, but not that often. In an amazing moment of thesis procrastination, today I sat down and figured out how to turn my del.icio.us bookmarks into a nice blogpost, with the plan that every week a post will appear with links I've recently read, or maybe I'll use the script to generate a draft for myself that I'll revise, or something.

But for this first such link post, I put in a whole bunch of them beyond just the last week -- why have just a few when you could have *all* of them? Future link posts will be shorter, I promise.

Ariel Rubinstein: Freak-Freakonomics July 2006
posted 8/19 under economics
sarcastic, critical review of levitt & dubner's Freakonomics
New Yorker review of Philip Tetlock's book on political expert judgment
posted 8/19 under judgment, psychology
experts suffer from all the standard JDM biases. very interesting
Arkes and Tetlock: Attributions of Implicit Prejudice, or âWould Jesse Jackson âFailâ the Implicit Association Test?â
posted 8/19 under psychology
G A P M I N D E R: HOME
posted 8/17 under visualization
currently has great visualizations of world development data on the frontpage
Worldmapper: The world as you've never seen it before
posted 8/17 under geography, visualization
ariel rubinstein critiquing behavioral economics
posted 8/17 under behavioral.economics
CS 224M: Multi-Agent Systems (Final Paper)
posted 8/17 under AI
multi-agent systems readings -- belief revision, update, auctions, knowledge
The New Yorker: Fact (on wikipedia)
posted 8/17 under wikipedia
Words and Other Things
posted 8/17 under blog, philosophy
Barney Pell's Weblog
posted 8/17 under AI, blog
visualization of lebanon-israel casualties by geography
posted 8/15 under graphic, israel, lebanon, statistics, visualization
another great NYT data visualization -- it's hard to imagine how else you could show this data.
Greg Mankiw's Blog
posted 8/6 under blog
Nick Bostrom's home page
posted 8/6 under philosophy
Brad DeLong's Semi-Daily Journal
posted 8/3 under blog
The Hive
posted 8/3 under wikipedia
Inside Higher Ed :: Crossing a Line
posted 8/1
deborah frisch episode
The Data Game: How Economists Can Learn From Online Video Games
posted 7/31
Jaron Lanier: hazards of the new online collectivism (digital maoism)
posted 6/4
I love the irony of posting this to del.icio.us
Social Science Statistics Blog: Human irrationality?
posted 5/14 under rationality
2020 Science
posted 4/3 under science
Argues that computer *science* -- algorithms and theory -- will be become an essential part of the sciences in general. Points to systems biology and physics as examples. Computers aren't just about data handling and number crunching.
Berkeley Groks Science Show
posted 3/29 under podcast
EDGE: THE SELFISH GENE: THIRTY YEARS ON
posted 3/28 under podcast
mindless.pdf (application/pdf Object)
posted 1/11 under behavioral.economics, cognitive.science, economics, neuroeconomics, psychology, social.science
A Primer on the Doomsday Argument
posted 1/9 under doomsday.argument, future, philosophy, statistics

Kudos to del.icio.us RSS feeds and Mark Pilgrim's Universal Feed Parser.

4-move rock, paper, scissors!

2006-07-30T16:12:00.000-07:00

Contrary to baseless speculation, it turns out it is possible to have a four-player, non-degenerate RPS game. The game below is assymetrical: B does better than others, but you don't want to play it all the time because that makes you vulnerable to A. By contrast, D ain't so hot. But if you never play D, then your opponent can get away with A. It's uneven, but the optimal mixed strategy plays everything with non-zero probability.

Notation: the arrow from A to B means that A beats B. Everyone ties themself. No arrow indicates a tie. There is a tie between A and C. The previous analysis indicated there is no non-degenerate 4-RPS with no ties. This has only 1 tie, so it seems to be the best possible.

(This game is just the previous one with the link from A to C removed. This makes it so B no longer dominates C, since C is now invulnerable from A, unlike B.)

Two Middle East politics visualizations

2006-07-25T11:59:00.000-07:00

These are both useful summaries. Slate has a chart of relationships between Hamas, Hezbollah, Israel, and Lebanon, versus a number of different actors in the region. NYTimes has something similar on a map, also showing the sizes of ethnic/religious groups. Seems the NYTimes is most interesting as far as visualization/graphic design goes. It also has rich relational information and lots of predicates: does group X have oil wealth? is group X in the government of country Y? Etc. Seems ripe for a relational concept-learning clustering analysis...

[From Metafilter]

neuroscience and economics both ways

2006-07-20T12:57:00.000-07:00

I previously posted two neuroeconomics reviews. Here's a new one from this year in Trends in Cognitive Sciences. It's interesting because not only does it look at using psychological knowledge to inform economics, but it also reviews work in the other direction: using economic decision and organizational theory to study brain systems. For example,
here's a paper that analyzes brain reward circuitry using labor supply theory.

The review:
Sanfey, Loewenstein, McClure, Cohen: "Neuroeconomics: cross-currents in research on decision-making"

Social network-ized economic markets

2006-06-28T15:18:00.000-07:00

Extremely interesting -- a generalization of Arrow-Debreu equilibrium in which interactions are restricted along a social network. Kakade et al 2005. (Found through NIPS 2004 (which looks like a great conference)). Also a longer and more detailed related version: Kakade et al 2004.

Rock, Paper, Scissors

2006-06-02T18:11:00.000-07:00

Pure game theory is debatably social science, so here goes.

Inspired by various 5-, 7-, and even 25-move extensions to Rock, Paper, Scissors, my friends and I were wondering whether it's possible to have an even-move version. It doesn't seem possible, at least for 4-RPS. I conjecture that all even-RPS's might be similarly impossible. Here's a very! rough writeup.

Neuroeconomics reviews

2006-06-02T17:16:00.000-07:00

Here are two great reviews, from 2003 then 2005.

1) PLoS Biology: Economy of the Mind nicely reviews the field and many interesting experiments.

One annoyance: They need to say "Banburismus" is more commonly known as Bayesian learning. (Banbury, England was a city near Bletchley Park they got their paper from when doing Bayesian statistical codebreaking of the Enigma cipher in World War II. Read the story here in MacKay's excellent free online textbook.) Thanks to neurodudes for the PLoS link.

2) Neuroeconomics: How neuroscience can inform economics is written by the leaders of the field, advocating their approach. I like the detail and their careful descriptions of how cognitive neuroscience findings can enhance our understanding of economic phenomena.

Also, the second is useful to read since it's the target of criticism by the more recent The case for mindless economics, which I view as an empire-strikes-back sort of paper. I'm waiting for Part III of this saga...

Lordi goes to Eurovision

2006-05-20T21:02:00.000-07:00

Continuing our previous story: they've won! Check out the interview at the end of the semifinals here.

NYT reports:

ATHENS, May 20 (AP) — In what some fans called a stunning upset, a Finnish heavy metal band with monster masks and apocalyptic lyrics won the Eurovision Song Contest late Saturday.

The band, Lordi, caused a bit of a national identity crisis in Finland, where opponents accused it of devil worship and cringed at the thought that it might win.

The annual kitsch extravaganza, which was the springboard for the Swedish group Abba and Celine Dion, is known for its bland dance music and bubble-gum pop acts. This year groups from 24 countries faced off before tens of millions on television.

The band's members do not disclose their real names. The lead singer, Mr. Lordi, said its win, Finland's first, was "a victory for open-mindedness." "We are not Satanists," he said. "This is entertainment."

Drunken monkeys experiment!

2006-05-10T17:09:00.000-07:00

Monkeys drink more alcohol when housed alone, and some like to end a long day in the lab with a boozy cocktail, according to a new analysis of alcohol consumption among members of a rhesus macaque social group.
...
"It was not unusual to see some of the monkeys stumble and fall, sway, and vomit," Chen added. "In a few of our heavy drinkers, they would drink until they fell asleep."
...
In yet another study, the scientists gave a group of male monkeys 24-hour access to the beverage dispensers. According to the researchers, a spike in consumption immediately followed the facility’s working hours.

"Like humans, monkeys are more likely to drink after stressful periods, such as soon after the daily 8-5 testing hours and after a long week of testing," said Chen.

Link (courtesy of digg)

Easterly vs. Sachs on global poverty

2006-04-28T00:58:00.000-07:00

I started reading Jeffrey Sachs' new book The End of Poverty. The first 30 pages are excellent, but it starts getting arrogant and annoying quick. Substantively, I'm uncertain whether a big new development aid push will solve things.

Since I enthusiastically bum around course websites I'm not taking (bad habit, will stop real soon now), I was fortunate to run across an excellent debate between Sachs and William Easterly:

Easterly's view on Africa: The West Can't Take The Lead which has some amazing anecdotes about African educators and entrepreneurs. (Or, I think they're amazing only because I'm a condescending Westerner?)

Easterly reviews Sachs. Choice quote:

"Success in ending the poverty trap," Sachs writes, "will be much easier than it appears." Really? If it's so easy, why haven't five decades of effort gotten the job done? Sachs should redirect some of his outrage at the question of why the previous $2.3 trillion didn't reach the poor so that the next $2.3 trillion does. In fact, ending poverty is not easy at all. In those five decades, poverty researchers have learned a great deal about the complexity of toxic politics, bad history (including exploitative or inept colonialism), ethnic and regional conflicts, elites' manipulation of politics and institutions, official corruption, dysfunctional public services, malevolent police forces and armies, the difficulty of honoring contracts and property rights, unaccountable and excessively bureaucratic donors and many other issues.

and finally a debate in the Letters section! Oh what fun. And dreadfully important. I've got to read Easterly's The Elusive Quest for Growth next...

high irony

2006-04-24T16:27:00.001-07:00

What do the newly enriched Chinese bourgeois spend their money on? Vacations to visit Marx's home in Trier, of course!

The identity politics of satananic zombie alien man-beasts

2006-04-24T16:27:00.000-07:00

I thought Eurovision was weird enough already. But in addition to the usual fun mix of kitschy pop and Cold War legacy nationalism in its telephone voting politics, this year will see Finland's satanic band Lordi:

HELSINKI, Finland — They have eight-foot retractable latex Satan wings, sing hits like "Chainsaw Buffet" and blow up slabs of smoking meat on stage. So members of the band Lordi expected a reaction when they beat a crooner of love ballads to represent Finland at the Eurovision song contest in Athens, the competition that was the springboard for Abba and Celine Dion.

"In Finland, we have no Eiffel Tower, few real famous artists, it is freezing cold and we suffer from low self-esteem," said Mr. Putaansuu, who, as Lordi, has horns protruding from his forehead and sports long black fingernails.

As he stuck out his tongue menacingly, his red demon eyes glaring, Lordi was surrounded by Kita, an alien-man-beast predator who plays flame-spitting drums inside a cage; Awa, a blood-splattered ghost who howls backup vocals; Ox, a zombie bull who plays bass; and Amen, a mummy in a rubber loincloth who plays guitar.

Dragging on a cigarette, Mr. Putaansuu added, "Finns nearly choked on their cereal when they realized we were the face Finland would be showing to the world."

...

But not everyone in this Nordic country of five million views the monster squad as un-Finnish. Some Finns say that Lordi is right at home and that the band's use of flaming dragon-encrusted swords and exploding baby dolls expresses the warrior spirit of the Vikings.

Link: NYTimes: Finland Squirms as Its Latest Export Steps Into Spotlight

new kind of science, for real

2006-03-26T01:11:00.000-08:00

Great Microsoft Research report (from a workshop they held?): 2020 Science where they argue that computer science will become part and parcel of science in general. For example, computation theory will be important to understand biological organisms as information processing systems. This is basically a much better version of Wolfram's New Kind of Science argument -- I believe this one. The big shared insight is that computers aren't just about data storage and number crunching. Wolfram and the some of the Santa Fe complex systems people are really in to simulations, which is fine. But there's tremendous potential in computation theory -- algorithms, formal representations, and more. Empirical scientists are going to have to learn this stuff!

Mark Turner: Toward the Founding of Cognitive Social Science

2006-03-18T02:44:00.000-08:00

Where is social science? Where should it go? How should it get there? My answer, in a nutshell, is that social science is headed for an alliance with cognitive science.

Mark Turner, 2001, Chronicle of Higher Education

Libertarianism and evolution don't mix

2006-02-21T01:41:00.000-08:00

The best bit from a paper by Paul Rubin on evolution and politics: libertarianism could be unpopular today because libertarian societies would get destroyed in competition with egalitarian militaristic tribes back in the hunter-gatherer days. The paper has some great points on what prehistoric society was like -- fierce intergroup wars and competition. Hobbes/Locke/Rousseau states of nature, not so much. (While we're at it, I have to plug Boyd and Richerson's gene-culture coevolution theory.)

Evolutionary psychology has of course its own special dangers, but apparently Rubin also wrote a book on the subject of the biological basis of politics. Interesting...

academic blogging

2005-11-21T06:12:00.000-08:00

Interesting aricle on Slate about the risks and rewards of academic blogging. I've added John Hawks' interesting anthropology weblog to the of ones to read...

science writing bad!

2005-11-20T15:54:00.000-08:00

An two-step explanation for distrust of science: (1) journalists write up poor science or take out the evidence and information from a scientific study, then (2) people read that and criticize science for being unfounded, arbitrary, etc. Link. Some fun quotes:

Statistics are what causes the most fear for reporters, and so they are usually just edited out, with interesting consequences. Because science isn't about something being true or not true: that's a humanities graduate parody. It's about the error bar, statistical significance, it's about how reliable and valid the experiment was, it's about coming to a verdict, about a hypothesis, on the back of lots of bits of evidence.

and

So how do the media work around their inability to deliver scientific evidence? They use authority figures, the very antithesis of what science is about, as if they were priests, or politicians, or parent figures. "Scientists today said ... scientists revealed ... scientists warned." And if they want balance, you'll get two scientists disagreeing, although with no explanation of why (an approach at its most dangerous with the myth that scientists were "divided" over the safety of MMR). One scientist will "reveal" something, and then another will "challenge" it. A bit like Jedi knights.
The danger of authority figure coverage, in the absence of real evidence, is that it leaves the field wide open for questionable authority figures to waltz in. Gillian McKeith, Andrew Wakefield, Kevin Warwick and the rest can all get a whole lot further, in an environment where their authority is taken
as read, because their reasoning and evidence is rarely publicly examined.

I think the deeper problem is that support for scientific findings is inevitably too complex for everyone to understand. Everyone hates statistics, so everyone has no choice
but to trust those that know do it.

2005-10-20T00:29:00.000-07:00

Bush job approval ratings by different polling houses. Link. Does aggregation across different polls make sense like this? Interesting general trend as well...

Kurzweil interview

2005-09-07T11:04:00.000-07:00

Ray Kurzweil interviewed on his new book, The Singularity Is Near. Good points on neuroscience, artificial intelligence, nanotech and the like. But man, I thought Age of Spiritual Machines was a bit wacky... Complete model of the human brain by 2030? Please. (Though the observation that brain scan resolutions are doubling yearly is interesting.)

I like the discussion about the relationship of power and intelligence of orgnizations. Thinking about Kurzweil's bizarre-sounding scenarios is good because in his world, humans and organizations start becoming the same thing... which leads to insights on the intelligence of normal organizations today.

cognitive modelling is rational choice++

2005-09-02T12:40:00.000-07:00

Rational choice has been a huge imperialistic success, growing in popularity and being applied to more and more fields. Why is this? It's not because the rational choice model of decision-making is particularly realistic. Rather, it's because rational choice is a completely specified theory of human behavior, and therefore is great at generating hypotheses. Given any situation involving people, rational choice can be used to generate a hypothesis about what to expect. That is, you just ask, "What would a person do to maximize their own benefit?"

Similar things have been said about evolutionary psychology: you can always predict behavior by asking "what would hunter-gatherers do?" Now, certainly both rational choice and evolutionary psychology don't always generate correct hypotheses, but they're incredibly useful because they at least give you a starting point.

Witness the theory of bounded rationality: just like rational choice, except amended to consider computational limits of humans and organizations. This is just as imperialistic as rational choice, because it's a fully specified theory of human behavior.

Rational choice and bounded rationality are both cognitive and behavioral models. Rational choice posits a particularly odd cognitive model, but it's still cognitive, or at least lays out a well-defined set of requirements on a rational agent's cognition. The next obvious step is to make more sophisticated and realistic cognitive models that take into account heuristics people use and display biases we observe. Cognitive modelling for social science is the logical extension of the rational choice research program.

Submit your poker data!

2005-09-01T22:28:00.000-07:00

Upload your poker hand histories to www.pokernomics.com, economist Stephen Levitt's fringe-of-economics project to study what are effective strategies in poker. This absolultely makes sense to me as an economics research project, only because I'm used to thinking of economics from the view of multi-agent systems and game theory... This is definitely all about game theory, maybe not economics. Game theory reaches beyond the bounds of the study of goods and services...

blog entry on it

Bayesian analysis of intelligent design (revised!)

2005-08-01T00:52:00.000-07:00

This is a revision of my earlier post. In Jaynes' awesome statistical manifesto book (another link), I just saw for the second time the odds ratio form of Bayes' rule, which is a lot cleaner for this sort of static analysis. So anyway...

Pick an organism. Two propositions, H and E, each may be either true or false about it.

H: the organism was designed by an intelligent creator.
E: the organism looks like it was designed by an intelligent creator.

Most of what I know about Intelligent Design theory (ID) is from seeing a talk by Michael Behe (may 2005). He had to major lines of argument: (1) it is implausible that an evolutionary process could produce life that looks as if it was intelligently designed. (2) Since it looks like it was intelligently designed, it was. He really emphasized the E component of the argument.

Justifications for E: Lots of organisms look like they were intelligently designed. They have complex and intricate mechanisms involving coordination among many components. Sometimes they look like things humans would design: for example, bacteria locomotion devices sometimes bear uncanny resemblance to human-designed motors or propellers.

Behe was really into showing all these quotes from pro-evolution authors like Dawkins who note this fact: many forms of life appear to us as if they were designed. Consider one of those organisms where E is true. This organism looks as if it was designed.

However, does that mean it actually was designed? That's a different proposition, the difference between H and E. Since I distrust human intuition on matters of intention ascription (we do it too often), I'd rather look towards a rational framework.

What is the plausibility that this ID-looking orgnanism actually was designed? That's asking to evaluate P(H|E). Bayes rule tells us how to find P(H|E): the plausibility of a hypothesis H, given the truth of a proposition E (evidence).

Bayes rule derived:


P(H|E) P(E)  =  P(E|H) P(H)

P(H|E)  =  P(E|H) P(H)  =  likelihood * prior
           -----------     --------------------
              P(E)         marginal likelihood

I'll explain the interesting conditional hypotheses H|E and E|H and the prior H in a second. P(E) denotes the likelihood to find an organism that looks like it was intelligently designed. Though P(H|E) denotes the plausibility H is true given E is true, to evaluate it we have to look at the probability E could be true independently. I guess we could take a poll of how many organisms picked at random look intelligently designed, but that confuses me greatly with what the events are and such.

Instead, let's eliminate the P(E). The standard way to do this is to expand it into a sum. But then it's hard to see how increaes and decreases in different variables affect the overall outcome. Instead, we use here the odds-ratio form of Bayes' rule.


P(H|E)   =  P(E|H)  P(H)  / P(E)  and of course  
P(~H|E)  =  P(E|~H) P(~H) / P(E)

so then divide...

P( H|E)     P(E| H)   P( H)
-------  =  ------- * -----
P(~H|E)     P(E|~H)   P(~H)

Let O(A) mean the "odds" of A. O(A) = P(A) / P(~A). If you want to roll a 6 on a die , your probability is 1/6 ("one out of six"), and your odds are 1:5 ("one to five against.") Note that P(A) > P(B) implies O(A) > O(B), so you can use your up and down intuitions about likeliness just fine with odds ratios. This notation makes Bayes' rule look really clean:


           P(E| H)
O(H|E)  =  -------- * O(H)
           P(E|~H)

It's easy to see: The "looks designed => was desgned" reasoning of ID is supported by high P(H), high P(E|H), and low P(E|~H).

H|E: if the organism looks like it was ID'd, the plausibility it actually was. (the core ID argument)

H: prior belief organism was designed
E|H: if the organism was ID'd, the plausibility it looks ID'd.
E|~H: if the organism was not ID'd (e.g. it evolved), the plausibility it looks ID'd.

P(H) is a pretty nasty prior: forgetting the evidence of whether it looks designed, what's the chance an organism was intelligently designed? That question seems to hinge on prior beliefs in the existence and activity of a creator. It's all that not up for debate. If you are already certain God exists, it may be reasonable to entertain the notion that organisms were intelligently designed. If you are less certain God exists, you may believe P(H) to be lower.

P(E|H) at first seems odd: certainly, if a creator intelligently designed an organism, doesn't that mean we'd be able to tell? Well, not necessarily: what if a designer makes decisions we cannot understand, or we can't divine the intelligence in the design of an organism? If that is likely to be the case, then P(E|H) decreases, and H|E becomes less likely. I don't think this question is mentioned that much in creationism/evolution debates.

P(E|~H) is a big point of controversy. Some evolutionary theorists argue P(E|~H) can be quite high. e.g. Dawkins' "Blind Watchmaker": Nature can create impressively complex and purposeful looking life through random chance and natural selection. Behe's presentation seemed to unfairly argue down E|~H by only considering gradualist Darwinist explanations of evolution. It seems implausible that one-at-a-time tiny mutations could produce big complex systems like the eye or the immune system. That is, it's too hard to get out of local minima. However, to examine E|~H you need to look at all alternatives to ID. Complexity theory explanations might note that great complexity and order can emerge out of randomness; thus, the formation of complex systems through evolution is more plausible than our intuitions might tell us. Or exaptation: old adaptations might be put to new uses.

So, here's how things line up for and against ID:

belief	pro-ID belief	reasons	anti-ID belief	reasons
E\|H	high	ID'd organisms will look ID'd to us	low	we may not understand a designer's designs; they may not look familiar or intelligent to us
E\|~H	low	gradualist adaptationism is unlikely to explain complex systems	high	blind watchmaker, complexity theory, exaptation... an evolutionary process could lead to outcomes that look as if they were designed.
H	high	prior belief in a creator and that creator's likelihood to design life	low	prior disbelief in a creator and that creator's likelihood to design life

Conclusions: (1) lots hinges on a hard-to-debate prior P(H). This implies that ID can be quite rational given a strong prior belief in a creator, or quite irrational given a strong prior disbelief in a creator. One of ID's aims is to rationally disprove evolution by arguing about E|~H and E a lot. But I don't think argument over H is going to get you anywhere; if your prior beliefs in God are very different, it may be difficult to reconcile your beliefs in evolution versus intelligent design.

(2) Don't forget the neglected E|H: if an organism was intelligently designed, the plausibility it looks that way to us.

Caveat: I'm confused how to analyze a given organism versus picking one at random. Does that make a difference?

Also, I'm wondering how to determine how much priors matter. When should argumentation over evidence for evolution force you to revise your beliefs about God? Is there a rational way to do this belief revision? If there isn't, are we all condemned to stick to our prior beliefs?

ID advocates are trying to establish a scientific/rational argument for intelligent design:

The theory of intelligent design (ID) holds that certain features of the universe and of living things are best explained by an intelligent cause rather than an undirected process such as natural selection. ID is thus a scientific disagreement with the core claim of evolutionary theory that the apparent design of living systems is an illusion.

In a broader sense, Intelligent Design is simply the science of design detection -- how to recognize patterns arranged by an intelligent cause for a purpose. Design detection is used in a number of scientific fields, including anthropology, forensic sciences that seek to explain the cause of events such as a death or fire, cryptanalysis and the search for extraterrestrial intelligence (SETI). An inference that certain biological information may be the product of an intelligent cause can be tested or evaluated in the same manner as scientists daily test for design in other sciences.

I'll put SETI in a similar category with ID, but those other quite fine fields mentioned all have reasonable and agreed-upon priors, like whether you suspect humans lived on a certain island, wanted to commit arson, etc. It seems reasonable to use prior hypotheses about the existence and intentions of human beings. In a given scientific community, there's probably decent consensus on the priors. A bunch of arson analysts might agree on H, that of all fires a given percentage of them are arsons. But given the evidence surrounding a specific fire, they can investigate and argue about H|E, whether it was an arson, by examining whether or not an arson is a likely explanation for the facts of the case: E|~H and E|H.

But of aliens and God, well, the H is a matter of faith. In evolution theory, E|~H is super controversial, and E|H is ignored. ID'ers and evolutionists just don't have enough common ground for there to be a good debate.

searchin' for our friend, homo economicus

2005-07-31T23:56:00.000-07:00

I must have seen a zillion draft versions of this study floating around online, but here's a terrific preprint: “Economic man” in cross-cultural perspective: Behavioral experiments in 15 small-scale societies (Henrich, Boyd, Bowles, Camerer, Fehr, Gintis, McElreath, Alvard, Barr, Ensminger, Henrich, Hill, Gil-White, Gurven, Marlowe, Patton, and Tracer 2005 (!)). So looks like we're now pretty sure, culture affects cooperation, you can see it in social practices. It's a really neat study. The writeup in this version is terrific, they talk about implications for culture-gene evolution and have great statistical analysis of cultural factors on ultimatum game performance.

balkanized USA

2005-07-31T02:30:00.000-07:00

From the same site, this is fun.

war death statistics

2005-07-31T02:12:00.000-07:00

What a project -- an impressively painstaking compilation of 20th century civilian and military casualties. Summary: lots of people were killed. Interesting are the comments on morality and how prejudgement leads to differing casualty estimations -- estimates vary wildly for controversial regimes, like Castro's Cuba (I've heard lots about), but are suspiciously round and agreed-upon for incidents that scholars seem to care less about, like the Congo Crisis of the 1960's (I've heard not so much about.)

guns, germs, & steel pbs show?!

2005-07-10T20:52:00.000-07:00

Looks like it's become a mini-series: Jared Diamond's Guns, Germs, and Steel has hit PBS! Great book, if repetitive and a little too ambitious -- he has a great environmental/technology explanation of the differences in societal development between Europe and the Americas, but he's pretty weak when trying to tackle Asia vs. Europe, or a number of other situations talked about near the end of the book. But anyway, it's fantastic social science.

Wish I had a TV around... it would be nice if the show made it onto the web. For some time, I remember hearing that all of Commanding Heights was available for free on the web, but it looks like they've taken it down. We shall see.

the psychology of design as explanation

2005-07-09T10:20:00.000-07:00

Since I posted the link to his blog, Baron just wrote about Cardinal Schönborn's anti-evolution Op-Ed piece. I agree absolutely that people should learn about the psychology of judgment and probability for these sorts of questions, where it's really hard to understand that random processes can generate things that seem not so random.

I'm still thinking about how the psychology of judgment plays in to the analysis below. I have a feeling that people's intuitions are usually too hospitable for explanations based on intention. E.g.: People are poor, therefore someone is trying to make them poor. Organizations (corportations, governments) do things, therefore someone (say, at the top) ordered them to do these things. Natural disasters happen, therefore someone is wishing them upon us. Etc., etc. I'm still not sure how a bayesian dissection of whether "looks intentful" implies "is intentful" shows us whether such an "intent-seeking" bias (hey, I have to call it something) is correct or erroneous. Hopefully more to come.

Also: there was an older posting by Tabarrok on MR arguing that theism makes ID quite reasonable, and atheism makes evolution quite reasonable. This would be the effect of the dominance of the P(H) prior, I believe.