Seth Roberts and academic blogging

I saw Seth Roberts briefly speak today (at an odd event) about self-experimentation. He tried drinking flavorless sugar water and it led him to lose lots of weight. He also did a great variety of other self-experiments over more than a decade, written up here (and IMHO the other ones are much more interesting).

I briefly spoke to him there and told him I heard about his work from Andrew Gelman's blog. He seemed surprised to (semi-)randomly meet someone who reads it. I think this is mistaken -- that particular blog seems quite popular in statistics/social science world. In fact, Gelman's blogging of Roberts' self-experimentation paper got picked up by the Freakonomics folks and it became a sensation and then a book deal. (Story.)

Also note, John Langford says of his own machine learning blog:

This blog currently receives about 3K unique visitors per day from about 13K unique sites per month. This number of visitors is large enough that it scares me somewhat—having several thousand people read a post is more attention than almost all papers published in academia get.

So true. Conclusion: blogs are a more effective medium for intellectual influence than journal articles.

(p.s. I wonder if free online journals will help fill the gap. E.g. PLOS or arXiv. Or in fields near and dear to Social Science++: theoretical economics, and judgment and decision-making ... and hopefully others? At least computer scientists are always good about posting their papers online... this seems to be the trend for younger researchers in general.)

Statistics is big-N logic?

I think I believe one of these things, but I'm not quite sure.

Statistics is just like logic, except with uncertainty.

This would be true if statistics is Bayesian statistics and you buy the Bayesian inductive logic story -- add induction to propositional logic, via a conditional credibility operator, and the Cox axioms imply standard probability theory as a consequence. (That is, probability theory is logic with uncertainty. And then a good Bayesian thinks probability theory and statistics are the same.) Links: Jaynes' explanation; SEP article; also Fitelson's article. (Though there are negative results; all I can think of right now is a Halpern article on Cox; and also interesting is Halpern and Koller.)

Secondly, here is another statement.

Statistics is just like logic, except with a big N.

This is a more data-driven view -- the world is full of things and they need to be described. Logical rules can help you describe things, but you also have to deal with averages, correlations, and other things based on counting.

I don't have any fancy cites or much thought yet in to this.

Here are two other views I've seen...

• Johan van Benthem: probability theory is "logic with numbers". I only saw this mentioned in passing in a subtitle of some lecture notes; this is not his official position or anything. Multi-valued and fuzzy logics can fit this description too. (Is fuzzy logic statistical? I don't know much about it, other than that the Bayesians claim a weakness of fuzzy logic is that it doesn't naturally relate to statistics.)
  
• Manning and Schütze: statistics has to do with counting. (In one of the intro chapters of FSNLP). Statistics-as-counting seems more intriguing than statistics-as-aggregate-randomness.
  

Not sure how all these different possibilities combine or interact.

Feminists, anarchists, computational complexity, bounded rationality, nethack, and other things to do

I was planning to write some WordNet lookup code tonight. But instead I've learned of too many intersecting things.

First, there are a zillion things to do this weekend (hooray flavorpill):

1. Picasso and American Art exhibit continuing at SFMOMA. I saw it very briefly last weekend but want some more. And Doug claims there's an interesting photography exhibit there too.
   
2. Reading from We Don't Need Another Wave: Dispatches from the Next Generation of Feminists, a fascinating looking book I've seen many times in the bookstores around here. By that I mean at least Modern Times (the neat Mission bookstore) and the Anarchist Collective Bookstore (out on the Haight). And the reading is at Modern Times, just down the street from my house! Amazing. Tomorrow at 7:30.
   
3. Since anarchists were just mentioned, fortuitously there also appears: the Bay Area Anarchist Bookfair this Saturday and Sunday! Speakers and books down by Golden Gate Park, oh my.
   

Can't say I'm a radical feminist or even an anarchist, but I *loved* reading that stuff back in high school. Went through waay too much of An Anarchist FAQ (which is amazingly comprehensive and highly recommended -- ironically, everyone who's read it falls into calling it "The anarchist faq" because, I guess, it's so good); and that, with snippets of MacKinnon, Foucault and other easy-to-misunderstand social theorists, I went around denouncing capitalism, statism, and patriarchy left and right. Temporarily believing and advocating radical views is the most educational thing I know how to do. (Guess I've come a long way since then, writing posts in praise of Hobbes... Though I will say reading the conservatives was rarely as fun, though Burke is pretty insightful.)

And THEN -- also tonight, I learned about the Kevin Kelly computability and induction work. I've been thinking about complexity and social behavior a lot -- computational complexity is a pervasive phenomenon in all sorts of human endeavors. It's impossible for people or organizations to solve certain types of problems, and many of the things we are our best responses under these constraints. We can't know everything, we can't consider all possibilities, we can't radically change our own beliefs, and we can't even enumerate what we already know.

There's *tremendous* potential for computational complexity theory to be applied to formal epistemology -- mathematical philosophy really needs to get away from being so reliant on pure logic. (Or maybe this is already happening.) To say nothing of cognitive modeling and theoretical neuroscience. And perhaps behavioral economics, if they ever get around to rediscovering Herbert Simon in the right way, not the small-tweaks-to-neoclassicalism that dominates mainstream behavioral theories (e.g. contrast Camerer vs. Rubinstein, p. 44 here).

But computational theory can solve this! It's a principled way to describe bounded rationality without resort to procedural rationality and all its messy algorithms and ad-hoc state machines. You could make bounded rationality models with substance and generalizable principles. In the same way that neoclassical microeconomics makes interesting insights about market behavior using convex optimization and other mathematical techniques that allow them to make statements about large classes of market situations, not just one particular market situation, I could imagine models that let you make statements about the behavior of large classes of, say, bounded-memory computational automata, not just some particular algorithm that happens to implement useful heuristics for a bounded-memory agent.

Anyways, all you hordes of Social Science++ readers may know that in the old days the my tag line was:

{social, political, economic} cross {cognition, behavior, systems}

That is, the cross-product with 9 combinations -- cells in a 3x3 matrix:

	social	political	economic
cognition
behavior
systems

In the spirit of insightful, serendipitous combinations of ideas, we can cross across feminism and anarchy versus computational complexity and, say, general equilibrium theory.


SOCIAL THEORY: Feminism
SYSTEM THEORY: Computational constraints (Turing, big-O, Chomsky hierarchy, etc.)
SOME RANDOM JOINT HYPOTHESIS: (1) Sexism is a result of computational constraints: A prerequisite to sexism is reasoning via gender. There's a bounded rational explanation: a person's gender is a neat little binary property of them, with which inferences can be made. It's more accurate than ignoring gender, but rather large mistakes can be made and when they are malicious or particularly suboptimal they are called sexism. (There are numerous issues with this hypothesis.)

SOCIAL THEORY: Anarchism
SYSTEM THEORY: Computational constraints
SOME RANDOM JOINT HYPOTHESES: (1) Anarchy is crappy because it's too hard to compute things in a distributed manner. Centralization makes coordination easier. (2) Anarchy could work better by examining and borrowing from distributed computation work. (3) There are certain problems for which it is really hard to find the solution, but it's easy to test if a proposed solution is right. (E.g. NP-complete problems, I think.) If there are lots of problems like this facing a society, more individual and organizational autonomy might increase the chances of solving them since there are lots of different approaches being tried. (A rather standard innovation argument, I suppose.)

SOCIAL THEORY: Feminism
SYSTEM THEORY: Neoclassical general equilibrium theory (e.g. Arrow and Debreu, greatest hits-style market behavior under optimizing rational trading agents, the sort of stuff in MWG (A-D joke here).
SOME RANDOM JOINT HYPOTHESES: (1) Economics isn't evil and patriarchal. Equilibrium theory, at least, is kinda innocuous -- just a pile of math that lightly suggests things about the world, e.g. markets could be efficient. (2) Economics is evil and patriarchal. Men in an ivory tower produced piles of math that lie about the world and convince (or rather, justify) free market policies that actually hurt the disadvantaged. [To be clear, this can turn in either direction:] (3) (The discipline of) economics is good because it encourages free markets which liberate women. (e.g. liberation from the domestic sphere) (4) (The discipline of) economics is bad because it encourages free markets which hurt women (due to the usual way capitalism hurts the disadvantaged, or maybe for more specific reasons.)

SOCIAL THEORY: Anarchism
SYSTEM THEORY: General equilibrium theory
SOME RANDOM JOINT HYPOTHESES: (1) Anarcho-capitalism (ok, so it's not anarchism proper) should work well because Arrow and Debreu say certain types of free markets achieve Pareto-optimal outcomes for their members. (2) General equilibrium theory is bunk because it doesn't take into account power relationships and the state (which are very important, according to leftist anarchist theory)

You can do this sort of thing forever and it is great. (I make no claim that these random hypotheses are actually true; they are merely interesting combinations of social and systems theory ideas.)


Finally: My roommate, Kevin, found nethack.alt.org, a telnet server for playing the most satirical, brilliant, and dramatic video game ever made, NetHack. Consider the following screenshots. He stepped on a polymorph trap and turned into a quivering blob, which forced him to drop all of his items. And he couldn't move, which was bad news when a troll started attacking. He managed to flee, but didn't have time to pick up all his dropped equipment. Naked and hungry, he made several attempts to slip past the troll -- now invisible and zapping him with magic missiles from a wand and other items it had jacked from the dropped pile of stuff. During one retrieval attempt, the troll moved to completely block the door, so Kevin went upstairs and found a pickaxe and, in a desperate final attempt, started chopping through a wall to create a shortcut on the other side. He almost made it, except for an invisible stalker -- appearing out of nowhere of course, since it's invisible -- that, with the troll, mercilessly destroyed him as he was starving from hunger.



Clearly this is the best game ever.

Finally^2: Muse in SF in a few weeks.

Computability and induction and ideal rationality and the simpsons

Don't have time to read much right now, but received word about a neat-looking paper: Uncomputability: The Problem of Induction Internalized by Kevin Kelly.

Kevin Kelly's website has an awesome statement that mirrors thoughts I've been having for the last few years -- the incredible importance of computational constraints applied to reasoning and rationality:

Kuhn teaches that a single, deep success suffices to keep a competing paradign on the table. Not surprisingly, computational learning theory shows its superiority over ideal theories of rationality when we trade in our ideal agents for more realistic, computable agents. The foundation of the deep success is a strong structural analogy between the halting problem and the problem of inductive generalization, allowing for a unified treatment of both, from the ground up. One consequence of the approach is that one can often show that computable agents are forced to choose between ideal rationality and finding the right answer. I say "so much the worse for ideal rationality". Another is that there are learning problems that cannot be solved by computational means unless the Humean barrier between theorem proving and the external, empirical data is torn down.

Right on.

To make this post worthwhile, here is an insightful Simpsons clip.



(thanks to Shawn)