Few people have had a better 2008 than Nate Silver. The 30-year-old numbers whiz fed exhaustive polling data and a half-century's worth of election returns into a model he designed, allowing him to call the presidential primaries and the general election with uncanny accuracy and turning his website, FiveThirtyEight.com - named for the total number of Electoral College votes -- into a must-read for political junkies. Meanwhile, the PECOTA system he developed for crunching baseball stats helped him predict a successful season for the previously horrible Tampa Bay Rays, who went on to make the World Series. I spoke with Silver about increasing the signal to noise ratio in political polling, and the implications of his work for business intelligence.

Know It All: A year ago you were a successful baseball wonk, now you're talking politics on national television. How did that happen?

Nate Silver: It used to be, four or eight years ago, you might get one poll of from a state in September or October, and that's all you'd get. Now you have 14 or 15 national survey research firms, they're all putting out polls in different states, you have the local pollsters, you have so much information to sort through that someone has to play traffic cop to some extent, and say this is what we should look at, this is how we can aggregate this information in a way that's sensible.

On one hand there was academic discourse about politics, very theoretical, some of it is good and some is not. On the other hand you have the campaigns, which are very secretive; if they know what they're doing they're not going to reveal it to anyone. I tried to bridge that gap, with numbers and analysis, but not being so intimidating about it, or relying on old clichés. I felt the same frustration you get with the sports media, with the conventional wisdom that doesn't hold up to scrutiny. That was a lot of it, just wanting to improve the political discourse.

Silver: We went back and looked at data from presidential elections in all the states, and we interpreted the polling data to evaluate who's been the most accurate, who has the bias - although there may be good reasons for bias, it might be based on philosophies on things like turnout. We have rules set up and algorithms set up for dealing with all these different polls.

Sometimes you might see two polls, one has Obama ahead by five in Pennsylvania and one has him ahead by 12, and if you know anything about the pollsters it might be what you expect, given the lean they have toward one candidate. If you get a poll that looks like an outlier, nine times out of ten you can understand why it came up with the numbers that it did - some weird assumption about turnout, or whatever else. You might have 15 polls that come out and one really weird number that comes out by chance or bad design. Those should be the kind that you kind of disregard, even though they're interesting and drive the most discussion.

Silver: There's no gut in the model itself. If we learn something that can improve the analysis, we'll change the model, but we won't just make an adjustment for one individual result. There may be some cases where we kind of put an asterisk out for the readers, and say we have a process and we're not going to kick this poll out, but frankly you should disregard it. There are standard procedures, and more anecdotal observations. When you have that many polls coming out, one bad poll won't skew things that much. It's kind of hard to cheat the system.

Silver: During primaries you get such a rich data set, you can really drill down to the congressional district level and see what helps a candidate in a particular place. You could look at the data in process - in the general election, you have Election Day and that's it, you have a terrific data set but not much to do with it. In the primaries, one state comes after another, you can use that to inform your predictions, and you have new information from one week to the next. And because Clinton and Obama were so similar ideologically, it brought out every possible kind of characteristic. We were literally looking at different ethnicities from European backgrounds, to see who was more inclined to one candidate or the other.

Silver: A lot of it is not all that complicated. You need a good model, and you need to not take shortcuts. You need to sweat the small stuff. The parameters of the problem are not all that large in politics, there are 50 states and a given number of polls, and you're trying to figure out who's going to win. It's a pretty clearly defined problem. I've got a problem-solving approach - my background as an undergrad was in economics, not statistics, I'm not that interested in the numbers for the numbers sake, but more in the question: Who's going to win the presidency?

There is a little finesse required here and there. The relationship between state polls and national polls is sometimes kind of hard to determine empirically, because we don't have much data -- before 2004 there weren't many state polls, so there isn't a big empirical basis for how you relate these two things. So there's some guesswork involved, you have to trust your intuition. When you are building a model, you have to understand the problem you're trying to solve.

Silver: Sometimes if you depend purely on the numbers, there's a kind of inverse selection at work. That applies to things like site selection for a business. A site might look great on paper for retail space, but maybe there's a porn store across the street, or a murder happened there two years ago. There are negative outliers you have to be very careful of.

Getting this right takes a combination of things. You want to be really meticulous, but also keep in mind the big picture, the problem you're trying to solve. You can't outsource all this stuff to India. You could have a million PhDs working on it, people who are very smart, but if they don't understand the landscape of American politics, or the way a baseball player's career looks like, they won't get it right. There's an art to it as well as a science.

There are a lot of parallels between this kind of political analysis and other areas. I went to a conference in the DC area about national security, and I could see parallels between predictions in baseball and predictions based on picking up terrorist chatter. There are overlaps with the way a company like McDonalds understands its reputation in different countries. The art is not in collecting information, but in being able to sort through it, and parse it, and find meaning within all that noise.

Silver: There's no real magic bullet. The difficult thing to do in baseball is to find premium talent. Last year, they had four or five All-Star caliber players, but they had a lot of dead weight on their roster, and their bad players were really bad and dragged down the whole team. So they'd already gotten the hard part right, and they decided to field a real baseball team instead of a random collection of talent.

Silver: Other factors do come into play

Silver: I'm still doing some baseball work, I still love baseball, I have to find a way to divvy up my time between that and the political stuff. I'm thinking about a couple of books. One concept is more politics, trying to define this science of electoral sabremetrics [statistical analysis of the sort used to great effect in baseball]. I'm also looking at how you create accurate predictions about things. The art and science of forecasting - hurricanes, air traffic control, people who predict the future in different ways, what are the commonalities? It's all driven by data, and it has tremendous implications for a lot of disciplines.