Last week I attended IDCC14 in San Francisco, where I was immersed in digital curation. Naturally, things like peer-review and domain knowledge/expertise were on my mind, as things to consider with research (data) publishing. Then I saw this story about a Twitter data scientist “hacking” BART. I immediately retweeted it with a remark about not understanding how BART works (which is true). Right now, particularly in the Bay Area, there are a lot of “hacks” to solve problems that aren’t actually problems. It’s just that the people who perceive the problems don’t have the full picture and “hack” the solution for them, which in the case of public transit is only a small segment of the users. Joe Eskenazi wrote a good column about this in SF Weekly that looks at both sides. (Disclosure: We play futsal together, or did until I broke my finger in a game. Miss you Kamikaze!)
Reading Haque’s paper on arXiv, it’s clear to me he’s got the math and science stuff down – it’s the transportation that he’s lacking. A common issue with data scientists is that they often have the analytic and technological skills, but lack the domain expertise. So they have to work with experts or learn enough to become an expert (which takes time). Even if he just ran some of these ideas past a transportation engineering or planning masters student, they could have helped him refine the “problem”. Haque seemingly wrote this in a vacuum, so when it saw the light of the internet the transportation folk just picked it apart based on the faulty assumptions of how commuter rail fares work. (Note: Everybody thinks they’re an expert on transportation because they use it. Sorry, you’re most likely not.)
This is where peer-review could have been a good thing. I looked at the paper on arXiv to see if it was published elsewhere, such as a journal or conference. arXiv is often used as an open access repository for pre-publication manuscripts. Haque’s paper was not (as of yet) published or submitted elsewhere, which means there’s been no obvious peer-review which explains a lot.
Peer-review would have pointed out the flaws in Haque’s methodology (assuming the reviewers had the expertise). Instead, he got the open peer-review of Twitter and lots of transportation professionals and advocates, many of whom are tired of tech workers “hacking” transportation in a way that doesn’t really help. (Seriously, fare evasion with the help of an app and surge pricing on transit? BART isn’t Uber!)
There is a lot broken with peer review, but this is one case where it could have helped. I really hope Haque can hook into the very passionate and knowledgeable transportation community here in the Bay Area and start “hacking” some real problems. Let’s do it!