damn statistics and sloppy journalism

December 25, 2009

in Flotsam and jetsam

As a former journalism major and current writer, the decline of journalism these days is very distressing to me. I could go on about this, but will cut to an item from one of the few papers that I still turn to for decent writing and fact-checking, The New York Times. It really depressed me.

The article was about the supposed increasing dissatisfaction with each of three versions of the Amazon Kindle. The author provided three pie charts using the starred reviews of each device on the Amazon site. First and foremost, these charts falsely represent the “data.” Here’s one of the charts:

Clearly, the slice representing one-star ratings, at 7%, if far larger than 7% of the pie. Here are these same data charted in Excel, without adjustment:

Further, as is pointed out in the comments on the piece, the one-star user ratings for the Kindle on the Amazon web site include comments from people who do not own a Kindle or are simply opposed to this type of technology versus real books, and do not represent actual product satisfaction. Amazon has started flagging ratings from verified purchasers; this commenter notes that 50% of the recent one-star ratings of the Kindle DX are not from Kindle owners. Other people note this is not a random sample and doesn’t really mean a whole lot.

The article also does not discuss — although commenters do — that there is typically a difference in satisfaction levels between early adopters of a new technology and later-generation users as expectations rise. There are other worthwhile comments about the methodology in the comments as well. It’s worth it to go read them.

Yes, I own a Kindle and like it very much. My point here isn’t to defend the product, or Amazon. I’d be (am) discouraged at reading an article in a major news source that so sloppily discussed any product in this manner. Shame on the Gray Lady.

{ 1 comment }

Comments on this entry are closed.

Previous post:

Next post: