Documentation Metrics
I recently had a telephone conversation regarding documentation practices that planted a seed. I’m pretty sure this isn’t a new idea, how could it be, but it’s something that I’ve never seen explicitly stated.
I’m a big fan of static analysis and code metrics in general. The reason I’m a fan is that there’s something in my constitution that likes to see improvement over time. If I think a number correlates strongly enough with software quality, I’ll enjoy seeing that number improve. It’s soothing to me.
What I haven’t seen around is a metric that can be used to measure the quality of a software library’s documentation. Without such a metric, any investment in time towards improving its documentation will never be a sexy thing for me to work on, and I’m probably not alone in these feelings, since most projects have terrible documentation.
I think the # 1 most important thing to measure about a library’s documentation is its web analytics.
Here’s my train of thought: Ultimately documentation is about connecting users to answers. If your page is a promising candidate for solving their problem, then it should receive more quality visits. Anything you do that increases the number and/or quality of those visits is a step in the right direction. If you attempt to game the search engines, it would negatively affect the quantity of hits you’re documentation gets since search engines already take these kinds of things into account.
I’m not exactly sure how the metric will be computed but I’m absolutely sure it’d involve hits, duration of visits and bounce back rate.
I’m imagining a 3rd party documentation analytics embed code that can be used freely on open source projects and that can display on each documentation page a measurement of its “quality” and how its trended over time. It’d allow you to compare two equivalent projects’ documentation, and more importantly tell how much effort is being spent on a projects documentation.
Wouldn’t that be nice? So many open questions. What do you think?
