Guest Post: Alternative Forms of Data

Today, Nick Bekas, professor of English at Valencia College, questions how far quantitative data can take us when we’re trying to understand what developmental education students need. He also suggests some qualitative methods that can make the picture more complete and thus improve decisions that affect individual students and entire institutions.

Every time I see a chart with a statistic, I immediately try to put it in context. I try to wrap it in details and identify its origin. And I don’t mean from a statistical perspective. What I think about are the people behind the stat:

•    Who are they? 
•    What do they think? 
•    What do they feel? 

A number cannot capture these things.

Yet, for the most part, statistics are the primary source for decision making. This stems from tradition and an ingrained suspicion of qualitative data prevalent amongst researchers. Also, numerical data are easy. They are readily digestible and “clean.”  How do you argue with 35 percent, beyond its method of collection? And when more sophisticated statistical analyses are shared, most people’s eyes glaze over because such studies are either beyond comprehension or simply not readable (meaning: not interesting).

The larger issue with such statistical studies is that they are rarely questioned to any great degree and are readily accepted as “truth” by policymakers. Alternative forms of data have developed as a reaction to the dominance of statistics, and because of a belief that statistics limit the complexity of things. Statistics should not be the only data point. In order for there to be an equilibrium of rigor and assurance, we have to present more than just statistical data. No one form should be the only lens through which we view the world. For this reason, at Valencia College, we use alternative forms of data to change the angle of repose on what we seek to understand. We incorporate methods such as:

•    focus groups
•    interviews
•    archival analysis
•    transcript analysis
•    journaling
•    video
•    film
•    photography
•    audio recordings 

We have to move away from viewing the apparent—a statistic—as sufficient evidence of the problematic. When alternative methods are applied with rigor and consistency, they can provide a valid and reliable picture of the things we evaluate. For example, having students keep a video journal for a term and then analyzing the content of those videos can provide a deeper understanding of why so many developmental education students are not successful. 

Thirty two percent can tell us there is a problem, but it cannot tell us why.

Nick Bekas is DEI project director and professor of English at Valencia College.


For another example of employing multiple data sources, check out this recent MDRC study on performance-based scholarships at the University of New Mexico. Behind the Study rounds out early quantitative findings on credit completion and financial aid uptake with student perspectives gathered in focus groups.