Making sure administrators’ interpretations of data produce informed decisions

Making sure administrators’ interpretations of data produce informed decisions

With a push for more data-driven decision-making by district administrators, it is important to take into consideration how administrator’s interpretations of that data can influence the decisions being made.

There seems to be an underlying assumption within the data-driven culture that that administrators interpretations will produce informed decisions. Yet, how do we know these interpretations are correct?

Administrators need to examine data from all angles prior to settling on one interpretation or another to ensure their interpretation leaves no stone unturned. Within the constraints of time and resources, here are a few questions that will help administrators think through the complexities of their interpretations.

How is the data displayed? Make sure the data is displayed in a way that is not skewing an administrator’s interpretation. Depending on things like the scale of the x and y-axis on a bar graph, changes over time in the data may appear substantial when really they are very small.

What lens are you using to view this data? Depending on administrators’ perspectives, they may argue for one interpretation of the data or for another. For example, an administrator’s knowledge and experience may affect whether a pie chart showing a 50/50 split shows a positive trend or a negative trend.

Is the data longitudinal in nature? Analyses using data collected over longer periods of time will give administrators more information on which to base their interpretations. Stanford Professor Sean Reardon recently published a study where he examined 12 sets of data sets dating back all the way to the 1960’s.[1] Part of the power of his findings is the longitudinal nature of the data sets.

How does this data categorize students? Data sets tend to lump students into categories that may overlook important characteristics and details that are not necessarily tracked through data collection methods. A couple of recent articles question how we categorize students within data sets, and specifically how that affects the identity of these students and how educators treat students.[2]

Is your interpretation based on three or more data points? Administrators will find more strength behind their interpretations if they base their findings on three or more data points. Researchers refer to this approach as triangulation in which they check results using three or more sets of data collected using different research methods. Triangulation can increase the reliability and validity of research findings as well as interpretations.

---

[1] Reardon, S. (Forthcoming, 2011) “The Widening Academic Achievement Gap Between the Rich and the Poor: New Evidence and Possible Explanations.” From Whither Opportunity? Rising Inequality, Schools, and Children’s Life Chances. Russell Sage Foundation. Retrieved on February 16, 2012 from http://cepa.stanford.edu/sites/default/files/reardon%20whither%20opportunity%20-%20chapter%205.pdf

[2] Artiles, Alfredo. J. (2011). “Toward an Interdisciplinary Understanding of Educational Equity and Difference: The case of the Racialization of Ability.” Educational Researcher, Vol. 40. No. 9., pp. 431-445. Baglieri, S., Bejoian, L.M., Broderick, A. A. Conner, D. J., Valle, A. (October 2011). “[Re]claiming ‘Inclusive Education’ Toward Cohesion in Educational Reform: Disabilities Studies Unravel the Myth of the Normal Child.” Teachers College Record. Vol. 113, No. 10. pp. 2122, 2154.