Learning appropriate methods for collecting, analyzing, and interpreting numerical information

Resources

In our increasingly data-rich information age, people need the ability to critically read and construct meaning from numerical information. From public discussions concerning scientific matters such as climate change, to interpreting research evidence assessing the performance of a business, there is a need to not only understand the meaning behind the numbers but to critically think about the interpretation of them.


This ability to interpret numbers, make judgments from data, and make arguments based on numerical evidence are skills that are needed for democratic participation in the 21st Century as well as for professionals in all manner of disciplines. This ability, sometimes called “numeracy,” or “quantitative literacy,” is a skill that must be specifically addressed in curriculum so that today’s college graduates are comfortable working with, understanding, and making arguments based on numbers. The Association of American Colleges & Universities (AAC&U) describes quantitative literacy as one of their essential outcomes. According to the AAC&U, quantitative literacy is much more than just knowing how to make computations. The emphasis is on solving real-world problems by use of numerical information. It includes comfort and skill in interpreting data, recognizing when numerical manipulation is useful, being competent to make computations when needed, using numbers to solve problems, and the ability to present information to others that involves numerical data (see instructional resource 1)

Numerical information is everywhere, including in those fields of study we might not typically think of as being mathematical (see the plethora of data in the Association of Religion Data Archives for instance). Consequently, quantitative literacy is something that should be addressed in a wide range of disciplines.  See, e.g. (8) and (9).  The National Numeracy Network (instructional resource 7) is an organization dedicated to spreading quantitative literacy across disciplines and publishes its own journal,Numeracy.

Beyond ensuring that students in a particular discipline have numerical skills sufficient to accomplish the tasks they need to in their field, quantitative literacy broadly applies to an individual’s ability to use numbers in any situation from personal to professional. Quantitative literacy, then, is an essential quality of an educated person and is a skill needed to manage one’s personal and professional life.

It is important to avoid assuming that quantitative literacy and mathematics are the same.  In Mathematics and Democracy: The Case for Quantitative Literacy, Steen wrote:

Quantitatively literate citizens need to know more than formulas and equations.  They need a predisposition to look at the world through mathematical eyes . . . and to approach complex problems with confidence in the value of careful reasoning.  Quantitative literacy empowers people by giving them tools to think for themselves, to ask intelligent questions of experts, and to confront authority confidently.  These are the skills required to thrive in the modern world. (15, pg. 2).

Steen’s reference to seeing the world through a mathematician’s eyes involve identifying patterns, asking questions about the interpretation of the data, questioning the methodology with which the data was collected, and being skeptical of arguments using data that appear to be one-sided.  Paulos’ A Mathematician Reads the Newspaper (14) illustrates this disposition.

Steen further broke quantitative literacy down into its “elements” listed here in no particular order since all are of equal importance (15, pgs. 8 – 9):

  • Confidence with Mathematics
  • Cultural Appreciation of Mathematics
  • Interpreting Data
  • Logical Thinking
  • Making Decisions
  • Mathematics in Context
  • Number Sense
  • Practical Skills
  • Prerequisite Knowledge
  • Symbol Sense

The difference between quantitative literacy and mathematics can be summarized by considering the following problem solving cycle for any problem that involves data.

problem solving cycleThis should be viewed as a “cycle.”  We start with observations and questions.  

We then determine what data will help us answer those questions and collect that data, then process that information with appropriate calculations.  Finally, we interpret the results within the context of our original questions.  Typically, the data and the interpretation will raise new questions, beginning another phase of the cycle.

Most mathematics courses focuses only on the middle step in the cycle: “compute.”  Quantitative literacy involves a focus on the entirety of the cycle, often allowing technology to handle the computations.

Ensuring that students are quantitatively literate may require instructors in fields beyond mathematics to more fully consider what quantitative skills should be taught in their courses and design deliberate learning and assessment activities directed at quantitative skills.  The cycle above could be mimicked with data in any field and any course in which numerical information is involved.

Teaching this Learning Objective

TEACHING THIS OBJECTIVE

As you are teaching this objective, it is important to appreciate the students’ need for understanding why this is valuable.  The experience that many students have in mathematics is one in which they memorized what to them were meaningless, computational algorithms and tried to match the memorized algorithm with each problem they confronted.  Don’t treat quantitative literacy as something to be addressed at the end of a  unit “if time permits.”  Instead, start with the numerical questions inherent in the content and use them as motivation to use quantitative skills to find answers.

In addition, it is helpful for students if you are sensitive to the prevalence of math anxiety.  There are exercises that can be used for students overwhelmed by a phobia of mathematics.  See (17) and (18) for these resources.

To begin preparing a lesson addressing quantitative literacy, identify realistic scenarios requiring numerical information.  Look at your own life and your profession.  Many of us pay taxes – how do we fill out a tax form?  Once it is completed, how can we analyze the data on the form to make decisions for next year?  Professionally, when do you use data?  Here are some examples in different disciplines:

Political Science/Law:  What does it mean for a law to have a “disparate impact” on different populations (such as African Americans) and what does evidence of a disparate impact look like?

Education:  How does one analyze standardized test results?  What is the difference between normative and positive data?  How do you make decisions based on this information?

History:  What does the Russian census of 1897 tell you about Russian society on the eve of the revolution?  How does that play into the way the Bolshevik regime evolved according to different scholars?

Music:  What are the ratios among the frequencies of various pitches in the 12-note scale?

Mathematics:  Why does our progressive tax code set marginal rates instead of average rates?  What does this have to do with continuity?

Health Professions:  What are the advantages and disadvantages of asking a patient to rate their pain on a scale of 1 to 10?

Business:  How did Enron misuse its data and why did that lead to its collapse?  What role did Arthur Andersen play in the misuse of data?
Consider addressing the following:

Assumptions

Quantifying situations often requires making assumptions. For example, maybe we assume that a particular type of model (exponential or linear) fits a situation. Or maybe we have to assume that the sampling distribution for our methodology is normal in order to draw conclusions from our data.

To illustrate, suppose we would like to estimate the world population in 100 years as part of an environmental study.  If the world population grew by 1.2 billion people in the last hundred years, we might assume that it will grow by 1.2 billion in the next hundred.  This would be an incorrect conclusion, since it assumes population growth is linear, when it is in fact exponential.  However, even using an exponential growth assumption, we should be careful as the parameters that we use (such as the growth rate) are only useful approximations for short-term analysis.

Make these assumptions explicit, and debate their merits.  The strongest quantitative conclusions will hold if the assumptions are mildly not true, but assumptions should also be used to limit our conclusions.

Communication

How we share our conclusions is as important as how we draw them.  Students should be encouraged to think about their audience when sharing their quantitative work and their conclusions, and consider how to ethically make their point without overstating it.  Provide students with examples from your profession where a case was made too strongly for a conclusion and discuss how it could have been better presented. There are lots of examples in the media where conclusions are incorrectly stated or exaggerated (see this amusing list of spurious correlations).

Ethics

There are many interesting ethical issues that arise with data collection, analysis, and reporting.  Many of them concern how to report your data to support your conclusions.  Here are some questions you can ask:

  • Is it appropriate to cherry-pick evidence to support a predetermined conclusion
  • What kinds of inappropriate ways can charts be used?
Collection Methods

Care needs to be taken when collecting and analyzing data.  Everything from sample sizes to sampling methods should be considered and scrutinized.  Case studies of poor methodology and overstated conclusions could be addressed and critiqued.

Data vs Modeling: Not a Zero-Sum Game

Mathematical modeling involves using a mathematical representation (typically an algebraic equation together with a graph) to examine quantitative relationships.

For example, we can use the equation:

FV = PV (1 + r)t

to model how the value of a fixed income investment (like a savings account) grows over time.  In this example, FV is the future value of the account at time t while PV(present value) is the initial deposit and r is the interest rate.  We could also represent this model with a graph where FV is on the vertical axis and t is on the horizontal axis.

Data and modeling go hand-in-hand. Traditionally, mathematical modeling is part of a course such as College Algebra while data analysis is part of Statistics.  For quantitatively literate citizens, however, data is used to estimate a model.  Students should learn how to start with data, find a model that fits well, and then use that model for prediction while avoiding inappropriate extrapolation.

If it fits within your instructional context, include numerical and quantitative matters in active learning activities.  We want our students to appreciate and develop a disposition for using quantitative information, so embedding numerical work in our active learning activities and case studies has the potential make progress toward this goal.  Within the mathematical community, Yoshinobu and Jones (19) argue that the benefits of active learning outweigh the costs of content coverage, while Kohen and Laursen (12) document the benefits of active learning in a systematic study.

Finally, consider the appropriate type of technology for your learning goals and your students’ abilities.  Graphing calculators are powerful but may not be used by those in a particular profession.  Alternatively, spreadsheets are commonplace and useful both professionally and personally. See instructional resource 6 below for best practices in using spreadsheets.

Assessing this Learning Objective

ASSESSing this learning objective

The Quantitative Literacy and Reasoning Assessment (10) is a scientifically constructed instrument used for assessment purposes.  This assessment is a standardized multiple choice test with 20 items as well as demographic questions.  The instrument includes benchmarks that are disaggregated for different types of institutions.  There is also a “promptless” quantitative literacy habits of mind assessment that involves reading a carefully selected news article (4).  Students respond to questions that don’t direct them to the data in the article (hence “promptless”), and the response is measured by how much the student refers to the data on their own.

Beyond standardized tests, the AAC&U promotes “grounding assessment in authentic artifacts of student work” evaluated using the VALUE rubric (instructional resource 1) to assess student progress (6).  One way to collect those artifacts is to have students deposit them into an online portfolio that can be evaluated by whoever is appropriate at your institution (7).  This is particularly useful for assignments that involve quantitative literacy in classes outside of the traditional general education coursework.  This could also include work done in capstone courses, internships, or other types of “signature work” completed at the end of a program (2).  What is important is that following the assessment, we “close the loop” by discussing the findings and using the results to improve our courses. (1)

Resources

  1. AAC&U’s Quantitative Literacy Value Rubric
  2. Teaching with Data
  3. Data driven learning experiences
  4. Mathematical Association of America Quantitative Literacy Resources
  5. Big data
  6. Spreadsheets Across the Curriculum
  7. National Numeracy Network
  8. Carnegie Foundation Quantway and Statway
  9. Dana Center New Mathways Project

References

  1. Arcario, Paul, Eynon, Bret, Klages, Marisa, and Polnariev, Bernard A. (2013). Closing the loop: How we better serve our students through a comprehensive assessment process. Metropolitan Universities Journal, 24(2), 21-37.
  2. Association of American Colleges and Universities (2015). The LEAP Challenge: Education for a World of Unscripted ProblemsWashington, D.C.
  3. Bickerstaff, S., Edgecombe, N., & the Scaling Innovations team (2012). Pathways to faculty learning and pedagogical improvement. Inside Out, 1(3).
  4. Boersma, Stuart and Klyve, Dominic (2013) “Measuring Habits of Mind: Toward a Prompt-less Instrument for Assessing Quantitative Literacy,”Numeracy: Vol. 6: Iss. 1, Article 6.
    DOI: http://dx.doi.org/10.5038/1936-4660.6.1.6  Available at: http://scholarcommons.usf.edu/numeracy/vol6/iss1/art6
  5. Boylan, H.R. (2002). What works: Reseach-based best practices in developmental education. Continuous Quality Improvement Network/National Center for Developmental Eduaction, Boone, NC.
  6. Eynon, B., Gambino, L.M., & Torok, J. (2014). Outcomes Assessment and Institutional Learning. Retrieved from http://c2l.mcnrc.org/oa/oa-analysis/
  7. Eynon, B., Gambino, L.M., & Torok, J. (2014). What difference can eportfolio make? A field report from the connect to learning project. International Journal of ePortfolio, 4(1), 95-114.
  8. Ganter, S.L., & Barker, W. (Eds.) (2011). Curriculum foundations project: Voices of the partner disciplines. Mathematical Association of America, Washington, DC.
  9. Ganter, S.L., & Haver, W.F. (Eds.) (2011). Partner discipline recommendations for introductory college mathematics and the implications for college algebra. The Mathematics Association of America, Washington, DC.
  10. Gaze, Eric C.; Montgomery, Aaron; Kilic-Bahi, Semra; Leoni, Deann; Misener, Linda; and Taylor, Corrine (2014) “Towards Developing a Quantitative Literacy/Reasoning Assessment Instrument,”Numeracy: Vol. 7: Iss. 2, Article 4.
    DOI: http://dx.doi.org/10.5038/1936-4660.7.2.4 ,Available at: http://scholarcommons.usf.edu/numeracy/vol7/iss2/art4
  11. Gillman, Rick (2006). Current Practices in Quantitative Literacy, Mathematics Association of America, Washington DC.
  12. Kohan, M. and Laursen, S. (2014). Assessing long-term effects of inquiry-based learning: A case study from college mathematics. Innovative Higher Education, 39, 183-199
  13. National Leadership Council for Liberal Education and America’s Promise (2008). College Learning for the New Global Century, Association of American Colleges and Universities.
  14. Paulos, John Allen. (1995). A Mathematician Reads the Newspaper, Basic Books, Philadelphia, PA.
  15. Steen, Lynn Aruthur ed. (2001). Mathematics and Democracy: The Case for Quantitative Literacy, Woodrow Wilson National Fellowship Program, Washington, DC.
  16. Steen, Lynn Arthur and Madison, Bernard L. (2011) “Reflections on the Tenth Anniversary ofMathematics and Democracy,”Numeracy: Vol. 4: Iss. 1, Article 1.
    DOI: http://dx.doi.org/10.5038/1936-4660.4.1.1  Available at: http://scholarcommons.usf.edu/numeracy/vol4/iss1/art1
  17. Tobias, Sheila (1978). Overcoming Math Anxiety, W.W. Norton and Company, New York.
  18. Tobias, Sheila and Piercey, Victor (1012). Banishing Math Anxiety, Kendall Hunt Publishing Company, Dubuque, IA.
  19. Yoshinobu, S. and Jones, M. (2012) The coverage issue. PRIMUS: Problems, Resources, and Issues in Mathematics Undergraduate Studies, 22:4, 303-316.

©2016 The IDEA Center

This document may be reproduced for educational/training activities. Reproduction for publication or sale may be done only with prior written permission of The IDEA Center.

Share

IDEA

301 South Fourth Street, Suite 200, Manhattan, KS 66502
Toll-Free: (800) 255-2757   Office: (785) 320-2400   Email Us

Connect
GuideStar Gold Participant