Improving Outcomes with Bloom’s Taxonomy: From Statistics Education to Research Partnerships

In recent years, many statistics courses have been restructured to focus on concepts and applications with the intent that students would be better prepared consumers and producers of statistical information [1]. However, even with a shift in focus, a gap still exists between student and instructor expectations (course objectives) versus what students are actually able to do after completing the course (student outcomes). Specifically, after completion of the coursework, students continue to struggle with applying statistical reasoning and thinking to their research problems. One explanation for this is that the method of assessment in introductory courses has not been updated with the course content; many of the assignments still focus on memorization and repetition. Using Bloom’s taxonomy for thinking skills, one may explain the gap between objectives and outcomes by noting that expectations involve higher order thinking skills (application, analysis, synthesis, and evaluation) while assignments and evaluation continue to focus on lower order thinking skills (knowledge and comprehension). One way to bridge the gap between objectives and outcomes is to construct assignments that provide students opportunities to employ higher order thinking.


Introduction
In recent years, many statistics courses have been restructured to focus on concepts and applications with the intent that students would be better prepared consumers and producers of statistical information [1]. However, even with a shift in focus, a gap still exists between student and instructor expectations (course objectives) versus what students are actually able to do after completing the course (student outcomes). Specifically, after completion of the coursework, students continue to struggle with applying statistical reasoning and thinking to their research problems. One explanation for this is that the method of assessment in introductory courses has not been updated with the course content; many of the assignments still focus on memorization and repetition. Using Bloom's taxonomy for thinking skills, one may explain the gap between objectives and outcomes by noting that expectations involve higher order thinking skills (application, analysis, synthesis, and evaluation) while assignments and evaluation continue to focus on lower order thinking skills (knowledge and comprehension). One way to bridge the gap between objectives and outcomes is to construct assignments that provide students opportunities to employ higher order thinking.

Teaching Reform
Guidelines for Assessment and Instruction in Statistics Education (GAISE) provide a framework for revising introductory biostatistics courses [2]. In these guidelines, there is greater emphasis on statistical literacy and developing skills in quantitative decision-making. The strategies for instruction are student-centered and require activelearning methods; the goal is application, not calculation. The purpose of GAISE is to promote statistical literacy and statistical thinking, where students not only understand concepts but are able to critically evaluate and make arguments based on quantitative information. Statistical literacy and statistical thinking have a variety of definitions, but all have some overarching themes. Statistical literacy aims for an educated consumer, one who can process everyday statistical information. Taking this idea one step further, statistical thinking is present in those who can see the "big picture"; they can take what was learned in class and apply it to specific areas, demonstrating higher order thinking skills and a higher understanding than what is present in statistical literacy [3,4].

Assessments
Assessment is an essential part of any statistics course [5]. Assessments are intended to evaluate student learning but should facilitate opportunities for student learning as well. Common course assessments include: homework problems, quizzes, exams, and group activities. Less often used in introductory courses are assessments requiring presentations (oral and written), data analysis projects, and article critiques. Although much attention has been given to reforming course content, less attention is paid to reforming assessments. In fact, many assessments used for the purpose of evaluation could limit meaningful learning.

Rote Learning vs. Meaningful Learning
Learning is a process where knowledge is created through experience, and three main types of learning have been identified: none, rote, and meaningful [6,7].
No learning occurs when the student is unable to recall key statistical constructs; students do not retain and cannot use statistical information. The ability to recall facts and list key components but an inability to use the information suggests that a student has experienced rote learning. In contrast, meaningful learning requires students to use the learned information in new ways to solve problems; meaningful learning gives students the tools for understanding new concepts [6].
Statistical literacy and thinking are achieved when students are able to transfer statistical concepts to new situations. Reformed instruction strives for meaningful learning, but what about assessments? At the end of the day, most students will put their time and energy into the measures of evaluation for a course. If the assessments do not provide students practice or opportunity for meaningful learning, reformed instruction and the course will fall short of teacher and student expectations ( Figure 1).
The expectations in Figure 1, although potentially ambitious for some introductory courses, clearly fall under meaningful learning, while the assessments are largely instruments of rote learning.

Bloom's Taxonomy
In 1956 Bloom's taxonomy provided a method for organizing *Corresponding author: Richard Charnigo, Department of Biostatistics, University of Kentucky, Room 203, Multidisciplinary Science Building, Lexington, Kentucky 40536, USA, Tel: 859-218-2072; E-mail: RJCharn2@aol.com educational goals [8]. These have been widely used to write objectives, activities, and assessments. Bloom's taxonomy distinguishes levels of learning, from knowledge (the lowest level) to evaluation (the highest level). Bloom's Taxonomy was later updated; major changes included replacing the nouns with verbs and categorizing various knowledge dimensions [9,10]. Briefly, lower order thinking skills are remember and understand with apply, analyze, evaluate, and create as higher order skills (Figure 2).
The revision of Bloom's taxonomy included knowledge dimensions factual, conceptual, procedural, and meta-cognitive (Table 1).
Assessments in introductory courses tend to fall in the factual and procedural knowledge dimensions and typically only require lower order thinking skills ( Table 2). Even with reformed instruction and a greater emphasis on the conceptual dimension, the assessments continue to remain in lower levels of cognition.
Projects and article critiques are often used in reformed classes and have the potential to move traditional assessments from lower to higher order thinking skills. These often require students to analyze, evaluate, and create. However, even these can be reduced to only taxing lower order thinking skills. As an example, consider two contrasting versions of an article critique assignment: Version 1: • Students find an article based on lecture • Critique the sample selection • Were the methods appropriate?
• Using the data presented, what would you conclude? Version 2: • Students are assigned an article • Describe the sample • What methods did the authors use?
• Interpret the results of Table X Though both versions are article critiques, Version 1 does a significantly better job of requiring higher order thinking skills. Types of assignments like Version 1 more closely align with the methods of reformed, conceptual teaching. As students put their time and energy into the measures of evaluation for a course, the extra effort needed for Version 1-type assignments reinforces the level of thinking emphasized in class, helping to meet student and teacher expectations. Unfortunately, assignments created with the intention of requiring higher order skills can easily revert to those which only demand listing, describing, and summarizing. This often occurs when instructors are juggling competing demands and large class sizes. Thus, it is not necessarily the type of assessment but what is required of the student that dictates the level of cognition.

Conclusion
The format of the assessment does not necessarily dictate higher order thinking. Artfully crafted multiple choice questions can achieve higher order levels of cognition. Likewise, assignments that should more naturally require higher order thinking (article critiques and projects) can be implemented in such a way that the assessments only require lower level thinking.
Statistical literacy and thinking cannot be achieved solely through the lower level constructs of understand and remember. Moreover, students are largely grade-oriented; misplaced emphases in assessment/ evaluation may override the intended objectives of the course. Instruction that complies with reformed, conceptual teaching may still fall short of expectations if assessments are not equally reformed. Using the taxonomies of cognition and knowledge provides a strategy for creating multi-faceted assessments so that courses meet expectations of statistical consumer and producer students.

Implications Beyond the Classroom
The principles for reforming assessments are not limited to improving outcomes in undergraduate courses. Bloom's taxonomy can be applied to training statisticians, developing workshops (e.g., continuing education), and even identifying productive research partnerships.
For our own graduate students, activities requiring higher levels of cognition provide practice at critical thinking. A student who can reason, reflect, and offer alternative arguments is well-prepared for jobs

Dimension Description
Factual Basic elements students need in academia, industry, and government; students who are successful rote learners may not fare as well.
The principles of Bloom's taxonomy can also be translated to a setting outside of assigned coursework. Unlike courses, trainings and workshops generally have a more focused agenda with participants seeking to learn a particular skill. Participants who seek to apply new knowledge to ongoing analytical problems are far more likely to find value in a workshop or training. For example, the participant who seeks to apply a new method to his/her existing data (higher-order) will benefit more than the participant who simply duplicates (lower-order) the instructions of the presenter.
For the applied, collaborating statistician, the constructs of Bloom's taxonomy provide a categorization of research partnership types. Statisticians who are asked to duplicate what has been done before or supply canned summaries are asked to remain at the lower levels of Bloom's taxonomy. In contrast, the upper levels of Bloom's taxonomy are required when statisticians collaborate as problem-solvers and fellow scientists. Clearly, the more fulfilling role is the research partnership that necessitates higher order skills. A collaboration that is motivated by data and understanding data within the context of the scientific problem is transformative and is by definition scholarship of integration and application [12].
For graduate students in other fields, introductory biostatistics courses may have a more significant impact than providing a quantitative foundation; introductory biostatistics courses may play a role in shaping the types of collaborations that come later. Collaborations with investigators are far more difficult when the statistician is viewed as a "number cruncher," the p-value producer, or the one who simply identifies the correct test. When an investigator's introduction to statistics requires only rote memorization and lower order cognition, one readily understands why the statistician is often (regrettably) viewed as less than a full research partner.
Statistical literacy and thinking outcomes can be improved by reforming assessments to utilize higher levels of Bloom's taxonomy. However, using the taxonomy to reform assessments in introductory graduate courses and other venues may have a far larger impact by instilling an appreciation for the art and science of statistics in future collaborators.