Get the complete book Thinking Strategically about MOOCs: The Role of Massive Open Online Courses in the College and University at Amazon in print or kindle version.
. . . Many will object to the use of data mining, learning analytics, and other methodologies that inform website advertising in the development of academic learning environments. Even some MOOC proprietors are dubious. Mike Feerick, CEO of Advance Learning Interactive Systems Online (ALISON), which provides interactive multimedia courseware for certification and standards-based learning, acknowledges the importance of data analytics while also asserting that expecting such tools to solve education’s problems is simply wrong. Feerick is just as adamant that talented and dedicated teachers who make effective use of these emerging tools are the key to pedagogical success as Koller and Ferreira are about the promise of data mining and analytics.
Media coverage of this issue highlights the technologies that these adaptive learning entrepreneurs promote, leading many to assume that they seek to supplant educators with software. Along with the persistent drumbeat coming out of these startups, the coverage inspires observations from scholars like Evgeny Morozov, who describes misplaced faith in technology as a “dangerous ideology.” In his article “The Perils of Perfection” (New York Times, March 2, 2013), Morozov labels this ideology “solutionism: an intellectual pathology that recognizes problems as problems based on just one criterion: whether they are ‘solvable’ with a nice and clean technological solution at our disposal.” He explains his ideas more fully in his book, To Save Everything, Click Here: The Folly of Technological Solutionism. Morozov raises important questions about the expectations our culture holds out for technological solutions to cultural and social problems. He specifically questions the impact of “nice and clean” big data solutions on our ability to negotiate the messy business of living and learning. Morozov argues that, in the effort to cleanse our social and cultural institutions of that messiness—“from education to publishing and from music to transportation” —in the name of mere efficiency is to lose the benefit of the struggle and decision-making that contribute to maturity. Morozov cites Sartre, who “celebrated the anguish of decision as a hallmark of responsibility,” and notes that celebrating the value of such inefficiency and struggle “has no place in Silicon Valley.”
Indeed, there is undeniable value in inefficiency and imperfection. We learn from our mistakes and we do well to foster spaces where we can mess up and gain insights from the process. Proponents of adaptive learning are confident that the digital spaces they are creating are just that—spaces where we learn from our mistakes. At issue for higher education is the extent to which we identify and implement big data and adaptive learning solutions. Morozov’s concerns notwithstanding, these resources will be part of whatever MOOCs and learning management systems colleges and universities put in place. We must ensure that we understand how to make the best use of these tools as supplements to the established value and success of educators. We need to listen to the cautionary tales of those like Morozov to understand the reasonable limits of such potentially invasive technologies. Campus leaders and stakeholders across all sectors of higher education and from all sorts of institutions need to fully understand the implications of these proposed solutions, insert appropriate checks and balances, and ensure continued appreciation for the necessarily awkward messiness of learning.
In addition to the concerns of Morozov and others about surrendering the benefits of traditional, albeit imperfect, spaces to the sanitized environments of adaptive learning, there are ethical issues regarding the use of data collected via MOOCs and other online learning environments. The expansive dimensions of big data will uncover new obligations on the part of the institution to act in the interest of the student once the institution “knows” and can predict something about that student’s performance. Accumulating and aggregating analyses of big data, by design, results in predictions about performance and may raise privacy concerns. Reviewing such analyses will require decisions regarding appropriate allocation of such institutional resources as faculty, curricular design, and staff support. An institution will need to determine whether there are reporting and/or disclosure issues. Policies may need to be drafted about informing students and faculty about the type, nature, and scope of data being collected about them. Existing policies will need to be reviewed to understand whether they address big data issues.
The May 6, 2013, edition of EDUCAUSE Review included an article by James Willis, John Campbell, and Matthew Pistilli, entitled “Ethics, Big Data, and Analytics: A Model for Application,” which offers an in-depth analysis of the implications of big data in higher education. With respect to emerging big data opportunities and questions, the authors enjoin campus leaders to “understand the dynamic nature of academic success and retention, provide an environment for open dialogue, and develop practices and policies to address these issues.” They outline the ethical issues involved in implementing big data solutions on campus and offer prescriptive guidelines for policy development.
As part of their research, the authors reviewed the outcomes of Purdue University’s Signals project, which uses big data and analytics to detect “early warning signs and provides intervention to students who may not be performing to the best of their abilities before they reach a critical point” (source: www.itap.purdue.edu/studio//signals/). The authors conclude that the Signals project has improved retention and graduation rates, illuminating an interesting set of decisions about resource allocation. They credit feedback from the big data component of the Signals project for increasing student success: “Students who are less prepared for college—as measured solely by standardized test score—are retained by and graduated from Purdue at higher rates than their better-prepared peers after having one or more courses in which Signals was used.” Knowing that the mediating impact of data analytics improves academic performance of students who are less well prepared for college raises questions about allocation of resources and the accountability of the institution. “With access to these predictive formulas, faculty members, students, and institutions must confront their responsibilities related to academic success and retention, elevating these key issues from a ‘general awareness’ to a quantified value.” Inserting big data and adaptive learning systems into MOOCs will likely enhance their potential for positive mediation of student learning outcomes. It will also amplify the impact and scope of these issues.
Francis Diebold is the Paul F. and Warren S. Miller Professor of Economics in the School of Arts and Sciences at the University of Pennsylvania. He is also Professor of Finance and Statistics in the Wharton School University of Pennsylvania. Diebold asserts that big data is “not merely taking us to bigger traditional places. Rather, it’s taking us to wildly new places, unimaginable only a short time ago.” If so, various current institutional policies are inadequate. To help with policy review and revision, Willis, Campbell, and Pistilli offer a set of questions that help inform implementation of big data in campus learning systems. They are worth sharing here:
- Does the college inform students that their academic behaviors are tracked?
- What and how much information should be provided to the student?
- What and how much information does the institution give faculty members?
- Does the institution provide a calculated probability of academic success or just a classification of success (e.g., above average, average, below average)?
- What guidelines are provided to faculty regarding the use of the student data?
- Should the faculty member contact students directly?
- Will the data influence perceptions of the student and the grading of assignments?
- What amount of resources should the institution invest in students who are unlikely to succeed in a course?
- What obligation does the student have to seek assistance?
Building on the research and related issues, the authors propose three specific responsibilities institutions must embrace in order to ensure academic success for faculty and students in an era of massive data aggregation in online adaptive learning environments in this new era:
- The institution is responsible for developing, refining, and using the massive amount of data it collects to improve student success and retention, as well as for providing the tools and information necessary to promote student academic success and retention.
- The institution is responsible for providing students and faculty members with the training and support necessary to use the tools in the most effective manner. It further is responsible for providing students with excellent instructional opportunities, student advising, and a supportive learning environment, as well as for providing faculty members with tools that allow them to deliver timely feedback to students on their progress within their courses.
- The institution is responsible for providing a campus climate that is both attractive and engaging and that enhances the likelihood that students will connect with faculty and other students, and for recognizing and rewarding faculty and staff who are committed to student academic success and retention.
Institutions will need to determine their capacity to manage the implications of big data. Commercial entry and expansion into the learning analytics and adaptive learning market is increasing dramatically and is already informing the development of MOOCs. That increase adds to the array of issues senior leadership in higher education must consider as part of the strategic integration of pedagogy and technology in the context of the campus mission.
Planning Questions: The authors of “Ethics, Big Data, and Analytics: A Model for Application,” suggest specific questions to address when considering adoption and implementation of big data on campus:
- What is the role of big data in education?
- How can big data enrich the student experience?
- Will the use of big data increase retention?
- To what extent can big data contribute to successful outcomes?
Further, as you consider the implications of learning analytics at your institution, questions to ask include:
- Are you already using data analytics?
- Do you have an organizational culture supporting the use of data analytics for decision-making?
- Does your institution have the organizational adaptability to implement analytics in the culture?
- Does your institution currently have the organizational capacity and skill sets to make good use of data analytics?
- Are you aware of and have you made use of the ECAR Analytics Maturity Index available from Educause?
- Have you reviewed your institution’s strategic plan to identify issues that would benefit from analytics?
- Do you view analytics as strategic investment or additional cost?
 James E. Willis, John P. Campbell, and Matthew D. Pistilli, “Ethics, Big Data, and Analytics: A Model for Application,” EDUCAUSE Review, May 6, 2013, http://www.educause.edu/ero/article/ethics-big-data-and-analytics-model-application?utm_source=Informz&utm_medium=Email+marketing&utm_campaign=ERO&utm_content=flip.