Using Evidence of Student Learning to Improve Higher Education E-Book

George D. Kuh

• Herausgeber: John Wiley & Sons
• Kategorie: Bildung
• Sprache: Englisch

Seitenzahl: 499

Veröffentlichungsjahr: 2014

Beschreibung

American higher education needs a major reframing of student learning outcomes assessment

Dynamic changes are underway in American higher education. New providers, emerging technologies, cost concerns, student debt, and nagging doubts about quality all call out the need for institutions to show evidence of student learning. From scholars at the National Institute for Learning Outcomes Assessment (NILOA), Using Evidence of Student Learning to Improve Higher Education presents a reframed conception and approach to student learning outcomes assessment. The authors explain why it is counterproductive to view collecting and using evidence of student accomplishment as primarily a compliance activity.

Today's circumstances demand a fresh and more strategic approach to the processes by which evidence about student learning is obtained and used to inform efforts to improve teaching, learning, and decision-making. Whether you're in the classroom, an administrative office, or on an assessment committee, data about what students know and are able to do are critical for guiding changes that are needed in institutional policies and practices to improve student learning and success.

Use this book to:

• Understand how and why student learning outcomes assessment can enhance student accomplishment and increase institutional effectiveness
• Shift the view of assessment from being externally driven to internally motivated
• Learn how assessment results can help inform decision-making
• Use assessment data to manage change and improve student success

Gauging student learning is necessary if institutions are to prepare students to meet the 21^st century needs of employers and live an economically independent, civically responsible life. For assessment professionals and educational leaders, Using Evidence of Student Learning to Improve Higher Education offers both a compelling rationale and practical advice for making student learning outcomes assessment more effective and efficient.

Leseprobe

USING EVIDENCE OF STUDENT LEARNING TO IMPROVE HIGHER EDUCATION

George D. Kuh, Stanley O. Ikenberry, Natasha A. Jankowski, Timothy Reese Cain,

Cover design by Wiley Cover image : © exoboy iStockphoto Copyright © 2015 by John Wiley & Sons, Inc. All rights reserved.

Published by Jossey-Bass A Wiley Brand One Montgomery Street, Suite 1200, San Francisco, CA 94104-4594—www.wiley.com, www.josseybass.com/highereducation

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400, fax [phone], or on the Web at www.copyright.com. Requests to the publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, 201-748-6011, fax [phone], or online at www.wiley.com/go/permissions.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. Readers should be aware that Internet Web sites offered as citations and/or sources for further information may have changed or disappeared between the time this was written and when it is read.

Jossey-Bass books and products are available through most bookstores. To contact Jossey-Bass directly call our Customer Care Department within the U.S. at 800-956-7739, outside the U.S. at 317-572-3986, or fax [phone].

Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material included with standard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com. For more information about Wiley products, visit www.wiley.com.

Library of Congress Cataloging-in-Publication Data

Library of Congress Cataloging-in-Publication Data has been applied for and is on file with the Library of Congress.

ISBN 9781118903391 (hardcover); ISBN 9781118903735 (ebk.); ISBN 9781118903667 (ebk.)

FIRST EDITION

The Jossey-Bass Higher and

Adult Education Series

CONTENTS

Preface

The Authors

The Organization of the Book

Acknowledgments

About the Authors

1 From Compliance to Ownership: Why and How Colleges and Universities Assess Student Learning

A Culture of Compliance

Realizing the Promise of Assessment

Harvesting Results

What This Book Promises

PART ONE WHAT WORKS? FINDING AND USING EVIDENCE

2 Evidence of Student Learning: What Counts and What Matters for Improvement

Sources and Properties of Assessment Evidence

Obstacles to the Effective Use of Evidence

What Counts as Evidence

Conclusion: Moving to What Matters for Improvement

3 Fostering Greater Use of Assessment Results: Principles for Effective Practice

A Brief History of Assessment Practice

Doing Assessment Versus Using Results

Good Practice: Examples of Effective Use

Seven Principles for Fostering Greater Use of Assessment Results

4 Making Assessment Consequential: Organizing to Yield Results

How Is Assessment Work Organized?

What Is Being Organized?

How Should Assessment Be Organized If Improvement Is the Goal?

Principles for Organizing Assessment

Conclusion

PART TWO WHO CARES? ENGAGING KEY STAKEHOLDERS

5 Faculty and Students: Assessment at the Intersection of Teaching and Learning

Faculty

Students

Students and Faculty Learning Together

Conclusion

6 leadership in Making Assessment Matter

Governing Boards

Presidents and Chancellors

Provosts and Chief Academic Officers

Deans and Department Chairs

Aligning Positions and Messages Across Multiple Roles

Conclusion

7 Accreditation as Opportunity: Serving Two Purposes with Assessment

Accreditation: The Context

Accreditation as Assessment Driver

The Role of Accreditation in Assessment

Institutional Roles and Responsibilities

Some Principles for Using Accreditation in Assessment

Concluding Thoughts

8 The Bigger Picture: Student Learning Outcomes Assessment and External Entities

The State-Level View

The Federal Policy Perspective

The National Organization Picture

The Philanthropic Frame

Implications

PART THREE WHAT NOW? FOCUSING ASSESSMENT ON LEARNING

9 Assessment and Initiative Fatigue: Keeping the Focus on Learning

What Is Initiative Fatigue?

Factors That Contribute to Assessment Fatigue

Assessment as One More Thing

Strategies for Dealing with Initiative Fatigue

It’s About the Learning

Conclusion

10 From Compliance Reporting to Effective Communication: Assessment and Transparency

What Is Transparency?

Internal and External Transparency

From Reporting to Transparent Communication

NILOA Transparency Framework

Final Thoughts

11 Making Assessment Matter

The Current Context

What's Around the Corner?

Mobilizing for Effective Use of Evidence of Student Learning

Some Final Thoughts

References

Appendix A: NILOA National Advisory Panel

Appendix B: NILOA Staff, 2008 to 2014

Index

Advert

Want to connect?

End User License Agreement

List of Tables

Chapter 6

Table 6.1

List of Illustrations

Chapter 2

Figure 2.1

Percentage of Institutions Employing Different Assessment Approaches at the Institution Level to Represent Undergraduate Learning in 2009 and 2013.

Figure 2.2

VALUE Rubric from the Association of American Colleges and Universities. Reprinted with permission from

Assessing Outcomes and Improving Achievement: Tips and Tools for Using Rubrics

, edited by Terrel Rhodes. Copyright 2010 by the Association of American Colleges and Universities.

Chapter 3

Figure 3.1

Comparison of Uses of Assessment Results in 2009 and 2013.

Figure 3.2

The Assessment Cycle.

Chapter 4

Figure 4.1

Various Factors Influencing Organization of Assessment.

Chapter 9

Figure 9.1

Comparison of AAC&U Essential Learning Outcomes and Degree Qualifications Profile Proficiencies. (From http://leap.aacu.org/toolkit/wp-content/uploads/2012/02/DQP_ELOs_onepage.pdf)

Chapter 10

Figure 10.1

NILOA Transparency Framework.

Guide

Cover

Table of Contents

Preface

Pages

ix

x

xi

xii

xiii

xiv

xv

xvii

xix

xx

xxi

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

19

22

25

27

28

29

30

31

32

34

35

36

37

39

40

41

42

43

44

45

46

47

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

82

83

84

85

86

87

88

89

90

91

93

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

120

121

122

125

126

127

128

129

130

131

132

133

134

135

136

137

138

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

183

184

185

186

187

189

190

191

193

195

196

197

198

199

201

202

203

205

206

207

208

209

210

211

212

213

215

216

217

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

PREFACE

UNDERSTANDING WHAT STUDENTS know and are able to do as a result of their college education is no simple task, yet it is fundamental to student success and to the quality and effectiveness of American higher education. This volume grows out of a deep concern that the practical value of otherwise well-conceived efforts to assess student learning in American higher education is often diminished by deeply nested misconceptions. Many in the academy—especially those most directly responsible for the assessment of student learning—still view the assessment of student learning as an obligatory, externally imposed chore of compliance and accountability. Yes, to be fair, the capacity and commitment of colleges and universities to assess student learning outcomes have grown substantially, especially over the last decade. But the fruits of these investments—the tangible benefits to students and academic institutions—are embarrassingly modest.

What is required, we believe, is a fundamental reframing of the conversation around assessment and a clearer focus on the use of evidence of student learning in more productive and targeted ways. As we explain in this book, a complex, evolving combination of trends and forces makes evidence of student learning essential to improving student success and strengthening the vitality of colleges and universities. The quality of student learning at colleges and universities is inadequate—even declining, some say—and the meaning and coherence of a college degree are threatened as most undergraduates attend multiple institutions. New providers of higher education, transformative emergent technologies, anxiety over college costs, scarce and constrained resources, high levels of student debt, and the growing concerns of governing board members, employers, policymakers, accreditors, donors, and others have placed the gathering and use of evidence of student learning in a new light. Often missed in this cacophony of voices is the fact that many institutions have been responding to these challenges for years, but with too little to show for their efforts.

It is the use of evidence of student learning—its utility and impact on the lives of students and the prospects of campuses—that is the focus of this book. Documenting student learning and the conditions that promote high levels of student performance is a daunting task. Knowing how to harness evidence of student learning to improve teaching and learning and propel students to greater accomplishment is ultimately what matters.

This is the central challenge we take up in this book: identifying what colleges and universities must do to move the assessment of student learning from an act of compliance to the use of assessment results to guide changes that foster stronger student and institutional performance. Rather than accept the conventional view that going through the motions of assessment is a necessary burden, we argue that evidence of student learning is essential to strengthen the impact of courses, programs, and collegiate experiences; to ensure that students acquire the intended knowledge, proficiencies, and dispositions; to continuously improve teaching and learning; and to document the value of higher education to individuals and society. Thus conceived, gathering evidence of student learning is not for compliance with external demands but, rather, an institutional strategy, a core function of continuous improvement, and a means for faculty and staff to elevate student success and strengthen institutional health.

The Authors

The contributors to this book are especially well suited to take up its challenge. All are actively engaged with the work of the National Institute for Learning Outcomes Assessment (NILOA), which is colocated at the University of Illinois and Indiana University. Founded in 2008, NILOA is the leading national voice supporting efforts by colleges and universities to obtain, use, and share evidence of student learning to strengthen student attainment and improve undergraduate education. NILOA’s monthly newsletter informs more than 6,500 college presidents, provosts, faculty, student affairs staff, institutional research directors, and assessment professionals about fresh thinking and new developments, resources including NILOA reports on special topics, case studies featuring best practices, and related topics. On average, more than 10,000 individuals each month visit the NILOA website (www.learningoutcomesassessment.org); most are from the United States, but academics from 120 countries and territories also draw on NILOA resources.

Since 2012, NILOA has tracked the use of Lumina Foundation’s Degree Qualifications Profile (DQP) and related efforts to calibrate teaching and learning activities with desired outcomes, including developing a library of exemplary course assignments from different disciplines that elicit essential learning outcomes. NILOA’s Occasional Paper series—with more than 20 releases at the time of this writing—has engaged the nation’s most prominent educational leaders and assessment scholars and practitioners in a dialogue around contemporary issues. All of these efforts are designed to increase the capacity of colleges and universities to gather and use evidence of student learning to guide change in ways that strengthen the quality and impact of American higher education.

Taken together, the contributors to this volume represent an exceptional blend of scholarly acumen and practical experience.

Tim Cain, a historian of higher education with a background in college student development, brings to his inquiries related to faculty involvement in outcomes assessment both expertise on faculty and students and experience codirecting a campus-wide undergraduate research initiative at the University of Illinois.

Peter Ewell, at the National Center on Higher Education Management Systems, inspired and chronicled many of the formative and contemporary events shaping assessment work since the mid-1980s by working with hundreds of campuses and providing policy advice on assessment to states and accreditors.

Pat Hutchings was among the pioneers at Alverno College in its early years of outcomes assessment, served as the inaugural director of the American Association for Higher Education Assessment Forum, and, as a senior scholar at the Carnegie Foundation for the Advancement of Teaching, worked with faculty who were studying their students’ learning.

Stan Ikenberry, president emeritus of the University of Illinois and the American Council of Education and NILOA co-principal investigator, has a lifetime of experience in American higher education and a deep understanding of why colleges and universities must harness evidence of student learning to confront the challenges facing students and institutions.

Natasha A. Jankowski manages the day-to-day work of NILOA and is among the best-informed scholar-practitioners about issues related to public reporting and use of assessment data to mobilize resources to realize the promises of data-informed efforts to promote student success and institutional improvement.

Jillian Kinzie, through her work with hundreds of colleges and universities at the National Survey of Student Engagement (NSSE) Institute for Effective Educational Practice and her experience with various accreditation organizations and review teams, brings deep insight into the applications of assessment results for institutional improvement.

George Kuh, also a NILOA co-principal investigator, with his leadership roles with national assessment programs such as NSSE, the Strategic National Arts Alumni Project (SNAAP), and the College Student Experiences Questionnaire (CSEQ) research program, coupled with a 35-year run as a university faculty member and academic administrator, brings another set of informed perspectives and expertise to the topics this book addresses.

Perhaps the most important qualification these authors share is a commitment to shift the functions and forms of assessment away from the conventional view that assessment is primarily an act of compliance to the realization that gathering and using evidence of student accomplishment are indispensable for addressing concerns about academic quality and informing institutional improvement.

The Organization of the Book

Stan Ikenberry and George Kuh open the book with their chapter “From Compliance to Ownership: Why and How Colleges and Universities Assess Student Learning.” They present the contextualized rationale for why it is imperative for the focus of assessment to shift from an act of mere compliance to one of institutional ownership in which evidence of student learning is harnessed to make decisions and guide change. As signaled earlier, the guiding premise is that assessment of student learning is essential to student success and institutional performance. While this same work may also confirm the quality and benefit of higher education and may be useful to regional accreditors and policymakers, the value of evidence of student learning lies on campus, within the academy, where it can be harnessed to make wiser decisions and improve the learning experience of all students.

This volume is then divided into three main parts.

Part I: Making Assessment Work

In Chapter 2, “Evidence of Student Learning: What Counts and What Matters for Improvement,” Pat Hutchings, Jillian Kinzie, and Kuh discuss what constitutes actionable evidence of student learning, as contrasted to other forms of data about the student experience, and consider the broad range of sources of relevant evidence, such as surveys, portfolios, classroom assignments, and external performances and their useful application for quality improvement.

In Chapter 3, “Fostering Greater Use of Assessment Results: Principles for Effective Practice,” Kinzie, Hutchings, and Natasha Jankowski illustrate the broad range of effective uses of assessment evidence, drawing on case studies and focus groups conducted by NILOA and reports from institutional consortia and individual institutions to describe approaches that prompted meaningful use of assessment results and the principles that undergird these efforts.

Chapter 4, “Making Assessment Work Consequential: Organizing to Yield Results,” examines different approaches to implementing assessment work as Kinzie and Jankowski illustrate how institutions with different missions use assessment committees, teaching and learning centers, faculty reward and governance structures, and institutional research and effectiveness offices to gather, use, and productively communicate evidence of student learning.

Part II: Who Cares? Key Stakeholders

In Chapter 5, “Faculty and Students: Assessment at the Intersection of Teaching and Learning,” Tim Cain and Hutchings focus on what may be the most important but often the most frequently overlooked dimensions of assessment—faculty collaboration and student participation in the design and implementation of assessment approaches. They offer principles for how to make assessment more meaningful and useful for those who are central to student learning and institutional improvement.

In Chapter 6, “Leadership in Making Assessment Matter,” Peter Ewell and Ikenberry explore the role of institutional leaders—specifically governing boards, presidents, provosts, deans, and department chairs—in managing and leading assessment efforts and in the practical use of evidence of student learning to inform institutional decision making and increase student learning and success.

In Chapter 7, “Accreditation as Opportunity: Serving Two Purposes with Assessment,” Ewell and Jankowski address accreditors’ need for evidence of student learning to assure quality and institutions’ role in meaningful engagement with accreditation standards, with special emphasis on the ways institutions implement and respond to accreditation requirements and the role of learning outcomes frameworks such as the Degree Qualifications Profile (DQP).

In Chapter 8, “The Bigger Picture: Student Learning Outcomes Assessment and External Entities,” Ikenberry, Kinzie, and Ewell examine state and federal policy related to assessing student learning as a means of quality improvement and consider the work of national organizations such as the American Association of State Colleges and Universities, the American Council on Education, the Association of American Colleges and Universities, the Association of Public and Land-Grant Universities, and the Council of Independent Colleges, among others.

Part III: What Now?

In Chapter 9, “Assessment and Initiative Fatigue: Keeping the Focus on Learning,” Kuh and Hutchings discuss what campuses can do to ameliorate the potential debilitating effects when faculty and staff find themselves overwhelmed trying to implement multiple assessment projects and improvement initiatives along with their regular responsibilities.

In Chapter 10, “From Compliance Reporting to Effective Communication: Assessment and Transparency,” Jankowski and Cain consider transparency not as a reporting or compliance exercise but as an effort to communicate to various internal and external audiences a variety of evidence on student learning. NILOA’s transparency framework is highlighted as a means for thinking about the several dimensions of assessment that can be made more transparent, the interests of various audiences in such information, and the ways higher education can more effectively communicate evidence of student learning to both internal and external stakeholders.

In the closing chapter, the contributors to this volume ponder what the assessment movement has accomplished in providing useful evidence of student outcomes and the work left to be done.

This volume is very much a collaborative endeavor. From the beginning, all the contributors helped shape the book’s purpose and structure. While certain authors took the lead on respective chapters, in every case, their good work benefitted from the comments and ideas of other contributors. And, in some instances, ideas that originally appeared in one chapter found their way to another when and where the material made for a stronger, more coherent, and persuasive presentation. The order of authorship reflects this collaborative nature. Kuh and Ikenberry, as NILOA co-principal investigators, and Jankowski, as associate director, are listed first. The other authors, all NILOA senior scholars, are listed in alphabetical order.

Audience

We intend for the book to spark a fresh and broad conversation on the future of higher education and the role of evidence of student learning in dealing with the contemporary challenges facing American higher education. Thus, the volume is especially relevant for those who lead, govern, and make America’s colleges and universities among the best in the world—presidents and provosts, governing board members, and education policymakers. They are key players in positioning assessment work within the broader framework of higher education so that it informs institutional decision making and quality improvement efforts.

We expect the book to be particularly useful for faculty members and assessment professionals, institutional researchers, and those new to assessment, as it provides both practical and conceptual advice for thinking about and undertaking student learning outcomes assessment. For this reason, the contents are also instructive for graduate students aiming for a position in postsecondary education, along with administrators and staff members seeking to better understand how gathering and using student learning outcomes data—done well—can contribute to their effectiveness and to overall institutional performance.

ACKNOWLEDGMENTS

FROM ITS INCEPTION, NILOA has benefitted from sage counsel from national thought leaders who comprise NILOA’s national advisory panel, listed in Appendix A. Their commitment to the core values of the academy and its obligations for societal betterment is inspiring and aspirational.

NILOA’s work, along with numerous advances in the assessment field, would not be possible without the visionary leadership and generous support of philanthropic organizations. We are especially grateful to Lumina Foundation and Jamie Merisotis for the leadership grant that helped launch NILOA. The Carnegie Corporation of New York and the Teagle Foundation also provided resources at critical junctures, and we are most thankful for their interest and support.

The University of Illinois at Urbana-Champaign has been our principal home and a congenial host for NILOA and a superb staff, including graduate research assistants and senior scholars (Appendix B). We have been warmly welcomed and supported by this rich academic community. NILOA’s impact and sustainability could not have been possible without the stimulating, engaged environment that makes it possible for us to thrive.

If there were an award for outstanding copyediting, it would surely belong to Sarah Martin from the Indiana University Center for Postsecondary Research. Her work and expertise have touched every page of this book, and the volume is much the better because of her. We also appreciate the good services provided by the Indiana University Center for Survey Research in conducting the three national NILOA surveys we refer to in this volume.

Finally, we tip our hats to countless colleagues across the country who responded to surveys, participated in focus groups, helped with case studies and reports, and otherwise shared their good work. It is through their efforts to assess student learning inside and outside the classroom that we have learned a great deal about the value of this important work and how it can be used to enhance student accomplishment and improve institutional performance. We are in their debt.

ABOUT THE AUTHORS

Timothy Reese Cain is associate professor at the University of Georgia’s Institute of Higher Education and a senior scholar at the National Institute of Learning Outcomes Assessment. He writes and teaches about the history of higher education, college and university faculty, campus speech, and learning outcomes assessment. He has published in Teachers College Record, Labor History, and the History of Education Quarterly, among numerous other outlets, and his first book, Establishing Academic Freedom: Politics, Principles, and the Development of Core Values, was released in 2012. He earned his A.B. in history at Duke University, his M.A. in higher education and student affairs at The Ohio State University, and his Ph.D. in education at the University of Michigan. From 2005 to 2013, he was on the faculty at the University of Illinois, where he coordinated the higher education program and codirected the Ethnography of the University Initiative.