Assessment Essentials E-Book

Trudy W. Banta

• Herausgeber: John Wiley & Sons
• Kategorie: Bildung
• Sprache: Englisch

Seitenzahl: 611

Veröffentlichungsjahr: 2014

Beschreibung

A comprehensive expansion to the essential higher education assessment text This second edition of Assessment Essentials updates the bestselling first edition, the go-to resource on outcomes assessment in higher education. In this thoroughly revised edition, you will find, in a familiar framework, nearly all new material, examples from more than 100 campuses, and indispensable descriptions of direct and indirect assessment methods that have helped to educate faculty, staff, and students about assessment. Outcomes assessment is of increasing importance in higher education, especially as new technologies and policy proposals spotlight performance-based success measures. Leading authorities Trudy Banta and Catherine Palomba draw on research, standards, and best practices to address the timeless and timeliest issues in higher education accountability. New topics include: * Using electronic portfolios in assessment * Rubrics and course-embedded assessment * Assessment in student affairs * Assessing institutional effectiveness As always, the step-by-step approach of Assessment Essentials will guide you through the process of developing an assessment program, from the research and planning phase to implementation and beyond, with more than 100 examples along the way. Assessment data are increasingly being used to guide everything from funding to hiring to curriculum decisions, and all faculty and staff will need to know how to use them effectively. Perfect for anyone new to the assessment process, as well as for the growing number of assessment professionals, this expanded edition of Assessment Essentials will be an essential resource on every college campus.

Leseprobe

Contents

List of Exhibits and Figures

Foreword

Preface

About the Authors

Chapter 1: Defining Assessment

Some Definitions

Pioneering in Assessment

Quality Assurance: An International Perspective

Assessment Purposes

Values and Guiding Principles

Chapter 2: The Essentials of Assessment

Planning Effective Assessment

Implementing Effective Assessment

Improving and Sustaining Assessment

Additional Thoughts

Chapter 3: Engaging Faculty and Students in Assessment

Involving Faculty in Assessment

Involving Students in Assessment

Acting with Integrity

Chapter 4: Setting Expectations and Preparing to Select Measures

Intentions for Learning: Goals, Objectives, Outcomes

Developing Statements of Expectations

Selecting Methods and Approaches

Designing Instruments

Determining Approaches for Implementation

Putting Everything Together

Chapter 5: Using Direct Measures

Using Classroom Assignments for Outcomes Assessment

Performance Assessment

Rubrics

Aggregating Assessment Results in and across Courses

Using Objective Tests for Outcomes Assessment

Electronic Portfolios

Chapter 6: Using Indirect Assessment Methods

Using Surveys in Assessment

Using Focus Groups in Assessment

Additional Indirect Methods

Qualitative versus Quantitative Approaches

Classroom Assessment Techniques

Chapter 7: Assessing Learning in the Major

Capstone Experiences and Courses

Portfolios

Experiential Education

Group Work and Team-Building Skills

Employer Involvement

Intentional Learning

Chapter 8: Assessing Learning in General Education

The Nature of General Education

Assessment Choices and Issues

Using Commercial Instruments and the Voluntary System of Accountability

Assessing Specific Aspects of General Education

The Degree Qualifications Profile

Assessing General Education Outcomes within the Major

Chapter 9: Assessing Student Learning and Program Effectiveness in Student Affairs

Foundations for Assessment in Student Affairs

Mission, Goals, and Objectives

Leadership and Preparation for Assessment in Student Affairs

Assessment Frameworks, Models, and Diagrams

Assessment Plans and Methods

Reporting and Sharing Results

Ethical Behavior

Improving Assessment

Rewards for Assessment

Chapter 10: Analyzing, Reporting, and Using Assessment Results

Helping Faculty and Staff Use Their Assessment Results

Assessment Reporting by Departments and Programs

Summarizing Reports

Managing Data

Assessing Unit Reports

Making the Process Transparent

Institutional Assessment Reporting

Analyzing Assessment Information

Displaying Results

Other Considerations

Chapter 11: Assessing Institutional Effectiveness

Linking Assessment and Institutional Planning: An Example

Organizing to Assess Institutional Effectiveness

Planning and Institutional Improvement at IUPUI

Administering an Assessment Plan

Considering Costs

Linking Assessment to Other Valued Processes

Chapter 12: Summing Up

A Time of Transition

Current Practice

Continuing Challenges

References

Name Index

Subject Index

Advertisement

End User License Agreement

List of Illustrations

Figure 11.1 Planning, Evaluation, and Improvement at IUPUI

List of Exhibits

Exhibit 4.1 Content-by-Process Matrix

Exhibit 4.2 Curriculum Map

Exhibit 4.3 Planning for Learning and Assessment

Exhibit 4.4 Selection Criteria Matrix

Exhibit 4.5 Objective-by-Measures Matrix

Exhibit 5.1 Planning Sheet for e-Portfolios

Exhibit 11.1 Assessment Plan Outline

Pages

i

ii

iii

iv

v

xv

xvi

xvii

xviii

xix

xx

xxi

xxii

xxiii

xxiv

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

21

22

23

24

25

26

27

28

29

30

31

32

33

34

35

36

37

38

39

40

41

42

43

44

45

46

47

48

49

50

51

52

53

54

55

56

57

58

59

60

61

62

63

64

65

66

67

68

69

70

71

72

73

74

75

76

77

78

79

80

81

82

83

84

85

86

87

88

89

90

91

92

93

94

95

96

97

98

99

100

101

102

103

104

105

106

107

108

109

110

111

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130

131

132

133

134

135

136

137

138

139

140

141

142

143

144

145

146

147

148

149

150

151

152

153

154

155

156

157

158

159

160

161

162

163

164

165

166

167

168

169

170

171

172

173

174

175

176

177

178

179

180

181

182

183

184

185

186

187

188

189

190

191

192

193

194

195

196

197

198

199

200

201

202

203

204

205

206

207

208

209

210

211

212

213

214

215

216

217

218

219

220

221

222

223

224

225

226

227

228

229

230

231

232

233

234

235

236

237

238

239

240

241

242

243

244

245

246

247

248

249

250

251

252

253

254

255

256

257

258

259

260

261

262

263

264

265

266

267

268

269

270

271

272

273

274

275

276

277

278

279

280

281

282

283

284

285

286

287

288

289

290

291

292

293

294

295

296

297

298

299

300

301

302

303

304

305

306

307

308

309

310

311

312

313

314

315

316

317

318

319

320

321

322

323

324

325

326

327

328

329

330

331

332

333

334

335

336

337

338

339

340

341

Guide

Cover

Table of Contents

Begin Reading

Praise for Assessment Essentials

“Banta and Palomba’s new edition of Assessment Essentials covers a wide range of current topics that are crucial for faculty, staff, and administrators to follow as they strive to sustain effective assessments. I highly recommend this comprehensive resource for academic leaders and others in higher education who want to maintain the quality of their assessment efforts.

I regularly teach graduate courses and used the original Banta and Palomba book as required reading that helped students learn critical knowledge and skills in implementing and sustaining assessments. I plan to use the comprehensive new edition of this book for my upcoming graduate courses and highly recommend this valuable resource to other faculty members.”

—Elizabeth A. Jones, Ph.D., professor of education and director, Doctoral Program, Holy Family University

“It’s hard to improve on a classic, but Banta and Palomba have done so with this updated edition of Assessment Essentials, which is replete with examples of what effective assessment work looks like in different types of institutions.”

—George D. Kuh, adjunct professor and director, National Institute for Learning Outcomes Assessment, University of Illinois and Indiana University

“A sweeping update that brilliantly reflects the evolution of the field. Institutions should use it to build the assessment infrastructure they will need in the brave new world of prior learning assessment and competency-based education.”

—Barbara D. Wright, vice president, WASC Senior College and University Commission, Alameda, California

Assessment Essentials

Planning, Implementing, and Improving Assessment in Higher Education

Second Edition

Trudy W. Banta

Catherine A. Palomba

Foreword by Jillian Kinzie

Cover Design: Lauryn Tom

Cover Image: © iStockphoto/VikaSuh

Copyright © 2015 by John Wiley & Sons, Inc. All rights reserved.

Published by Jossey-Bass

A Wiley Brand

One Montgomery Street, Suite 1200, San Francisco, CA 94104-4594

www.josseybass.com/highereducation

No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400, fax [phone], or on the Web at www.copyright.com. Requests to the publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, 201-748-6011, fax [phone], or online at www.wiley.com/go/permissions.

Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, incidental, consequential, or other damages. Readers should be aware that Internet Web sites offered as citations and/or sources for further information may have changed or disappeared between the time this was written and when it is read.

Jossey-Bass books and products are available through most bookstores. To contact Jossey-Bass directly call our Customer Care Department within the U.S. at 800-956-7739, outside the U.S. at 317-572-3986, or fax [phone].

Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material included with standard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com. For more information about Wiley products, visit www.wiley.com.

Library of Congress Cataloging-in-Publication Data has been applied for and is on file with the Library of Congress.

ISBN 9781118903322 (cloth); ISBN 9781118903766 (ebk.); ISBN 9781118903650 (ebk.)

THE JOSSEY-BASS HIGHER AND ADULT EDUCATION SERIES

LIST OF EXHIBITS AND FIGURES

Exhibit 4.1

Content-by-Process Matrix

Exhibit 4.2

Curriculum Map

Exhibit 4.3

Planning for Learning and Assessment

Exhibit 4.4

Selection Criteria Matrix

Exhibit 4.5

Objective-by-Measures Matrix

Exhibit 5.1

Planning Sheet for e-Portfolios

Exhibit 11.1

Assessment Plan Outline

Figure 11.1

Planning, Evaluation, and Improvement at IUPUI

FOREWORD

The pressures for colleges and universities to demonstrate educational effectiveness and use empirical evidence to make improvements have only intensified since 1999, when Assessment Essentials debuted as a comprehensive resource for those responsible for assessment in higher education. Now, with the prospect of the tenth reauthorization of the Higher Education Act, lawmakers are pushing to enact new ways to hold colleges and universities accountable for their costs and outcomes. How are colleges and universities navigating this period of heightened demands for accountability?

Results from the 2009 and 2013 National Institute for Learning Outcomes Assessment (NILOA) survey of provosts suggest that most institutions are engaged in considerable assessment activity and are using multiple approaches to gather evidence in response to a variety of drivers, most notably accreditation. In 2009, the vast majority (92 percent) of all colleges and universities participating in the survey used at least one assessment approach at the institution level, and the average was three. In 2013, that average increased to five. Institutional use of assessment evidence for every category—including accreditation, external accountability, strategic planning, and institutional improvement—was also higher in 2013 than in 2009.

Indeed, much has changed in the world of assessment since the first edition of Assessment Essentials was published, and results demonstrating the amount of assessment activity going on at most institutions suggest significant progress on the assessment agenda. Yet strong interest in learning more about how to effectively implement, organize, support, and sustain assessment activities continues to grow. In this second edition, Trudy Banta and Catherine Palomba make a substantial and timely contribution to enriching the knowledge base regarding how assessment is carried out and the best ways to organize and structure effective assessment work.

This updated edition of Assessment Essentials serves as a good checkup of the robustness of assessment practice. The practical examples it presents are a source of reassurance that assessment in colleges and universities is healthy and that institutions are not ignoring important assessment behaviors—while also serving as an alarm system, raising questions about current conditions of assessment before they become chronic. The Contents list alone could function as an annual checklist for healthy assessment practice in colleges and universities. These respected general practitioners of assessment have done a thorough examination of the overall health of assessment.

Banta and Palomba are exactly right to place considerable emphasis throughout this book on increasing the involvement of faculty in assessment. They make the case that effective assessment of student learning cannot occur without the participation of faculty in every step of the assessment process. Quite simply, faculty involvement is key to meaningful assessment, and the meaning of assessment is only as good as the scope and quality of faculty involvement. Notably, the authors also acknowledge the importance of assessing student learning and program effectiveness in student affairs and have added a new chapter to address this expansion in assessment work. In particular, the chapters dedicated to engaging faculty and students in assessment and evidence of focused efforts to assess learning in student affairs make a significant contribution to documenting effective work and discussing current needs in the field.

It is heartening to see that ensuring the use of assessment results is a consistent theme across all chapters in Assessment Essentials. For almost a dozen years, I have had the good fortune to work with the National Survey of Student Engagement (NSSE) to further the assessment of student engagement in educationally effective practice and provide colleges and universities with diagnostic, actionable information to inform efforts to improve the quality of undergraduate education. While it is rewarding to see campuses adopt NSSE as an important component of their assessment programs that provides them concise summaries of the strengths and shortcomings of their students’ experience, it is disappointing when campuses fail to share results with faculty, draw clear lessons from their assessment data, or formulate concrete improvement priorities. Motivated by the understanding that campuses are much better at gathering data than using assessment to improve, Banta and Palomba dedicate much-needed attention to outlining approaches that campuses can use to encourage faculty and staff to take action on assessment information, and in several chapters they highlight examples of closing the assessment loop, or taking action and then determining the results of these actions. Quite simply, assessment has little to no value if results are not shared and used in meaningful ways. This book aims to make assessment results meaningful and used.

Assessment has taken on greater importance across all areas and units of colleges and universities. The widespread use of national surveys, rubrics, and portfolios to assess learning; the emergence of assessment technologies, including an array of data management systems; and the increase in assessment specialists are all signals of serious investment in assessment. While “assessment cynics” do still exist, the shallow compliance approach to assessment—simply to satisfy the expectations of accreditation—is clearly untenable and no longer holds sway. Although previously common, the compliance approach to assessment has been supplanted by an approach favoring assessment for learning and improvement. Now, assessment more often than not is viewed as vital to improving educational quality and effectiveness. The first edition of Assessment Essentials helped guide this shift, and this new edition updates and extends the guidance. More important, Banta and Palomba demonstrate how faculty and staff have made meaning of assessment results to demonstrate educational effectiveness and inform institutional improvement.

Improving quality in undergraduate education to foster learning and success for all students is imperative for US higher education. The challenge that this presents to institutional leaders, faculty, and staff demands meaningful assessment and concerted action to enhance educational effectiveness. The stock-taking activities undertaken by NILOA indicate that assessment has become a permanent fixture in the structure of colleges and universities, yet there is clearly more to do for assessment at these institutions to advance and mature. Moreover, assessment in higher education has grown increasingly more complicated as demands for evidence and expectations for ensuring student learning and institutional improvement have intensified. Faculty and staff with assessment responsibilities and students of assessment need practical resources and achievable assessment examples to advance their assessment skills and repertoire. Assessment Essentials is precisely the resource to guide the field in these demanding times.

Jillian Kinzie

associate director, Indiana University Center for Postsecondary Research, and NILOA senior scholar Bloomington, Indiana

PREFACE

When we were considering the possibility of undertaking the task of revising the 1999 version of Assessment Essentials, our Jossey-Bass editor, David Brightman, offered to ask three experienced assessment professionals to review the original and give us advice about approaching a revision. We are indebted to this anonymous trio because they gave us a perspective that opened our eyes to the distinctive character of the first edition and to the possibilities for continuing to make a unique contribution to the literature with a revision.

Our reviewers told us that we had offered a practical guide to assessment practice, with principles, examples, and advice about decisions that must be made in the course of implementing an outcomes assessment initiative. In addition to the guidance for practice, we had provided history and context with perspective and vision. The reviewers asked us for a similar approach in our revision, and we have tried to fulfill that request.

When we wrote the 1999 edition, there were few such resources to guide faculty and staff who needed a basic introduction to outcomes assessment, with connections to current references that would help even experienced assessment leaders acquire fresh examples and extended understanding. Now there are many books, several journals, numerous conferences, new organizations, and a panoply of examples of good practice available on institutions’ websites. This has made it extremely difficult to produce, as we were instructed by our publisher, a work of as few pages as the first edition!

In addition to the principles, examples, and perspectives offered in the first edition, our three reviewers asked us to add information about these topics:

The use of technology in assessment, including electronic student portfolios

Capstone courses as assessment vehicles

Assessment in student affairs

The link between outcomes assessment and such valued institutional processes as strategic planning, curriculum revision, and comprehensive program review

So much has changed since 1999 that we have replaced virtually all of our original examples, and of course this required new surrounding text. So while we have retained much of our original organization, most of the words are new. We have expanded our original focus on assessment of student learning to include institutional effectiveness. Accordingly, there are new chapters on assessment in student affairs and assessing institutional effectiveness.

Audience

The three reviewers of our first edition identified our audience as learners: faculty and staff engaged in leading outcomes assessment on their campuses, faculty and staff new to assessment and seeking a comprehensive overview, faculty and staff teaching master’s and doctoral students in higher education and student affairs. We have attempted to provide some history, context, perspective, and vision for these populations.

The Contents

In Chapter 1 we introduce our broad definition of outcomes assessment, encompassing all institutional programs and services. We also include some history and perspective on assessment’s progress. Chapter 2 presents our assessment essentials envisioned in three phases: planning, implementing, and improving/sustaining assessment. The essential step of engaging faculty and students in outcomes assessment is the subject of Chapter 3.

Chapters 4, 5, and 6 prepare readers to develop outcomes statements and make decisions about appropriate outcomes. Curriculum mapping, instrument validity and reliability, rubrics, and examples of direct and indirect measures are introduced.

Chapters 7, 8, and 9 contain illustrations of the ways measures can be applied in assessing student learning in the major, general education, and student affairs, respectively. Chapter 10 addresses the essential processes of analyzing, reporting, and using assessment findings.

Chapter 11 offers examples of comprehensive institution-wide assessment programs. In Chapter 12 we characterize the current assessment scene as one fraught with uncertainties, making it difficult to predict a clear way forward. Yet much has been achieved in the four-decade history of outcomes assessment in higher education, and we conclude with some of those achievements, as well as some continuing challenges.

Acknowledgments

We are indebted to the thousands of assessment professionals who have moved this field forward over the past four decades and have unselfishly shared their successes, failures, and continuing challenges in books, journals, and conference presentations, as well as on websites. They have provided the hundreds of examples we cite in illustrating our Assessment Essentials.

Cindy Ahonen Cogswell, a doctoral candidate in Indiana University’s higher education program, has contributed her considerable research skills and editing expertise on our behalf. We could not have produced this book without the amazing clerical assistance of Shirley Yorger. And finally we appreciate the review of the final manuscript by Kenneth Gilliam, a master’s-level student in the student affairs program at Indiana University.

Trudy W. Banta

Indianapolis, Indiana

Catherine A. Palomba

Durham, North Carolina

To Our Essentials:

Ruth and Ky

Logan, Holly, and T.J.

Mady and Max

Neil, Mary Frances, and Nick

ABOUT THE AUTHORS

Trudy W. Banta is professor of higher education and senior advisor to the chancellor for academic planning and evaluation at Indiana University-Purdue University Indianapolis (IUPUI). Considered a pioneer in outcomes assessment in higher education, Banta has received ten national awards for her work in this field. Before moving to IUPUI in 1992, she was the founding director of the Center for Assessment Research and Development at the University of Tennessee, Knoxville.

Banta has developed and coordinated twenty-seven national conferences and fifteen international conferences on the topic of assessing quality in higher education. She has written or edited eighteen books on assessment, contributed thirty chapters to other published works, and written more than three hundred articles and reports. She is the founding editor of Assessment Update, a bimonthly periodical published since 1989 by Jossey-Bass.

Catherine A. Palomba is director emeritus of assessment and institutional research at Ball State University. Prior to leading a nationally recognized assessment program on that campus, she taught economics at West Virginia University and Iowa State University. She also was a research analyst at the Center for Naval Analyses in Virginia. She earned her PhD in economics from Iowa State University and her master’s degree in economics from the University of Minnesota. Her assessment work includes two books, as well as several articles and presentations.

CHAPTER 1DEFINING ASSESSMENT

The concept of assessment resides in the eye of the beholder. It many definitions, so it is essential that anyone who writes or speaks about assessment defines it at the outset.

Some Definitions

In common parlance, assessment as applied in education describes the measurement of what an individual knows and can do. Over the past three decades, the term outcomes assessment in higher education has come to imply aggregating individual measures for the purpose of discovering group strengths and weaknesses that can guide improvement actions.

Some higher education scholars have focused their attention on the assessment of student learning. Linda Suskie, for instance, in the second edition of her book Assessing Student Learning: A Common Sense Guide (2009) tells us that for her, the term assessment “refers to the assessment of student learning.” In the first edition of this book, we also adopted the focus on student learning: