Home Process Presentation
  Preparatory Review
Introduction
Reflective Essays

1(a). Undergraduate Admissions
1(b). Campus Diversity
2(a). Support for Student Learning
2(b). Teaching Effectiveness
3(a). Delivery of Education
3(b). Educational Applications of Technology
–>4(a). Institutional Uses of Data
4(b). Program Review
Conclusion/Executive Summary
Appendices


line

 

4(a). Institutional Uses of Data

1.2 (indicators and evidence), 4.1 (data linked to decision-making), 4.3 (data informs planning), 4.4 (data collection for improvement), 4.5 (strategic data needs), 4.6 (assessments used), Question 3(data collected and used effectively) under Strategic Thinking and Planning (4), Question 6 (culture of evidence) under Commitment to Learning and Improvement (4)

Introduction

The collection, dissemination, and use of Institutional Research data in decision-making are critical to improving educational effectiveness. The management of institutional data is decentralized at Berkeley. This permits decision-makers to use data effectively in their planning processes, but also creates a number of challenges. For example, the campus has not explored in depth the possible development of a campus-wide strategy for assessing student learning outcomes. This essay discusses the challenges posed by decentralization and efforts underway to integrate institutional data systems, and proposes some ways to address the need to assess comprehensively student learning outcomes.

Institutional Data Management in a Decentralized Environment

Institutional research and planning at Berkeley take place in a variety of venues and not simply, as at many other universities, in a centralized institutional research office. Decentralization of institutional data ensures that decision makers throughout the campus have efficient and reliable access to the data and the analyses that they require.

Six offices provide most of the institutional data to the Berkeley campus: Planning and Analysis, Student Research, Admissions and Enrollment, Graduate Division, Space Management and Capital Projects, and Sponsored Projects. Several of the larger colleges and schools also employ analytical staff. Managers and analysts from the institutional units that provide data also serve on standing or ad hoc campus committees. Their primary function is to provide appropriate data to support committee decisions. Their participation ensures that committee members fully understand complex statistical information and allows campus-wide as well as departmental decisions to be "data driven."

For example, in the past year the Strategic Planning Committee (SPC) and Near Term Planning Subcommittee requested from the Office of Planning and Analysis a number of special data analyses to identify departments where faculty workload and student demand for a major were highest. Similarly, a special report prepared by Admissions and Enrollment on the academic success of athletes at Berkeley for the Admissions, Enrollment, and Preparatory Education Committee and the Undergraduate Admissions Coordination Board will lead to a more systematic review of athletic admissions by faculty, especially in the admission of high risk athletes. At the request of the Affordability Steering Committee, the OSR conducted surveys to evaluate the impact of a variety of costs of attending Berkeley on freshmen and transfer student populations. In many cases, data provided for a single committee's use may be shared with other decision-makers or committees as appropriate.

Challenges of Decentralization

Having multiple campus units perform institutional research functions creates a number of challenges, including content and technical issues, integration of systems and data, and issues of access and ownership. Some of the pitfalls of decentralization are that (a) institutional data are at times not shared effectively with planners and decision-makers, with the result that planning efforts are not fully informed; (b) institutional data elements are not fully shared among data providers or analysts across campus; (c) efforts may be duplicated as several units collect and analyze similar data; and (d) data generated for reports to departments or campus committees are often provided on a piecemeal basis, designed to answer a specific question or set of questions rather than being part of a broader evaluation or research agenda.

As are most large institutions nationally, the campus is now engaged in the transition to relational database and web-based technologies which, in effect, combine the advantages of the centralized and distributed models of computing. This transition requires significant changes in business policies and practices that ultimately will increase efficiency and accountability by minimizing redundancy and improving data quality. Among the many technical and policy issues raised by the new technologies, two seem particularly germane to the evolving organization of institutional data collection and analysis at Berkeley: (a) how to make data accessible while ensuring security and confidentiality, and (b) whether the present quantity and configuration of analytical and technical staff remain optimal. Both issues are the subjects of continuing discussion and evaluation.

Efforts Toward Data Integration

Recognizing that data integration is an important requisite for improving the campus's organizational and operational effectiveness, the Chancellor launched a Data Integration Initiative to standardize data and data access policies across campus, and to improve access to campus information through the use of Web-enabled technologies and on-line analytical processing tools. The Working Group's May 2002 report on data integration included five major recommendations (Data Integration Workgroup):

  1. Create a permanent Data Stewardship Council to provide a forum for resolving data integration issues such as inter-system communication, data definition, and integrity;

  2. Analyze shadow systems in use across the campus;

  3. Direct the Data Stewardship Council to create a mechanism by which a campus-wide report will be produced that articulates a vision for a logical data architecture for the campus;

  4. Recognize that the work associated with the goals of the Data Integration Initiative and the recommendations of the Data Stewardship Council will require dedicated attention to progress and staff support;

  5. Recognize the Office of Planning and Analysis as the repository of official campus-wide aggregate and institutional data.

The Data Stewardship Council will also address technical and systems issues including the development of policies related to data access and security, a student data warehouse, and campus data dictionaries.

Establishment of a Data Gateway

The Institutional Data Gateway, established in Spring 2002, provides the campus community with instantaneous access to basic institutional data and links to UC system and national data. The following five campus databases are included in the Data Gateway:

Cal Profiles, established in November 1998, is a comprehensive, longitudinal view of more than 400 data elements for all campus units, which can be viewed at any level from the campus down to a specific unit. The Office of Planning and Analysis provides hands-on training and an on-line guide for campus staff.

Cal Profiles Plus, created in November 2001, accesses the same database as Cal Profiles, but provides more detail and allows the user to drill down and create charts.

The Common Data Set (CDS) is a collaborative effort between publishers and educational institutions to standardize the set of annual data elements requested for educational publications. The campus web site was launched in August 2000.

The Performance Metrics web site, on line since September 2001, provides trend data on campus metrics used to assess the degree to which organizational goals are being met. It also provides a one-year snapshot comparing Berkeley to a group of 12 peer institutions—5 privates, 5 publics, and 2 UC campuses. Comparative data is available on a series of measures.

The Student Data site maintained by the OSR allows users to select and view summary data on applicants, registered undergraduate students, and undergraduate degree granted.

Student Learning Outcomes

Building on the Data Integration Initiative, as part of its Educational Effectiveness Review, the campus will explore the possible development of a campus-wide strategy for assessing student learning outcomes. The following steps have been identified as ways the campus can improve assessment of student learning:

  1. Establish protocols by which student data sources may be identified and shared across campus. This has been accomplished in some venues (data distributed by Student Information Systems), but needs to be campus-wide. Expansion of the Institutional Data Gateway is another solution for assuring that key data are shared.

  2. Encourage committees that supervise broad areas of the institution, such as admissions, enrollment planning, undergraduate education, or graduate education to develop plans for evaluating their processes. This will allow data analysts to collect data for more than ad hoc queries and provide time for in depth analysis.

  3. Develop a centralized plan for identifying and collecting undergraduate student learning outcome data. Give the responsibility for collecting these data to a single unit. It is also important to create a structure for reviewing and evaluating learning outcome data. There should be some common measures that all departments are assessed on, and some specific to their individual needs. Course evaluations, assessments of student work, an assessment of the student learning experience (in the new SERU-21 Survey), and measures of alumni success, may form a backbone for the "common data" collected in this process. Review and evaluation of learning outcome data should start at a central level, but there also must be a formal method of communicating findings at the college and departmental level.

 

Relevant WASC Standard
graphic ornament

Standard 4

Creating an Organization Committed to Learning and Improvement

• Strategic Thinking and Planning
• Commitment to Learning and Improvement


The institution conducts sustained, evidence-based, and participatory discussions about how effectively it is accomplishing its purposes and achieving its educational objectives. These activities inform both institutional planning and systematic evaluations of educational effectiveness. The results of institutional inquiry, research, and data collection are used to establish priorities at different levels of the institution, and to revise institutional purposes, structures, and approaches to teaching, learning, and scholarly work.


line
  [Accreditation Home][Undergraduate Education Home][UC Berkeley Home]

© Copyright 2002 The Regents of the University of California at Berkeley. All Rights Reserved.
To report problems with this website, please contact the
webmaster.