Job Description

Position Overview

The Assessment System Analyst collaborates with Undergraduate Medical Education leadership, course and clerkship directors, and faculty on the development of clinical assessments (e.g., EPAs), objective structured clinical examinations (OSCEs), and simulation. This person will assist in the implementation of testing software, will collaborate on building exams, and work with dashboards and assessment systems and software, including Item Response Theory and computer adaptive testing (CAT).

The Assessment System Analyst will be knowledgeable and experienced in the use of statistical and analytic software such as SPSS, R, Python, etc. and be able to conduct statistical and qualitative data analyses.

The Assessment System Analyst will train and support students, faculty, and staff on the use of medical school assessment tools. This person will serve as a contact point with collaborators in the office of information technology on issues related to assessment systems and software.

Duties/Responsibilities

30%: Development of Clinical Assessments

Analyze and refine the Clinical Skills Assessment (CSA) protocol
Assemble EPA assessment data and reports as requested
Communicate effectively with students, staff, faculty, clerkship directors and leadership on the results of data analysis
30%: Develop and Maintain Dashboards and Assessment Systems

Prepare data tables, charts, statistical significance
Analyze and interpret data tables, charts, statistical significance, dashboards using software such as SPSS, R, Python
Prepare narrative reports
20%: Implementation and Management of Testing Software

Manage ExamSoft item analysis
Explore the use of Fastest for potential use in (CAT)
10%: Collaborate on Building Exams

Consult with Foundational Course directors, instructional designers and instructors regarding effective test construction and results of item analysis for continuous improvement
Advise Clerkship Directors
10%: Maintain Clinical Exams

Analyze results of objective structured clinical examinations (OSCEs) and collaborate with faculty and assessment colleagues to refine OSCEs based on data
Qualifications
Required Qualifications

BA/BS in psychology, education, or related fields with expertise in assessment and measurement, psychometrics, statistics, data analytics, or related areas and two years of work experience, or master's degree
Experience administering and analyzing assessments
Preferred Qualifications

Master’s degree in psychology, education, or related fields with expertise in assessment and measurement, psychometrics, statistics, data analytics, or related areas
Training and experience in educational evaluation and measurement and/or research and assessment.
Work experience in assessment and evaluation, preferably in medical education
Proficiency with data analysis and data handling, storage and cleaning.
Ability to utilize automated statistical programs
High degree of proficiency in the construction and utilization of database, spreadsheet, word processing applications and other data analysis tools along with basic statistical analysis skills.