Session Descriptions (2018)

Sessions and presentations (PDFs where available) for your reference.

Untitled Document

Plenary Sessions

Assessment Plenary: Conceptual Basis, Evidence Base and Future Implications of the Australian Professional Performance Framework for Continuing Professional Development

Professor Elizabeth Ann (Liz) Farmer, BSc Hons, MBBS, PhD, FRACGP, MAICD, Immediate past Chair, Medical Board of Australia Expert Advisory Group on Revalidation 2015–17

CEO Plenary: Professional Self-Regulation in the Post-Truth Era

Richard E. Hawkins, MD, President and Chief Executive Officer, American Board of Medical Specialties

Operations Plenary: The Art and Science of Leading People through Change and Transition: Achieving Improvement by Optimizing the People-side of Change

Stephanie Waite, BS Ed, MSLOC, Director of Leadership and Organizational Development at Ann and Robert H. Lurie Children's Hospital of Chicago

The Inaugural Lois Margaret Nora, MD, JD, MBA Endowment Lecture:
Las Vegas Reflections on Authenticity

Lois Margaret Nora, MD, JD, MBA

▲ Go to Top

Health Policy Research Sessions

Assessing Assessment: Exploring Impact and Improvement Opportunities

Michelle Lin, MD, MPH, Icahn School of Medicine at Mount Sinai Hospital (moderator)
Johnathan A. Bernard, MD, MPH, National Sports Medicine Institute
Regan Bergmark, MD, Harvard Medical School, Massachusetts Eye and Ear Infirmary
Martin F. Casey, MD, MPH, Mount Sinai Hospital/Elmhurst Hospital, New York
Souvik Chatterjee, MD, National Institutes of Health Clinical Center and Johns Hopkins Medicine
Brian T. Garibaldi, MD, Johns Hopkins School of Medicine and Johns Hopkins Biocontainment Unit

Session Summary
The ABMS Visiting Scholars Program fosters scholarship by early-career physician scientists and other researchers to inform physician assessment, enable certification program improvement, and advance patient care quality. In this session, Visiting Scholars will highlight findings from their research exploring assessment of residents’ technical and arthroscopic skill in treating meniscal injuries, inter-rater reliability of bedside skills assessment, and the comparative accuracy of assessment of trainees’ diagnostic reasoning skills using direct observation and web-based, crowd-sourced technology. Visiting Scholars also will discuss findings regarding the extent to which physician specialists use outcomes information to improve practice, the association between variations in emergency department charges and revisit rates, and how participation in continuing certification affects physicians’ resource use, confidence, and professional growth.

Learning Objectives

  • Gain an understanding of tools used to assess physicians’ clinical and surgical skills
  • Identify internet-based diagnostic resources
  • Understand physician specialists’ use of outcomes measures to inform daily clinical practice
  • Recognize the impact of Maintenance of Certification participation on internal medicine physicians’ cost-effectiveness and professional satisfaction
Engaging Millennial Physicians in Specialty Certification and Professional Leadership

Michelle Lin, MD, MPH, Icahn School of Medicine at Mount Sinai Hospital
Dave Emory, Kantar Media
Julie Raymond, American Academy of Pediatricians

Session Summary
Physicians entering practice today, those born between the early 1980s and 2000, bring the attitudes and expectations of a new generation of doctors—one raised with the promise of technology, in the shadow of 9/11 and the economic recession of 2008. Today’s early-career physicians bring expectations, priorities, attitudes, and skills that may be unique to their cohort. This session will explore the differences between “millennial docs” and the physicians that came before them, and consider how the organizations and institutions that seek to serve them may need to change.

Learning Objectives

  • Understand how to engage early-career physicians in certification
  • Identify attitudes and priorities of millennial doctors
Engaging Physicians in Diagnostic Process Improvement: Innovative Interventions [PDF]

Diana Rusz, MPH, Society to Improve Diagnosis in Medicine
Rena T. Kasick, MD, Nationwide Children’s Hospital, The Ohio State University College of Medicine
Katie Raffel, MD, University of California, San Francisco

Session Summary
Led by the Society to Improve Diagnosis in Medicine (SIDM) in collaboration with the Institute for Healthcare Improvement, six leading health care organizations have implemented pilot interventions to improve the diagnostic process by engaging physicians in practice-relevant professional development and involving patients as partners. In this session, SIDM project director Diana Rusz and Nationwide Children’s Hospital and University of California, San Francisco pilot site leaders Rena T. Kasick, MD, and Katie Raffel, MD, will describe their physician engagement strategies, share initial findings, and discuss next steps.

Learning Objectives

  • Better understand the use of Plan-Do-Study-Act rapid-cycle intervention in quality improvement
  • Explore the effectiveness of specific strategies to improve the diagnostic process
  • Identify strategies to evaluate complex interventions
Federalism in Health Care: The Increasing Role of States in Setting National Policy

Rachel Nuzum, The Commonwealth Fund

Session Summary
While congressional efforts to repeal and replace the Affordable Care Act are seemingly on hold for the remainder of 2018, substantial policy changes are being advanced through executive orders and administrative actions at the federal level, many of which provide considerable new discretion to states. In this session, Rachel Nuzum, The Commonwealth Fund’s vice president for federal and state health policy, will describe the federal health policy landscape, the variation in health system performance across the states, the increasing role of state health policymakers, and ways in which The Commonwealth Fund is helping legislators develop the capacity they need to adopt and implement policies at the state and federal levels to ensure access to affordable, high-quality care.

Learning Objectives

  • Understand national and state health policy environments
  • Identify federal and state influence in health care financing, delivery, and access
Finding Your Voice: Why Legislative Engagement Matters

Kim Van Brunt, American Board of Physical Medicine and Rehabilitation
George D. Wendel, MD, American Board of Obstetrics and Gynecology
Alice Coyle Hughes, American Board of Internal Medicine

Additional Presenters
Anne Marie Irani, MD, ABMS Board Member, Board-Certified Allergist, ABAI
Panelists: ABIM, ACS, ABOG

Session Summary
This panel discussion, moderated by a Board-certified allergist, features physician volunteers who will share their experiences testifying before state legislatures on physician assessment and continuing certification issues. Topics will include how to develop a grassroots volunteer network, creating an effective advocacy toolkit, and useful software tools. Panelists certified by the American Board of Internal Medicine and the American Board of Obstetrics and Gynecology, as well as from the American College of Physicians, will provide real-world examples of the effectiveness of grassroots engagement on legislative issues.

Learning Objectives

  • Learn about a range of legislative communications approaches
  • Understand how volunteers and legislators respond to grassroots campaigns by physicians
  • Find out what you need in your advocacy toolkit to create a successful campaign
Medical Professionalism and Accountability: Platforms for High-Value Physician Practice [PDF]

Primary Presenter
Robert Phillips, MD, Vice President, American Board of Family Medicine, Executive Director, Center for Professionalism and Value in Healthcare

Additional Presenters
Matt Austin, PhD, Assistant Professor in the Armstrong Institute for Patient Safety and Quality at Johns Hopkins Medicine

Session Summary
Physicians everywhere chafe under the weight of goals they set for themselves, expectations of their patients, standards of their profession, and requirements of the sites and systems in which they practice. Bombarded by programs that impose external priorities on their efforts, physicians are disillusioned, disgruntled, dissatisfied—and disinclined to participate in either individual or institutional improvement initiatives.

In this session, representatives from the American Board of Family Medicine and the Armstrong Institute for Patient Safety and Quality at Johns Hopkins Medicine will discuss ways to align the professional and organizational programs to better support physicians’ efforts to provide high-quality, high-value patient care. We will explore strategies these organizations are taking to leverage the power of professionalism to improve patient care and achieve greater value.

Session Learning Objectives

  • Quality improvement
  • Improvement in diagnosis
  • System transformation
Patients as Partners in Policy and Program Improvement: Myths, Challenges, Opportunities and Rewards [PDF]

Laurel K. Leslie, MD, MPH, American Board of Pediatrics
Susan Sheridan, MIM, MBA, DHL, Society to Improve Diagnosis in Medicine

Session Summary
Patients and family members are often best positioned to assess professional performance and inform health system improvement. Nevertheless, the value of the patient perspective is often underestimated or overlooked. Session presenters and participants will work together to identify barriers that limit patient engagement, consider how organizations have met those challenges, and discuss the rewards of patient engagement in policy development and program improvement.

Learning Objectives

  • Understand the role patients can play in organizations’ policy development and program improvement efforts
  • Recognize what an organization might need to put in place (e.g., commitment, infrastructure) to support effective patient involvement
  • Identify ways an organization might involve patients, such as ad hoc work groups, standing committees, advisory boards, and focus groups
Strategic Engagement in Research [PDF]

Andrew Jones, PhD, American Board of Surgery
Robert Phillips, MD, MSPH, American Board of Family Medicine
Adam Turner, MPH, PMP, American Board of Pediatrics

Session Summary
The ABMS Member Boards have access to data that have value to internal and external stakeholders (e.g., boards of directors, committees, physicians, the public). These data include physician exam performance, demographics, workforce trends, and information about what physicians do in practice. Different boards have varying strategies and resources for analyzing and disseminating these data publicly.

The purpose of this session will be to discuss how three boards have developed strategic plans for addressing research and evaluation within their respective organizations. Presenters from the American Board of Surgery, the American Board of Pediatrics, and the American Board of Family Medicine will discuss the following topics:

  • Developing a strategic plan for engagement in research and evaluation
  • Developing a research agenda
  • Developing policies for working with external collaborators and stakeholders
  • Managing requests for data
  • Evaluating external requests for research and data
  • Allocating resources for research, including decisions about establishing a foundation and/or an internal research team
  • Partnering with an Institutional Review Board
  • Disseminating research to external stakeholders

The three boards vary in their size, the maturity of their research programs, the type of data to which they have access, and their processes for engaging in research. These varied perspectives will help to inform how other boards might develop or modify a research program at their own organization.

Learning Objectives

  • Identify and summarize key components for developing a systematic plan for engaging in research
  • Learn how to develop a process for collaborating with external collaborators
  • Recommend strategies for working with external collaborators and managing requests for data

▲ Go to Top

Initial and Continuing Certification Sessions

A Collaborative Model for Journal-Based Continuous Assessment [PDF]

Mary Carol Badat, MAdEd, American College of Allergy, Asthma and Immunology
Steve Folstein, MFA, MEd, American Academy of Allergy, Asthma and Immunology
Lawrence Vapniarek, MBA, American Board of Allergy and Immunology

Session Summary
Many physician certification boards are moving away from the traditional high-stakes, secure exam in response to feedback from their diplomates that these assessment models place too high an emphasis on memorization of rote facts. Instead, physicians are looking for assessment methods that are relevant to their practices and reward them for the ongoing, self-directed learning they undertake as part of their professional responsibilities. A common resource used by physicians for this purpose is the multitude of medical journals now available. Medical boards can leverage medical society journals and their educational programming to implement and support continuous assessment programs for Maintenance of Certification (MOC) Part III. Journals can provide an effective and efficient method for both acquiring and assessing knowledge, skills, and attitudes.

This session will explore the American Board of Allergy and Immunology’s Continuous Assessment Program (CAP), the collaboration with the related medical societies, and the educational strategies for supporting the program, which include the use of journal content as the basis for assessment. This session will explore the challenges that prompted the development of the CAP program. We will outline the process of design and implementation along with other project contingencies. We will demonstrate the mechanics of the program and each society’s role. Initial (cycle 1) observations will be discussed.

Additionally, we will look at the approach the American Board of Allergy and Immunology took in collaborating with the societies and how the societies both collaboratively and independently developed a variety of educational approaches for their members. Broader applications and implications will be explored.

Learning Objectives

  • Describe a journal-based model for continuous assessment of knowledge, judgment, and skills (MOC Part III)
  • Utilize both formative and summative assessment in the design of a Part III activity that builds on and links to Part II
  • Design supporting educational activities for journal-based continuous assessment programs
Accredited Improvement Activities: Improving Practice or Quality for MOC, CME, and MIPS [PDF]

Kate Regnier, MA, MBA, Accreditation Council for Continuing Medical Education
Natalie Trahey, ABIM

Session Summary
To facilitate the engagement of clinicians in practice improvement, the Accreditation Council for Continuing Medical Education (ACCME) is collaborating with several stakeholders to align and simplify approaches for physicians to meet continuing certification expectations with the performance and quality improvement work that they engage in where they work and learn. By harmonizing accreditation requirements with professional expectations, this collaboration has created the opportunity for accredited providers to award American Board of Internal Medicine (ABIM) Maintenance of Certification Practice Assessment (i.e., Part IV) points for Continuing Medical Education (CME) activities that address performance and/or quality improvement. The Centers for Medicare & Medicaid Services (CMS) have also recognized these accredited improvement activities as an approved Clinical Practice Improvement Activity (CPIA) that physicians and other health professionals can use to meet the Merit-based Incentive Payment System (MIPS) requirements in the Medicare Access and CHIP Reauthorization Act.

In this session, we will share the opportunity this collaboration presents for boards and accredited CME providers. We will also conduct some hands-on exercises to explore simple ways to plan performance and quality improvement activities that meet the expectations of ACCME, ABIM, and CMS.

Learning Objectives

  • Describe strategic approaches that specialty boards can take to simplify the engagement of physicians and other health professionals in performance and quality improvement by using harmonized requirements and ACCME’s Program and Activity Reporting System
  • Explain the requirements and values that are shared across educational (i.e., CME), medical specialty board, and regulatory stakeholders with respect to learning and improvement and payment incentives
  • Identify alignment between ACCME requirements and the requirements for accredited improvement activities that meet ABIM Practice Assessment and CME MIPS CPIA expectations
  • Reflect on what existing performance and quality improvement CME activities could be easily offered for ABIM Practice Assessment and CMS MIPS requirements
The American Academy of Orthopaedic Surgeons and the ABOS: Together We Are Better

Primary Presenter
David Martin, MD, American Board of Orthopaedic Surgery

Additional Presenter
David Halsey, MD, American Academy of Orthopaedic Surgeons

Session Summary
Between 2015 and 2018, American Academy of Orthopaedic Surgeons (AAOS) members indicated increasing dissatisfaction with the Maintenance of Certification (MOC) program of the American Board of Orthopaedic Surgery (ABOS). Numerous groups had submitted negative advisory opinions to the AAOS over the past 10 years, but the more recent advisory opinions were increasingly making demands for alternative pathways that improved or eliminated the high-stakes examination as a knowledge assessment method. Several of the advisory opinions asked for an alternative certification pathway to be developed by the AAOS. AAOS members were unhappy with the high-stakes examination and were frustrated by the lack of alternatives.

An existing AAOS/ABOS MOC task force explored the development of an AAOS alternative MOC pathway. With significant leadership cooperation and communication between the organizations and significant input from AAOS leadership and membership, change has occurred, and the dissatisfaction has decreased.

The increased options for diplomates to satisfy the knowledge assessment portion of the ABOS MOC program have been added rapidly to meet the changing needs of AAOS members. ABOS has established Practice-Profiled Examinations for certain orthopaedic subspecialties; these examinations do not include general orthopaedic questions, as the diplomates are tested only in their subspecialty to reflect their practice. Practice-Profiled Examinations are also available in orthopaedic sports medicine and surgery of the hand, even for those diplomates who do not have those subspecialty certifications but wish to be examined in these practice areas.

A recent survey indicated that nearly 40 percent of diplomates would prefer a longitudinal assessment program for the next time they need to recertify. ABOS committed to implementing a web-based longitudinal assessment program as an alternative knowledge assessment pathway in the ABOS MOC program.

Learning Objectives

  • Articulate strategies for collaboration between a specialty society and the corresponding certification board to meet the needs of the members and diplomates
  • Understand that cooperation and collaboration are critical to an ongoing continuing certification program that is relevant and valuable
  • Learn how member organizations and certifying boards can work together to affirm, maintain, and improve physician self-regulation
Assessing Professionalism [PDF]

Carolyn Kinney, MD, American Board of Physical Medicine and Rehabilitation
Susan Ramin, MD, American Board of Obstetrics and Gynecology
David Martin, MD, American Board of Orthopaedic Surgery
Leonie Gordon, MD, American Board of Nuclear Medicine

The assessment of professionalism has been a topic of discussion for many Member Boards of the American Board of Medical Specialties. Professional behavior is necessary to gain the trust of our patients and the public. A shared definition of knowledge, skills, and attitude that encompass professionalism will be discussed using the model of three core elements: devotion to medical service; public profession of values; and negotiation regarding professional values and other social values.

Learning Objectives

  • Define medical professionalism.
  • Describe what constitutes being professional in the practice of medicine. What aspects of professionalism do we intend to measure? What knowledge, attitudes, and behaviors?
  • Develop a tool that can be used for evaluating professionalism for all Boards, and which looks at the reasons for a behavior, rather than just the behavior itself.
  • Discuss integrating clinical vignettes with multiple choice questions that all Boards can use for the assessment.
Assessment for Health Systems Science: Accomplishments, Gaps, and Future Directions

Michael Dekhtyar, American Medical Association
Jean D’Angelo, National Board of Medical Examiners
Linette Ross, MA, National Board of Medical Examiners

Session Summary
To bridge the gap between education and practice and to address the quadruple aim of medicine, medical schools are integrating health systems science (HSS) content into curricula to form a triad alongside the basic and clinical sciences. HSS is defined as the principles, methods, and practice of improving quality, outcomes, and costs of care. HSS will help learners and practitioners contextualize the patient’s health concerns within their health system experience. Examples of HSS interventions to improve knowledge and skills include student participation as patient advocates.

To build a foundation of learner assessment and program evaluation for HSS curricula, the American Medical Association (AMA) Accelerating Change in Medical Education consortium and the National Board of Medical Examiners (NBME) have piloted a high-fidelity, vignette-based multiple-choice examination. Areas of focus of the piloted version of the examination included quality improvement, patient safety, evidence-based medicine, and teamwork. Future areas will include population and social determinants of health; clinical informatics; health care structures, policy, and economics; and ethics/professionalism.

This session will describe the development and initial validity evidence supporting the HSS examination and will address concerns related to creating adequate questions that encompass HSS principles. Presenters will also articulate next steps for the continued development and quality improvement of this examination and the relevance of HSS principles and assessment throughout graduate medical education, credentialing, and practice. Presenters will also leverage the experience of the AMA’s 32-school consortium and NBME’s rich history of developing innovative assessment to address and discuss gaps and potential strategies to develop robust assessment and evaluation programs focused on HSS education and practice.

Learning Objectives

  • Understand how initial validity evidence was gathered for the HSS examination to demonstrate the relevance of content and its appropriate use in formative and program evaluation settings
  • Recognize the unique needs of assessment and evaluation programs related to teaching and evaluation of HSS
  • Identify future possibilities for expanding use of the examination in new contexts through graduate medical education and practice
Behind the Scenes: Lessons from an Article-Based Approach to Lifelong Learning and Self-Assessment

Sylvia Aziz, MHA, CSSGB, American Board of Obstetrics and Gynecology
Michael Nelson, MD, American Board of Allergy and Immunology
Patti Vondrak, American Board of Psychiatry and Neurology

Session Summary
Whether you’re thinking about implementing an article-based approach to Lifelong Learning and Self-Assessment (LLSA) or whether you’ve utilized it for several years, this session will take you on a journey from the eyes of three Member Boards (American Board of Obstetrics and Gynecology, American Board of Psychiatry and Neurology, and American Board of Allergy and Immunology) that are all at different stages of the LLSA article-based process. We will explore the course of development of the programs as well as implementation techniques and logistics. Finally, we’ll take a look at lessons learned, ongoing challenges that each board faces, and tips for any boards interested in pursuing or continuing an article-based approach to their LLSA requirements.

Learning Objectives

  • Describe the infrastructure of an article-based method as part of LLSA
  • Identify key preparations necessary to implement an article-based approach to avoid barriers and complications
Catching Up: How Principled Assessment Design Can Welcome Assessments to the 21st Century [PDF]

Karen Hoeve, MEd, American Board of Pediatrics
Robert Brucia, MEd, American Board of Pediatrics

Session Summary
For the past century, methods for developing high-stakes certification exams have remained relatively unchanged. Traditional methods have stood the test of time and continue to produce valid and reliable exams. Given the stability and success of traditional design methods, testing professionals are inclined to take an “If it’s not broke, don’t fix it” stance. While the presenters will neither argue for nor against traditional methods, they will provide an alternative approach that may be considered for future assessments.

In this session, presenters from the American Board of Pediatrics will discuss the various components of principled assessment design (PAD). A compare-and-contrast approach will show how PAD may improve on the current test development methods used by many organizations. Acknowledging that a “redesign the wheel” approach is not feasible, the presenters will explain ways to incorporate PAD methods with current practice, with the goal of demonstrating how improvements can be made without tearing existing assessments down and starting from scratch. Defining a construct, blueprinting, writing and reviewing items, and scoring will be explored, and examples of PAD approaches to these topics will be provided. Session attendees will leave with an understanding of what PAD is, how PAD may improve current development practices, and how to implement PAD methods in their own assessment design activities.

Learning Objectives

  • Understand the basics of principled assessment design (PAD) and how it differs from traditional design methods
  • Determine the feasibility of PAD approaches and how they may improve current design methods
  • Learn how to incorporate PAD into current design methods
CertLink—A Small Board’s Perspective [PDF]

Maria Watts, MBA, American Board of Nuclear Medicine
Cecily Marroquin, MA, American Board of Medical Genetics and Genomics
Patrick Murphy, BS, RT(R)(CT), American Board of Nuclear Medicine

Session Summary
This session explores the small-board perspective on the process of developing and preparing the pilot CertLink™ program (longitudinal assessment program) and ultimately launching it to the American Board of Medical Genetics and Genomics (ABMGG) and the American Board of Nuclear Medicine (ABNM) diplomates. The presentation will depict how ABMGG and ABNM approached program development, item writing, communication with diplomates, the registration process, deploying surveys, the launch itself, and post-launch challenges and decisions. In addition, this session will address how the Member Boards worked with the ABMS CertLink team to develop and implement the CertLink platform. This session will give insight to all boards, but particularly to boards with fewer than 10 staff members.

Learning Objectives

  • Learn why following the timeline is key
  • Understand the importance of open communication with ABMS, presence during meetings and conference calls, and communication with diplomates
  • Explain the details of the stages and processes involved in the CertLink launch
  • Identify the lessons learned for application in other boards’ pilot programs, particularly longitudinal assessment pilots
Clinician Burnout: Case Studies from Minnesota’s Statewide Effort to Improve Culture in Health Care [PDF]

Rahul Koranne, MD, MBA, Minnesota Hospital Association

Session Summary
Facing a rising epidemic of burnout in physicians and advanced practice professionals (APPs), health care leaders across Minnesota collaborated to launch a statewide improvement effort focused on improving work culture and increasing well-being. Since 2016, the MHA health care burnout survey reveals that more than half of physicians and APPs feel great stress because of their work, and 37 percent report feeling burned out.

When exploring the drivers of burnout, we find that more than one-third of Minnesota physicians and APPs disagree with the notion that they have enough time for what is important. Close to one-third report they disagree with the idea that they have control over how they work or that resources are allocated fairly. This driver data, coupled with the burnout prevalence data, reveal a problem that is ripe for response and growing in magnitude but likely responsive to change.

Efforts to develop a statewide quality improvement roadmap around burnout, focused on organizational-level interventions and actions, practice-level interventions and actions, and individual-level interventions and actions, are underway, in partnership with the 50+ chief medical officers from hospitals and health care systems across the state.

We will highlight eight Minnesota hospitals and health systems that have seen an improvement in their work environments because of intentional action. Specific strategies used by sites will be presented, coupled with improvement data showing changes.

Learning Objectives

  • Describe the prevalence of burnout across the country and in a region, including appreciation of the distinct experiences of physicians and APPs, as well as the primary drivers of burnout in health care
  • Value a framework for action around burnout that is synthesized from multiple existing frameworks and sets the context for hospitals and health systems, large and small, to act around the issue
  • Develop a personal action plan around burnout based upon case study information from sites ranging from small, rural hospitals to large, metropolitan health systems and their successful interventions to reduce this phenomenon
Cohort-Based Longitudinal Surveys of Anesthesiologists’ Burnout, Distress, and Depression

Huaping Sun, PhD, American Board of Anesthesiology
Yan Zhou, PhD, American Board of Anesthesiology
Ann Harman, PhD, American Board of Anesthesiology

Session Summary
Physician burnout, distress, and depression can cause concerns for physician well-being and patient safety. Long working hours and high student debt are commonly believed to be risk factors, particularly for residents in training or junior physicians just entering practice. To better understand characteristics that could be associated with anesthesiologists’ burnout, distress, and depression, the American Board of Anesthesiology has been sending cohort-based longitudinal surveys to anesthesia residents and post-graduate year 5 (PGY-5) fellows or practicing anesthesiologists since 2013.

Three validated scales—the Maslach Burnout Inventory (MBI), Physician Well-Being Index (PWBI), and Harvard Department of Psychiatry/National Depression Screening Day Scale (HANDS)—were used to measure the risk of burnout, distress, and depression. Additional questions were developed to gather information about demographic characteristics, personal life situations, professional experiences, and self-perception of work-life balance and social support system. A total of 11 surveys were collected across four cohorts in four years.

About 37% to 57% of anesthesiologists were identified to have risk of burnout, 17% to 37% were found to be in distress, and 9% to 15% were found to have depression. The three consistently protective factors across the three scales were whether the workplace provides appropriate resources to address burnout/depression, whether the physician is maintaining a balance between his or her personal and professional lives, and whether the physician has a strong social support system.

More detailed results will be presented during the session, and implications of the findings will be discussed. The presenters will also share their experiences with designing and sending large-scale surveys and strategies to increase response rates.

Learning Objectives

  • Better understand anesthesiologists’ burnout, distress, and depression and associated characteristics
  • Learn how to design large-scale surveys to better understand the issues of interest while protecting participants’ anonymity and confidentiality
  • Identify strategies for analyzing multiple surveys across cohorts and years
Designing Secure Continuous Assessment Strategies for MOC Programs [PDF]

Linda Althouse, PhD, American Board of Pediatrics
Benjamin Chesluk, PhD, American Board of Internal Medicine
Kirk Diepenbrock, American Board of Obstetrics and Gynecology

Session Summary
Examinees are asking for more convenient and flexible assessments that offer a learning component. As technology and knowledge rapidly change within a domain, it is important to stay current. Programs need to assess whether individuals have kept current and whether they continue to demonstrate appropriate levels of competence. This assessment practice is affecting medical specialty boards’ Maintenance of Certification (MOC) programs in a big way. As many of these boards make the shift to providing continuous assessment programs, they are considering how they will secure these assessments and authenticate their examinees.

This session will present case studies from three medical specialty boards on their approaches to ongoing MOC programs for their board candidates. The presenters will discuss the market demand for more timely and relevant education, describe the challenges and considerations for test security of continuous assessment, and provide the measures and the results of the methodologies implemented.

Learning Objectives

  • Describe the market demand for more timely continuous assessments from certification holders in the medical board MOC space
  • Define secure assessment strategies that support continuous assessment
  • Discuss the challenges of implementing continuous assessment in a certification program
Ensuring a Relevant and Results-Oriented LLSA Program through a Creative Data-Driven Framework

Pooja Shivraj, MS, PhD, American Board of Obstetrics and Gynecology
Kirk Diepenbrock, American Board of Obstetrics and Gynecology

Session Summary
The American Board of Obstetrics and Gynecology (ABOG) has a Lifelong Learning and Self-Assessment (LLSA) program that is article-based and showcases new studies, practice guidelines, recommendations, and up-to-date reviews. Vetted articles, accompanied by questions, are intended to contribute to better patient care by requiring ongoing diplomate participation in continuous learning and self-assessment activities. Given the intended use of these article-based assessments, care needs to be taken to ensure that the measures of these assessments are reliable, of high quality, and of relevance and value to diplomates. In addition, interpretations from these assessments need to be valid to warrant appropriate use of data.

Ronka, Geier, and Marciniak (2010) propose a framework for effective data interpretation and use. Three components of the framework include data quality, capacity, and culture. Though the framework was initially proposed for use in K–12 education, this session generalizes the framework and applies it to reflect on the interpretability and use of decisions made from ABOG’s LLSA program.

During this session, we will:

  • Provide a brief overview of ABOG’s LLSA article-based continuing certification program while focusing on the type of data obtained from it.
  • Describe the three components of the framework, and how it could be generalized and used by other Member Boards to reflect on decisions made from their assessments.
  • Discuss measures to ensure data quality.
  • Analyze the psychometric quality of data, appropriateness of cut scores, and relevance to practice.
  • Review data capacity, and impediments within our program.
  • Assess how we are driving the data culture within the organization to support both quality and capacity.

Learning Objectives

  • Provide a broad overview of the structure of ABOG’s article-based continuing certification program
  • Gain insight into the type of data obtained from ABOG’s program, and how the data are interpreted and used
  • Build an understanding of a creative and generalizable framework that ensures the integration of data quality, and build capacity and culture for effective data interpretation and use
Examinee References and Resources: Steps toward Open-Book Testing and Innovative Item Development

Matt Ferris, MA, ELS, National Board of Certification and Recertification for Nurse Anesthetists
Dot Horber, PhD, National Board of Osteopathic Medical Examiners

Session Summary
As certification organizations move toward nontraditional assessments, provision of reference resources during assessment is one of many areas of uncertainty. Though reference material availability probably increases an assessment’s fidelity to clinical situations, it raises many implementation questions: What references will be useful to examinees, and what is the best way to make them available within any assessment design? What references are acceptable resources? What effect will reference availability have on test time needs, test outcomes, or test validity? Searching for evidence, organizations face the meta-question: How do we base our decisions on empirical data?

This session follows the experiences of the National Board of Certification and Recertification for Nurse Anesthetists (NBCRNA) and the National Board of Osteopathic Medical Examiners (NBOME) in answering these questions about examinee reference use.

Presenters will discuss how NBCRNA has used empirical data to guide beta research design of a Continued Professional Certification Exam (CPCE) for its 50,000-plus certificants, incorporating open-book features, and report results of smaller-scale related studies.

Presenters from NBOME will discuss development of the Point-of-Care Knowledge, Education, and Testing (POCKET) program, an innovative format that focuses on competency domains other than clinician knowledge recall. POCKET is being developed for the Comprehensive Osteopathic Medical Licensure Examination series for osteopathic physicians and incorporates online resources to locate clinical diagnostic and treatment information to answer questions.

Presenters will discuss the item development process and present data obtained from a proof-of-concept study of osteopathic medical residents in late 2016. Lessons from this study and planned next steps will provide insights to organizations seeking more authentic modes of assessment of physician clinical behavior and decision-making.

Learning Objectives

  • Identify methods for using data to guide decisions about innovative test design and item development and to reduce risk
  • Describe important potential impacts of introducing resource availability to certification assessments
Geriatrics Virtual Patient Case Modules for Surgical and Related Medical Subspecialties [PDF]

Thomas N. Robinson, MD, University of Colorado School of Medicine/American Geriatrics Society
Andrew G. Lee, MD, The Methodist Hospital/American Geriatrics Society
Myron Miller, MD, Sinai Hospital of Baltimore/American Geriatrics Society

Session Summary
The Geriatrics Virtual Patient Case (VPC) Modules for Surgical and Related Medical Subspecialties focus on cross-cutting geriatric specialty care issues important to specialty practitioners. This new, multimedia educational curriculum emphasizes evidence-based care of the geriatric patient with the aim of improving the delivery of care for individual older adults in addition to raising the quality of geriatric health care across the system as a whole.

The goals of this session include introducing the Board Certification community to these Modules; demonstrating the multimedia educational modules focused on cross-cutting geriatric specialty topics most relevant to surgical and medical specialists; and showcasing the accessibility and usability of the new educational curriculum.

Ten geriatrics VPC modules were developed over the past two years to launch in 2018. This project is a collaborative effort between the American Geriatrics Society Geriatrics-for-Specialists Initiative and the American Board of Medical Specialties and funded by The John A. Hartford Foundation. The VPCs are designed to use multimedia and realistically presented clinical tasks to pose key challenges in caring for older patients; provide feedback and education applicable across specialties in the delivery of high-quality care to older adults; and fulfill Maintenance of Certification requirements for Lifelong Learning and Self-Assessment. Five geriatricians and 16 other specialist physicians participated in the development of the VPC modules.

This presentation will highlight the background for the project, describe which specialists the cases are geared toward, review the multimedia educational curriculum content, and demonstrate how to access the cases.

Learning Objectives

  • Participate in realistically presented clinical tasks that pose key challenges in caring for older patients
  • Provide contemporaneous branching logic feedback and educational content applicable across specialties in the delivery of high-quality care to older adults
  • Describe a means to fulfill Maintenance of Certification requirements for Lifelong Learning and Self-Assessment
Incorporating the ACGME Competencies into the Certification Process through Nonwritten Examinations

Poojah Shivraj, MS, PhD, American Board of Obstetrics and Gynecology
Huaping Sun, PhD, American Board of Anesthesiology
Wilma Larsen, MD, American Board of Obstetrics and Gynecology

Session Summary
In 2002, the Accreditation Council for Graduate Medical Education (ACGME) launched the Outcomes Project, which instituted six core competencies for all residencies and fellowships. Since then, the competencies have been refined and updated. The competencies currently include (1) patient care and procedural skills, (2) medical knowledge, (3) practice-based learning and improvement, (4) systems-based practice, (5) interpersonal and communication skills, and (6) professionalism. ABMS Member Boards attempt to incorporate these competencies into their both their written and nonwritten certification examinations.

Written examinations, in general, tend to assess the first two competencies adequately; it is more difficult to adequately assess the latter four competencies, sometimes referred to as soft skills. Presenters from the American Board of Anesthesiology (ABA) and the American Board of Obstetrics and Gynecology (ABOG) will discuss mechanisms they have developed to incorporate assessments of these competencies into their certification process.

ABA launched a program in 2018 to assess two domains that may be difficult to evaluate in written or oral exams—communication and professionalism, and technical skills related to patient care. Interactions with a standardized patient or clinician simulate practical physician performance in real clinical care settings.

ABOG incorporated the assessment of these competencies through structured cases in each of its subject areas. A committee is developing cases that test a physician’s ability to communicate with patients and peers as well as how a physician manages ethically complex situations. By specifically developing scenarios to assess these skills, the board is able to ensure that those physicians who become certified are demonstrating basic skills in these areas.

Learning Objectives

  • Learn how to develop questions on nonwritten examinations that will demonstrate physicians’ competence on the ACGME core competencies
  • Understand the pitfalls and benefits of structured cases in evaluating soft skills
  • Gain awareness of the possible research that could be conducted to validate the nonwritten certification examinations using ACGME competency assessment from training programs
Item Review Strategies: Utilizing Volunteer Subject-Matter Experts to Assemble High-Quality Exams [PDF]

Bailey Tinkle, American Board of Radiology
Dagoberto Esquer, American Board of Radiology
Vivian Malek, American Board of Radiology

Session Summary
Subject-matter expert (SME) involvement is an essential component of the assembly of exams and assessments for the American Board of Radiology (ABR). Exam content is written and reviewed by practicing physicians and professionals who are active in the fields of diagnostic radiology, radiation oncology, interventional radiology, and medical physics. SME involvement in the writing and review of the exam items promotes pride in the products, demonstrates responsibility to their communities and the public, and drives the composition of exams and assessments that are both clinically relevant and reliable across their four disciplines.

ABR utilizes more than 1,000 volunteer SMEs in different roles. Some are writers or reviewers only, while others are writers and reviewers who finalize the items that make up the exams. The presenters will discuss old and new processes, from handwritten submissions and radiologic film to electronic submission of items and images. They will also discuss the struggles of motivating volunteers through the ups and downs of refining the item review process to better integrate with the often chaotic daily schedule of medical professionals, while still encouraging mastery of item writing skills. They will discuss the pros and cons of utilizing volunteers as SMEs, and how it has shaped the review process. Analyzing solutions and success from previous processes, they will discuss future endeavors that incorporate SMEs into all aspects of the item review process, in addition to new challenges with the new online longitudinal assessment.

The hope is to present an alternative methodology for reviewing items and open a dialogue about existing struggles in content development.

Learning Objectives

  • Explain how to solicit and leverage volunteer SMEs in the exam content development process
  • Demonstrate how item review strategies are applied to the different production processes for different examination and assessment products
  • Discuss common issues and solutions encountered during item review by ABR
Lessons Learned: Writing Items and Critiques for Question-Based Longitudinal Assessment Programs [PDF]

Carolyn Kinney, MD, American Board of Physical Medicine and Rehabilitation
Rebecca Johnson, MD, American Board of Pathology
Mikaela Raddatz, PhD, American Board of Physical Medicine and Rehabilitation
David Swanson, PhD, ABMS

Session Summary
As a part of their continuing certification program, many ABMS Member Boards are implementing a longitudinal assessment to assist diplomates in keeping up to date. A substantial implementation challenge is development of substantial numbers of high-quality, clinically relevant questions and “critiques” (explanations of answers) for use in the assessment. This session will address the approaches used in preparation of assessment material by two Member Boards that recently launched their programs. The presenters will provide tips for systematically developing questions, “clones” (superficially different questions used to assess learning and retention), and critiques, as well as procedures for assuring quality through a structured review process.

Participating Member Boards are using a question-based approach to longitudinal assessment in which items are predominantly structured as clinical vignettes describing patient care situations and challenging diplomates to make clinical decisions (most likely diagnosis, next step in workup/management). Critiques are structured to explain why the keyed answer is correct and the distractors are incorrect; links to reference material are also provided for diplomates wishing to learn more about a topic. Presentations will include sample material and lessons learned in the development and review of questions and critiques.

Learning Objectives

  • Be able to identify key features of clinically relevant, well-written questions and critiques for use in longitudinal assessment programs
  • Appreciate the importance of systematic approaches to development and review of questions and critiques consistent with characteristics of the practice of one’s specialty
Milestones 2.0: Using the Past to Define the Future

Laura Edgar, EdD, Accreditation Council for Graduate Medical Education
Stanley Hamstra, PhD, Accreditation Council for Graduate Medical Education

Session Summary
The Accreditation Council for Graduate Medical Education (ACGME) launched the Milestones in 2013 as part of the Next Accreditation System. At the time of the launch, ACGME committed to starting a thorough review and revision process three to five years later. The review and revision process formally began in 2016 with four interdisciplinary workgroups looking at four competencies: Interpersonal and Communication Skills, Practice-based Learning and Improvement, Professionalism, and Systems-based Practice. We began forming workgroups for specialty-specific Patient Care and Medical Knowledge Milestones in 2017. As of February 2018, 15 specialties were at various stages of the process.

Before each of the workgroups could begin, a review of the data had to be completed. Data were available from two areas. Milestone evaluations are reported to ACGME twice each year, and research about the Milestones has been completed by ACGME and others. The data will help inform Milestones 2.0. The availability of Milestones data in the context of national trends not only provides program directors with a tool for making the best decisions regarding learner progression and graduation but also can be used to reflect on the applicability of national curricula within a specialty.

In this session, we will review some of the data collected through the Milestones, discuss ongoing research, and examine how this information affects Milestones 2.0.

Learning Objectives

  • Discuss the value of Milestones research
  • Explain how Milestones 2.0 are being developed
MOCA-Peds 2017 Pilot Results: Outcomes, Learning, Clinical Practice Changes, and What Is to Come [PDF]

Valerie E. Haig, American Board of Pediatrics
Adam Turner, MPH, American Board of Pediatrics
Robert Furter, PhD, American Board of Pediatrics

Session Summary
Advances in technology have greatly advanced the possibilities for assessing medical knowledge and clinical judgment through more convenient, continuous, and dynamic models that also incorporate learning into the assessment process. The American Board of Pediatrics (ABP) is one of the first medical specialty boards to pilot a continuous, web-based assessment program through its new program, Maintenance of Certification for Pediatrics (MOCA-Peds), which will serve as an alternate to the prior secure, proctored exam.

The purpose of this presentation is to provide the results of the 2017 MOCA-Peds pilot and how these have led to learning and clinical practice changes. We will discuss challenges and lessons learned, and highlight change strategies to improve the model being implemented in 2019 and beyond.

Key information to be considered include performance, platform usage, and survey self-reported data; participant item-level feedback; and test components (including item development) from the platform.

Within these data elements, early indicators show a positive response to the pilot. Both item-level feedback and diplomate surveys have shown that item rationales and references are being well received and commended as informative and beneficial. Similarly, in the most recent survey of 2017 pilot participants who met the passing standard, 62.0% (1,727) of survey respondents self-reported at least one practice change related to participation, with another 16.8% (468) reporting a future planned change. In the same survey, 96.2% plan to maintain their general pediatrics certification using MOCA-Peds going forward. These available data are being used to determine both successes and areas for improvement moving forward.

Learning Objectives

  • Recognize the challenges and benefits associated with continuous assessment models
  • Articulate unique attributes of longitudinal assessment components and their impact on learning and potential practice change
  • Summarize the ABP’s methods to further augment their own methodology in creating and/or analyzing longitudinal assessment and survey data
Multiple Approaches to Studying Board Certification and Disciplinary Actions [PDF]

Andrew Jones, PhD, American Board of Surgery
Michael Peabody, PhD, American Board of Family Medicine
Mikaela Raddatz, PhD, American Board of Physical Medicine and Rehabilitation

Session Summary
ABMS Member Boards are under increasing pressure to link their certification processes to external measures of quality. One such measure that has received increased attention in recent years has been physician disciplinary actions. Disciplinary actions, unlike patient outcomes or other quality metrics, are readily accessible to Member Boards for research purposes.

Previous sessions at the ABMS Conference have discussed research that focused on the relationship between Board Certification and disciplinary actions. This session will build upon previous work by describing studies by three different boards that have taken unique approaches to studying disciplinary actions. Presenters from the American Board of Family Medicine, the American Board of Physical Medicine and Rehabilitation, and the American Board of Surgery will discuss the following:

  • The different aggregate sources for disciplinary actions data provided by the Disciplinary Action Notification System (DANS)
  • The process of obtaining information about noncertified physicians that may not exist in board databases and how to incorporate that information into a study on disciplinary actions
  • The relationship of both written and oral exams to disciplinary actions
  • Different strategies for analyzing data on disciplinary actions

The presenters will share their insights into working with DANS data and other data sources to provide lessons for other boards who are interested in studying this topic. Empirical findings from their studies will also be presented to show the variability in the relationship between certification and disciplinary actions across boards.

Learning Objectives

  • Learn about the data sources for obtaining disciplinary actions and other physician data and the challenges associated with each source
  • Understand how different types of exams (written and oral) relate to disciplinary actions
  • Discuss how both initial certification and Maintenance of Certification relate to disciplinary actions
Promoting Practice Improvement through Faculty, Forums, Reflection and Learning Management Systems [PDF]

Brian Block, American Society of Anesthesiologists
Gerry Higgins, American Society of Anesthesiologists

Session Summary
The American Society of Anesthesiologists was pleasantly surprised by the buzz occurring within two recently released education programs. Why the surprise? We learned from our mistakes and gave it another go. Attend this session to learn how previous failure doesn’t mean you can’t be successful in designing a program that helps learners affirm what they do well, address areas requiring improvement, and explore new knowledge, skills, and behaviors to improve. Online course configuration and learning management system (LMS) logistics will be explored. Four activities take learners from what they think they know to reflecting and measuring how well they are performing.

Learning Objectives

  • Apply strategies to design online education that challenges learners to reflect and improve their current practice
  • Identify opportunities to leverage LMS features and functionality
Testing Models Incorporating Item Families from Automatic Item Generation

Kirk Becker, PhD, Pearson
Andrew Dallas, PhDM, National Commission on Certification of Physician Assistants

Session Summary
Automatic item generation (AIG) represents a range of computer-assisted or fully automated approaches to the development of test content that typically allows for the systematic creation of a large body of content relating to a focused area of practice. The availability of such content may contribute to test security, construct representation, study aids and formative feedback, and other uses.

This session will provide detailed information on the incorporation of AIG methods in item development at the National Commission on Certification of Physician Assistants (NCCPA) and will explore a range of models for managing, analyzing, and using that content.

Because of changes to Maintenance of Certification for health care professionals, alternative approaches to content development are being explored. AIG represents a paradigm shift that uses an item model as the unit of currency. Whereas existing content models produce a single item, AIG produces many items, all of which measure the same knowledge, skill, or ability. Therefore, if a candidate answers an item incorrectly, an item from the same item model, which would measure the same knowledge but look different to the candidate, could be administered to reassess that knowledge after remediation.

The ultimate goal of NCCPA’s use of AIG is to build items using principled assessment methodologies such as assessment engineering (AE; Luecht, 2006). Yet traditional methods of item analysis and calibration will no longer serve for the massive numbers of items possible using AIG. Methods such as item family calibration and item difficulty modeling present options for this situation, and will be discussed.

Learning Objectives

  • Learn about the types of test items that can be generated using AIG, and the development process necessary to successfully implement AIG for a program
  • Evaluate the pros and cons of different approaches for using AIG-generated items in a testing or learning program
  • Acquire knowledge of new test designs and psychometric approaches, such as item family calibration and item difficulty modeling, to enable full utilization of AIG items
Transforming Continuing Professional Development Opportunities in Clinical Practice [PDF]

Bonnie Jortberg, PhD, RD, CDE, University of Colorado Department of Family Medicine
Elizabeth Bishop, PhD, MS, PMP, American Board of Family Medicine
Robyn Wearner, RD, MA, University of Colorado Department of Family Medicine

Session Summary
The goal of this session is to present the continuing professional development opportunities created by the American Board of Family Medicine (ABFM) and the University of Colorado Department of Family Medicine (CU-DFM). Many physicians across the United States are participating in quality improvement and practice improvement initiatives, yet are having difficulty receiving Performance Improvement Activity or Maintenance of Certification (MOC) Part IV credits. In response to this need, ABFM and CU-DFM have partnered to create opportunities for physicians to receive these credits. Professional development activities have been developed for practices participating in the Transforming Clinical Practice Initiative and are available for physicians from all specialties to complete for Performance Improvement Activity and MOC Part IV credits. The activities include innovative e-learning modules on topics important to practice transformation, such as cost and value of care, patient engagement, coordinating care across the medical neighborhood, population management, patient self-management support, quality improvement, team approach to care, and access to care.

During this session, we will demonstrate the e-learning modules and how learners can access the modules to receive credit. We will also describe how physicians can integrate their practice teams to complete these activities. Examples of a team approach to completing these activities will be provided as part of the session.

Learning Objectives

  • Explore the learning opportunities for physicians to improve their knowledge and skills for transforming clinical practice
  • Discuss the role of integrating the practice team as part of completing professional development activities
  • Describe access to the learning opportunities to receive Performance Improvement Activity and MOC Part IV credits
Update on International Activities: ECFMG, ACGME International, and ABMS International™ [PDF]

Ronald R. Blanck, DO MACP, Chair, Board of Trustees, ECFMG
Mira Irons, MD, ABMS
Lorraine Lewis, EdD, RD, ACGME International

Session Summary
This session will provide information and advances related to the international work being done by the Educational Commission for Foreign Medical Graduates (ECFMG), Accreditation Council for Graduate Medical Education (ACGME), and the American Board of Medical Specialties. The session will focus on the impact of U.S. policy on international medical graduates seeking training or work in the United States, accreditation of residency programs internationally, and providing certification internationally.

Learning Objectives

  • Improve the understanding of current advances across the spectrum of undergraduate medical education, graduate medical education, and board certification internationally
  • Advance knowledge related to future international initiatives
  • Identify potential opportunities for collaboration through international initiatives
Using Virtual Patient Cases for ABMS Certification and Continuing Certification

Richard Bell, MD, Discourse LLC Virtual Patient Cases and Lewis Katz School of Medicine at Temple University
Nina Hoe, PhD, University of Pennsylvania
Norman Berman, MD, Geisel School of Medicine at Dartmouth

Session Summary
Considerable feedback has indicated that diplomates of the ABMS Member Boards want their examinations and continuing certification exercises to be clinically oriented and relevant to their practice. Virtual patients (VPs) are an “interactive computer simulation of real-life clinical scenarios for the purpose of [healthcare and] medical training, education, or assessment” (MedBiquitous 2006). We propose that the ABMS Member Boards explore the use of VPs for both initial certification and continuing certification.

VP cases can be more engaging and better reflect practice-based experiences than most other forms of assessment. VP cases can test data-gathering skills, the ability to formulate a diagnosis, and management of the patient. Cases can be developed for a wide range of conditions, allowing the use of both core and specialty cases to provide a more practice-relevant examination/assessment. VP cases are machine-scored, so interexaminer variation and bias are eliminated.

Our three panelists have extensive experience with the development and use of VP cases. In this one-hour session, we will discuss the development of VPs, present the case for using them in assessment, and discuss the requirements for creating cases that are reliable and valid tools to assess clinical skills. We will describe the process of building a VP for assessment and will demonstrate a case with audience participation. We will describe users’ acceptance of VP cases as an educational and assessment tool.

Learning Objectives

  • Define a VP assessment case
  • List the advantages of VP as assessment tools
  • Explain the importance of developing cases that can be validated
  • Learn the basics of creating a VP case
  • Understand the degree of end-user acceptance of virtual patient cases
  • Conceive of and describe approaches that ABMS and the Member Boards of ABMS could use to introduce VPs into the certification and continuing certification process
When Lifelong Learning Meets Accredited CME [PDF]

Kate Regnier, MA, MBA, Accreditation Council for Continuing Medical Education
Ann Harman, PhD, American Board of Anesthesiology
Melissa Lamaffar, MA, American Board of Pediatrics
Thomas (Ty) McCarthy, American Board of Pathology
Brian Nussenbaum, American Board of Otolaryngology - Head and Neck Surgery

Session Summary
Since 2015, the Accreditation Council for Continuing Medical Education (ACCME) has been collaborating with various specialty boards to meet the evolving needs of physicians and accredited Continuing Medical Education (CME) providers by simplifying the integration of Maintenance of Certification (MOC) and accredited CME.

Our collaboration has grown through shared recognition of the value of accredited CME and has been facilitated by data-sharing and a search tool that allows diplomates to find activities that meet self-identified educational needs ( Together, we will share how this collaboration is reducing the burden on diplomates, fostering flexibility and creativity among CME providers, and engaging physicians and educators in a culture that promotes lifelong learning and improvement. Beginning in 2015 with the American Board of Internal Medicine (ABIM), the goal of our collaboration was to expand the number and diversity of accredited CME activities that could also count for ABIM MOC Medical Knowledge points and to streamline the process for registering CME activities in ABIM’s MOC program. Since then, the American Board of Anesthesiology, American Board of Pediatrics, American Board of Pathology, and American Board of Otolaryngology have joined in. As of March 2018, this initiative had resulted in 17,024 accredited CME activities—delivering 6,440,429 MOC points/credits via 768,589 physician interactions.

In this session, we will describe the collaborative approaches taken to simplify and align processes and to identify and implement improvements, the reactions from the diplomate community, and the efforts to expand on the success of this collaboration. Presentations include identification of opportunities for streamlined processes, evaluation and process improvement strategies, and evidence of physician engagement and improvement within the system.

Learning Objectives

  • Understand the strategic approach to align and simplify requirements and processes for diplomates and accredited CME providers
  • Reflect on your own approaches in order to identify opportunities to better address the needs of diplomates and CME providers
  • Pursue next steps to explore collaboration with ACCME and other specialty boards to simplify the provision of CME activities that count for MOC
Workplace-Based Assessment: Informing Important Decisions about Clinician Competence [PDF]

Patricia Hicks, MD, MHPE, National Board of Medical Examiners
Melissa Margolis, PhD, National Board of Medical Examiners
Thomas Rebbecchi, MD, National Board of Medical Examiners

Session Summary
Workplace-based assessment allows for the collection of data about clinician performance in the authentic clinical environment. Individual clinicians, those responsible for oversight of clinician performance, and patients all require evidence of competence in the tasks and activities required by the profession. Despite the importance of this evidence, there is a distinct lack of availability of assessment systems that can provide such evidence in a valid and reliable way. The National Board of Medical Examiners, the American Board of Pediatrics, and the Association of Pediatric Program Directors have developed a workplace-based assessment system that both provides critical evidence about the performance of trainees at different points along the continuum of educational and professional development and collects analytic evidence to support the validity of interpretations or decisions that are made based on the results.

Development of assessment system content centers on identification and inclusion of the content that is critical to key stakeholders. Content experts within the pediatrics community participated in a series of iterative activities to develop targeted items, instruments, and reports for each phase of assessment.

The presentation will include a description of the content development process that was designed to connect the right observer to the appropriate content within a specific context/learning activity. A demonstration of the Pediatrics Milestones Assessment Collaborative assessment system will provide a clearer picture of the structure and focus of the assessment instruments, and sample reports will be reviewed.

The discussion will focus on evidence that supports the validity of assessment system content, processes, and outcomes. Group dialogue will explore perspectives regarding best practices and future directions for development.

Learning Objectives

  • Describe the important steps for developing the content of a workplace-based assessment system
  • Identify challenges in identifying and refining content for assessment items
  • Evaluate the utility of assessment outcome reports for reinforcing and correcting performance in the workplace

▲ Go to Top

Operations and Communications Sessions

The Art and Science of Leading People through Change and Transition: Follow-up to Plenary Session

Stephanie Waite, BS, ED, MSLOC, Lurie Children’s Hospital

Session Summary
The health care industry continues to be affected by enormous changes, especially as it moves to continuous assessment and competency-based frameworks. These changes are happening at all levels—individual, team, organizational, clinical, business, and environmental. It is critical to understand the concept of change, methodologies for moving through change efficiently, and, most important, how people experience change in order to ensure effective preparation, adoption, implementation, and sustainability. If there’s one thing we know, it’s that change will always occur, so let’s do it with less resistance, more energy, higher adoption, and people who are champions because they understand why a change is occurring.

In this session, you will discover the high-level strategies, frameworks, and tactics to utilize during your next change event, project, or execution of a new idea to ensure buy-in, successful adoption and implementation, and creation of an environment that can sustain the change. You will walk away with ideas and actions that support your ability to more effectively influence and lead people through change. During a breakout session, participants will address individual projects or key changes occurring within their own organizations and discuss deeper-level applications of change and transition concepts to their varied specialties, leadership positions, and cultures.

Learning Objectives

  • Identify the three phases of transition that people move through during change
  • Differentiate between the concepts of change and transition
  • Discuss direct applications of change and transition management to catalyze your success
  • Identify tactics to influence and lead others through change, removing barriers and obstacles that hinder success
Benefits of Shared Data Services

Dimitri Goldstein, American Board of Medical Specialties
Shannon Dunahue, American Board of Family Medicine

Session Summary
This session will dive into the current offerings by the data team at the American Board of Medical Specialties (ABMS): how to effectively utilize ABMS’ shared data services for Primary Source Verification (PSV) and credentialing activities by ABMS and its Member Boards. Q&A will be encouraged.

This closed session is for Member Board staff only.

Learning Objectives

  • Biographical Updates: Custom monthly reports deliver the latest data corrections for maintaining board certification and brand awareness among patients, doctors, and the credentialing community
  • Disciplinary Action Notification Service (DANS): A brief history of the evolution of this service, an overview of its current components, and data quality dependencies for accurate monitoring outcomes
  • Licensure Info Delivery Service: A use case by the American Board of Family Medicine in its current implementation and added value application of PSV data, as it is sourced through a partnership between ABMS and the Federation of State Medical Boards
Branding and Communication of Complex Initiatives: A Member Board’s Practice Analysis

Sandra Hodgson, American Board of Obstetrics and Gynecology
Amanda Novak, MEd, American Board of Obstetrics and Gynecology

Session Summary
Communicating major board initiatives can be complex, information-loaded challenges. Being strategic and comprehensive in your approach is critical for ensuring immediate understanding and easy recollection of your initiative. Timely planning and preparation of communication elements and materials—and including all team members in the project from start to finish—foster a successful campaign effort.

In addition to productive planning and preparation, branding can play an integral role in creating a successful communications plan. Knowing when to brand a new initiative (Is it big enough? Is this going overboard?) can be tough, but depending on your board’s goals and purpose for any given initiative—and understanding the strategic definition of success for that initiative—will aid in determining how “big” to go and whether a full branding effort is needed. Branding and creating a visual identity for your initiative serves multiple purposes:

  • Creates excitement and enthusiasm across all stakeholder groups
  • Facilitates engagement/connection
  • Provides a communications platform on which all materials and tools associated with the project can stand
  • Creates synergy and provides cohesiveness
  • Builds board brand awareness and clearly defines who is behind the initiative

This session, led by members of the American Board of Obstetrics and Gynecology, is designed to help board staff tasked with spearheading major initiatives to successfully launch and implement communications campaigns, including creating a branding platform for the initiative that will help ensure a clear direction and maximum impact.

Learning Objectives

  • Apply a strategic communications approach to effectively communicate complex initiatives
  • Create branding for your initiative to create a “wow” factor and facilitate understanding and recall
  • Use a team-oriented approach across all aspects of the project, including design and communication plan development
  • Maximize communication results using cost-effective internal and external resources
Building a New Diplomate Portal: From Concept to Creation to Implementation

Cecily Marroquin, MA, American Board of Medical Genetics and Genomics

Session Summary
Board diplomate portals provide a necessary platform for tracking diplomate progress as they fulfill requirements for continuing certification. The American Board of Medical Genetics and Genomics (AMBGG) developed a diplomate portal in 2007 to facilitate the needs of the then newly implemented MOC program. Over the course of a decade, the MOC program evolved and outgrew the capacity of the initial portal. During this time of transition from a time-limited, 10-year program toward ongoing continuing certification, ABMGG realized that a new portal was required to best support the long-term vision of the improved program.

This session provides an “in-the-weeds” presentation of the development process, including choosing a vendor, shaping an improved program, designing the user experience, communicating program and platform changes to diplomates, migrating data, and bringing the concept to fruition.

This closed session is for Member Board staff only.

Learning Objectives

  • Learn the details of the stages and processes involved in the ABMGG portal development
  • Identify key lessons learned
Coupling Improvement and Innovation: Results from the AHA Innovation Academy Pilot

Patrick Dolan, MD, Comer Children’s Hospital
Andy Shin, JD, MPH, American Hospital Association

Session Summary
This session will focus on linking the improvement and innovation sides of medicine within a health care system. The vehicle of change was a pilot project of the American Hospital Association’s AHA Innovation Academy, conducted at Comer Children’s Hospital.

This session will review the lessons learned from the AHA pilot, including how using this model can help enhance highly diverse teams and team dynamic and flow. With a mission of leveraging innovation as a potential cure for problems, important barriers and gaps were identified through the improvement project.

The presentation will be followed by a panel discussion featuring members of the AHA Innovation Academy pilot along with the SVP/CMO of AHA.

Learning Objectives

  • Learn why innovation and improvement in health systems are a good marriage
  • Hear the outcomes of the AHA Innovation Academy pilot
  • Explore the benefits of a highly diverse team
  • Find a spark for innovation within your department
Gathering User Feedback to Inform the Design and Usability of Websites and Mobile Apps

Chris Perry, American Board of Pediatrics
Mike Adams, American Board of Pediatrics

Session Summary
Even when communicators and marketers use best practices and usability standards to inform design decisions, it is impossible to guess how the navigation and layout will be interpreted by the end user—or is it? In this session, Mike Adams and Chris Perry will describe and demonstrate the many ways the American Board of Pediatrics gathers user input and feedback to inform, mold, and tweak content, navigation, and design decisions. Some of the tools discussed will be focus groups, online surveys, first-click tests, tree tests, and other usability tests and web tools.

This session will appeal especially to communicators, marketers, designers, developers, information architects, and others who participate in developing websites and/or mobile apps.

Learning Objectives

  • Understand several common and easy ways to gather user input and feedback
  • Be able to choose and select the right tool for each unique situation
  • Learn how to get buy-in from leadership
Having the Right Total Rewards Strategy Will Move Your Organization Forward [PDF]

Jennifer Hopp, SPHR, SHRM-CP, American Board of Anesthesiology

Session Summary
The total rewards mix is the driver for attracting, retaining, and engaging talent, but too often it is approached in piecemeal fashion and misunderstood and undervalued by employees. Organizations are encouraged to create an overarching total rewards strategy that evaluates the effectiveness of each reward element, reviewing how it aligns—or doesn’t—with business strategy and employee needs. An optimal mix of reward elements includes not only compensation and benefits, but also work/life balance, career development, and social recognition, among other offerings. This session will guide HR professionals and management in evaluating their current rewards system and determining if it is supporting the needs of their organization.

Learning Objectives

  • Learn what total rewards means and determine if your total rewards strategy is designed to influence the right behaviors to drive business results
  • Identify ways to ensure your employees understand the real value of their total rewards package
  • Hear what’s new and gaining traction in the millennial-driven workplace
Influencer Marketing: Leveraging Allies to Tell Your Board’s Story

Kim Van Brunt, American Board of Physical Medicine and Rehabilitation

Session Summary
This session will provide a brief introduction to influencer marketing, including engagement strategies, incentives, content generation, and metrics. Attendees will then learn about a recent influencer marketing campaign by the American Board of Physical Medicine and Rehabilitation promoting an examination currently under temporary criteria. The session will end with a brainstorming session on other applications of influencer marketing for the boards community, including examples in Board Certification and Maintenance of Certification.

Session Learning Objectives

  • Understand the basic premise of influencer marketing
  • Learn about a successful implementation of influencer marketing in the boards community
  • Leave with a plan to implement a small influencer marketing campaign at your own board
Innovations and Evolution: Modernizing Systems for Standardization

Kristin Gudenkauf, American Board of Radiology
Kevin Graves, American Board of Family Medicine
Cecily Marroquin, MA, American Board of Medical Genetics and Genomics

Session Summary
Initial Certification (IC) and Maintenance of Certification (MOC) processes are an integral part of all board processes, and a crucial milestone in professional development for residents. Despite the vital role of certification, however, some facets of the processes have not been revisited since conception, doing a disservice to residents, diplomates, and board staff. Developments in technology and board innovations offer opportunities for increasing administrative efficiency, clarifying and streamlining the process for residents, and improving the experience of initial certification for all.

The presenters in this session will walk through how different boards have implemented changes to their MOC and IC processes through standardization, technical and non-technical solutions, and organizational transformation to improve customer service, data integrity, and efficiency. They will focus on process evaluation, development of software and candidate portals, conversion to digital processes, and quick wins that can be utilized to streamline a process without the assistance of an IT development team.

This closed session is for Member Board staff only.

Learning Objectives

  • Recognize how vision, direction, and standardization can simplify processes, improve customer service, and increase organizational efficacy
  • Learn how communicating a consistent message, both internally and externally, can eliminate ambiguity and fortify standard processes
  • Understand how custom software can change the board-constituent relationship
  • Identify existing low- to no-cost solutions that can save time and simplify a process
Learning from Your Web Users: The Key to an Effective Online Presence [PDF]

Lamar Goodenough, Digital Wave Technologies, Inc.
Sandra Hodgson, American Board of Obstetrics and Gynecology

Session Summary
How can you make sure people can easily use your board website? Talk to them!

Keeping up with your constantly evolving web content can be a daunting task. And web users, particularly medical professionals, are becoming increasingly demanding and impatient with content or functionality that doesn’t make sense to them. In order to make life easier for both you and your users, it is critical to build your web experience around their needs and expectations. This means getting their input early and often, and letting it guide each site update.

This presentation will focus on how end-user discovery can become an integral part of your web process. By observing, testing, and talking to your users, and then responding to their insights, you can build your content architecture, design, and functionality in ways that make sense to them.

This approach has had a direct impact on protocols and standards developed for the new American Board of Obstetrics and Gynecology (ABOG) web presence currently under development. The presenters will provide a sneak peek at the work in process and show how its approaches to content strategy and design can be attributed to insights gained from user testing and feedback.

Learning Objectives

  • Learn how to ensure a truly user-focused website by incorporating user discovery and feedback throughout the development process
  • Gain insights into how user discovery led to specific content and design decisions in ABOG’s web redesign project
The Nitty Gritty of Time Management, Productivity, and Planning for Success

Primary Presenter
Cecily Marroquin, MA, American Board of Medical Genetics and Genomics

Additional Presenters
Nicole Kendall, American Board of Medical Specialties
Trevor Fleck, American Board of Medical Specialties

Session Summary
Keeping a project on track requires a wide breadth of skills, from listening and planning to organizing and managing follow-through. Often, Member Board staff of all levels of job responsibility and expertise are called upon to wear many hats, making project management skills invaluable. This session aims to distill the principles of effective project management down to practical strategies for everyday application.

This closed session is for Member Board staff only.

Learning Objectives

  • Distill the principles of effective project management down to practical strategies
  • Learn the time management practices to apply every day

▲ Go to Top

Quality Improvement Sessions

Accreditation as a Mechanism to Assess and Improve Quality [PDF]

Mary Lally, MS, CAE, Intersocietal Accreditation Commission

Session Summary
When the first Intersocietal Accreditation Commission (IAC) accreditation program was developed in 1991, the goal was to offer the field of noninvasive vascular technology an independent, objective evaluation in an educational, nonpunitive manner for process improvement. The purpose was to help facilities standardize their processes to provide better patient care and improved outcomes.

Today more than 14,000 facilities throughout the United States and Canada are accredited by the IAC in multiple diagnostic imaging and procedure-based modalities, including vascular testing, echocardiography, nuclear/PET, and many more. In addition to a comprehensive peer review of all clinical and operational aspects of the facility, each of the IAC accreditation programs places significant emphasis on continuous quality improvement. The quality metrics reviewed during the accreditation process include staff credentials, patient safety, equipment quality control, image quality, and report completeness and accuracy.

In 2012, the IAC began a research initiative to explore the impact of accreditation on patient care and to strengthen the accreditation process. Through several research studies, including many published in peer-reviewed journals, we now have evidence that IAC accreditation leads to awareness of safety, standardization of processes, adherence to guidelines, and, most importantly, improved patient care.

This session will review the published research findings from several studies that demonstrate the effect of accreditation on patient safety, including radiation exposure and safety practices, procedure documentation, image quality, study comprehensiveness, interpretation accuracy, reporting timeliness and completeness, and adherence to best practices.

Learning Objectives

  • Describe the mechanisms by which the IAC accreditation process assesses and improves quality
  • List the quality metrics evaluated during the process
  • Explain how facilities can use the IAC’s QI self-assessment tool to evaluate and improve their processes, imaging studies, and reports
  • Discuss research findings that demonstrate the positive effects of accreditation
Addressing the Opioid Epidemic through Quality Improvement

Teena Nelson, MHA, American Board of Medical Specialties
Sean Powell, MD, Kaiser Permanente

Session Summary
This town-hall session will provide attendees the opportunity to share, discuss, and ask questions about quality improvement (QI) work focused on the nation’s opioid epidemic. Session attendees can expect to hear about, discuss, and share the following:

  • Alignment of opioid work with national measures and guidelines
  • Organizational collaboration toward common opioid goals
  • Team-member and stakeholder involvement
  • Common metrics and interventions
  • Resource and tool utilization
  • Lessons, barriers, and best practices

Learning Objectives

  • Identify opioid QI project components that were successfully implemented
  • Recognize opioid QI project components that led to improvement
  • Understand how to structure and implement opioid QI projects
Continuous Quality Improvement [PDF]

Rahul Koranne, MD, MBA, FACP, Minnesota Hospital Association

Session Summary
A major focus of the Minnesota Hospital Association (MHA) over the past 17 years has been to collaboratively lead continuous quality and patient safety improvement. MHA identified this need and has made it a strategic priority since 2000 to work with our members to improve patient care in each community. This process of improvement begins with identifying quality or safety issues and then working to help improve the outcomes for our patients.

In this session, presenters will engage the audience in dialogue about the continuous quality improvement (CQI) process that MHA has used with leaders and staff from health care facilities, including how this model was applied to address the hot-topic issue of preventing workplace violence through the creation of strong partnerships between health care and law enforcement organizations.

MHA leaders will highlight the PDCAC (Plan-Do-Check-Adjust-Celebrate) model used to drive CQI through the creation of road maps that provide evidence-based recommendations for preventing and reducing harm. The Plan phase begins with identification of gaps, and problem solving by statewide expert committees results in current, evidence-based best practices. These recommendations are compiled into road maps that are pushed out through the MHA web-based portal for statewide use. In the Do phase, organizations begin implementing road maps and reporting data to MHA on the process, outcome, and adherence measures contained in the road maps. MHA provides ongoing education and assistance to members to help implement the road maps. In the Check phase, MHA analyzes process and outcome data from multiple sources. The data are brought back on a continuous basis to the statewide expert committee for discussion and action planning in the Adjust phase. To ensure CQI across the state, adjustments are continuously made to road maps, education, and other support areas in response to data and changing member needs. Finally, in the Celebrate phase, MHA identifies small and big wins so that staff, team, leaders, and organizations can be recognized publicly. This important step ensures that human beings fueling the CQI machinery are continuously nourished and remain engaged.

Learning Objectives

  • Gain familiarity with how the PDCAC model is used for CQI in Minnesota
  • Hear examples of how PDCAC has been used for CQI
  • Learn about the critical role physicians can play in CQI
Engagement of Primary Care Physicians in a National Ambulatory Quality Improvement Collaborative [PDF]

Selam Wubu, MPH, American College of Physicians
Doron Schneider, MD, FACP, Abington Hospital – Jefferson Health

Session Summary
This session will provide a framework for engaging primary care physicians in a national quality improvement (QI) collaborative based on the American College of Physicians (ACP) Quality Connect experience. ACP Quality Connect is a national network of physicians and health care teams who are dedicated to improving quality, care delivery, and outcomes. ACP has implemented Quality Connect programs across 22 states, focused on improving care in the areas of diabetes, chronic pain, atrial fibrillation, and adult immunizations. More than 350 QI champions and 6,000 participants have been engaged in Quality Connect programs since 2014.

ACP presenters will share results from an evaluation designed to identify which elements of the Quality Connect approach are most effective in engaging clinicians and supporting the implementation of their QI activities. The Quality Connect approach includes identifying QI champions who lead the implementation of the QI program in their practice. Technology solutions to support QI via registries and online practice management tools are offered in addition to educational resources that are tailored to address specific gaps identified in the self-assessment stage. Biweekly coaching calls with national expert faculty and periodic check-in calls with ACP staff are offered to support champions through the implementation phase.

The session will highlight the importance of professional development and recognition as part of a successful QI collaborative, including opportunities to publish results in peer-reviewed and ACP publications and to present at state and national ACP meetings.

Results from past programs will be shared to highlight the effectiveness of the Quality Connect programs, which have realized significant improvements in quality care and patient outcomes across diverse practice settings and patient populations. Examples of program outcomes include increased measurement of patients’ HbA1c levels at an internal medicine residency clinic in Virginia.

Learning Objectives

  • Understand key drivers for engaging primary care physicians in a QI collaborative
  • Identify opportunities to integrate professional development into QI learning collaboratives and initiatives
Engaging Physicians in Meaningful Practice Improvement [PDF]

Grant Lynde, MD, MBA, Emory University

Session Summary
Engaging physicians in meaningful quality and practice improvement projects can be a daunting task. How should we select, design, execute, and sustain a project? How can we identify meaningful quality improvement efforts that teams will find exciting? What are the best practices for achieving sustainable results?

We will discuss how to select projects that will be both strategically aligned with your organization and clinically relevant to physicians. We will contrast process and outcomes metrics and identify which will help your project achieve success. After the discussion of metric selection, we will identify data sources and methods of presenting the data to ensure provider engagement. The application of practice improvement and quality improvement tool sets, such as lean and Six Sigma, will be described. Finally, methods to sustain the improvement will be identified and compared.

Learning Objectives

  • Propose a quality or process improvement project that is both meaningful and engaging
  • Appraise various metrics and benchmarks used to identify success
  • Plan actions to ensure sustainability of the project’s achievements
Helping Clinicians Practice at the Top of Their License through Quality Improvement

Stephen Davidow, MBA-HCM, CPHQ, APR, PCPI

Session Summary
Clinicians continue to feel overwhelmed by the continuing pace of change, and they see a lack of human and financial resources as a major barrier to quality improvement, according to a survey of PCPI’s member organizations.

Although not a routine part of most clinicians’ training, quality and process improvement methods and tools have the power to transform care and reduce the sense of burden and of being overwhelmed that many clinicians feel. Attendees will learn how to determine roles and responsibilities that allow clinicians and other members of the care team to meet the needs of patients while gaining a greater sense of control for themselves and for their practices.

Learning Objectives

  • Describe how waste in health care operational and clinical activities leads to poor performance, poor outcomes, and dissatisfaction
  • Identify the common forms of waste
  • Recognize activities that improve patient care and those that do not
  • Determine which team members need to be involved in quality and process improvement, and when
How Mayo Clinic Improved the Quality Improvement Process Using Voice-of-the-Customer Feedback [PDF]

Peggy Paulson, MS, Mayo Clinic
Kathleen Olmstead, Mayo Clinic

Session Summary
At Mayo Clinic, we couldn’t call ourselves quality improvement experts without continually reviewing our own processes. Since 2009, we have been reviewing projects to submit for Maintenance of Certification and Performance Improvement Continuing Medical Education credits. Over those years the portfolio program has changed, and so have we. Our program has grown in size and scope, and yet we continue to have great work completed by teams across the institution. In order to keep up with the growth and changes to the program, we have to continually assess our processes and make improvements.

In this session, we will discuss how we improved our standardized project review process using voice-of-the-customer (VOC) feedback. We will review the following:

  • Process for gathering feedback
  • Methodology
  • Rationale
  • Result of the VOC exercises
  • Impact on project improvement ideas
  • Development of our new quality improvement application system

We will also discuss lessons learned during the analysis and how these lessons informed the improvements to the quality improvement process.

Attendees should leave this session with a sense of how to evaluate a program and make improvements. This session is intended for those whose organizations are a Portfolio Program sponsor or are looking to become one.

Learning Objectives

  • Identify standard components of a quality project review process
  • Learn how to apply standards to a process
  • Demonstrate awareness of the importance of VOC feedback
  • Select methods for collecting VOC feedback
How to Get Started in Health Care Quality Improvement [PDF]

Stephen Davidow, MBA-HCM, CPHQ, APR, PCPI

Session Summary
In this session, participants will learn tips on how to identify and prioritize improvement projects for greatest impact. Then they will see how to test improvement ideas using the Plan-Do-Study-Act (PDSA) methodology to ensure that their improvements are actually improvements and ready for full implementation.

Learning Objectives

  • Identify common types of health care improvement projects
  • Examine the use of three types of brainstorming techniques
  • Explore different prioritization methods and the importance of return on investment in final decisions
  • Define the specific components of the PDSA methodology
  • Describe how to move through successive, linked PDSA cycles starting with small tests and ultimately reaching full implementation for continuous quality improvement
Improve Documentation Quality and Efficiency Using Peer Coaching and Peer Chart Review [PDF]

Sue Turner, MBA, Massachusetts General Physicians Organization
David Y. Ting, MD, Massachusetts General Physicians Organization

Session Summary
Nationally, the administrative burden in health care is a leading factor in physician burnout and attrition. At Massachusetts General Hospital, a biannual physician survey of members of the Massachusetts General Physicians Organization (MGPO) indicated that documentation is a leading contributor to administrative burden and frustration. Quality assessment demonstrated that many clinical notes were bloated with redundant or non-value-added elements and were often late or occasionally missing key components.

This session will introduce the audience to the ChartCraft method, a set of robust guiding principles and best practices for clinical documentation, and will provide a framework for using ChartCraft principles to support high quality and efficient physician documentation. In addition, the session will review the MGPO Documentation Improvement Program in Primary Care, which utilized peer chart review, educational interventions, and peer coaching to help beleaguered physicians improve their documentation efficiency and effectiveness.

This pilot program had a demonstrated impact on physicians’ perceptions of burden. The participants completed surveys before and after the program to report their level of burden with documentation, and the results showed some self-reported improvements in documentation efficiency.
The online peer review program and the MGPO documentation improvement project have provided greater insight into areas of documentation risk, and the outcomes will translate into better cross-coverage of patients, especially during high-risk transitions. Also, coaching participants to focus on the “most important” part of the note (the Assessment and Plan) at the top, and transitioning to problem-based charting, will ensure more complete and useful documentation.

Finally, through this program, the MGPO engaged busy primary care providers by aligning their work with additional requirements, specifically Continuing Medical Education and Maintenance of Certification Part IV credits. This alignment of multiple requirements helped to further reduce the administrative burden.

Learning Objectives

  • Describe the benefits of peer coaching to improve physicians’ efficiency with using the EHR
  • Employ the ChartCraft methodology to assess provider documentation
  • Utilize peer review to align requirements and incentives for physicians
Lessons from the Kaiser Permanente Maintenance of Certification Portfolio

Sean Powell, MD, Kaiser Permanente
Jo Ann Townson, Kaiser Permanente

Session Summary
The Kaiser Permanente Federation has a diverse portfolio of Maintenance of Certification (MOC) projects. Projects are offered for physicians in all 20 participating board specialties. In the last five years, more than 4,500 physicians have completed projects, accounting for 20 percent of the projects offered by affiliate organizations. Projects are focused on improving the care delivered to our 12.1 million members in the United States.

In this session, we will offer our portfolio as a model for smaller organizations with less extensive portfolios. We will discuss the foundations of our MOC program. We will describe how our portfolio has grown over the years and how projects have become more rigorous (incorporating advanced metrics) and more complicated (employing at least two Plan-Do-Study-Act cycles). Finally, we will discuss how projects align with local, regional, and national quality improvement goals, which allows the MOC program to be an important part of the quality improvement efforts of the organization as a whole.

With such a great number and diversity of projects that improve the health of our members, the Kaiser Permanente Federation is in an excellent position to act as a model for affiliate organizations on how to manage their portfolio. We believe that our experience may help other health care organizations to improve their MOC offerings for their physicians and improve the care delivered to their patients.

Learning Objectives

  • Design and implement MOC projects with an aim of improving patient care
  • Use appropriate metrics in the design and implementation of MOC projects
MOC-PEAKS: A Participant Survey Informs Improvement of Part IV Programs and Projects

Joel Tieder, MD, MPH, Seattle Children’s Hospital

Session Summary
Maintenance of Certification–Practice, Engagement, Attitude, and Knowledge Survey (MOC-PEAKS) was developed by Seattle Children’s Hospital in partnership with Dr. Jan Carline of the University of Washington. The goal of the survey is to systematically evaluate MOC activities to inform project and program improvement.

The survey is designed to be taken before beginning and upon completion of an MOC project, with questions that solicit physicians’ perceptions of (1) engagement and attitudes about quality improvement (QI) and MOC, (2) their ability to apply QI methods, and (3) the impact of the activity/project on patient care.

Seattle Children’s Hospital implemented the survey in 2015 (see Hospital Pediatrics, PubMed PMID 29046431) and is currently piloting the survey at six ABMS Multi-Specialty Portfolio Programs (MSPPs). We hope to gain user experience to develop the survey further for use in various settings to assist other MSPPs in providing data to inform improvement of their programs, and ultimately to understand the qualities of program and project that lead to engaged participants who are learning QI methods and improving the care of their patients.

In this session, we will cover the development of the survey, describe how it is currently used at Seattle Children’s Hospital to improve the program and projects, and share early findings from the national pilot. We will also share the survey and enroll additional sites for the pilot.

Learning Objectives

  • Use systematic evaluation of the participant experience to improve Part IV MOC projects and portfolio programs
  • Overcome barriers to systematic evaluation
Paving the Path Together: Using Collaboration to Unlock the Medical Education Continuum [PDF]

Kristin Gudenkauf, American Board of Radiology
Cecily Marroquin, MA, American Board of Medical Genetics and Genomics
Patrick Dolan, MD, Comer Children’s Hospital

Session Summary
A physician’s medical education continuum extends from the undergraduate experience through medical school, residency training programs, fellowships, and ultimately the transition to ongoing participation in continuing certification. Throughout each stage, physicians encounter various organizations (e.g., medical specialty societies, health care facilities, regulatory organizations) that influence their perception of quality care and professionalism. This session seeks to explore the opportunity for collaboration across health care institutions and organizations to both understand and enrich that influence throughout the medical education continuum.

To open this discussion, this session will highlight one university’s initiative that launched a nationwide collaborative, using the umbrella of the American Academy of Pediatrics’ REVISE project to unite residents, fellows, and attendings under one common goal. The discussion will highlight lessons learned and opportunities discovered regarding development and maintenance of a collaborative. The discussion will then focus on unlocking potential areas for collaboration between the various organizations on the medical education continuum (e.g., Accreditation Council for Graduate Medical Education, programs, Member Boards) to better understand how we can enhance the physician experience throughout the continuum.

Learning Objectives

  • Understand perspectives from the distinct phases of the medical education continuum, including physicians at all levels
  • Identify lessons learned in coordinating a large collaborative to ensure meaningful impact for the participants
  • Discuss opportunities and strategies for engaging all parties throughout the continuum
Portfolio Program Sponsors’ Affiliations with Other Organizations for Practice Improvement [PDF]

R. Van Harrison, PhD, University of Michigan

Session Summary
The ABMS Multi-Specialty Portfolio Program allows program sponsors to approve quality improvement (QI) efforts for Maintenance of Certification (MOC) Part IV (Improvement in Medical Practice) to improve care in affiliated organizations. The program’s policies outline specific structural requirements that must be in place. Within this framework, Portfolio Program sponsors must address a number of practical and operational decisions on whether to establish an affiliation for MOC Part IV and how to implement and manage affiliation arrangements. The purpose of this presentation and group discussion is to share experiences in implementing and managing affiliation arrangements and identify successful practices with Portfolio Program sponsors interested in initiating or improving affiliation arrangements.

In summary, the policies of the ABMS program for organizational affiliations for MOC IV are as follows:

  • A formal agreement recognizing commitment to QI/performance improvement and support of MOC
  • Identified oversight arrangements and process for prioritizing QI
  • Meaningful participation by individuals
  • Adjudication of disputes by the program sponsor

Beyond this framework, operational issues to consider include the following:

  • Value/priority of affiliation arrangements
  • Nature of the affiliation
  • Formal agreements
  • Oversight relationship
  • Management of ongoing relationships

The presenter will discuss and provide examples illustrating the above issues and how they were addressed at one institution; discussion groups will help participants initiate and improve successful affiliation arrangements as Portfolio Program sponsors across a variety of organizations and contexts.

Learning Objectives

  • Discuss with institutional leaders whether affiliation relationships for MOC Part IV should be pursued given the institution’s priorities and the effort involved in establishing and maintaining these relationships
  • Identify several aspects of organizations and their relationships that indicate that an affiliation for MOC Part IV will be easily established and managed or will require special attention to function well
The American College of Cardiology Journey into Quality and Practice Improvement: Challenges, Lessons Learned, and Strategies to Engage Stakeholders and Improve Outcomes

Maria Ortiz, CHCP, American College of Cardiology
Akua Asare-Danso, MD, American College of Cardiology

Session Summary
Bleeding after percutaneous coronary intervention (PCI) is a common, often preventable complication that is associated with increased short-term and long-term risks of morbidity and mortality and higher hospital costs. American College of Cardiology (ACC) staff will recount how a solution to a specific problem was created to address patient safety in the inpatient setting using the ACC National Cardiovascular Data Registry®. The goals were to show how tools from the ACC Post-PCI Bleeding Risk Checklist could decrease bleeding rates at various facilities; learn about the gaps that were discovered along the way; and be able to share innovative ideas on how the checklist can be integrated into the development, planning, and implementation of educational offerings across an organization.

The checklist was created in response to a grant proposal through the American Board of Internal Medicine Foundation to develop a patient safety initiative.

ACC-developed tools are clinically useful, and hospitals may benefit from developing their own quality improvement infrastructure to help facilitate adoption of quality initiatives. A hospital system that has an existing quality team and has a robust documentation process seems to be more successful in implementing the tools provided, compared with a hospital system that does not have the resources to implement the tool in their facility.

With this in mind, ACC could offer specialized tool kits in the future for hospitals that already have an established quality team in place, focusing more on working in partnership with the hospitals as they continue their efforts rather than taking a lead role in teaching the hospitals how to develop these structures. Potentially, for hospital systems with limited resources, ACC could package the tools to differentiate among levels of hospital familiarity with quality improvement projects and tailor the solutions to their needs in order for the hospitals to successfully implement specific tools that can improve patient care.

Learning Objectives

  • Explore how the ACC Quality Improvement Solutions team developed the ACC Post-PCI Bleeding Risk Pilot Study
  • Identify techniques used to engage participant facilities and how those facilities approached the implementation of the study
  • Highlight the challenges, lessons learned, and recommendations identified
Systems-Based Improvements in Laboratory Stewardship: Improving Quality, Care While Lowering Costs

Gary Procop, MD, MS, Cleveland Clinic

Session Summary
The Laboratory Stewardship Committee at Cleveland Clinic is a multidisciplinary team of clinicians, pathologists, administrators, and other caregivers. This team is committed to optimizing test utilization to promote best practices and ensure optimal patient care. The goals include working to decrease unnecessary phlebotomy; reduction of unnecessary duplicate orders, including “daily” orders; the conscientious use of molecular testing; the promotion of best practices; the development of evidence-based guidelines for optimal testing; and the reduction of health care costs through the thoughtful and judicious use of health care resources. Part IV (Improvement in Medical Practice) credit has been given for projects in this portfolio.

Nine ongoing interventions have become integrated into the care-delivery culture at Cleveland Clinic. The first intervention that was implemented blocks or notifies providers of duplicate tests that are likely unnecessary, which led to a direct impact on quality. Other interventions notify providers of expensive tests, restrict genetic tests to certain physicians, and provide guidance for the use of complex molecular genetic testing.

Since the beginning of this program in 2011, interventions have resulted in the avoidance of 78,555 unnecessary tests and saved $4,218,337.

This session will demonstrate the value of establishing and promoting a multidisciplinary approach to laboratory stewardship for systems-based improvements in medical practice. Guidance will be given regarding how to establish and manage such a committee, select projects, measure the impact of interventions, and report outcomes. A laboratory stewardship committee provides an excellent vehicle to promote and implement evidence-based practices as health care migrates from volume-based, transactional practices to value-based care. Opportunities wherein an effort can improve patient care, patient satisfaction, and quality while simultaneously decreasing health care costs are rare.

Learning Objectives

  • Identify the benefits of a multidisciplinary approach to laboratory stewardship
  • List interventions that can improve patient care and satisfaction while lowering cost
  • Describe how laboratory stewardship projects may help individuals fulfill Part IV Maintenance of Certification requirements in a systems-based manner
ASA's Innovative Initiatives to Address Quality and Performance Improvement [PDF]

Michelle Michelotti, MHA, American Society of Anesthesiologists
Roseanne Fischoff, MPP, American Society of Anesthesiologists

Session Summary
As health care reform continues to gain momentum nationwide, providers across the continuum of care are facing increasing pressure to demonstrate that they deliver value-based care—namely cost-effective, high-quality care and exceptional patient experiences. However, making the transition to value-based care requires training and coaching in areas such as change management, data analysis, care delivery, and leadership. In today’s environment, this training needs to be more innovative and better integrated, taking into account the needs of the entire care team. The Perioperative Surgical Home (PSH) Learning Collaborative and SimSTAT programs of the American Society of Anesthesiologists (ASA) fulfill these needs.

In 2014, ASA partnered with Premier, Inc., a leading health care improvement company, to establish a national learning collaborative to develop, pilot, and evaluate the PSH model—a patient-centric, team-based system of coordinated care that guides patients through the entire surgical experience, from the decision to undergo surgery to discharge and beyond.

The presentation will provide information about the PSH model and the PSH Portfolio, a comprehensive suite of services, resources, and tools that support team-based care through education, consultation, engagement and economics. We will focus on the Learning Collaborative and why it has been so successful in helping practices transition to coordinated care.

We will also discuss the SimSTAT program. Through this screen-based simulation platform, we will be able to leverage a modality that can be delivered on demand, continuously pushing learners to learn from their mistakes in the virtual environment, allowing for deliberate, repetitive practice until they have moved from competency to mastery.

Learning Objectives

  • Identify new and innovative modalities to simultaneously meet the educational and practice improvement needs of clinicians
  • Describe a patient-centric, team-based model of care created to help meet the demands of a rapidly approaching health care paradigm that will emphasize gratified providers, improvements in population health, reduced care costs, and satisfied patients
  • Devise possible strategies to allow providers to demonstrate that they deliver cost-effective, high-quality care and exceptional patient experiences