It’s Time to Change the MOC Process

Image of woman taking notes

Everyone knows the problems with the maintenance of certification testing. The tests, no matter what specialty, ask arcane details of obscure issues in medicine, information that easily could be – and is — looked up. It doesn’t test most physicians’ stock-in-trade: decision-making. The score has little to do with performance in practice.

However, to totally forgo ongoing assessment and certification turns the quality control of medical practice over to payers, including the government. In addition to it affecting our income, do we really want our abilities to be represented by how well we check off boxes in the electronic health record?

The simple issue is this: Testing recall of information has little correlation with actual practice, especially in this environment in which information changes so quickly. Ongoing physician assessment must assure that board-certified physicians can make decisions based on the best, currently-available evidence that should inform their practice.

The process of making decisions goes way beyond simply recalling previously-learned information. Daniel Kahneman, in his popular book Thinking Fast and Slow, describes two distinct ways that humans make decisions.

In System 1 Thinking, we rely mainly on our memory and recent experience to make decisions almost automatically without deliberate, conscious thought. For example, most primary care physicians, when faced with treating a cat bite, will immediately recall the most common organisms likely to be causing the infection and the appropriate first-line treatment.

The other type of thinking, System 2 Thinking, is a slow, deliberate approach, often accompanied by a realization that new information is needed to solve the problem. If a patient is allergic to the first-line treatment of a cat bite, many physicians will need to find, using information tools, appropriate second-line options. System 2 thinking requires two steps: 1) recognition that we don’t know the right answer; and, 2) using information sources to find the information quickly and accurately.

System 2 Thinking requires physicians to manage their “information inventory” to determine whether they can make decisions based on their current knowledge or whether they need to use high quality information tools to find the appropriate information at the time the information is needed.

Current testing, even that which is cased-based, evaluates recall of information and previously-developed “illness scripts.” While this knowledge is important, it is also important to test whether physicians two additional components of decision-making: 1) knowing when to question one’s existing knowledge; and 2) where to obtain new information when a patient case requires one to go “off script.”

Ongoing certification processes of physicians should focus more on clinical decision-making ability, assessing when current knowledge may not be sufficient or when it is not based on the best current evidence. It should also test the ability to use commonly-available, evidence-based tools to quickly find the answers to questions. In other words, tests should reflect the real world of medicine.

Some boards are considering “open book” assessments.  This is just a start. As in the real world of clinical practice, there has to be an element of time involved in the assessment. Testing in the future should be more closely aligned with clinical practice, in which physicians ask themselves: 1) Am I confident enough in my decision to go forward; and, 2) If not, where can I quickly find the best evidence? In this way, the new exam will test examinees’ ability and practice of keeping up with changes in medicine, as demonstrated by their ability to determine when their existing knowledge might not be sufficient, triggering their use of information tools to determine or confirm their initial decision, and the ability to use tools to find the best currently-available evidence.

These changes will do two things. First, it will reflect actual practice, and will feel like what most physicians do day to day (or should be doing). Second, it will assess lifelong learning skills. In this way it should be more predictive of performance in practices in which information is constantly evolving and comes from a wide variety of sources.

This approach will not be easy – the “right” answer on a test question may change quickly as evidence evolves. Still, this approach, rather than asking physicians to commit to their short-term memory various facts they no longer use, while not giving them credit for the decision-making and information retrieval skills they’ve developed in their practice, will give all of us a more confidence in the process.

3 comments on “It’s Time to Change the MOC Process

  1. Charlotte A. Paolini, D.O. (Associate Professor of Family Medicine and Geriatrics, CUSOM) on

    “Tests should reflect the real world of medicine.” What a concept! Of course! Testing for a bunch of memorized facts, that may or may not be relevant to one’s work, is ludicrous. In my mind, it devalues the well-developed and well-structured clinical reasoning process of the clinician. Who practices medicine sitting in front of a monitor screen without ANY access to resources? Even medical students learn early on to keep their electronic devices close at hand in order to check their facts or to be sure they are dosing that medication correctly. It is embarrassing to me that the finely tuned reasoning process of the physician is not even considered in today’s credentialing exams. Thank you, Dr. Shaughnessy, for your very insightful comments regarding this frustrating topic. I am hoping this will stir the winds of change for the development of more appropriate and relevant MOC testing.

    Reply
  2. Jose R Nino on

    I feel for my physician colleagues, and for the public, both groups might be being mislead by an outdated and unreliable system of keeping doctors and patients safe.

    Reply
  3. Allen Shaughnessy on

    While other aspects of the maintenance of certification attempt to document physicians’ clinical reasoning, I agree that the current exam does not do so. The trick is to come up with a way to measure reasoning skills and, as I mentioned in the blog, the knowledge and skill to question one’s current knowledge and check the answer.

    Reply

Leave a Reply

Your email address will not be published. Required fields are marked *