How Do You Keep Your Question Banks Current?

Med-Challenger is, in fact, both a content company and technology company. Ongoing development and quality maintenance is always underway.  Effectively balancing maintenance and update concerns is one of the harshest realities for medical education providers.  Nothing is ever “finished” – things keep evolving.

If you had to pick one component as most the important and challenging overall, it’s the ongoing management of our clinical content.  No matter how functional, useful, or relied upon our technical platform is, the educational content it delivers must be of quality and kept up-to-date and relevant to current practice guidelines at all times.  

To stay in step with the ongoing changes in medicine, guidelines, exam specifications, and MOC requirements, Med-Challenger has several editorial processes in action at any given time.  These editorial changes are prioritized by clinical importance, education impact, and the clinical knowledge depth required of editors. 

We essentially have
two levels of editorial work happening at all times…

Core Clinical Updates

Core clinical content updates that ensure clinical information is correct, relevant, based on current practice guidelines - the hardcore practice content and clinician-based editorial work aimed at clinical accuracy and today’s practice requirements.  Editorial clinical quality processes include:

• Top-to-bottom clinical review of all content by Editor-in-Chief and board-certified, practicing clinicians per specialty/discipline, once every two years at least
  (better than ACCME recommendations)
• Daily topic updates and new content development
• User-reported errata and corrections – we invite users to report any content
  for editorial review.  Revised content updates live materials automatically.

Supportive Editorial Updates 

Supportive editorial updates are cases where information does not impact clinical accuracy or validity, but where updates are desired as material may present as outdated when it is not. This typically relates to textbook edition-based content references.  Editorial support processes include:

• Product front matter updates (contributor listings, etc)
• Reference citations where clinical review has not found erata, but there
  is a new “edition” of a referenced publication.

That last item is the most frustrating – references, especially where no clinical information change has occurred or where the citation reflects a textbook edition.  

Our clinical authors focus on the body, shape, and quality of the clinical content as it pertains to best current practice.  Clinical editors add and update references where needed to support clinical practice updates as well, but - if a reference update is not critical, or deemed a research effort based on a publisher cycle, not medicine, those changes occur in another process.

Thus, when quickly browsing content, it’s a common misconception to assume that if a reference is over 2 years old, the content is.  Not so.  Our content is updated constantly. 

A Word on Reference Dates

References can be longstanding.  Some things don’t change that often.  Others do.  If references listed under an author-revised question have not changed clinically – for example, the same information is just on a different page number in the new edition - there’s no reason to urgently change the still-valid reference but for perception (and maximizing the impact of clinical revision time.)

Both of editorial processes require time, costs, and labor.  They involve author relations management and assorted “human problems” related to procuring timely revisions of certain information and getting those changes into live products.
 

From one of our Chief Emergency Medicine Editor, Dr. Andrea Eberly, MD, FAAEM:

"Sometimes, very important medical topics are researched with great intensity and then the research dries up, but the relevance of the topic remains. In that case, the best references are going to be a bit older, and should remain during successive updates.

We do try to keep important teaching points in our Q&A, and if we were to delete older references that reflect those points, we would need to alter the question content accordingly, which could potentially result in the deletion of the important points in the question.

If an older reference supports the key point, then it needs to remain one of the references. Newer references can be added to questions as secondary references, while the older key reference is left in place.

Topic reviews are conducted on a continual basis, and we try to ensure that an update of each entire topic is conducted  every 2 years or sooner. During these updates, the content is reviewed against the most recent edition of a clinical textbook resource, as well as current clinical practice guidelines, and the content is edited accordingly.

As a question ages, the content will be completely replaced, and our authors will then add a new key reference as they subject the question to a more fundamental rewrite."

Our Commitment to Content Quality

We invest in content quality.  In 2023 alone, nearly 1/2 of revenues went BACK directly into product development. Out of that, 1/2 went directly to content quality, namely to our team of clinical authors, contributors, editors, and proofers.  It’s the largest single expense we have – and it will remain so.

Thanks to our editorial and QA workflow, we are able to get new information live for consumption up to 5x faster to supply updates and errata corrections than other online CME content provider.  If your program relies on licensed textbook releases…we’re 20x faster to
market. 

Our editorial process allows updates to live materials seamlessly, in as little as 48 hours from the time the issue is initially reported. 

Of interest, roughly 60% of reported errors turn out to be erroneous claims - a simple “misread” of clinically correct content.  Of the remainder of content reports, 75% are “minor,” non-clinical issues or suggested additional clarifications.  The clinical “hot” items receive peer-review treatment immediately. 

In the end, we do everything possible with programs in place to ensure our clinical materials are up-to-date as they should be.