Validity

This MedEd Model covers validity with assistance from @Midwest_MedPeds

This #MedEd Model is on validity. Validity describes how justifiable the decisions are that result from a test, whether we are referring to an exam, a rating scale, or a clinical diagnostic.

We’ll focus on describing current validity discourses, then list the components of a validity argument, and wrap up by applying validity to #MedEd.

Let’s review 3 validity discourses in #MedEd:

  1. Validity is a property of a given test

  2. Validity is an argument supporting a proposed interpretation & use of assessment data

  3. Validity is a social imperative that considers individual & societal consequences of assessment

Let’s dive deeper into the second discourse: validity as an argument. First, we need to understand what we mean by “argument.”

The core components of an argument have been described in the argumentation literature by Toulmin in a 3-part framework:

  1. Claim

  2. Warrant

  3. Grounds

Now, let’s consider what it would mean for validity to be presented as an argument.

Check out the image to see how the claim, warrant, and grounds connect to validity in #MedEd.

•

•

•

In other words, whenever we use an assessment to make a decision in #MedEd, we need to 1) articulate the inferences that justify using that assessment for the proposed decision and 2) generate evidence supporting those inferences.

•

This matters, because if we use an assessment to make a decision *without* ensuring we have evidence supporting our inferences, we could end up making misguided (or even harmful) decisions in #MedEd.

•

•

•

Making a plan to gather evidence for a validity argument can be daunting. Luckily, Kane’s Framework can help us!

Kane’s framework identifies four inferences in using a test for decision-making:

  • Scoring

  • Generalization

  • Extrapolation

  • Implications

By applying Kane’s Framework, we can uncover the weakest assumptions in a validity argument, and then gather evidence to explicitly/robustly address those assumptions.

If the evidence is refuting, we revise our plan.

What constitutes evidence for each of the four inferences?

We’re glad you asked! Check out the table for some ideas/examples of quant & qual validity evidence!

Now let’s practice applying the concept of validity to #MedEd.

Check out the prompt below and share your answers by replying to this tweet. We can’t wait to hear your thoughts!

We’re super lucky for this #MEM to have support and a comment from


@Midwest_MedPeds

!

Thank you for suggesting this topic (http://mededmodels.com/submit-ideas) & contributing, Dr. Kinnear!

Here are the take homes.

What did you learn from this?

Check out & amplify the original Tweetorial:
Validity