Learn more about the SCT

Main purpose of the SCT

In every day practice, health professionals use their knowledge; experience and clinical reasoning, to resolve the various problems they meet.

Application of rules and principles enable to resolve «well defined» problems: data are accurate, goal is easily reachable, and solution is available. In this context, the problem is solved with technical competence.

However, in daily practice, some important data about the problem are often missing, or ambiguous. In this case, the problem is «ill defined»: goal is not always clear, data could be interpreted in different ways and many solutions are likely to be proposed. This type of problem requires the capacity to reason in an uncertainty context. This ability constitutes the professional competence.

Traditional written tools for assessing clinical reasoning, such as rich-context, multiple-choice questions, properly and reliably test the ability of students to apply well-known solutions to well defined problems. But comprehensive assessment of clinical reasoning should include tools other than those assessing well defined problems, tools which measure the ability to rationally solve ill defined problems. Some evaluation tools, such as oral examinations, can assess this aspect of clinical competence but they have limitations such as the difficulty of standardization, objectivity of scoring, or practicability for large groups of examinees.The Script Concordance Test (TCS) was developed with the purpose of expanding the material assessed in clinical reasoning to include ill-defined problems. It is designed to be added to existing tools, not substituting for them.

For instance, this is an example of a rich-context MCQ (source: G. Norman):

A young man of 20 years old has been stabbed in the arm with a knife. The dorsal face of the forearm and the hand between the thumb and the index finger are insensitive. The wrist extensors are paralyzed and he cannot extend his thumb between the metacarpophalangian and interphalangian articulations. Which is the damaged nerve?

  1. Median nerve
  2. Radial nerve
  3. Ulnar nerve
  4. Posterior interosseous nerve
  5. Posterior brachial cutaneous nerve

In a rich-context MCQ, we evaluate the result of the reasoning process and there is only one correct answer.

In a SCT, we evaluate the reasoning itself, not only its result. Scoring involves comparing answers provided by examinees with those of a reference panel composed of physicians with experience in the field being assessed. Panel members are asked to complete the test individually, and their answers are used to develop the scoring key. Credits for each question are derived from the answers given by the reference panel. Therefore, more than one answer can receive points.

Principles behind the SCT

SCT are based on the hypothetico-deductive theory of the clinical reasoning and on the scripts theory

The script theory stems from cognitive psychology. According to theory, scripts are made up of links between illnesses, clinical features and management options. Health professionals progress toward solutions to clinical problems with hypotheses (or management options) and their related knowledge networks (scripts) in mind.

Scripts appear when the student starts to be confronted to real clinical tasks. They are then improved, perfected and refined throughout the professional life. Scripts are used to understand and to treat actively the medical information.

Facing a clinical situation, clinicians quickly manage hypothesis that they later try to confirm or to invalidate by means of clinical signs (imaging study, physical sign, laboratory or complementary test result) that reinforce or eliminate them. This is how they progress towards the resolution of the clinical problems.

The clinical reasoning is therefore made of a multitude of clinical judgements that can be measured and then compared, objectively, with those of an expert panel. In this respect, the principle behind the SCT is to compare the scripts of examinees to those of experienced clinicians using a series of clinical tasks set in specific contexts. The SCT probes the quality of examinees’s scripts:

  • For each item of the test, the examinee receives a credit that depends on the number of members of the panel that chose the same answer.
  • This approach considers the variability of the answers of the expert panel to calculate examinees’ scores. The aggregate scoring method is the most commonly used method.

The construction of an SCT first requires two or three persons with good knowledge of the field to be tested. Clinical situations that are representative of the field, and that need not be unusual or rare, are elicited through a semi-structured interview. Diagnostic hypothesis, investigative strategies, and treatment options are specified for each situation. Short clinical vignettes, each followed by a series of test questions, make up the SCT. Each case is described in a short clinical vignette, followed by three parts. The first part (« If you were thinking of ») contains a diagnosis, investigation, or treatment that is relevant to the clinical vignette. The second part (« And then you find ») presents information, like a physical sign, a pre-existing condition, an imaging study, or a laboratory test result that may have an effect on the first part. The third part (« This hypothesis becomes ») is a five-point Likert scale that the examinee uses to indicate what effect this information (part 2) has on the proposed diagnosis, investigation, or treatment (part 1). An example of a diagnostic test is provided in Figure 1.

component-300x211

The construction of items follows the key features approach, meaning that the choice of questions is focused on the elements that are most critical or more likely to lead to error in solving a clinical problem. Each question is built to foster reflection before answering and each question is assumed to be independent of the others. This is clearly specified in the test instructions. However, questions that are related to the same vignette might not be independent. This aspect has never been studied in previous studies relating to the use of SCT.

To prevent a rating bias on examinees, the questions across various cases should be constructed to spread answers among all anchors on the Likert scale. As with other assessment tools, the number of questions that are necessary depends on the purpose of the test. The higher the stakes of the examination, the more questions will be needed in order to reach a reliability alpha coefficient of 0.80 or higher. Published studies using the SCT, and using questions as the unit of measurement, have shown fairly good reliability, with alpha coefficients ranging from 0.54 to 0.82. From generalizability D studies, it can be expected that 50 to 60 questions are needed for an SCT to reach an alpha reliability coefficient of 0.8.

SCT uses in different domains

The TCS is used in the following contexts:

In medicine, at the predoctoral level (2nd cycle studies in France):

In the speciality formation

In the continuing formation for competence maintenance and development: