What Standardized Tests Can't Tell Us: The Case for Qualitative Assessment

Formal or static standardized measures in speech-language pathology are readily accepted and widely used by school-based speech-language pathologists (SLPs). And for good reason! With explicit directions, quick scoring, as well as clear cut-offs for “average” and “outside of average” performance or skills, these measures fill a real need for the school SLP.

‍

Unfortunately, informal or qualitative assessments do not receive the same attention. Often viewed as the sidekick rather than the hero of an evaluation, qualitative assessments are generally used less frequently or, in some cases, not at all.

‍

In reality, qualitative assessments are just as valuable and accessible as standardized measures. Let’s address some of the barriers potentially preventing school SLPs from embracing these assessments.

‍

Misunderstandings about evaluation requirements

‍

School SLPs may be prioritizing standardized measures over qualitative assessments simply because they believe they are required. However, this is a myth.

‍

The Individuals with Disabilities Act (IDEA) requires that a “variety of assessment tools and strategies” be used to gather relevant functional, developmental, and academic information to determine eligibility for special education. However, it does not specify that a standardized measure must be used. Assessment tools and strategies may include standardized tests, criterion-based assessments, dynamic assessment, interviews, observations, and so on. Additionally, IDEA discourages overreliance on any single measure or assessment as the determinant for the appropriate educational program for a child.

‍

In short, school SLPs are free to choose the battery of diagnostic tools and strategies that best meets the needs of each individual student. SLPs are not legally required to include or prioritize standardized measures over qualitative measures.

‍

Misconceptions that standardized assessments are inherently “better”

‍

Per IDEA, eligibility for special education services involves answering the following three questions (ASHA):

1. Is there a disability?
2. If so, is there an adverse effect on educational performance resulting from the disability?
3. If so, are specially designed instruction and/or related services and supports needed to help the student make progress in the general education curriculum?

‍

When used appropriately, standardized tests can answer, at best, the first question listed above. Standardized tests can identify if an individual’s communication skills in specific areas are significantly different than others in their peer group; however, it cannot provide information on how that difference relates to the individual’s real-life experiences or affects their educational performance.

‍

Simply put, the remaining two questions regarding adverse effects and specially designed instruction cannot be answered by most standardized tests. Qualitative measures, in contrast, can answer all three questions above because they are designed to elicit descriptive information about real-world settings, providing a better representation of the student’s strengths and needs in real-life situations. For students who stutter, qualitative assessments also demonstrate that we value the individual’s experience and the impact of their stutter in everyday life more than how their stutter appears on the outside.

‍

Concerns that qualitative assessments are time consuming

‍

We recognize that incredible workloads are placed on school SLPs; which means their time and energy must be allocated mindfully. Here are some ideas for streamlining qualitative measures to promote time efficiency.

‍

⏰ Create and reuse Google forms (or similar) to collect data from parents and teachers. This practice may be more time effective than scheduling and conducting meetings or phone calls. Additionally, for students who stutter, there are several ready-made rating scales, checklists, and questionnaires that may be used.

‍

⏰ Record speech/language samples so that they can be reused to gather information about two or more areas of communication. For example, an SLP could record an interview with a student about their experiences as a person who stutters. That same speech sample could be used to measure any articulation errors that may be present in connected speech. In another situation, an SLP could record a child describing a wordless picture book. That recording could be analyzed for language (vocabulary, morphology, syntax, narrative structure, etc.) and also be used to measure observable characteristics of the child’s stuttering. We will address the limitations of measuring observable stuttering behaviors later in this article.

‍

⏰ If using standardized assessments, SLPs should select subtests that directly relate to the parent’s, teacher’s, and/or student’s concerns. In other words, it may not be necessary to administer an entire battery of an assessment, which can be time consuming.

‍

⏰ Create and reuse templates to communicate the results of frequently used qualitative assessments, such as parent and teacher questionnaires. 

‍

Uncertainty in what to look for in qualitative assessments

‍

School SLPs may be open to incorporating more qualitative assessments into their practice but feel uncertain in how to proceed. Below are suggested qualitative assessments one could use with a student who stutters or is suspected of having a communication difference, and what the SLP may “look for” in determining eligibility.

‍

📋 Classroom observation

• Observe the student’s verbal participation across settings and speaking opportunities. Is their verbal participation similar, more than, or less than their peers?
• Are outward signs of struggle or avoidance present?
  
  • Secondary Behaviors (e.g., fist clenching, head nodding, blinking, etc.)
  • Avoidance (e.g., word switching, not raising their hand, saying “I don’t know” frequently, requesting to go to the bathroom frequently, etc.)
• Are there environmental, curricular, or instructional barriers present that may be negatively impacting the child who stutters? This could range from classroom speaking demands to time pressure to participation expectations.

📋 Interview with the student

• Tell me about what talking is like for you…
  
  • at home and at school
  • one-on-one, in small and large groups
  • in person, during video calls, and on the phone 
• Which individuals are easy to talk to? Which are hard to talk to? 
• Which situations are easier for you? Which are more challenging?
• Why do you think that is?
• What would make talking and participating in class easier for you?

📋 Teacher and parent questionnaires and/or interviews 

• Spero Stuttering, Inc. offers a helpful resource to gauge adverse effects: Teacher Questionnaire Nonacademic Adverse Effects of Speech Impairment on Educational Performance 
• It’s important that parent interviews address the communication environment at home, family communication styles, and how stuttering affects the child's participation in activities, social interactions, and overall quality of life. This includes emotional responses and coping strategies observed in the child and in the parents themselves.

📋 Gather diagnostic information using authentic materials and in settings that make sense for the student.

• The SLP may choose to use materials from the curriculum rather than a standardized test. For example, the student may be asked to read a passage from their English Language Arts textbook rather than a passage from the Stuttering Severity Instrument (SSI-4).
• The SLP may also choose to collect a communication sample in the regular education setting during authentic interactions, rather than in the therapy setting during artificial speaking tasks. For example, the SLP may record the student during “morning meeting” with the rest of their class.
• It should be noted that any communication sample captures only a single moment in time. Because stuttering is highly variable, its observable characteristics may differ from day to day, week to week, or even across different situations. Therefore, these observable features are less diagnostically significant than the individual's internal experience of stuttering, which is more effectively assessed through qualitative measures.

📋 Complement formal questionnaires with qualitative questions

• Administration of assessments such as the Overall Assessment of the Speaker’s Experience of Stuttering (OASES) or the Communication Attitude Test (CAT) could be completed in an interview-style. This approach allows the SLP to ask follow-up or clarifying questions throughout.
  
  • For example, if the student indicates avoidance behaviors on a question about avoiding words, the SLP might say, “Can you give me an example of when that has happened?” Or “Tell me what that might look like for you…”

In summary, evidence-based evaluations should include qualitative assessments. There are no legal requirements that preclude them from being used. Qualitative assessments also provide information about how communication differences affect the individual’s educational experience and real-life interactions, which is necessary to determine special education eligibility. Qualitative assessments can be included in a way that is time effective and intentional.