The first aspect of MSBI that distinguishes it from other forms of schema instruction is attention to barriers in accessing the problem, including reading level, problem structure, quantities/content, and vocabulary (Spooner et al., 2017). The flexibility of MSBI is critical here, as the individual supports needed exist on a continuum and likely differ within and between students based on the task and their phase of learning (Root et al., 2022).

As a specific type of schema instruction, MSBI teach students to use schematic diagrams (i.e., graphic organizers) to model their conceptual understanding of each schema. In contrast to other forms of schema instruction wherein students copy or draw schematic diagrams, MSBI provides physical or virtual schematic diagrams with tailored visual supports such as color-coding and icons paired with text as antecedent cues. A universally designed schematic diagram enables students to model the relationship represented in the problem using concrete, representational, and abstract methods regardless of their communication, language, or fine motor skills.

In schema instruction, students often learn a mnemonic to remember an attack strategy consisting of logically sequenced steps for understanding and solving word problems, such as “FOPS” (Find the problem, Organize information using a diagram, Plan to solve the problem, and Solve the problem; Jitendra & Star, 2012). The level of literacy skills necessary to successfully use a mnemonic-based strategy is a barrier for many students. Relatedly, multiple interdependent decisions are encompassed within a step like “Solve the Problem.” In MSBI, a student-friendly task analysis presents the attack strategy as a sequence of distinct, discrete behaviors in the form of a checklist that may include visual supports as needed.

The guidelines for word problems put forward by Spooner et al. (2017) emphasize making explicit connections to real-life activities and routines by providing multiple examples of each schema within personally relevant contexts. MSBI mediates response generalization by teaching a “rule” for each schema, a metacognitive strategy consisting of a verbal phrase (e.g., “Small group and small group combine into big group.”) with corresponding hand gestures (e.g., bringing two fists together to represent a part-part-whole relationship).

Given the individualized and flexible nature of MSBI and heterogeneity of this population, researchers have primarily used single-case experimental designs (SCED) to evaluate its impact on word problem solving (Root et al., 2021). One unique feature of SCED that is not shared by group experimental designs is the ability to make changes to specific aspects of an intervention based on participant data (Tincani & Travers, 2022). This flexibility has allowed researchers to maintain experimental control while simultaneously tailoring MSBI using the UDL framework and Spooner et al.’s (2017) conceptual framework.

Published MSBI studies have consistently reported response-guided changes based on categorization of data as indicating a skill or performance deficit (Cox et al., 2024). Changes have been made to stimuli (e.g., materials, task analysis format), behavioral supports (e.g., self-management strategies, individualized reinforcement systems), and instructional strategies (e.g., massed trials, system of least prompts). For example, Root et al. (2020) determined the variable data of one middle school student with ID was not due to a skill deficit (i.e., he knew what was expected) but rather a performance deficit stemming from a reluctance to talk aloud. Authors drew on the UDL framework to remove this barrier within the task by teaching an alternative means of action and expression (pointing).

A second example is seen in Cox et al. (2024). They attributed stagnant performance of two high school participants with ASD/ID to a skill deficit, as they were both making consistent errors reflecting procedural misunderstandings in calculating percent of change. In response, interventionists increased intensity of instruction by providing step-level feedback using a system of least prompts rather than waiting to deliver all feedback after the entire problem was solved independently. In contrast, Root et al. (2020) categorized the stagnant performance of a middle school student with ASD/ID as a performance deficit, indicating the intervention as planned was not sufficiently engaging because the participant was successful during guided practice but rushed through independent practice problems. The researchers and teacher collaboratively implemented an alternate contingent reinforcement system to increase motivation.

Two literature reviews on MSBI indicate that these examples are not isolated cases. Root and colleagues (2021) discussed this as an area for future research to understand “under what conditions” MSBI is effective. Relatedly, Clausen et al. (2021) concluded more evidence is needed to demonstrate how to support students who have limited literacy or complex communication needs. Understanding when, why, and how response-guided changes should be made is particularly relevant to MSBI, as it can provide a framework for using the principles of UDL to identify barriers and implement aligned supports.

This manuscript reports findings from a feasibility test of three units of Math Scene Investigators, a word problem solving curriculum that uses MSBI to teach multiplicative schemas to secondary students whose mathematical instruction is based on alternate achievement standards. The primary purpose was to examine the effectiveness of MSBI as planned by asking “Is there a functional relation between MSBI and word problem solving behaviors?” The secondary purpose was to explore the need for and impact of response-guided changes to MSBI.

The study took place in a large suburban public high school located in the southeastern United States. Researchers asked one special education teacher who taught a mathematics course aligned to the state’s alternate achievement standards to send consents home with all of his students who were able to communicate their basic wants and needs in English either vocally or with AAC. Although researchers did not have access to educational records, all of his students received academic instruction aligned to alternate achievement standards, indicating they met the state’s criteria for a “significant cognitive disability.” In total, four students returned consents.

Researchers screened all four consented students using a researcher-created prescreening tool to assess the following skills: (a) receptive and expressive identification of double-digit numbers, (b) transferring numbers to a calculator, (c) using a calculator to solve single- and double-digit equations with whole numbers, and (d) solving ratio/proportion word problems. Table 1 details demographics of the four students with ID who were invited to participate in this study based on their performance on the prescreening tool. All invited participants demonstrated a basic level of prerequisite mathematics skills (a–c) during the screening as well as a need for the intervention (d). Three participants previously participated in studies with the research team that targeted different schemas between one and three years prior to the onset of the current study (Phillip and Esteban, multiplicative comparison; Jayla, percent of change). We did not expect generalization to ratio and proportion word problems given performance on the prescreening (d).

All participants followed the school’s bell schedule to attend both core-content courses aligned to alternate achievement standards that were taught by special education teachers and elective courses with general education peers taught by general education teachers (e.g., Culinary). Mathematics instruction focused on computational fluency in a small group format but did not use a specific curriculum. The intervention occurred three to four times per week either in a teacher’s lounge or conference room during each participant’s math block for 15–35 min.

Two members of the research team rotated as the interventionist who had prior experience with MSBI and contributed to curriculum development (discussed further below). Before the study, interventionists reviewed procedural fidelity and role-played procedures for each condition. At the time, both interventionists were White females enrolled in a special education doctoral program, held current teaching certifications in special education, and had experience teaching students with ASD/ID as classroom teachers and researchers.

A NCMB single-case experimental design (Harvey et al., 2004; Ledford & Gast, 2018; Slocum et al., 2022) measured the effect of MSBI on word problem solving behaviors. This variation of multiple-baseline design has garnered recent attention, with multiple author groups agreeing that it is a “practically useful and experimentally adequate design for answering some research questions” (Ledford, 2022, p. 662) as it still “provides verification with baseline logic, contrary to arguments made by prior researchers” (Smith et al., 2022, p. 667). Given our research questions, we used the logic of multiple baseline design and our own expertise in this content area to make design decisions that considered “the myriad of subtleties of research priorities and contextual factors” (Slocum et al., 2022, p. 681), rather than historical precedent for strict adherence to a specific set of rules (e.g., synchronized tiers; Ledford, 2022; Smith et al., 2022). As such, we aimed for a “balance between experimental rigor and innovativeness” (Harvey et al., 2004, p. 275).

Participants were assigned to tiers based on the sequential timing (i.e., first come first served) of when they turned in their consents and passed the prescreening tool (n = 4). Participants were randomly assigned to intervention start-point (i.e., baseline length) after screening using an online random generator. The online random generator randomized the number of baseline sessions between a range of possible baseline lengths: tier 1 between 3–4, tier 2 between 5–6, tier 3 between 7–8, and tier 4 between 9–10. The random generator selected 3, 6, 7, and 9 respectively.

There were three phases of intervention: ratio (unit 1), proportion (unit 2), and discrimination (unit 3). Participants completed one probe between each intervention phase wherein they solved one ratio and one proportion problem under baseline conditions (see procedures for details). Intervention phase change decisions (e.g., Unit 1 to Unit 2) were based on participant data meeting the established mastery criteria for the primary dependent variable (described below). Based on Browder et al.’s (1986) guidelines, response-guided changes were considered when data had a stagnant or descending trend of three data points.

The primary dependent variable was the number of critical independent problem-solving behaviors, measured by a researcher-created rubric that aligned with the problem-solving routine displayed on participant worksheets (Supplemental File 1). Although the problem-solving routine was shown as 8 steps on the student worksheets, Table 2 details how we measured 10 behaviors by splitting steps 5 and 7 into discrete behaviors for a total of 10 operationally defined and repeatedly measured problem-solving behaviors. Four behaviors were considered critical because they were required to arrive at the correct final answer (identified on Figure 1). The six non-critical behaviors were taught as a part of the problem-solving routine to support conceptual and procedural knowledge and measured to enable fine-grained tracking of participants success with MSBI as planned (see procedures for details). The mastery criteria for each unit were set a priori as three out of four critical problem-solving behaviors for two out of three sessions.

This study was a feasibility test of three units of the Math Scene Investigators, a curriculum that uses MSBI to teach students to solve word problems. The curriculum was developed by the research team through an IEP-funded Early Career Development and mentoring grant. All members of the research team contributed to the development of the materials, procedures, and protocols. Each unit had two introduction lessons, followed by daily lessons that consisted of one model problem, one guided practice problem, and one independent practice problem. Researchers delivered instruction using scripted lesson plans that incorporated the 16 elements of explicit instruction (Archer & Hughes, 2011). Lesson plan outlines can be found in Supplemental File 2.

Participant materials included worksheets, a choice board for schematic diagrams, a smartphone to access a calculator app, and self-management folders. Participants solved novel word problems each session that were written by researchers using Spooner et al.’s (2017) guideline and validated by a content area expert; no word problems were repeated throughout the study. Participants selected the schematic diagram from a laminated menu with the two options (ratio and proportion) and were given the corresponding schematic diagram as a 1” × 3” label sticker to place onto the worksheet. Vocabulary cards displayed a clear and concise definition and a visual representation with color coding for the following terms: ratio, proportion, equivalence, cross multiply, isolate the variable, and inverse operation (Supplemental File 3).

Participants were given two worksheets (one for each problem type), a smartphone to access a calculator app, the schematic diagram laminated menu, and a pencil. The interventionist began by reading the word problem and provided the following statement: “Show me how to solve this problem. Remember, if you need anything read aloud again, you may ask me.” As they independently completed both worksheets, the interventionist gave non-specific verbal praise (e.g., “You are working really hard!”) but no corrective feedback.

After solving the independent practice problem(s), participants checked their work with assistance from the interventionist. They self-monitored accuracy of each problem solving behavior directly on the worksheet by giving themselves a check mark for those completed independently correct. Any errors were discussed and revised using a green pen. Participants recorded the total number of check marks (i.e., independent correct behaviors) and used their self-reflection folder to: (a) self-record and self-graph their score from the independent practice problem(s), (b) self-evaluate whether or not they met the goal they had set, (c) set a goal for the next session, and (d) self-reflect on their problem solving experience using two Likert scales represented with emojis and by circling icons representing the components of MSBI they enjoyed (e.g., goal setting, checking off their work on the task analysis)

To reduce bias, we used an online random number generator to identify a minimum of 30% of sessions for each participant for IOA and PF before the study began (Ledford et al., 2023). A graduate research assistant not involved in data collection was trained to use the coding manual to independently score worksheets. Disagreements were discussed in weekly meetings. IOA was calculated using point-by-point agreement (Ledford & Gast, 2018), with an overall average of 96% agreement in baseline and probes (range of 90–100%), 90% for intervention sessions (range of 80–100%), and 90% for discrimination (range of 60–100%) across all scored sessions and interventionist. A PF checklist was used to measure critical elements of the intervention (Supplemental File 4). A total of 41% of all sessions were scored for PF for both interventionist across all sessions; PF was overall average of 98% for baseline and probes (range of 80–100%), 96% for intervention sessions (range of 83–100%), and 99% for discrimination (range of 94–100%). Supplemental File 5 details IOA and PF broken down by Unit across each participant.

Following Kratochwill et al. (2022) recommendations for NCMB data presentation, we chose to display the data as if it were collected in real-time by aligning sessions across each tier (participants) to enable vertical analysis (Figure 1). Dates on the x-axis indicate when each participant’s data collection began. To address long-standing concerns in SCED regarding interrater agreement in visual analysis, we followed Wolfe et al.’s (2019) protocol for systematic visual analysis of level, trend, variability, immediacy, overlap, and consistency. Separate visual analysis for ratio and proportion enabled interpretation of effects within and between phases.

After the intervention concluded, interventionists engaged participants in social validity interviews using a semi-structured protocol of six open-ended questions and five Likert rating scale questions with emojis and text to represent response options (Supplemental File 6). All questions and response options were read aloud and displayed to participants one at a time. Pictures of study materials (e.g., calculator, hand motion rules, schematic diagrams, task analysis) were displayed on a printed matrix as a visual reference. Participants could request for a break between open ended and Likert scale questions. Social validity data was analyzed using content-analysis.

As anticipated, response-guided changes to MSBI were necessary for two participants. Symbols in Figure 1 identify the onset of each change, which are described further below.

Social validity interviews ranged from 15–21 minutes (average of 18 minutes). All participants liked setting daily goals, choosing word problem locations, checking their work with the interventionists, and graphing their progress. Phillip shared that he liked checking his work because, “I like seeing the results” and Jayla recalled that she enjoyed graphing her scores because “I like seeing how I’m doing with my math. I like seeing my progress.” Participants identified which mathematics tools were the most helpful during their learning: schematic diagrams (all), the extended checklist (Aaliyah), and the text-to-speech app (Esteban). Esteban shared that he was using the text-to-speech app in other settings outside of the study, and that his caregivers planned to purchase the paid version of the app. Three of the four participants expressed a desire for their mathematics teacher to teach them the way they learned it with us in the research study.

We applied this logic of systematic replication in SCED (Travers et al., 2016) by assuming that if our participant characteristics, task demands, and instructional supports were similar to those in prior published studies then MSBI as planned would at minimum result in an increase in measured behaviors for the first problem type (ratio) over baseline levels for all participants within the first few sessions. Each of our participants met this expectation in the first unit on ratios as demonstrated by an immediate change in level and trend replicated across all four participants. Regarding the proportions, we anticipated all participants would need an increase in dosage of the intervention to reach criteria due to the more complex syntax and procedures (i.e., cross multiplication). This expectation was also met, as participants required more instructional sessions to meet criteria for proportions (range 15–25) over ratios (range 4–6).

Finally, our expectations were met in terms of participants requiring explicit discrimination training (Unit 3), given that prior MSBI studies that targeted more than one problem type documented this need (e.g., Browder et al., 2018; Cox et al., 2021; Root et al., 2020; Saunders, 2014). While schema instruction is intentionally designed to support generalization and transfer (Fuchs et al., 2004; Spooner et al., 2017), probes showed our participants overgeneralized, meaning they applied the most recently taught strategy to all problems until given explicit instruction and deliberate practice with planned examples.

Whereas hypothesizing after results are known (i.e., Harking) is a questionable research practice, Hollenbeck et al. (2016) argue there is value in transparently reporting new understandings or hypotheses derived from post-hoc analyses when they are “driven by informed reconsiderations that prompt interesting questions…particularly in high-stake contexts where data are resource-intensive to obtain” (p. 11). They refer to this as “Tharking,” defined as “clearly and transparently reporting new hypotheses that were derived from post-hoc results in the Discussion section of an article”, with emphasis on how (transparently) and where (in the Discussion section) this takes place (p. 11). In this study, post-hoc exploratory analyses informed our understanding of the effectiveness of MSBI as planned as well as the need for and outcome of response-guided changes. To guard against Harking, the content of the results section reflects our rationale for and interpretation of response-guided changes as they were made during the study. Here we aim to transparently discuss how our post-hoc analyses impacted our interpretation of participants’ need for and response to response-guided changes.

First, it is important to reiterate that our a priori rules for making response-guided changes were based on long-established recommendations for making data-based decisions (Browder et al., 1986) and those reported in prior published MSBI studies (see Cox et al., 2024). We attributed variability to a skill deficit and zero to minimal trends to a performance deficit (e.g., can-do vs. won’t do; Codding et al., 2017). The UDL framework was then used to consider the barriers in the task and corresponding solutions based on that categorization. For example, Esteban’s stagnant data pattern in the beginning of Unit 2 was considered a skill deficit; he was unable to read the problem and therefore made errors in filling out the schematic diagram. MSBI is designed to address this barrier by having the student ask for things to be read aloud by an adult or peer (Spooner et al., 2017), but this alternate mode of representation presented a barrier to engagement for Esteban because he was self-conscious about asking for this support. Therefore, the first response-guided change made for Esteban was a self-directed option for accessing the problem (i.e., text-to-speech app) that minimized threats and distractions. As reflected in the results above, access to the problem was not considered as a barrier after he began using the app.

At the conclusion of data collection, a member of the research team conducted an error analysis of all worksheets for Aaliyah, Jayla, and Esteban. They discovered direct alignment between dips in the dependent variable and independent practice problems that had multi-word labels (e.g., ice cream cone and bowl of ice cream), units (e.g., cups of rice, ounces of water), or had labels beginning with the same letter (e.g., apple and avocado). We did not anticipate language to be a barrier within the task because we followed the same guidelines as prior MSBI studies and had our problems validated by an expert in mathematics education. We concluded Spooner et al.’s (2017) guidelines for writing word problems may not sufficiently encompass the universe of variables that influence cognitive accessibility for students with ASD/ID, particularly those with limited literacy skills who must rely on listening comprehension.

This discovery prompted post-hoc analysis of both the need for and outcomes of response-guided changes as we generated new exploratory hypotheses regarding for whom and under what conditions MSBI “works.” Executive functioning emerged as a potential barrier, specifically demands on working memory, impulse control, and cognitive flexibility as these are three components of executive functioning that Purpura et al. (2017) identified as being distinctly related to development of mathematics skills. Our exploratory post-hoc analyses found evidence of executive dysfunction during Unit 2 for Aaliyah and Esteban. For example, they had more difficulty staying on task (inhibition control), transferring quantities and labels from the problem into the schematic diagram (cognitive flexibility, working memory), and executing the multi-step cross multiplication procedure (working memory) than in Unit 1.

Our inductive approach led us to consider whether cognitive load theory may give a helpful theoretical and empirical basis for proactively identifying barriers in the learning environment and making response-guided changes to MSBI. Cognitive load theory is conceptually aligned with schema-induction theory, which focuses on the development and activation of mental models and schemas. Cognitive load theory assumes the finite capacity of an individual’s working memory is a barrier to storage of new information in long-term memory due to an intricate relationship between working memory and three types of cognitive load (Sweller, 2010). Intrinsic cognitive load represents the inherent mental effort required by the task and extraneous cognitive load refers to the unnecessary mental effort caused by factors unrelated to the learning task. For example, intrinsic cognitive load of a word problem solving task refers to mental effort required to process and understand the information presented in the problem itself. Extraneous cognitive load results from information that must be processed but is not directly relevant, such as extraneous information in a word problem. High intrinsic and extrinsic cognitive loads reduce germane cognitive load, which is the mental effort dedicated to constructing meaningful relationships between new information and existing knowledge. Both cognitive load theory and schema induction theory aim to optimize learning by activating relevant schemas and provide complementary frameworks for reducing reliance on working memory by retrieving and activating existing schemas from long-term memory.

We hypothesize the variability in language demands and lexical ambiguity of the proportion word problems likely increased their intrinsic cognitive load (Gupta & Zheng, 2020), particularly for Esteban, the only participant with a secondary language impairment. Relatedly, the expanded task analysis and massed trial practice of the cross-multiplication procedure may have decreased extraneous load by reducing demands on working memory. Given that the contribution of working memory to individual differences in word problem solving for students with learning disabilities is well documented (Fuchs et al., 2020; Lee et al., 2009; Swanson, 2011) and individuals with ID perform significantly lower on executive functioning tasks compared to typically developing peers (Spaniol & Danielsson, 2022), this is an area that warrants further inquiry.

Word problem solving is one way for students ASD/ID to apply mathematical knowledge and skills aligned with situations they could encounter in everyday life. Additionally, while the alternate achievement standards are reduced in depth, breadth, and complexity when compared to the grade-level standards, they still emphasis mathematical word problem solving. Our study highlights the effectiveness of MSBI for teaching word problem-solving to students with ID while emphasizing the need for proactive and reactive interventions tailored to individual needs. Practitioners can adapt mathematical instruction to support the cognitive load of their learners by strategically minimizing intrinsic cognitive load (i.e., aligning instruction to students learning level, iteratively introducing new concepts, and explicitly teaching important vocabulary) and extraneous cognitive load (i.e., limiting extraneous information, chunking instruction, limiting distractions) to support student learning.

Practitioners should proactively follow a universal design approach using Spooner et al.’s (2017) model for designing the instruction and materials, and reactively use student data to address specific difficulties encountered during instruction. To do so, discrete problem-solving behaviors must be operationally defined and repeatedly measured to identify specific difficulties (e.g., cross multiplication, information transfer). Observed difficulties should be practiced in isolation using massed trials with frequent opportunities to respond.