School-Based Social Skills Training for Young People with Autism Spectrum Disorders

Background: The Secret Agent Society (SAS) Program, an intervention to enhance social–emotional skills, was provided by schools for children with autism spectrum disorder (ASD). The program was assessed to determine if it improved social skills at school and home, and whether improvements were maintained. Methods: Eighty-four students participated. Key outcomes were parent and teacher ratings of emotion regulation, social skills, and direct child social problem-solving measures. The standard school curriculum served as the control condition. Phase 1 was a two-group waitlist-control comparison of SAS versus the standard curriculum. Phase 2 was a follow up of all participants before and after the intervention and at 12-months post-intervention. Results: Parent and child measures improved after the intervention but not in the waitlist condition. Improvements in parent, child, and teacher measures were apparent at 12 months. Conclusions: The SAS Program warrants further research as a potential program for schools that serve children with ASD.

Keywords: Autism spectrum disorder; social skills training; school

As many as 1 in 68 children may have an autism spectrum disorder (ASD) according to the Centres for Disease Control and Prevention (Autism and Developmental Disabilities Monitoring Network Surveillance Year, [5] Principal Investigators, [5]). Impairments in social functioning and emotion recognition are a feature of ASD across the spectrum of cognitive and language abilities and have a pervasive and life-long impact (Gray et al., [16]; Howlin, Moss, Savage, & Rutter, [17]; Sofronoff, Attwood, Hinton, & Levin, [39]; Tonge, Brereton, Gray, & Einfeld, [43]). Social deficits in ASD include problems with nonverbal communication, initiating and maintaining conversations, social–emotional reciprocity, interpreting and expressing emotions correctly, recognising and responding appropriately to others' feelings, thoughts and intentions, developing and maintaining relationships, adjusting behaviour to social contexts, and engaging in interactive play with peers (American Psychiatric Association, [2]).

Social deficits are usually apparent from the earliest years but often become more evident when children begin attending school, where problems in social interaction and understanding can have a profound impact on peer acceptance, academic achievement and mental health (Cappadocia, Weiss, & Pepler, [11]). Social difficulties frequently result in high rates of bullying (Little, [27]; van Roekel, Scholte, & Didden, [45]), social isolation (Kasari, Rotheram-Fuller, Locke, & Gulsrud, [21]) and significant levels of anxiety, anger, and depression (Sofronoff et al., [39]). These difficulties frequently persist into adulthood and have a negative impact on the ability to secure employment and live independently (Howlin et al., [17]). For teachers, too, the substantial demands of supporting pupils with ASD can result in high levels of stress and burnout (Jennett, Harris, & Mesibov, [18]; Spears, Tollefson, & Simpson, [41]).

While intervention programs designed for very young children with ASD have shown promise in addressing cognitive and behavioural deficits (Magiati, Tay, & Howlin, [32]; Reed, [35]), social behaviour has proved more resistant to change (Kasari & Patterson, [20]). There is no agreed approach for teaching social skills to young people with ASD and improvements in social functioning observed in trial settings do not reliably generalise to real-life situations (Kasari & Patterson, [20]). Recent reviews (Cappadocia et al., [11]; Flynn & Healy, [15]; Kasari & Smith, [22]; Otero, Schatz, Merrill, & Bellini, [33]; Reichow, Steiner, & Volkmar, [36]; Wong et al., [50]) describe a range of different interventions, including peer-mediated programs, social skills groups, script fading procedures, pivotal response training, and video modelling, but there is no evidence that any one program is more effective than others. Similarly, there is little evidence that social skills curricula developed for children with other psychological disorders are effective for children with ASD, as these often provide insufficient explicit instruction in nonverbal communication, emotion recognition or regulation, or in the steps that constitute prosocial behaviours (Krasny, Williams, Provencal, & Ozonoff, [25]; Tse, Strulovitch, Tagalakis, Meng, & Fombonne, [44]).

A significant limitation of most ASD-specific social skills intervention programs is the lack of evidence demonstrating that any gains observed in the trial situation generalise to social functioning in real life. For this reason, recent research has begun to focus on interventions that are conducted in school settings. These have included strategies to improve executive functions (Kenworthy et al., [24]), social engagement with peers (Kasari et al., [21], [19]; Laugeson, Ellingsen, Sanderson, Tucci, & Bates, [26]; Wood, Fujii, Renno, & Van Dyke, [51]); emotional development (Ratcliffe, Wong, Dossetor, & Hayes, [34]), and general social understanding and competence (Lopata et al., [30]). However, conducting intervention research in schools presents many challenges. An earlier systematic review by Bellini, Peters, Benner, and Hopf ([8]) of school-based social skills interventions concluded these were only minimally effective for children with ASD and that there was little generalisation across persons, settings, or play stimuli. In a recent study, Locke, Kasari, and Wood ([28]) also note that autism-related interventions with proven efficacy are rarely adopted or successfully implemented in public schools due, at least in part, to the "lack of fit" between the intervention and the needs, structures, and capacities of the school setting. Barriers to implementation in schools include limitations of time, space and resources, school policies and timetables, competing staff demands, staff and pupil movements, and many other factors. Additionally, schools may be unwilling to take part in randomised control trials because of the view that all pupils should be given the opportunity to participate in a potentially helpful intervention. Finally, within less populated areas, the amount of time needed to travel between geographically dispersed schools to complete training or assessments also adds to the challenges of conducting school-based trials.

An earlier meta-analysis of single case, school-based social skills interventions (Bellini et al., [8]) concluded that such programs were only minimally effective, and a more recent meta-analysis (Whalon, Conroy, Martinez, & Werch, [49]) reported very mixed results, with effect sizes ranging from weak to strong. de Bruin, Deppeler, Moore, and Diamond ([12]) reviewed the impact of behaviourally based interventions to improve skills or minimise behaviour problems and although they report some positive effects, they highlight the need for more systematic evaluative research in schools. However, each of these reviews focuses on single case studies, rather than randomised controlled trials.

Overall, there is only limited evidence for the effectiveness of treatments conducted in school-based settings for young people with autism (Locke et al., [29]).

The present study is based on an evaluation of the "Secret Agent Society (SAS) Program" (Beaumont, [6]). An earlier trial (Beaumont & Sofronoff, [7]) examined the efficacy of this intervention (see details below) with young people with Asperger syndrome aged 8–11 years, all of whom had intellectual abilities (IQ) in the low average range or above. Compared with controls (n = 23), the treatment group (n = 26) showed greater improvements in parent-reported social skills and greater gains in knowledge of effective anxiety and anger management strategies. Teacher ratings of pupils' social competence also supported the efficacy of the program. Treatment gains were maintained at 5-month follow up without further intervention.

Some of the ways in which SAS may have advantages compared with some other school-based social–emotional skills training programs for children include its fun and engaging game-based delivery format, the active involvement of parents and school staff and the provision of visual tools (e.g., skill code cards) to support skill generalisation to daily life. The hierarchical structure of the program is also unique – teaching emotion recognition and emotion regulation skills (which often contribute to social skill performance deficits in peer interactions) prior to step-by-step social interaction skills.

However, the original Beaumont and Sofronoff ([7]) study was limited by a number of factors. These included a lack of standardised diagnostic protocols, potential reporting biases arising from parents and teachers being both intervention participants and the primary evaluators of the program, and lack of information on the longer term benefits of the intervention. Participants, too, were not representative of the wider population of students with ASD, all being of at least low average IQ and within a fairly narrow age range. Finally, this was an efficacy study delivered within a university setting and did not assess the effectiveness of the program in an applied service delivery context, such as schools.

The primary aims were to determine if the Secret Agent Society (SAS) program led to improvements in: (i) social skills at home and school; (ii) emotional regulation and awareness; and (iii) social problem-solving ability; and whether any improvements were sustained 12 months after the cessation of the weekly intervention program.

The present study was designed to evaluate the program within a broader age and intellectual ability range than in the original trial of Beaumont and Sofronoff ([7]) and to employ both objective and subjective measures of social competence. It was also intended to conduct and assess the program within the school setting. However, it was recognised from the outset that this would limit the possibilities of conducting a randomised trial. Staff training in the program could only take place during already designated training sessions; the timing of assessments and data collection would need to be flexible in order to fit in with school terms and timetables, and all participating schools would expect their pupils to have access to the intervention at some stage. The study design reflects the ways in which these challenges impacted on the trial.

Ethics approval was granted by the University of Sydney Human Research Ethics Committee and the Aspect Research Approvals Committee.

Aspect (Autism Spectrum Australia) is the largest provider of autism-specific education in Australia. Aspect operates satellite classes on the grounds of Department of Education and Communities and Catholic Education Commission regular schools throughout the Australian state of New South Wales, Australia. Aspect classes are small (5–15 students per classroom) and have at least one teacher and one teacher's aide.

Participants were recruited from 15 ASD-specific specialist primary- and high-school satellite classes run by Aspect. Families were sent study information sheets and parent consent forms. Requirements for participation were attendance at an Aspect satellite class and a diagnosis of ASD confirmed by a student's enrolment with Aspect. For admission to Aspect classes students must have a recent (<12 months) clinically confirmed diagnosis of ASD (including ASD, autism, autistic disorder, Asperger's syndrome or pervasive developmental disorder – not otherwise specified) from a specialist medical practitioner or clinical psychologist. At the time of this study, students were required to meet the diagnostic criteria for autistic disorder, Asperger disorder or pervasive developmental disorder not otherwise specified as per the DSM-IV-TR (Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition, Text Revision; American Psychiatric Association, [1] or ICD-10 International Classification of Diseases 10th Revision; World Health Organization, [52]) and this was required to be supported by a standardised diagnostic assessment.

In total 84 children (75 male; 9 female) and their families were enrolled in the project. Children's mean age was 10.7 years (range 8.2–14.6); mean receptive language age equivalent was 9.6 years (range 4.8 –20.3) and mean full scale IQ was 91.3 (range 48–136). Most participants (73; 87%) had an IQ score ≥ 70; seven (8%) had an IQ score <70 and four (5%) had no IQ score available. The mean parent-reported score for current autism symptomatology, as measured by the Social Communication Questionnaire – Current form (SCQ; Rutter, Bailey, & Lord, [37]), was 14.0, in the middle range of severity for this measure. The average socio-economic status rating, derived from postcode decile ratings using the Australian Bureau of Statistics Socio-economic Indexes for Areas, was 6.5, similar to the average of 6.0 for NSW and Australia as a whole. Full details of participant characteristics are outlined in Table 1.

Table 1. Participant characteristics by group in Phase 1 of the study.

Note: TAU: treatment as usual; ITG: Immediate Treatment Group.

The SAS Program is a multimedia social skills program for children with ASD (Beaumont & Sofronoff, [7]). The curriculum uses an espionage-themed computer game and other activities to teach children to recognise emotions in themselves and others; express feelings in appropriate ways; talk and play with others; solve social problems; and detect and manage bullying. Skills are taught through weekly group sessions with SAS trained facilitators. Generalisation of skills is facilitated through between-session practice tasks, pocket-sized "code cards" and visual supports for children to use across settings, parent information sessions, and handouts for children's classroom teachers.

Group session activities include discussion, role-playing, and skills practice using supplementary materials, such as posters, secret agent "gadgets", and a board game. Children are encouraged to complete weekly "home missions" that involve playing the SAS computer game and practicing learnt skills in everyday contexts. A Home-School Diary is used to monitor and reward children's skill use at home and at school.

Prior to program delivery, facilitators participated in a two-day training course. The SAS Program was delivered to groups of three to six students by one or two facilitators over a 10–13-week period. Facilitators received weekly phone supervision sessions with CI Beaumont throughout. The program consisted of 9 × 90 minute child sessions; 4 × 2 hour parent sessions; weekly teacher tip sheets provided to children's satellite class teachers; and child booster sessions and parent phone calls 3 and 6 months after the end of weekly sessions.

When treatment ceased, parents and teachers were provided with recommendations on how to continue to use the program strategies, including review of strategies and continued use of visual supports at home and school. After the end of SAS Program delivery, new teachers received a 2-hour training and information session at the start of the next school year to ensure all students had a classroom teacher familiar with SAS principles.

During the treatment-as-usual period participants (called here the Treatment –As –Usual [TAU] group) received the standard Aspect curriculum. Aspect provides specialist education to students with ASD via the Aspect Comprehensive Approach for Education (ACAE). The ACAE is based upon the core competencies of autism, including social, communication, sensory, learning, and behavioural needs, which are integrated into the NSW Board of Studies Curriculum (Keane, Aldridge, Costley, & Clark, [23]). Teaching strategies include play, cognitive strategies, peer-mediated intervention, using children's strengths and interests, direct instruction, social skills groups, and extra-curricular activities. The Aspect curriculum involves several elements of social skills development, including recognition and expression of emotions, sharing attention, social interaction and play with peers, social perception, self-regulation, perspective-taking, and collateral skills. However, it does not include the SAS Program's extensive targeted focus on the development of emotion regulation and key-related social skills.

The two-day SAS training was provided to teachers in four Aspect classes during their next scheduled available training days. These teachers then provided the SAS Program immediately to their students (Immediate Treatment Group (ITG); n = 26). The 58 students whose teachers were not yet trained in the SAS program continued to receive the standard Aspect curriculum forming a "treatment as usual" (TAU) control group. Participant characteristics for each group can be seen in Table 1.

The ITG was assessed immediately prior to commencing the program (pre-intervention) and immediately post-intervention, 20 weeks later (post-intervention). The TAU group was assessed at the time of their recruitment to the study (pre-wait) and then 23 weeks later (pre-intervention [see Phase 2 following]).

After the treatment-as-usual period, these students were given the SAS program, and then completed a post-intervention assessment. All participants later received a 12-month follow up. Table 2 shows average duration of times between observations.

Table 2. Duration of inter-measurement periods in weeks.

Note: TAU: treatment as usual.

Of the 84 students who completed initial assessments, 70 participated in the SAS Program. 10 participants, all in the TAU group, left their Aspect school prior to receiving the program; 4 withdrew from the study due to family commitments; 2 students withdrew from the study during the intervention assessment and 66 families participated in the 12-month followup assessment (2 students left their Aspect school and ceased study involvement during the follow-up period).

Child assessments took place at school. Cognitive and language assessments were administered by trained research assistants and a supervising psychologist. The social problem-solving measures were scored by a research psychologist blind to pre- or post-intervention status. Parent measures were completed by the child's primary caregiver. Where possible, the same parent completed the questionnaires at each assessment point. Teacher measures of social skills were completed by teacher aides, if available, and by the same respondent at each assessment point, where possible. This was done to provide an independent measure of children's social skills, given that some classroom teachers were also SAS Program facilitators.

Wechsler Abbreviated Scale of Intelligence (WASI; Wechsler, [47]) and Wechsler Intelligence Scale for Children: Fourth Edition (WISC-IV; Wechsler, [48]). In most cases IQ was assessed using the WASI, a brief measure of intelligence commonly used in research settings to provide an estimate of verbal, performance and full scale IQ scores. However, if the child had been assessed on the WISC-IV within the past 2 years, these scores were used instead.

Peabody Picture Vocabulary Test, Fourth Edition (PPVT-4; Dunn & Dunn, [13]). Receptive vocabulary skills were assessed using the PPVT-4.

Social Communication Questionnaire (SCQ; Rutter et al., [37]). Children's level of autism symptomatology was assessed using the SCQ Current form. The SCQ Current form focuses on autism symptomatology over the past 3 months. The SCQ Current form is primarily used for treatment planning or assessment of change; there is no clinical cut-off.

Social Skills Questionnaire – Parent and Teacher forms (SSQ-P; SSQ-T; Spence, [42]). The parent and teacher forms of the SSQ examine children's competence in domains such as emotion regulation, nonverbal communication, interactive play, talking to others, and assertiveness. The 30-item questionnaires have good internal consistency (Spence, [42]), and have been used in previous studies examining the effectiveness of social skills programs for high-functioning individuals with ASD (e.g., Beaumont & Sofronoff, [7]; Broderick, Caswell, Gregory, Marzolini, & Wilson, [9]; Mackay, Knott, & Dunlop, [31]; Sofronoff, Leslie, & Brown, [40]). Scores range from 0 (poor) to 60 (good).

Emotion Regulation and Social Skills Questionnaire – Parent & Teacher forms. (ERSSQ- P; ERSSQ-T; Beaumont & Sofronoff, [7]; Butterworth et al., [10]). The ERSSQ-P is a 27-item social skills and emotion regulation measure. The ERSSQ was specifically designed to capture the subtle social–emotional challenges of children with high-functioning ASD. The parent version of the questionnaire has good internal consistency (α = .89) and good concurrent validity with the parent version of the SSQ (Beaumont & Sofronoff, [7]). Scores range from 0 (poor) to 108 (good), with higher scores reflecting greater levels of social skill and emotion regulation ability. A 25-item teacher version of the ERSSQ was developed for this trial and demonstrated high internal consistency (α = .92) and good concurrent validity (r = .72) with the teacher version of the SSQ-T (Butterworth et al., [10]). Scores range from 0 (poor) to 100 (good), with higher scores reflecting greater levels of social skill and emotion regulation ability.

Since parents and teachers were involved with both intervention delivery and assessment completion, two further vignette-based tasks (described below) were included as independent measures of change in children's skills. Both have been used in previous evaluations of treatment outcome for children with ASD (e.g., Beaumont & Sofronoff, [7]; Sofronoff, Attwood, & Hinton, [38]). The scoring is the same for both tasks and responses were recorded by a research assistant to eliminate the need for the child to write. Responses were scored by a rater blind to the time point at which the task was administered.

James and the Maths Test (Attwood, [3]). This task examines children's knowledge of appropriate anxiety management strategies. A story about a boy (James) who feels anxious about a maths test is read aloud to the child. The child is then asked to suggest strategies for James to cope with his anxiety; one point is awarded for each appropriate response. For a strategy to be scored as 1 it must be appropriate to the situation, so "speaking to the teacher" is appropriate in this situation "talking to mum and dad" is not. Strategies are only scored once, "taking some deep breaths to relax" and "using relaxation" would only be 1 point not 2.

Dylan is Being Teased (Attwood, [4]). To examine participants' knowledge of appropriate anger management strategies, the child is asked to generate ideas for how a story character (Dylan) can effectively cope with anger about being teased at school. Similarly only 1 point scored for strategies that are repetitions, "ignore them and walk away" scores 1 point not 2. Talking to Dylan to help him calm down and finding a teacher are both appropriate responses as are telling the bullies to stop and moving closer to friendly peers.

To check treatment fidelity, program facilitators completed checklists covering the content and processes for each child or parent session. The mean percentage of program activities completed by facilitators was 94% per child session (SD = 8.52, range = 50–100%) and 97% (SD = 4.66, range = 82–100%) per parent information session. A minimum of 27% of child sessions and 50% of parent information sessions for each group were randomly videotaped.

Videotaped sessions were coded by one of three independent raters to assess accuracy of self- reported checklist data. Two of the raters were trained to a 94% inter-rater agreement level with the first primary rater before independently coding session footage. The mean percentage agreement between the self-report session checklist and independently coded session footage data was 93% (SD = 6.31, range = 75–100%). Treatment fidelity was measured in the same way for both phases of the study.

Two analyses of repeated measures data are presented.

A random effects regression was used to model each of the six outcome measures (SSQ-P & T; ESSRQ-P& T; James and Dylan tasks) as functions of group membership (TAU, ITG), time (pre [pre-wait for TAU, pre-intervention for ITG], post [pre-intervention for TAU, post- intervention for ITG]), group by time interaction, age, gender, performance IQ, Peabody (PPVT-4) standard score and autism severity (SCQ Current) score. The interaction term is a measure of the difference between treatment groups in the pre to post trajectory of response.

The six outcomes (SSQ-P & T; ESSRQ-P& T; James and Dylan tasks) were modelled as functions of time (pre-wait, pre-intervention, post-intervention, and 12-month follow up), age, gender, performance IQ, Peabody (PPVT-4) standard score and autism severity (SCQ Current) score. In this analysis the post- vs. pre-intervention time coefficient measures the effect on outcome of the SAS Program and the 12-month follow up vs. pre time coefficient measures the improvement in outcome a year after the Program, compared with pre-intervention. For those in the original TAU group, the pre-wait vs. pre-intervention time coefficient measures the change in outcome during their "waitlist" period.

 Table 3 summarises the outcome measures by treatment group before and after intervention (Phase 1 summaries). None of the differences between the TAU group and the ITG group were significant at pre-intervention.

Table 3. Summaries of total score outcomes by occasion and intervention group in Phase 1.

Table 3. Summaries of total score outcomes by occasion and intervention group in Phase 1.

Table 3. Summaries of total score outcomes by occasion and intervention group in Phase 1.

Note: TAU: treatment as usual; ITG: Immediate Treatment Group.

 Table 4 gives the results of the random effects regressions for Phase 1 of the study.

Table 4. Regression analysis of Phase 1 data – controlled study with two time points.

Notes: p-value: *<.01; ** <.001. Random effects regressions of parent, teacher, and child total score outcomes on occasion, intervention, occasion by intervention interaction, age at pre-intervention, gender, Performance IQ, Peabody (PPVT-IV) standard score and SCQ current score. A negative value for the regression coefficient means that ITG > TAU. TAU: treatment as usual; ITG: Immediate Treatment Group.

There are significant group (SAS/TAU) by time (pre/post) interactions for the parent SSQ and ERSSQ measures and for the child James and Dylan measures. The interaction coefficients may be interpreted as the excess Post–Pre difference in the SAS group compared with the TAU or the excess SAS–TAU difference at Post-intervention compared with Pre-intervention, after demographic, cognitive and autism severity measures are accounted for. There is evidence that the SAS intervention was associated with greater pre- to post-intervention gains in the parent and child measures than was TAU, though not in the school staff SSQ and ERSSQ measures.

There is also evidence of a negative association between both parent measures and SCQ current, and between the child James measure and SCQ current; better parent rated social skills and child knowledge of anger management strategies is associated with lesser autism symptom severity.

Table 5 summarises total score outcomes by occasions of measurement (pre-wait, pre- intervention, post-intervention, and 12-month follow up) on all outcome measures. Table 6 provides a summary of the random effects regression analyses for Phase 2 of the study (all participants over 4 time points).

Table 5. Summaries of total score outcomes by occasion of measurement after both phases.

Table 5. Summaries of total score outcomes by occasion of measurement after both phases.

Table 5. Summaries of total score outcomes by occasion of measurement after both phases.

Table 6. Regression analysis of phase 2 data – single group follow up with four time points.

Notes: p-value: *<.01; **<.001. Random effects regressions of parent, teacher, and child total score outcomes on occasion of measurement, demographic attributes, performance IQ, Peabody (PPVT-IV) standard score and SCQ Current total score.

After controlling for demographic, cognitive, and autism severity characteristics of the child, parent-reported SSQ and ERSSQ total scores were found to be higher at post-intervention and at 12-month follow up than at pre-intervention. The corresponding school staff measures did not show a significant rise at post-intervention but did so at 12-month follow up. Like the parent measures, the child-based James and Dylan scores were significantly higher at both post-intervention and 12-month follow up than at pre-intervention.

These results are consistent with those of the Phase 1 two-group analysis at the post- intervention time point, in showing that there were significant gains evident in pre- to post-SAS scores on the parent and child measures, though not in the school staff measures. They further show that at 12-month follow up, the gains in parent and child measures were maintained, and that gains were now also evident in the school staff measures. In addition, this more powerful analysis indicates that the James and Dylan measures and the parent measures are negatively associated with the autism severity measure across all occasions. There is no evidence of change in any of the six measures during the TAU period.

The 14 significant coefficients relating to between group effect differences in Phase 1 (see Table 4) and to differences from pre-intervention in Phase 2 (see Table 6) have effect sizes ranging from.44 to.97, averaging.70, with a standard deviation of.15. According to Cohen (as cited in Dunst & Hamby, [14]), this is a medium effect size.

The principal aim of the present study was to determine whether delivering the SAS Program to young people with ASD within a school setting could lead to (i) enhanced social skills in both home and school settings; (ii) improvements in emotional regulation and awareness; and (iii) improvements in social problem-solving ability. Improvements sustained at 12-month follow up were also examined.

The results for the two phases are in agreement. No effect was observed without the SAS program, while an effect of the SAS program is evident in both the controlled, experimental analyses (Phase 1) and the uncontrolled quasi-experimental analyses (Phase 2). Furthermore, the follow-up evidence showed that parent- and child-measured post-intervention gains were maintained at 12-month follow up and that school-measured gains were also evident at this point.

The study found significant and positive changes on measures designed to assess social competence (see Tables 4 and 6) following participation in the SAS Program. These changes were not seen during the curriculum-as-usual period. Further, whereas data on the longer term effects of other interventions have often been very limited, this study showed that any gains found immediately post-intervention were maintained up to 12 months. Indeed there were small continued improvements for some parent and school staff ratings of social skills following the end of the program.

The inclusion of a relatively large sample of young people with a mixed age and IQ range also allowed exploration of associations between child characteristics and treatment outcome. No correlation between intervention response and variables such as age, gender, performance IQ, or receptive language skills was found. Moreover, although participants with higher autism symptomatology had lower scores on average, over all occasions, on measures of social skills than those with lower autism symptomatology scores, there was no evidence to indicate that children with more severe autism symptoms were less likely to respond to the program.

The finding of improvements in at least some of the parent and school staff ratings of social skills after the period of active program delivery is of interest. This may reflect the possibility that the techniques learned during the intervention period continued to be applied afterwards and produced further skill learning. As school is arguably a more socially challenging environment than home, it may be that students needed more practise consolidating their social–emotional skills before they could optimally apply them at school, leading to statistically significant intervention gains on school staff report measures. In this sense, the follow-up evaluation in this study is not the same as a typical test of maintenance of effect after an intervention is entirely stopped.

Despite the positive findings, the study has some limitations that raise questions about the conclusions that can be drawn. In particular, the classes were not randomised in Phase 1 due to the practical requirement to provide the intervention first to class teachers who had the next available training day. Therefore, is it justified to attribute improvements in social skills scores to participation in the program? For example, could the reported changes simply be a consequence of children's natural development over time? This is unlikely, given that there was no significant improvement among TAU participants during the wait list period of nearly 6 months. Nevertheless, maturation could account for at least some of the improvements reported in our uncontrolled Phase 2 analyses. Several outcome measures also depended on non-blinded teacher and parent reports raising the possibility that improvements could be due to a placebo effect. Hence, it is possible that positive expectations of the program among parents and school staff might have influenced their ratings on the two principal outcome measures, the SSQ and the ERSSQ. However, two other child-based measures (the James and Dylan tests) were scored by a researcher blind to the child's pre- or post-intervention status and changes on both these measures corresponded with the unblinded teacher and parent measures. Moreover, school staff measures were primarily completed by teachers' aides who were not directly involved with program delivery. Improvements on some measures also continued to be found during the post-program- follow-up period and there is no ready reason why a placebo effect would act in this way (Walach & Maidhof, [46]). Finally, outcome measures did not involve any direct assessment of children's functioning in real-life social situations and the principal outcome measures used, that is, the James and Dylan tests, are measures of children's knowledge of emotion regulation strategies, not evidence of their use of these strategies when needed. Quantification of the clinical significance of students' improvements in social–emotional functioning must await the development of new real-world measures of disability relevant to the autism field.

The study sought to extend data on the efficacy of the SAS Program derived from previous trials by examining its effectiveness in a real-world setting, namely in a number of classes for young people with ASD. It is a considerable challenge to do this, given the constraints of a school curriculum and the many contingencies and unpredictable eventualities of school life. Nevertheless, measures of emotion regulation and social skills of young people with ASD showed sustained improvements up to 12 months following delivery of the SAS Program; these improvements were not seen following participation in the curriculum- as-usual arm of the trial. This suggests the SAS Program warrants further research as a potential adjunctive program for use in schools that serve young people with ASD.

The authors acknowledge the contributions of participating children, parents and Aspect school staff. R. Beaumont was affiliated with the School of Psychology, The University of Queensland, Brisbane, Australia at the time of the research. D. Costley was affiliated with Autism Spectrum Australia (Aspect), Sydney, Australia at the time of the research.

Dr Renae Beaumont is the author of the Secret Agent Society Program and receives royalties on all program materials sold.