Introduction:  .doc file

The Edmonton Narrative Norms
Instrument (ENNI) Introduction


Rationale for Developing the Instrument

           Narratives have become a common feature of clinical assessment and intervention in the field of speech-language pathology. There are many good reasons for the use of narratives in the clinic.  For example, as compared to most tests of language which look at the use of words and sentences in isolation, narratives require children to combine words and sentences for a particular purpose.  In addition, they are a form of language that is commonly found in everyday life, both in interaction with others and in educational and recreational media (books, television, film). Oral narratives are considered to be a form of literate language, and to serve as a bridge between oral and written language styles (Westby, 1989). Researchers have found that there are general organizational principles that we use when we produce and understand stories, and have described how these principles develop throughout childhood (Stein & Albro, 1997). The ability to produce and understand narratives have been found to be impaired in children with learning disabilities, even when the children have not previously been found to have problems in basic language skills (Ripich & Griffiths, 1988; Roth & Spekman, 1986). Similar problems have been identified in children with specific language impairments or language/learning disabilities (Bishop & Adams, 1992; Boudreau & Chapman, 2000; Girolametto, Wiigs, Smyth, Weitzman, & Pearce, 2001; Liles, 1985, 1987; Klecan-Aker, 1985; Merritt & Liles, 1987, 1989; Norbury & Bishop, 2003; Paul & Smith, 1993; Strong, 1998; Tsai & Chang, 2008).  Narratives have been found to predict academic achievement of children with learning disabilities (Feagans & Appelbaum, 1986) and children at risk for language impairments (Fazio, Naremore, & Connell, 1996); for children with early language impairments, preschool narrative performance appears to predict later language development (Bishop & Edmondson, 1987) and reading comprehension (Bishop & Adams, 1990). Thus narrative skills appear to discriminate between children with impairments and those without in a real-life language context, making them valuable tools for assessment and intervention.

             Given the large amount of research devoted to the subject in recent years, it would seem feasible to derive information about typical storytelling skills at different ages from the research literature.  However, a variety of methods have been used by researchers to collect narrative samples from children, including oral retell (Schneider, Williams, & Hickmann, 1997; Stein & Glenn, 1979), narration from pictures (Girolametto, Wiigs, Smyth, Weitzman, & Pearce, 2001; Paul, Hernandez, Taylor, & Johnson, 1996; Pellegrini, Galda, & Rubin, 1984), oral retell with pictures (Girolametto et al., 2001; Paul, Hernandez, Taylor, & Johnson, 1996; Strong, 1998), retell from film (Dollaghan, Campbell, & Tomlin, 1990; Liles, 1985, 1987), completion of story stems (Merritt & Liles, 1987), and re-enactment using props (Feagans & Applebaum, 1986).  It is difficult to generalize across the results of these studies because of the possibility that the way that stories are collected will affect the quality of the stories children tell or retell. Even when studies used more than one way of eliciting stories (e.g., Paul, Hernandez, Taylor, & Johnson, 1996), the stories were not structurally comparable across tasks – i.e., the same stories did not appear in all formats. Using oral and pictorial stories that were structurally comparable, Schneider (1996) and Schneider and Dubé (1997, 2003) found that children tell stories that vary in quality depending on how they were originally presented.  Schneider (1996) found that children with language impairments aged 5 to 9 provided more story information when they retold stories they had heard than when they had to formulate stories from pictures without hearing an oral version.  Schneider and Dubé (2003) found a similar pattern of results with typically developing children in Kindergarten and Grade 2, with children in both grades providing more information when retelling oral stories than when formulating stories from pictures.  Schneider and Dubé (2003) argue that both oral and pictorial tasks are analogous to real-life storytelling situations, and thus both should be assessed, along with a variety of other storytelling tasks. Picture stories appear to tap the child's ability to formulate the story as opposed to the ability to recall a story formulated by someone else as in retell tasks.  

Existing tests of storytelling

        One normed test, the Renfrew Bus Story (Cowley & Glasgow, 1994), has been developed to assess narrative abilities from oral retell.  Based on a British version of the same instrument by Renfrew (1995), it consists of a picture story which is presented along with an oral version of the story; the child is then asked to retell the story while looking at the pictures again.  The test yields scores of amount of story information and sentence length.  One limitation of this test is that is normed only to the age of 6;11.  The Bus Story is appealing to children; however, neither the pictures nor the oral story were devised according to any theoretical model of storytelling. The story scoring matches the child’s retell to the originally presented oral story, according to story-specific information that the authors considered important; again, no model of storytelling was used to guide the scoring. 


Another narrative instrument available for use is the Strong Narrative Assessment Procedure (SNAP; Strong, 1998).  The instrument consists of four storybooks of the Frog stories by Mercer Mayer, which have been extensively used in research on children’s narrative abilities, along with audiotapes of oral versions of the stories.  Children listen to each story while looking at the storybook and then tell the story to the examiner without the storybook.  The instrument is accompanied by “field data” from 39 typically developing children and 39 children with language impairments, aged 8-10 years.  Scoring procedures and field data results are available for a number of measures, including amount of Story Grammar information, numbers of different types of clauses, percentage of incomplete/erroneous referential cohesive ties, and counts of words, C-units, and words per C-unit.  Findings from individual children aged 8-10 can be compared to these field data.  Although a model of story knowledge (Story Grammar) was used to score children’s retellings, the stories themselves were not constructed to be examples of ‘good’ stories, and are not structurally similar to one another.  Inspection of the field data reveal that some scores, such as Story Grammar information and referential cohesive ties, vary widely across the stories, most likely due to differences in the stories as originally depicted in the books.  The requirement for children to tell the story without looking at the pictures means that the SNAP requires adequate memory skills, and narrative skill deficits may be difficult to distinguish from memory problems.


Before we began to develop the ENNI, no valid normed instrument was available that involves story formulation from visual material only.  Such an instrument would help to provide a more comprehensive picture of a child’s independent storytelling skills.  The materials for such an instrument should be specifically designed according to a model of language, to facilitate both children’s understanding of the story and the ability to score children’s story narrations reliably.  The narrative instruments previously available covered small age ranges, either all preschool or all school age.  If narratives truly are related to literate language skills, and if preschool storytelling skills can predict later reading comprehension, then it is helpful to have narrative tasks that spanned the age range from preschool to the school years.  


            Since the process of developing the ENNI began, another instrument for narrative assessment has been developed: The Test of Narrative Language (TNL; Gillam & Pearson, 2004).  The TNL is normed on children aged 5-12 from a number of states in the United States.  It consists of four storytelling tasks, one involving oral retell of a simple script-based story and the others involving pictures. One of the picture sets depicts a story with one main protagonist and repeated attempts at a goal; it appears to be intermediate in complexity between the ENNI simple story with 2 main characters but a simple episode structure and the ENNI complex story with 4 main characters and repeated attempts at a goal.  The TNL's other two picture storytelling stimuli consist of a single picture each.  One picture is used for a comprehension task, while the other is used to elicit the production of a story.  Storytelling from a single picture requires children to formulate more of the story than is depicted; it requires formulation with less structuring than picture sequence tasks and thus should prove quite useful for older children.  It appears that together the ENNI and TNL offer a wide range of story formulation tasks with picture support.  One difference between the instruments is that the TNL's scoring mixes several types of scoring in its measures, while the ENNI offers distinct measures of several different aspects of storytelling, which we believe make it more useful for understanding a child's pattern of strengths and weaknesses and hence for planning intervention.


            We designed the Edmonton Narrative Norms Instrument to collect norms using narrative stimuli designed for the purpose.  We used the ENNI to collect local norms from children in the city of Edmonton, Alberta.  Local norms have been proposed as a valuable resource for the interpretation of children's story data (Riddle, 1996).  Even if good North American norms existed, local norms would be valuable because of the possibility of variations in storytelling abilities across the continent due to demographic factors and cultural mix (Hughes, McGillivray, & Schmidek, 1997). 


To enable reliable and valid scoring of the stories told from pictures, it was important that the stimuli clearly depicted stories that fit some model of a good story.  Stimuli for the ENNI were carefully designed according to a model of story knowledge, Story Grammar.

Story Grammar Model

The Story Grammar (SG) model describes the information that adults identify as essential to “good” stories and that adults and older children typically include in their stories (Stein & Policastro, 1984). Although different researchers have posited somewhat different variations (Mandler & Johnson, 1977; Rumelhart, 1975; Stein & Glenn, 1979; Thorndyke, 1977), there is agreement on the basic components of the model.  Stories consist of sets of sequentially related categories and each category refers to different types of information that serve specific functions in the story.  Table 1 describes these categories and functions. 


Within the model, there is a central character who is motivated to carry out some type of goal-directed action.  A story revolves around an attempt or attempts by the central character to attain a goal and the story usually ends when the goal is successfully achieved.


There are two major components to the Story Grammar model:  structural patterns and Story Grammar units or elements (Stein & Albro, 1997; Stein & Glenn, 1979).   Structural patterns describe the overall content and organization of stories; the basic pattern that would be considered adequate is  called a Complete Episode, with some patterns being less well developed and others being elaborations of the Complete Episode.  Story Grammar units are the categories of information that are typically provided in a certain order within episodes; they can be considered core story content that would commonly be included in good stories (although some units, such as Initiating Events and Outcomes, are included more often than others, such as units describing inner thoughts and feelings).


Research has found that after hearing stories corresponding to the SG episodic structure, children as young as four recall stories in ways that fit the idealized schema (Stein & Glenn, 1979).  In addition, when children and adults are asked to retell a story that does not conform to the Story Grammar model they recall the story in a manner that does conform to it (Stein & Glenn, 1979).  When asked to retell stories in which specific categories are omitted, adults and children add information that correspond to the omitted components (Mandler & Johnson, 1977; Stein & Glenn,1979). 


In general, the experimental evidence supports the distinctions among the categories of the model.  Thus the Story Grammar model appears to be a valid representation of how individuals organize story information in order to encode, understand and retrieve stories.  The acquisition of story schema knowledge appears to develop as a function of age with older children’s stories approximating the competency observed in adults. 


The Story Grammar model has been recommended for clinical use by several authors (e.g., Hedberg & Stoel-Gammon, 1986; Hughes, McGillivray, & Schmidek, 1997).  Researchers have applied the model to compare stories told by children with and without language impairments, using a variety of elicitation techniques such as story-stem completion (e.g., Merritt & Liles, 1987), retell of orally presented stories (Ripich & Griffith, 1988), and stories formulated from film viewings (e.g., Liles, 1987; Merritt & Liles, 1987) and pictures (e.g., Ripich & Griffith, 1988).  Analyses of the ENNI normative data have revealed statistically significant developmental trends for the two stories (one single-episode story and one 3-episode story) that have been analysed thus far, as well as significant differences between children with and without language impairments within each age group except for the oldest age group, the 9 year olds (Schneider, Hayward, & Dubé, 2003).  ENNI SG scoring forms contain detailed information on how to score each SG unit.

Other Aspects of Storytelling Skills 

The Story Grammar model provides useful information about the organization of information within stories.  However, other aspects of stories also need to be considered when evaluating stories.  Story Grammar is a way to evaluate the macrostructure of stories.  Macrostructural analyses focus on the overall content and organization of stories.  In contrast, microstructure approaches focus on relationships among parts of stories (Hughes, McGillivray & Schmidek, 1997).  These types of measures thus each contribute to our overall analysis of story quality; no one measure appears to capture everything that contributes to perceived story quality (McCabe & Peterson, 1984).   In order to evaluate story quality, attention must be given to both the macrostructure and microstructure of stories.  To assess the microstructure, the ENNI includes one measure of cohesion, First Mentions, with which the referring expressions a child uses to introduce characters and objects in a story can be evaluated.  Referring expressions are linguistic forms used to refer to animate beings (the elephant, Ella, she), objects (the train, it), places (the park, there) and concepts (an idea).  They can be considered adequate if they are appropriate for the listener’s knowledge, shared physical context, and the preceding linguistic context.  For example, an indefinite noun phrase such as an elephant or a proper name is appropriate for a new character in a story, while the elephant or she would only be appropriate for mentioning the character later on in the story.  Young children frequently introduce referents in a confusing way, often using pronouns such as she, which are appropriate only if the speaker can presuppose that the listener already has the referent in his or her consciousness.  The ability to introduce referents appropriately develops gradually through the early school years. A scoring system has been developed which focuses on first mentions of the characters and selected objects, with detailed information on scoring each.  First Mention usage in the ENNI stories distinguishes among age groups as well as between children with and without language impairments (Schneider & Hayward, in preparation).  The ENNI will facilitate analysis of this important skill. 


Since the ENNI stories are a type of language sample, the stories elicited with the ENNI can also be analysed using standard language sampling measures.  The ENNI includes instructions and normative tables for many of the standard analyses in common use.


                                                                                                                    Updated April 2011