In times when we as practitioners are continually asked for ‘evidence-based’ practice or proof of our ‘value added’ it is no wonder that we turn to formal testing as a way of trying to prove our worth. Perhaps this is one of the main reasons why children today are some of the most ‘tested’. So, have we lost confidence in our own skills as professionals?
Formal assessments are carried out with published tests that have been standardised using a sample of people in order to gain an idea of the possible responses to test items. They are described as norm referenced because they relate the performance of the testee to peers of the same age. It is important to note that very few of the standardised language assessments on the market currently are standardised on deaf children. Those that are include the Test of Reception of Grammar (TROG) and the Test of Language Development (TOLD).
It is important to distinguish formal tests from informal published procedures, which are also a useful part of our assessment tool kit and can provide frameworks, guidelines and/or indicators of how individuals are performing using criterion references. Such approaches give a picture of whether a person does or does not have a particular skill and some of these are also specific for deaf children, such as the Early Support Monitoring Protocol. Informal procedures are wide ranging and have many advantages over standardised tests. These warrant in-depth discussion themselves. However, the focus of this particular article is to encourage us to reflect on the pros and cons of our own decisions in choosing and interpreting formal tests. In this endeavour the following questions need consideration before we open our test cupboards.
Why are we testing? No one test will be able to give us a whole picture of the child or young person’s communication, language and speech skills. Most formal assessments will only be looking at one small part of the child or young person’s communication. We need to fully understand not only what a test is testing, but also its limitations and what we do with the information gleaned.
What is the test testing? Tests help us to focus on specific aspects of communication. This might be: • throughout the whole test, for example the TROG looks at the understanding of grammar only • through various subtests, for example the Assessment of Comprehension and Expression (ACE), which assesses eight different aspects of communication, including some analysis of semantic, syntactic and narrative skills
How available or convenient is the test? Are we only using the tests that we have available or accessible to us? If this is the case we need to refer back to the previous point. We need to consider that carrying out the test may give us some insight into a small and particular aspect of the whole communication process and may provide information about the child’s response to a test situation but are there better or complementary ways to find out about that specific skill in the child?
Who else uses the test? Many of the assessments on the market are being used by ToDs and speech and language therapists. It is essential therefore that there are clear protocols set up for how, when and which assessments are carried out and how any results are to be documented and shared. Not only is this good joint working practice but frequent use of the same test will invalidate results. In addition, some tests, such as the Clinical Evaluation of Language Fundamentals-Revised (CELF –R) and the new Pre-School Language Scales 4 (PLS-4), specify qualifications that testers must hold, like a recognised degree in speech and language therapy. Thus any protocol needs to take account of such issues and also ensure that all involved professionals, such as ToDs, SLTs, members of cochlear implant teams and auditory verbal therapists, are included.
How long does the test take to administer? Some tests are preferred as they are short and, therefore, quick to administer, such as the Renfrew Action Picture Test or Renfrew Bus Story. It is important to understand that most of these are screening assessments and will only give a rough idea of where the child or young person is in terms of one aspect of their language development. Further formal and informal assessment is often necessary in these cases.
What use are the age level/standard scores? These can be very useful when we are required to compare our deaf child or young person to their peer group, especially when we are trying to get extra support for them or a statement of special educational need. However, care needs to be taken with the interpretation of these results as most tests are not standardised on deaf children. Any adaptations that might be used for the deaf child, for example, repeating a command or signing, will affect the results. In addition, age levels can easily be misunderstood and may be seen as a measure of general performance, which can be very disheartening for families.
How do we record results and set appropriate targets? It is important to ensure that the results are summarised and shared and that developmentally related targets are set with regard to further development of the child or young person’s language and communication skills. Whichever assessment we choose we have a responsibility to know what we are going to do with the results in terms of our management of the deaf child or young person. If, for example, after having completed a Boehm, Bracken or British Picture Vocabulary Scales assessment and having ascertained that the pupil is two years behind his or her peer group in relation to vocabulary/concept development, do we then have a plan or programme based on developmental norms ready to put in place for that child? Without this an age score becomes valueless as it does not indicate what vocabulary we should teach that will be pertinent to the everyday lives of individuals.
How useful is re-testing? Even when a test is re-administered, the child/young person may still not know some of the items and any naturalistic progress will not be shown. The use of informal assessment to supplement testing is therefore often more informative as it can be carefully structured in smaller incremental, developmentallybased steps allowing for re-evaluation after a period of teaching or exposure to a particular aspect of language. This would be more sensitive to the valueadded nature of the child or young person’s learning than a formal standardised assessment.
How might we use a holistic approach combining formal and informal assessment? Some examples The Assessment of Comprehension and Expression (ACE) has become a more popular test in recent years as it covers a wider range of linguistic skills and tests slightly older children (6–11 years). It gives a useful profile so it is easy to see which areas a child might need most help with. It is likely that the child will have difficulties with many, if not all, these linguistic aspects. Once we have this test information, a decision then has to be made about which aspects to address first and which skills might need to build on others. It may also be that these skills (for example inferential and non-literal understanding) have not developed because there are still gaps in earlier semantic knowledge, in which case we need to be prepared to use informal assessment and professional knowledge to determine the sequence in which these skills develop and at what stage we should begin. We also need to consider the value of talking to the child’s teacher or to the child to ascertain difficulties in these areas using robust informal procedures alongside, or indeed instead of, the test.
The use of the Test of the Reception of Grammar (TROG) or the South Tyneside Assessment of Syntactic Structures (STASS) can be an effective way to provide a structure for beginning to look at grammatical development. As with any single assessment, each of these has its limitations and we need to be mindful of what each does and does not allow us to find out. These tests can be supplemented and/or replaced by informal approaches, such as grammatical analysis of the child or young person’s spoken/written language. Using a framework for normal grammatical development can help to understand where the sticking points are likely to be and to give details of how to provide the necessary opportunities to practise and work with specific grammatical structures.
It is important to remember that, whatever test we use, it cannot give us a comprehensive picture of the child/young person’s communication skills and it will only form one specific tool in our overall assessment kit. Informal assessment, through our observations within natural contexts, the use of other criterion-referenced assessments and the employment of our own professional skills and judgements in colaboration with those of others are equally essential in the wholer assessment process.
Published in the BATOD Association Magazine Focus on Assessment January 2010