|
Editor’s Note: The following article appeared in the Summer 2000 issue of The Independent Practitioner. Unfortunately the IP inadvertently omitted part of the conclusions and the references. To keep the article in tact, we are reprinting it in it’s entirety. |
|
Use of Computerized Continuous Performance Tasks for Assessment of ADHD: A Guide for Practioners
ADHD is the most common psychiatric disorder of childhood (NIH, 1998) and over the past several years there has been a significant increase in public attention and interest in Attention-Deficit/Hyperactivity Disorders (ADHD) not only in children but also in people across the lifespan. While the public interest has spawned controversy about the diagnosis and the extensive use of medication for treatment, it has also led to an increase in referrals for assessment and treatment of ADHD from schools, physicians, and self-referrals. Because of increases in demand for services, assessment and treatment of ADHD has been seen as a worthwhile practice niche for independent practitioners. Division 42 included two guides addressing assessment and treatment of ADHD (Resnick, 1999; Sussman, 1999) in the original offerings of the Practice Information Clearinghouse of Knowledge (PICK 42).
The extensive empirical research on ADHD indicates that assessment for the disorder is best accomplished through multi-method procedures, including norm-based behavior rating scales completed by parents and teachers (or self-report if an adult client), a structured interview, and assessment of intellectual and academic functioning (Barkley, 1998). Computerized Continuous Performance Tasks (CPTs) are becoming increasingly popular as an additional source of information regarding the ability to maintain attention and inhibit impulsive responding over time. CPTs are assessment tools that yield valuable and unique information that can easily be integrated into ADHD assessments and treatment monitoring. This article provides 1) a review of CPTs to provide background about their contribution to services for ADHD and 2) a summary and evaluation of specific products currently available to assist practitioners to select among the options available to them.
CPT’s provide a quick assessment of current abilities for sustained attention. CPTs require the client to monitor a sequence of visually or aurally presented stimuli over a period of 15 to 20 minutes and to respond by clicking on the mouse whenever a designated target stimulus appears. Omission errors, or the failure to respond to target stimuli, reflect inattention whereas commission errors (responding to nontarget stimuli) are thought to reflect impulsive tendencies. CPT omission and commission variables have consistently been found to discriminate between children with ADHD and normal controls (see Losier, McGrath, & Klein, 1996 for a review).
Standardized, clinic-based measures of attention, such as CPTs can provide objective data less influenced by factors such as rater bias, which can significantly effect the validity of parent and teacher ratings of behavior (Barkley, 1987). CPTs also can detect attention problems in children who are not overtly hyperactive or impulsive (e.g. ADHD predominantly inattentive type) and may thus fall within the normal range on parent or teacher ratings of behavior. Observations made during the CPT can also be valuable in providing information regarding motor activity, ability to comprehend and follow simple directions, and response to boredom. Finally, the “objective” presentation of CPT results, combined with other test data and behavioral observations, can provide data valued by teachers and physicians and aid in explaining a diagnosis of ADHD, especially in cases in which clients or their parents are doubtful of the ADHD diagnosis in general.
Review of Popular CPTs
There are a number of CPTs currently available to private practitioners. The present article will be limited to a brief review of four CPTs which are widely-used and commercially available: Gordon Diagnostic System (GDS; Gordon, 1983), Test of Variables of Attention (TOVA; Greenberg, 1996), Conners’ Continuous Performance Task (CCPT; Conners, 1995), and the Intermediate Visual and Auditory Continuous Performance Test (IVA; Sandford, 1995). Table 1 provides basic information on each of these CPTs including age range for norms, cost, computer requirements, and administration time. The following overview will provide a brief description of the general task and type of stimuli used for each of CPTs, major variables, and brief psychometric information.
Gordon Diagnostic System. The GDS was one of the first commercial CPTs to become widely available. As opposed to a software program, the GDS is a self-contained portable unit, which administers two attention tasks and a test of impulse control. Separate adult, child and preschool versions of each of the three tasks are included in the purchase price of the test. Parallel forms of each task are also included to reduce practice effects when retesting. In the non-distracting version of the vigilance task, the client is told to press a button when a certain combination of numbers flashes on the screen. In the distracting version target digits are shown surrounded by distracting numbers. A third delay task requires the client to inhibit responding to earn points. The GDS provides a printout that includes number of correct responses, incorrect response and failures to respond. Normative data, based on approximately 1,300 non-ADHD boys and girls, is available for children ages 4-16. Norms are also available for college students, adults and geriatric populations. However, these norms are based on small samples and the test has been found to be less sensitive for clients over age 14 (Oehler-Stinnett, 1998). Acceptable test-retest reliability and validity for the GDS has been established through a significant number of published research studies and a long history of use in clinical settings. A false positive rate of 2% and a false negative rate of 15 to 35% have been reported (Gordon, Mettelman & Di Niro, 1989). Although an auditory version of the GDS is expected to become available in the near future, currently the GDS includes only visual stimuli and does not include an auditory component.
The Test of Variables of Attention. The TOVA was designed for use in diagnosing and monitoring medication effectiveness in children and adults with attention deficit disorders. In order to avoid confounding language processing or short-term memory skills with the assessment of attention, the TOVA does not use numbers or letters as test stimuli. Rather, the TOVA presents a large square on the computer screen with a smaller square embedded near the top or bottom edge of the large square. Subjects are asked to respond when the small square is near the top of the larger square and not to respond when the small square is near the bottom of the larger square. Stimuli are presented in a 2-second fixed-interval format, with targets presented infrequently in the first half of the test (prompting omission errors) and frequently in the second half of the test (prompting commission errors). The primary TOVA variables include errors of omission (inattention), errors of commission (impulsivity), response time and response time standard deviation.
The TOVA interpretive report program allows the user to choose up to seven different forms of information. Information alternatives include raw scores and standard scores on each of the variables for each quarter of the test, analysis graph, signal detection data (which yields an ADHD Score), notes to the Clinician and a School Intervention Report. Normative data, based on a sample of 1,500 normal children and adults, are presented in separate tables for boys and girls ages 4 to 20, adults 20-70 (in ten year increments) and seniors (age 80 and up). Information presented in the technical manual includes several studies, largely unpublished, in support of the validity and reliability of the TOVA. A false positive and false negative rate of 20% have been reported for children 6 to 15 (Greenberg, 1996).
Although the TOVA was initially developed as a visual CPT, an auditory version, the TOVA-A, was released in 1996. Although the TOVA-A is currently included in the purchase price of the TOVA-V, the cost of scoring and interpretation of the TOVA-A is in addition to the per-test cost of scoring and interpretation of the TOVA-V. During the TOVA-A, clients are asked to discriminate between two different tones (Middle C and Middle G). Like the TOVA, stimuli are presented in fixed 2-second intervals with target stimuli occurring infrequently in the first half of the test and infrequently in the second half of the test. The total test time for the TOVA-A is 21.6 minutes. The TOVA-A has been normed on children 6 to 19 years of age. Information on validity of the TOVA-A is limited to a factor analysis of TOVA-A data and a comparison of variables scores between the TOVA and TOVA-A. In addition, information on sensitivity and specificity of the TOVA-A is not reported in the manual.
Conners Continuous Performance Test. The CCPT is a vigilance task in which respondents are asked to press a button when any letter but “X’ appears on the screen. The test includes 6 blocks with different interstimulus intervals and takes 14 minutes to complete. The major variables include number correct, omission errors, commission errors, and various reaction times. Test results can be obtained in six different forms that provide interpretive guidelines, reaction times as well as raw scores, t-score and percentiles for all of the major variables. T-scores and percentiles are available comparing results to the general population norms or the ADHD normative group. However, it is highly recommended that the general population norms be used for interpreting the CCPT results. The norms for the CCPT are based on a sample of over 1200 children and adults ranging in age from 4 to 70 (majority of adult sample between 18 and 30 years old). Conners (1995) indicates that the CCPT may be most effective for individuals 6 to 17 years of age. A false positive and false negative rate of 10-15% or lower is reported in the CCPT manual. Conner’s presents brief summaries of several published research articles as support for the validity of the CCPT. Specific information regarding reliability is not reported.
An updated Windows version of the CPPT, the CPT-II, has been recently developed and will be available for purchase in the Spring of 2000. Available information on the CPT-II indicates that the revised test includes a larger normative sample, newly designed reports, a more comprehensive software manual and inclusion of a validity scale. A preschool version of the CPT-II, the K-CPT, will also be available soon. The K-CPT uses pictures of common objects as test stimuli rather than letters. Psychometric information on the CPT-II and the K-CPT are not currently available.
The Intermediate Visual and Auditory Continuous Performance Test. The IVA is an integrated 13-minute auditory and visual CPT. In addition to the typical demands of clicking in response to a designated target, the IVA requires the test taker to “shift sets” and to make discriminatory responses to mixed auditory and visual stimuli (e.g. click if you see or hear a “1”, do not click if you see or hear a “2”). A preschool version of the IVA, the PREVA, is currently being normed and presents auditory and visual stimuli consecutively rather than concurrently. The normative database for the IVA consists of 487 individuals ranging in age from 5 to 90 years old. Norms are located in a read-only file that is included with the IVA software.
The IVA calculates errors of commission (“prudence”), errors of omission (“vigilance”), as well as response speed and response speed variability. Results for six primary subscales and three validity scales are presented separately for auditory and visual stimuli allowing for comparison between the two modalities. Full-scale response control (impulsive errors) and attention quotients (inattention errors) are calculated from a combination of select subscales. In addition, a fine motor regulation scale is calculated by summing off-task behaviors with the mouse (e.g. excessive clicking). Three validity scales are included which assess for neurological and/or learning problems, poor motivation or motor fatigue and lack of comprehension, all of which could invalidate the test results.
Test results include raw score percentages (percent of correct trials) and quotient scores for all the major variables, as well as a histogram of auditory and visual response speeds. The printout of results does not provide any interpretive notes, thus the clinician must be quite familiar with the interpretive guidelines contained in the Interpretation Manual in order to draw conclusions from the test. A step-by-step guide is provided in the manual to assist in interpretation of the IVA when assessing for ADHD. For an additional one-time fee ($195), software which produces an interpretive report for the IVA can be purchased. Sanford (1995) reports a false negative rate of 8% and a false positive rate of 10%.
Cautions in Using CPTs
Three of the primary criticisms of the use of CPTs in the diagnosis of ADHD involve ecological validity, high rate of false negatives, and the limited ability of CPTs to discriminate between ADHD and other clinical disorders. Examiner presence may result in better than average performance for children who respond well to one-to-one attention and reinforcement. The novelty and attractiveness of a computer-based task may also result in performance that is an overestimation of typical ability to maintain attention. On the other hand, lack of familiarity with a computer mouse and performance anxiety induced by the lab situation may yield results suggesting greater deficits than would be typical in a natural environment. To address the problem of ecological validity, it is suggested that the clinician gather samples of the client’s behavior in a number of settings including lab tests, observations made in the waiting room and during testing, and a school observation if possible. A child who performs unusually well on an individually administered computer test of attention may display very different behavior while in the waiting room or during less stimulating portions of the assessment battery. Careful review of teacher and parent behavioral observations will also allow the clinician to better determine if results of a CPT are in fact reflective of behavior in other settings.
Various CPTs have yielded false negative rates from 20 to 37% or higher (Greenberg, 1996; Barkley, 1991), leading researchers to caution that “normal” CPT performance should not be used as evidence to rule out a diagnosis of ADHD. Rather than interpreting false negatives as test error, it has been suggested that clinicians more closely examine any discrepancies between CPT performance and other assessment data (Ingersoll, 1996). The interpretive guidelines for many of the CPT’s (GDS, TOVA, IVA) caution that individuals with above average IQs are likely to perform well despite attention and impulsivity problems in other settings. Likewise, individuals with below average cognitive ability may have difficulty comprehending the task directions, resulting in abnormal performance that may be primarily due to cognitive deficits. A recent study of false negatives on the CPT found that children with elevated teacher and parent ratings, but average CPT performance, are twice as likely as referred children with abnormal CPTs to be medication nonresponders (Fischer, Newby & Gordon, 1995). These results indicate that “average” performance on a CPT may have implications for patients diagnosed with ADHD and referred for a medication evaluation.
The ability of CPTs to distinguish between ADHD and other clinical disorders is an issue of major concern to most practitioners since recommended psychotherapy interventions may vary greatly depending on the true diagnosis (e.g. anxiety disorder versus ADHD). A number of studies have found that CPTs have been inconsistent in differentiating ADHD from other clinical groups (e.g. Halperin, Matier, Bedi, Sharma & Newcorn, 1992). However, the majority of research in this area has considered the CPT in isolation of other test data in determining clinical accuracy. When the results of a CPT are considered in the context of other assessment information, CPT data can be extremely useful in differential diagnosis. For example, CPT data and observations made during the CPT may assist the clinician in differentiating between conduct problems and ADHD by providing information about intentionality of impulsive responding (Oehler-Stinnett, 1998).
Use of CPTs in Monitoring Responses to Medication
In addition to screening and diagnosis for ADHD, a major feature of many CPTs is to assist the clinician in monitoring medication in children and adults with ADHD. Stimulant medication has been found to significantly reduce the number of CPT omission and commission errors made by children diagnosed with ADHD (see Losier, McGrath & Klein for a review). It has also been suggested that results of CPT tasks may provide information regarding the effectiveness of differing doses of stimulant medication. The ability of CPTs to assist in monitoring dose effectiveness may be particularly noteworthy given the recent findings from the NIMH multi-modal treatment study (NIMH, 1999) that stress the importance of determining the optimal medication dosage in addressing ADHD symptoms. All of the CPTs described in this article include one or more features, such as the ability to compare the results of multiple administrations, designed to assist in monitoring medication effectiveness.
Comparisons of Available CPTs
Thoughtful integration of CPT results with other assessment data allows for maximum use of multiple sources of information regarding attention, impulsivity and hyperactivity in arriving at an accurate diagnosis of ADHD. In addition, CPTs can be a useful tool in evaluating the effectiveness of medications used to treat ADHD symptoms. The GDS, TOVA, CCPT and IVA are four CPTs that are widely used in clinical practice. Although many of the major features of these CPTs are quite similar, there are differences in cost, ease of interpretation, psychometric properties and research support, and additional features such as inclusion of auditory stimuli.
In evaluating the cost of the various CPTs, anticipated frequency of use is an important factor to consider. Although the initial cost of both the GDS and IVA is somewhat high (e.g. about $1,500), this cost includes unlimited use and scoring. In addition, the cost of the GDS includes the portable microprocessor-based unit, eliminating the need for a computer or a printer. For a much lower cost ($495), the new version of the CCPT, the CPT-II, also includes unlimited use, scoring, and interpretation. In order to test preschool-age children, the clinician can purchase both the CPT-II and the K-CPT (Kiddie version) for $675.
The TOVA-V and TOVA-A do not provide an unlimited-use purchase option. Rather, the practitioner pays an initial fee of $295 for the software and an additional $15 scoring fee for each test administered, after the first five tests. Thus after 85-90 administrations of the TOVA, the practitioner will have invested the same amount of cost as the GDS or IVA.
Ease of interpretation is another significant factor in discriminating between CPTs, particularly for practitioners with limited time. Both the TOVA and CCPT provide extensive reports that include interpretive guidelines. The TOVA also provides the option of generating a school intervention report that can be individualized by including and or editing paragraphs as needed. The GDS and IVA reports do not provide printed interpretative guidelines. However assistance in interpreting the test scores is provided in the interpretation manuals. In addition, interpretive software is available for the IVA at an additional charge. There are both advantages and disadvantages of interpretations not being included in the test printout. An obvious disadvantage is the greater time required to consult the manual in order to make interpretations. However, in the process of repeatedly consulting an interpretation manual, the clinician may acquire greater expertise in the use and interpretation of the test than may occur when interpretations are printed in the report. There is also the possibility of CPT results being misinterpreted or overemphasized in the diagnosis of ADHD when specific diagnostic information is suggested in the printed report (as in the case with the TOVA).
In regard to psychometric properties of the various CPTs, the GDS appears to have the strongest psychometric support. Both the TOVA and CCPT present limited psychometric information and the IVA has a much smaller normative sample than the other CPTs. In addition, there are many more published studies that support the GDS as compared with the TOVA, CCPT and the IVA. The greater number of published studies for the GDS is due in part to its longer history and recent changes and developments in the TOVA, CCPT and the IVA.
Finally, the CPTs discussed in this article differ in their types of features and age ranges. Both the IVA and the CCPT have recently developed preschool versions of their original CPT tests that are shorter and simpler. The K-CPT (CCPT kiddie version) uses pictures rather than numbers or letters, which may be more effective in engaging young children in the task. The TOVA and GDS have both recently developed auditory versions of their CPT task in order to assess deficits in auditory attention. Inclusion of an auditory component also provides information on the presence of a possible central auditory processing disorder (CAPD), although it is often quite difficult to differentiate between a diagnosis of ADHD and CAPD (e.g. Riccio, Cohen, Hynd & Keith, 1996). The IVA includes auditory stimuli intermixed with visual stimuli requiring alternating attention or mental flexibility in shifting sets. Sanford and Turner (1995) cite research suggesting that ADHD males are more affected than controls by stimuli presented with uncertain temporal modality. However, further research is needed to determine if difficulty in shifting sets is specifically characteristic of ADHD.
Summary
All of the available instruments reviewed provide clinically useful information when used with other measures and sources of information. Practitioners must consider several factors in selecting from among the alternatives. The first factor is the value placed on the auditory component and the shift between auditory stimuli and visual stimuli while attending. All systems except the CPPT and the GDS currently have options for presentation of auditory stimuli and the IVA is the only one that requires shifting of set.
The second factor is the available norms. All have acceptable norms but the GDS has been studies more extensively and the IVA has the fewest individuals in the normative sample. The IVA and CPPT also have tests specifically designed for preschoolers.
The third factor is the information provided in the reports. The TOVA and the CCPT both provide detailed reports with interpretive guidelines and suggestions that are not provided by the other instruments.
The fourth factor is hardware requirements. If you do not have a computer that can be used to administer the instruments or would want to give the tests outside the office setting and do not have computer that can easily be moved, you may want to consider the GDS. This is a self-contained unit and does not require additional hardware. However, all of the others can operate on computers that have few features. Machines that can be dedicated to assessment as a consequence of providers upgrading their systems can accommodate the requirements for most testing including administration of the CPTs. However, both the TOVA-A and the IVA require either a sound card or additional computer hardware.
The final factor is cost. The CCPT and GDS have unlimited use. Because it includes the hardware, the GDS costs three times as much as the CCPT. The IVA also has an option for unlimited use that places the cost close to that of the GDS. The TOVA and the IVA have pricing that provides software at a lower price than the other applications but require a per administration charge. The details of the price structure are contained in Table 1. The relative merits of alternatives depend upon the anticipated number of administrations. Thus, if greater than 85 administrations are anticipated, the unlimited administration options are clearly preferable.
References
Barkley (1987). Can neuropsychological tests help diagnose ADD/ADHD ? The ADHD Report. 1 (2), 1-3.
Barkley (1991). The ecological validity of the laboratory and analogue assessment methods of ADHD symptoms. Journal of Abnormal Child Psychology, 19, 149-178.
Barkley (1998). Attention-Deficit/Hyperactivity Disorder. A Handbook for Diagnosis and Treatment, Second Edition, Guilford Press: NY.
Conners, K. (1995). Conners’ Continuous Performance Test Computer Program 3.0: Users Manual. Canada: Muli-Health Systems, Inc.
Fischer, M., Newby, R.F., & Gordon, M. (1995). Who are the false negatives on continuous performance tests? Journal of Clinical Child Psychology, 24, 427-433.
Gordon, M. (1983). The Gordon Diagnostic System. DeWitt, NY: Gordon Systems.
Gordon, M., Mettelman, B.B., & Di Niro, D. (1989). Are continuous performance tests valid in the diagnosis of ADHD/hyperactivity? Paper presented at the 97th annual convention of the American Psychological Association, New Orleans.
Greenberg, L.M. (1996). Test of Variables of Attention: Professional Manual (Version 7.0). Los Alamitos, CA: Universal Attention Disorders.
Halperin, J.M., Mattier, K., Bedi, G., Sharma, V., & Newcorn, J.H. (1992). Specificity of inattention, impulsivity and hyperactivity to the diagnosis of attention –deficit hyperactivity disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 31. 190-196.
Diagnosis and Treatment of Attention Deficit Hyperactivity Disorder. NIH Consensus Statement 1998, Nov 16-18; 16(2): In press.
Losier , B.J., McGrath, P.J., & Klein, R.M. (1996). Error patterns on the continuous performance test in non-medicated and medicated samples of children with and without ADHD: A meta-analytic review. Journal of Child Psychology and Psychiatry, 37, 971-987.
Oehler-Stinnett, J.J. (1998). Review of the Gordon Diagnostic System. In Mental Measurements Yearbook (13th Ed.) James C. Impara & B.S. Plake (Eds.). Lincoln, NK: Burros Institute of Mental Measurement.
Resnick (1999). ADHD through the life span: They don’t outgrow it. In Division 42, Practice Information Clearinghouse of Knowledge. Phoenix, AZ: Division 42 of the American Psychological Association.
Sandford, J.A. (1995). Intermediate Visual and Auditory Continuous Performance Test: Administration and Interpretation Manual (Version 1.2). Richmond, VA: BrainTrain.
Sussman (1999). Working with AD/HD children and their parents. In Division 42, Practice Information Clearinghouse of Knowledge. Phoenix, AZ: Division 42 of the American Psychological Association.
|
|
