On October 29, the Senate Health, Education, Labor and Pensions (HELP) Committee approved legislation by a vote of 21 to 0 to reauthorize Head Start. The Senate version enables more poor children to get a head start; strengthens the Head Start workforce; enhances Head Start coordination and collaboration; helps improve outcomes for children; strengthens the academic portion of Head Start; and enhances the accountability of Head Start programs. One of the most controversial proposals - assessment of young children - deserves further attention and analysis because accountability for program performance is crucial, but the role of assessments in that context is a difficult one.

Background

Measuring what and how well students learn is an important building block in the process of adequately preparing young children for school. Just over a year ago, the Administration began developing a national child assessment for Head Start, called the National Reporting System (NRS) - a test every four year-old participating in Head Start must take twice a year. The NRS assessment is not even mentioned in the Senate bill, yet it has potentially major repercussions for Head Start - at best, strengthening programs by improving the way teachers teach; at worst, leading to de-funding of high-quality programs, and excluding some of our neediest children. More expert guidance on and review of the development, implementation, and uses of this major new initiative are warranted.

Currently, children in Head Start are assessed three times a year on all of the domains of early learning and development, including literacy and math. The NRS is an assessment developed by the Department of Health and Human Services (HHS), which creates an additional test for all four year-olds in Head Start (roughly 500,000 children) on literacy and math skills only. Rather than the traditional "observational assessment" used by current Head Start programs, the NRS is a one-on-one question and answer test for each child. Children are assessed twice a year and, according to Administration documents, changes in children's scores would be used to judge the success of individual Head Start programs. The new testing program is expected to cost about $20 million each year. Some pilot testing was begun in April and May of 2003 and HHS has begun to fully implement this assessment for every four year-old this fall.

Top Ten Concerns

1. The NRS has been developed and implemented too quickly, compromising quality. Assessing young children is extremely difficult and requires carefully crafted assessments. Young children are not used to taking tests and often do not have the emotional maturity to sit still and focus on the task at hand. Their test scores tend to fluctuate across time and can reflect many factors unrelated to their skills (their mood, whether they sleep enough the night before a test, their susceptibility to distraction depending on the test location, their ability to respond to questions posed by a stranger, etc.).

The NRS assessment was both developed and implemented within a year; by contrast, the time frame for the development of tests under the No Child Left Behind Act (NCLB) is three to four years. As a result of the Department's haste, the quality is suspect. Hundreds of experts have concerns about the foundation of the assessment, including its validity, reliability, and fairness. Earlier this year, over 200 researchers in early childhood development and assessment signed a letter to Congress expressing significant reservations about the NRS, and the rush to put an assessment system in place in six to eight months.

2. More time is critical to develop and pilot the assessment in an appropriate way. Academic experts and Head Start staff alike are requesting a more scientific, thoughtful approach to the development of this assessment - one that includes some Congressional oversight. The National Academy of Sciences (NAS) report, "Eager to Learn: Educating Our Preschoolers," made it clear that more research on assessing young children is needed before such assessments should be used for accountability purposes. Because of this, it is crucial that the assessment instruments to be used in the NRS are properly validated and deemed to be appropriate for four year-old children. At this point, we simply do not know that this is the case.

3. The assessment has been developed and implemented in secrecy. This process has been surrounded by an air of secrecy. An assessment of this magnitude and importance must be developed and implemented with great care and openness. An independent body, such as the NAS, should examine how to appropriately assess young children. We know that good and useful assessments can be developed, even for young children, but they must be developed carefully and appropriately if they are to resolve, and not create, educational problems.

The process of developing assessment instruments should involve consultation with the public and experts in early childhood development and research methodology. Although HHS claims that they have had many meetings with "experts," there is little or no information publicly available that clarifies what went on at these meetings, what decisions were reached, and whether the advice of the experts was or was not heeded in developing the NRS. This process must be more open and public.

4. How the data will be analyzed and used is unclear. The Administration has been inconsistent in its statements about the use of the data, at times saying it's for program improvement and accountability and at other times saying it's for measuring individual child progress. Because results from this assessment could lead to de-funding programs, the assessment has the features of a high-stakes test - putting test pressures on young children and negatively impacting the way they view themselves and their ability to learn and achieve. In addition, linking test results to grantees' award amounts would create a disincentive to serving those children who most need Head Start. The Administration needs to be explicit about its intentions on uses of the data.

5. Expert reviews and pilot testing have been inadequate. The Administration claims there has been appropriate expert review and pilot testing. The process has not been responsive to expert and public concerns, leading to concerns about the adequacy of the review. Since the assessment has been made public, many experts in early childhood education and assessment have raised concerns about the test. Supposed feedback periods overlapped with the data collection and analysis phase of the pilot study, making feedback uninformed.

The Erikson Institute, a prestigious Chicago graduate school specializing in child development, surveyed national leaders in the early childhood field about the best way to measure achievement in preschoolers. In this survey, standardized testing (the method utilized by the NRS) was ranked tenth. Clearly, these experts do not view the methodology in use by the NRS as optimal.

6. The assessment is not balanced. The current NRS assessment is narrowly focused on literacy, language, and mathematics skills. Head Start is a comprehensive program that addresses child development through a whole-child approach, including social, emotional, physical, and cognitive development. The NRS at this time does not reflect development of the whole child. The assessment targets only a few of the skills that Head Start seeks to instill in children. For example, social skills are not being assessed; without them, children are simply not ready to learn.

7. The assessment may not be valid and reliable. Many questions surround the scientific soundness of the test. For example, the early math scale was adapted from a test for kindergarteners rather than from a test known to be valid for preschoolers. As another example, sections of the Spanish-version assessment have extremely low predictive validity. Many individuals have expressed concerns about the scientific soundness of the training for test givers. The NRS uses a "train-the-trainer" approach, which means that a relatively small number of Head Start staff nationwide receive training and then they are expected to return to their states and train the vast majority of test-givers. This is especially questionable since Head Start staff varies greatly in terms of education and ethnicity in ways that may lead to testing inconsistencies.

Sufficient time must be taken to ensure that English language learners are not placed at a disadvantage by having to take a test that is inappropriate for them. The test is currently available in English and Spanish, yet many Head Start children speak Asian or other languages. In addition, some Native American Head Start Directors are concerned that the NRS, in its current form, is not appropriate for their students, who often do not speak English in the home. Many Head Start directors question the appropriateness of these tests for children with developmental delays, speech and language delays, or other disabilities. We should take time to ensure that the assessment tool ultimately used is valid and reliable; assesses the gamut of skills that children acquire in Head Start; and is appropriate for children from a wide variety of cultural backgrounds.

8. The assessment could be culturally-biased and unfair. Many reviewers feel the assessment is not fair to children from diverse cultural and geographic backgrounds. For example, one question asks children to identify a swamp from a set of four drawings. Yet, how many children living in an inner city or on an arid Indian reservation have ever seen or experienced a swamp? Another question asks a four year-old in Head Start to distinguish between a vase and a decanter.

9. This year's assessment will not measure progress across the full year. The Administration has had to extend the deadline for receipt of fall data because it was unable to get testing materials to programs in a timely way. As a result, some children will not be tested until November, so their progress from August and September through November will be missed. Teachers and directors are very upset about this situation, wanting to ensure fair measuring of children's progress.

10. The assessment fails to include observations and teacher and parent input. The way data are measured is too narrow since it only includes a direct (face-to-face) child assessment. Direct assessments are useful, but they do not capture the full progress of the child at the preschool age. Multiple methods of data collection should be used for high-quality and accurate assessments, in which the NRS does not yet engage.

While the Senate bill is silent on the NRS - it does not mandate its use in determining program quality or in de-funding decisions - given the Administration's investment in this assessment and plans for the future, Congress must ensure that this assessment is not fully implemented until the test can be shown to be a reliable and appropriate measure, and be administered in a fair and even-handed manner.