The Tennessee Department of Education is trying to regroup from a rash of spring failures with TNReady testing, but state lawmakers question whether the state and its vendor can pull off a smooth test despite promises to do better.
“Implementation and execution of the TNReady testing has been flawed since its inception. Because of the problems, it is difficult to assess if the internal structure or test content is valid,” State Rep. Bryan Terry said after sitting in on a summer presentation to House Education Committee members.
Some 650,000 students in grades 3-12 took TNReady this spring, the statewide test used to see how students are progressing, and about 300,000 of them did so online, including all high school students. The results are used to evaluate schools, administrators and teachers, but that is being put on hold this year because of legislative action amid foul-ups with the spring testing, situations in which some students were unable to log on or, in some cases, had their test results vanish.
Initially, officials said the problems stemmed from a cyberattack. But they recanted that after determining botched testing was caused by several factors: interference between a test platform and practice test; an unauthorized change by the vendor, Questar, in text-to-speech required for online connection; an unscheduled maintenance by a Questar employee; and a fiber-optic cable outage.
The state responded by cutting Questar’s $30 million contract by $2.5 million, amending Questar’s contract and putting together a new request for proposals, in addition to hiring a consultant to analyze Questar’s technological capability. The department of education also will be part of a multi-state “stress test” this fall to see how the testing system works.
In addition, the state is putting together a TNReady Ambassadors Program within the department to focus on “the customer experience.”
“We really made a concerted effort to get feedback to prepare” for the next round of testing, said Elizabeth Fiveash, who heads policy and legislative affairs for the education department.
House Education Committee members were unimpressed with the presentation, pointing out the spring 2018 failures continued a troubling pattern of miscues over three years.
State Rep. Terry Lynn Weaver, a Lancaster Republican, said she felt the state should get its money back from Questar. She also raised questions about evaluations for prekindergarten, kindergarten and first-grade teachers that are causing low teacher scores because of problems logging student test results into a computer system, a matter separate from TNReady.
“Right now my morale with you guys is zip, nada, nothing, so that’s all I’ve got to say,” Weaver told education officials. “I just don’t have any confidence in what you’re bringing here with these numbers and slides, because basically right now they don’t mean doodly-squat.”
An analysis done by Hum-RRO determined online disruptions had “minimal impact” on overall test scores, and Fiveash said the number affected was in the “hundreds,” not in the “thousands.”
Other factors such as “local conversations and media coverage” might have hurt student performance and motivation, according to the state’s presentation.
House Education Committee members were unimpressed, for the most part, with the presentation. But even though several legislators said they wanted to get rid of Questar, Fiveash said there isn’t enough time this year to switch to another vendor.
Instead, the education department will move forward on the premise TNReady “is the right test for Tennessee students,” Fiveash said.
Terry contends the department of education took the wrong track by moving forward without trials on testing, leading to “invalid results” and “frustration from students, parents and teachers.”
“Plans, ideas and rhetoric, often without a track record can inspire hope, but confidence is something that is earned,” Terry said. “Our department of education and the vendors have yet to earn my confidence nor the confidence of much of those impacted by their failures. At this point, folks are tired of the rhetoric and want to see results.”
Overall, state scores from TNReady showed poor results. Grades 3-5 improved slightly in English and language arts, stood pat in math but dropped in science. Grades 6-8 dropped slightly in all three areas, and grades 9-12 fell off in English and language arts, science and U.S. history while making minimal improvement in math.