LIVINGSTON, NJ – In response to several Livingston residents who raised concerns over the validity of the recently administered Alcohol & Drug Abuse Prevention Team (ADAPT) Coalition survey at last week’s Livingston Board of Education (LBOE) meeting, Superintendent of Schools Christina Steffner followed up with both ADAPT and Epiphany Community Services (ECS) to get answers to the community’s questions and presented her findings at Monday’s LBOE meeting.
The survey was fielded last month to students who voluntarily opted into taking it during their school day in health classes at Mt. Pleasant Middle School, Heritage Middle School and Livingston High School.
Last administered in the township in 2014, the ADAPT survey strives to take a snapshot of attitudes that students have toward alcohol, drugs, tobacco use and substance abuse at two-to three-year intervals. Since the survey’s inception in 2012, it has been used in 17 unique districts in New Jersey and has been administered to nearly 8,200 students.
ADAPT works in concert with ECS to analyze survey results and give feedback on survey methodology and data corruption, upon request.
In response to issues raised around the potential of data corruption during the public comment portion of last week’s LBOE meeting and the claim by one parent that he was able to repeatedly take and submit the survey while off school premises, Steffner reached out to ADAPT and ECS in follow up.
According to Steffner, in order for ECS to include a survey in its final analysis, each survey must have been taken on school grounds at the appointed time, be 100-percent completed and include consistent responses on sets of questions designed to be paired and to yield like answers.
ECS was given all three of Livingston schools’ respondent data to review and analyze. The first step in that process was to put the data through a “cleaning process,” which tests for validity, reliability, accuracy and identification of potential limitations of collected data, according to Steffner.
Of the nearly 3,000 surveys originally submitted, 756 were found by ECS to be unusable. Of the unusable data, 425 surveys were found to be less than 75-percent complete.
Sixteen survey respondents failed a test question formulated to assess a respondent’s level of honesty when taking the survey by looking at whether the respondent replied affirmatively to having knowledge of a fictitious drug used in surveys of this type. An additional 90 surveys were rejected for including inconsistent responses between pairs of questions asking the same thing in a slightly different way.
“We then asked ECS to go back and look at surveys that weren’t started within 10 minutes of their being administered in the classroom,” said Steffner. “Those 225 surveys were also rejected, as we assumed that if the students were taking the survey with the teacher in the room, they would have started within that time frame.”
Ultimately, 2,270 surveys were found to be usable. These surveys are considered valid and reliable by the industry standard with a margin of error of 95 percent—a very high confidence rating, according to Steffner.
“At this point, ECS feels they have a great deal of data they can count on as being accurate, valid and reliable,” reported Steffner. “They also feel that Livingston Schools provided some really good data that is representative of the students who took the survey.”
ADAPT Coalition Senior Coordinator Joel Torres was also on hand at the meeting to speak about the methodology and eventual goal of the survey.
Torres discussed the limited funding associated with the administration of the ADAPT Survey as a reason why it hasn’t been used more in Essex County. Because the survey is free to take and utilizes limited grant dollars, it is typically administered to only two or three school districts within Essex County every few years. This year, the survey is being distributed in both Livingston and East Orange.
Data from the surveys is not publicly shared, although schools can request a private report for their own uses. Additionally, because each school’s information is kept private, the county-wide assessment that ADAPT eventually releases focuses on trends, rather than on data specific to each district queried.
“The goal of the survey isn’t to pinpoint what various towns are doing right or wrong when it comes to substance abuse,” said Torres. “It’s more to show that the programs that ADAPT provides are effective, as well as both evidence and research-based. These results will help us make sure that the programming we’re offering is going to be helping and supporting the community.”
Results from the Livingston survey will be ready within a month, with the results of the county-wide survey becoming available a few months following that.