Two Years Before Online Attack, Auditors Warned Los Angeles Unified About Cybersecurity. How Bad Were The Problems They Found?
In September 2020, cybersecurity auditors convinced Los Angeles Unified School District staff to hand over their passwords, tricked those employees into “execut[ing] malicious codes,” and gained access to a “limited number of Social Security numbers” stored on school district systems.
Those security liabilities were documented in a September 2020 report that will help district leaders decipher why the district fell victim to a ransomware attack over the Labor Day holiday weekend — 24 months to the day after the auditors finished their work.
Those old findings highlight real fears among teachers and parents about the recent breach: that hackers might have been able to access sensitive data on staff or students. Superintendent Alberto Carvalho has acknowledged that possibility is hard to rule out — though he has also expressed cautious optimism that a rapid response was able to safeguard some information.
How Did LAUSD's Cybersecurity Measure Up To Other Districts?
It’s unclear — at least from the deliberately vague public version of the cybersecurity audit released in January 2021 — whether there’s a link between the Labor Day weekend breach of LAUSD’s systems and the vulnerabilities auditors found two years ago.
However, several cybersecurity experts who reviewed the report also told LAist that the audit findings suggested LAUSD was taking online threats relatively seriously — and that some of the deficiencies in the district’s defenses in 2020, while perhaps alarming on some levels, are still commonplace among comparable institutions.
- “I’ve seen audits where the house is on fire. This is not one of those,” said Nick Merrill, a research fellow at the Center for Long-Term Cybersecurity at UC Berkeley.
- “I didn’t read this audit as being particularly damning of the cybersecurity practices of the district. That’s not to say they couldn’t be doing more and better, but compared to the … average school district, they’re not doing worse,” said Doug Levin, director of the K-12 Security Information Exchange, a non-profit that advises districts and tracks cybersecurity threats against schools.
- “There are things that clearly [LAUSD] could've done better,” said Clifford Neuman, director of the USC Center for Computer Systems Security, “but in general, my impression was that while they’re not the best, they’re not the worst here."
LAUSD’s Office of the Inspector General, the district’s independent internal watchdog, originally commissioned the cybersecurity audit and published the report. William Stern, who led the office at the time, declined to comment to LAist this week. Deputy IG Austin Onwualu has not responded to a request for comment.
Carvalho issued a statement saying that LAUSD plans to convene an independent task force that would “review all previous network audits and reports,” including the findings from 2020.
“I want our new task force to take a deep dive into the recommendations and implementation progress of this security audit,” Carvalho’s statement said. “This incident has been a firm reminder that cybersecurity threats pose a real risk for our district — and districts across the nation.”
How Bad Were The Weaknesses Auditors Found In 2020?
The firm that conducted the cybersecurity audit, Crowe LLP, shared its full findings with LAUSD officials in a confidential report. The public, “redacted” version of the report withheld many key details to avoid laying out a roadmap to LAUSD’s systems for hackers. As a consequence, however, critical context about the weaknesses in LAUSD’s defenses is not part of the public record.
Despite this lack of specifics, the auditors’ 2020 finding that LAUSD’s password and credential controls were lacking stood out to experts.
Levin said it’s a common weakness: “Time and time again, we see districts compromised via poor password management practices.”
Beefing up credential systems can provide a critical line of defense. Under a “multi-factor authentication” login system, users must present both a password and a second credential to gain access. Ideally, this second “factor” comes from the user responding to a push notification on their phone through a specialized app — though there are other methods, like entering a fingerprint or a physical key, or responding to a text message or phone call.
LAUSD has been rolling out multi-factor authentication — and reportedly was in the process of making it mandatory. In a statement this week, Carvalho said the district is now “expediting the rollout of a multi-factor authentication process.”
Two years ago, auditors were able to access a “limited” number of Social Security numbers stored on LAUSD systems — a finding that likely stemmed directly from lax password protections in the district at the time, Levin said.
Many parents, many educators presume that schools are doing more to protect that information and data … than they actually are.
A Social Security number alone is not necessarily critical to stealing someone’s identity, Levin said. Practically speaking, Social Security numbers are widely available on the dark web for cheap. Merrill said it’s likely the apps on students’ phones — ahem, TikTok — are likely more pressing cybersecurity risks.
However, the missing context from the report becomes crucial here: it matters which user accounts the auditor was able to breach — an individual student, or a high-ranking administrator? — and what systems they had permission to access.
“My concern is that if they’ve got access to a Social Security number, they’ve got access to a lot of other things,” Neuman said.
While it’s early in the investigation, Carvalho has said investigators have no evidence that “critical health information” or employee Social Security numbers were “compromised” in this weekend’s attack. He said the district’s payroll system is “undisturbed.” He said the intrusion was first detected in a system that stored data on facilities contracts, most of which wasn’t confidential and likely publicly disclosable under open records laws.
Earlier this week, Chief Information Officer Soheil Katal said LAUSD does not collect student Social Security numbers, though the district does have all kinds of sensitive information on students.
Does The 2020 Report Show Whether LAUSD Was Prepared?
The experts weren’t surprised that, two years ago, auditors were able to successfully trick LAUSD employees into sharing their passwords and “unknowingly execute malicious codes.”
“Yeah, this is bad. We don’t like it,” said Merrill. However, “putting it in the context of real institutions … this is totally normal.”
Levin agreed, saying this is also exactly the sort of cybersecurity test that “99%” of institutions would not pass.
Few institutions can completely harden themselves against hackers intent on using “social engineering” to break into an online system. In some ways, a school district is uniquely vulnerable to these scams, with so many teachers, students and staff regularly using its systems — and certain groups of users, like parents, who can’t be expected to, for example, sit through cybersecurity training.
“There is no system that is 100% secure,” said Neuman. “A system that is as large as this, and so unstructured … there are always going to be ways in.”
Experts said the standard isn’t for an institution to prevent every user from opening every suspicious attachment or clicking every bad link. Rather, the key question is: what does the institution do when a threat first seeps through?
“Once someone clicks, then what happens?” Levin said. “What protections are in place to keep malware from executing? What practices are in place to stop it from spreading? What practices are in place to alert the cybersecurity team in the district to warn other users not to click?”
The 2020 audit report does fault LAUSD for not providing training to key personnel, like the Information Technology Division’s security team, on what to do in the event of a cybersecurity breach.
LAUSD did not make officials available for an interview on this story — but experts said that what looks like a rapid response to the Labor Day weekend ransomware attack suggests that officials did have a response plan in place.
Merrill also noted the audit found that, even in 2020, LAUSD had “segmented” its network. The process of dividing the district’s network into smaller, distinct sub-networks is a fundamental cybersecurity step, Merrill said.
“I’m seeing an organization that was more-or-less taking security seriously,” said Merrill, who has been working in cybersecurity for about 10 years. “I’m not going to say this was perfect, I’m not going to say they were as aware as a school district, or any institution, should be.”
“Am I confident they got this audit report back and improved their security substantively? Yeah,” Merrill added, “Looking at this ransomware attack, they recovered pretty quickly.”
Experts said it’s plausible for officials to suggest that LAUSD’s fast response — which involved a total shutdown of all online systems — may have limited the damage in the attack, or prevented sensitive information from being compromised.
But how far along was the attack before LAUSD caught it?
In ransomware attacks in school districts over the last year-and-a-half, Levin said the malicious actors — in virtually every case — have compromised the school district’s systems “weeks or even months” before the attack itself, often quietly stealing data all along.
“The very last step is to execute the ransomware — to encrypt the files and demand the ransom,” Levin said.
“If they caught the threat actor in the process of exfiltrating the data, perhaps they have greater confidence that they caught it in time and that things were not exposed,” Levin said. “If, however, it was after the ransomware started to execute and files were encrypted, I would have many more questions then.”
Carvalho said on Tuesday that the district had received no ransom demand.
Growing Risk, Lack Of Standards
As the cyberattack begat hastily rewritten lesson plans, a plodding process of changing 700,000 district passwords lurched forward in fits and starts. Many students have been locked out of their data for a week now as they wait to reset their passwords.
For some teachers, the superintendent’s statements have done little to reassure them so far.
“In my experience with the analysis of cyberattacks, especially ransomware attacks in the past, there’s oftentimes personal data that has been exposed,” said Wil Page, an instructional technology coach at King Middle School in Silver Lake. “While I hope that this was able to get stopped in time, I just don’t know that that’s true.”
Levin said the attack in LAUSD highlights a growing risk to school districts. In a 2022 report, his organization documented more than 1,300 data breaches, ransomware attacks and other instructions into school district systems — a figure that has grown every year since the K-12 Security Exchange began tracking these incidents.
The school systems in Las Vegas; Baltimore County, Md.; and Fairfax County, Va., have all faced ransomware attacks recently. So did Carvalho’s former district, the Miami-Dade County Public Schools, while he was superintendent.
In the cybersecurity world, Levin said there are few minimum standards to which school districts must adhere.
“Many parents, many educators presume that schools are doing more to protect that information and data … than they actually are,” Levin said.
“We trust [schools] with a lot of sensitive information … There should be an expectation that that information is appropriately secured from unauthorized access or cybercriminals,” Levin added. “There isn’t a culture or a history or a regulatory expectation that schools provide that sort of protection in any way like we would hold banks to or hospitals to.”
“And that, in and of itself, is the risk with schools.”
Stephanie Moran Reed had to say goodbye in January to the bookstore she founded with her husband. The MiJa Books owner opens up on customer experiences, mom guilt, and a favorite book recommendation.
Anti-Latino slurs were published on the cover of a CSU student newspaper in October. The painful incident led to protests and soul searching at the mostly Latino campus.
Many California students live doubled-up with other families and friends.
Madeline wrote to the county in mid-November asking for approval to have a unicorn in her backyard. Now the hunt begins.
This museum helps students explore the connections between prejudice, anti-Semitism and the murder of more than six million Jewish people and so many others.
Prowl with P-22, trace the life of a famed Black architect, Paul R. Williams, and meet a flower-spouting monster. There’s an illustrated story for readers of all kinds.