Did California’s Massive COVID Homeless Shelter Program Work? A New Evaluation Probes the Results

By Marisa Kendall, Calmatters

California’s massive effort to shelter homeless residents during the COVID-19 pandemic was a success, according to a new report that says the effort changed the state’s homeless services system for the better.

But at the same time, the researchers pointed out a troubling dearth of available data on the program. With the little information they were able to access, they found that people who left the program had at least a 40% chance of returning to homelessness.

Project Roomkey, one of Gov. Gavin Newsom’s signature pandemic initiatives, temporarily moved about 62,000 homeless Californians into hotel rooms in an effort to prevent them from catching COVID. The model, which provided residents with their own private room and bathroom, as well as (in some cases) healthcare and other services, was a world apart from traditional shelters where dozens of people sleep together in a room and services are limited.

The independent evaluation of the program, released Monday evening, says Roomkey succeeded in saving lives while also shifting the way the state thinks about homeless shelters.“I believe the program was successful and I believe it has a further reach beyond just what happened during the program itself, because many communities and many providers have seen the value of this type of program and are trying to continue with it,” said Nichole Fiore, a principal associate with research firm Abt Global who co-authored the report. The report was funded by the California Health Care Foundation and the Conrad N. Hilton Foundation.

But the researchers acknowledged large holes in the accessible state and local data. For example, they couldn’t cross-reference death records or information about participants’ health, meaning they had no way to prove Roomkey prevented deaths or kept people healthier.

Initial responses to public records requests filed by CalMatters revealed dozens of deaths in Roomkey programs spread across the state.

In Ventura County, 32 people died in Roomkey hotel rooms, the majority “due to health conditions and COVID related,” said Jennifer Harkey, program director of the Ventura County Continuum of Care. Twelve people died in San Diego County, eight in Tulare County and seven elderly people in Riverside County, officials told CalMatters.

Still, officials say the individual hotel rooms saved many more people from the more hazardous alternatives of staying on the street or crowding into traditional group shelters.

“This has been a highly successful program by offering safety and stability to highly vulnerable unsheltered people,” Harkey told CalMatters in a statement. “This has proven that non-congregate shelter is a model we need to continue in our community.”

In addition, though many program operators tried to move people into permanent housing as COVID vaccines were released and Roomkey hotels closed, they succeeded less than a quarter of the time.

“Project Roomkey exemplifies what’s possible when government thinks outside the box and acts with urgency,” Gov. Gavin Newsom said in an emailed statement responding to the evaluation. “By leveraging existing hotels and motels, the state, in partnership with local jurisdictions, rapidly housed thousands of individuals in need of assistance.”

Did Project Roomkey end people’s homelessness?

Project Roomkey launched in March 2020 and was most heavily used between April 2020 and June 2021. It primarily accepted residents who were considered particularly vulnerable to COVID because they were 65 and older or had medical conditions such as asthma, diabetes or heart conditions.

It started as a hastily-deployed health program — its primary goal was to prevent people from dying of COVID-19, not to permanently end their homelessness. At the outset, both state and local officials expected it would operate for just a few months. As the pandemic dragged on and the hotels stayed open much longer than anticipated, many counties began trying to move Roomkey participants into permanent housing.

As of January, 22% of people leaving Roomkey programs throughout the state went to permanent housing, and 11% to temporary housing, according to state data compiled by the researchers. Another 40% remained homeless: 25% went into other emergency shelters and 15% went back to the street. Another 18% went to unknown destinations, meaning they didn’t tell staff where they were going, or staff didn’t track it.

The positive connotation the researchers gave those results shows just how low the bar is for homelessness programs. Housing is notoriously scarce and expensive in California, and there is a massive shortage of rental subsidies and other resources to help pay people’s rent.

“22% is actually pretty good for an exit into permanent housing,” Fiore said.

The state spent $24 billion on housing and homelessness over the past five years, but as the crisis continues to get worse, lawmakers increasingly are demanding to know where that money is going and why it isn’t getting better results. A recent state audit found huge gaps in the data collected on massive California-wide homelessness programs. And legislators in the state assembly held an oversight hearing on homelessness Monday.

Because Roomkey was so large — and started when counties had access to an unprecedented amount of federal COVID relief dollars that could be spent on housing — the sheer number of people Roomkey permanently housed was impressive, said Vivian Wan, chief executive officer of Abode Services, which ran 16 Roomkey sites in Santa Clara, Alameda and Napa counties. More than 14,000 Californians moved from Roomkey into a permanent home, according to state data obtained by CalMatters.

“More people got housed during this time than any other time that I know of,” Wan said.

Even so, she was hesitant to say 22% was a “good” result, percentage-wise. By comparison, people who left Abode’s shelters in Alameda County last year moved into permanent housing 24% of the time.

San Francisco’s largest navigation center (a shelter that provides beds and other services) reported just 8% of the people who left its program ended up in permanent housing in 2022.

Roomkey also didn’t always do a better job of ending people’s homelessness than other shelters, and housing placements varied widely county by county.

Some counties struggled even to track that data.

Officials in Fresno County, for example, said that in the frenzy of opening nine Roomkey projects originally planned to operate for just a few weeks or months, not all site operators tracked where residents went.

There was no data available for four of the county’s nine sites. Officials reported that about 120 out of the nearly 670 people served by the remaining sites — or roughly 18% — moved into permanent housing.

Amina Flores-Becker grew up in Fresno and has watched housing become unaffordable for many middle class and low-income neighbors. One major challenge with Roomkey, the deputy county administrative officer said, was a lack of housing options for people who left hotels, leading many back to the street.

“There was definitely a mass exodus from those shelters when that went away,” Flores-Becker told CalMatters in an interview earlier this year.

One takeaway emphasized by the Abt researchers: The longer people stayed in Roomkey, the more likely they were to move on to permanent housing (at least in Tulare, Los Angeles and Ventura counties, the only counties for which the researchers obtained detailed outcome data). In Los Angeles County, more than half of those who stayed in Roomkey 18 months or longer moved into permanent housing.

The researchers also compared the demographics of Roomkey residents with residents of other shelter programs in Los Angeles County. Roomkey residents were more likely to be White and less likely to be Black .

Researchers struggled with a lack of data

The evaluation determined Roomkey met its goal of saving lives (a claim anecdotally backed by experts who say the state didn’t see the flood of homeless deaths during the pandemic that they initially feared). But the Abt researchers weren’t able to obtain data on deaths or health outcomes as proof. They tried to get participant data from the state’s homeless services system to cross-reference with Medi-Cal and death record data — but they couldn’t, due to data privacy restrictions and other bureaucratic roadblocks, Fiore said.

Many people came to Roomkey from crowded shelters where as many as 300 people might share a room — perfect conditions for COVID to spread, Wan said. Without Roomkey, Wan suspects homeless shelters might have seen COVID mortality rates similar to those of nursing homes – which, at the height of the pandemic, accounted for more than 40% of California’s COVID-related deaths.

“I can’t imagine the spread had we not been able to decompress (homeless shelters),” she said. “People sleep nose to nose in many of these sites.”

In the absence of extensive statewide data, the Abt researchers surveyed and interviewed Roomkey providers and participants throughout California. They sent an online survey to all counties that participated in the program, and conducted interviews with city, county and program staff in 15 of those communities. They also went on site visits to Los Angeles, San Francisco, Tulare, Ventura and Santa Cruz counties.

The researchers asked for data on participant outcomes from the five counties where they conducted site visits, but Los Angeles, Tulare and Ventura were the only ones that complied.

That lack of access to data is a problem, Fiore said.

“It’s critical to be able to collect, analyze and evaluate that data across systems,” she said. “And right now all the data for those benefits are sitting in different systems.”

Roomkey gets positive reviews, despite problems

Most Roomkey participants surveyed reported having a “positive experience,” according to the Abt report. They said they particularly appreciated having their own bathroom.

“It really was a place for people to stabilize,” Fiore said. “Get rest. Get nourishment. Get services.”

One participant from Ventura County told the researchers that he could refrigerate his insulin, consistently take his medication and make his doctor’s appointments.

But the Roomkey hotels were not designed for long-term living, which posed challenges as the pandemic continued for multiple years, according to the evaluation. Residents couldn’t cook (not even on hotplates) and the minifridges in the rooms could store only enough food for one or two days. Roomkey staff told the researchers the motels tended to be “decrepit” and didn’t have enough rooms accessible for people with disabilities. Some residents complained of cockroaches and rodents.

Roomkey programs also struggled to provide enough services. Many residents had been living on the street for a long time and suffered from chronic health conditions such as kidney failure, cancer and seizures, as well as PTSD and other mental health issues. Some counties, such as Los Angeles, provided on-site nurses. San Francisco provided counseling and therapy.

Wan said some of her team’s Roomkey sites could have used more health services. “Frankly, because there just weren’t enough resources to go around,” she said.

Statewide, Roomkey staff also enforced strict rules to prevent the spread of COVID, such as requiring residents to mostly stay in their rooms and prohibiting guests. In some cases, the isolation exacerbated residents’ mental health issues.

“Physically I was getting better when I was at the (shelter-in-place) hotel, but mentally…I was too isolated,”one participant told the researchers. “I needed more interaction. It eventually came to a point where I was talking to the TV and arguing with myself in my room.”

Some participants left the program because of those rules.

Funding was another challenge. Cities and counties ramped up their Roomkey programs with the understanding that the federal government would foot a large chunk of the bill. But the Federal Emergency Management Agency recently said it wouldn’t reimburse them for Roomkey stays of longer than 20 days between June 11, 2021 and May 11, 2023.

That limit, which state and local officials are urging the federal government to reverse, will cost California and local governments more than $300 million, according to an estimate from the governor’s Office of Emergency Services.

But the program has made its mark on California. For example, hotels are a key part of the homelessness strategy in Los Angeles, where the new Inside Safe initiative moves people from encampments into hotels. But like Roomkey, that program has struggled to provide residents with enough health and mental health services, and to move people from the hotels into permanent housing.

Roomkey also helped dispel a myth that Fiore says is often perpetuated about homeless residents: That they don’t want to live inside. People who never accepted beds in traditional shelters were willing to try Roomkey, she said.

“People will come indoors if they are offered autonomy, safety, privacy, if they’re able to keep their partners, their pets, their possessions,” Fiore said. “When their needs are met and their needs are considered, then people will come indoors.”

CalMatters investigative reporter Lauren Hepler contributed to this story.

For the record: Both the California Health Care Foundation and the Conrad N. Hilton Foundation, which funded the independent report, also are among funders of CalMatters. Our news judgments are made independently and not on the basis of donor support.

This article was originally published by CalMatters.