The summer surge in federal immigration raids is hitting the housing market, pushing up labor costs and freezing projects.

Nearly two-thirds of builders said immigration enforcement is affecting their labor force, up from 23% in March and 13% a year ago, according to a September John Burns Research & Consulting survey.

“We’re almost dead in the water because our workforce is gone,” said Scott Brannock, president of Jacksonville-based Tidewater Homes and the Northeast Florida Builders Association. “We have national builders who are missing projections, losing hundreds of thousands of dollars because they have slabs sitting on the ground and no one to frame.”A September survey from the National Association of Home Builders found the cost of labor had increased a median of 4.2% from the previous year.

The shortages come as the country desperately tries to drive down sky-high housing costs that have trapped millions of Americans out of homeownership and stalled sales.

Immigration raids have primarily hit trades such as roofers, painters and framers. Meanwhile, the data center build-out in some areas has siphoned off specialized trades, drawing HVAC technicians and electricians with lucrative wages that home builders struggle to match.

Immigration-enforcement advocates say that deportations will drive down housing costs by removing competition for housing. They say builders should hire U.S. citizens.

“It is an absurd contention that we should stop deporting immigration lawbreakers simply because construction companies want to employ them,” said Matthew O’Brien, deputy executive director of the Federation for American Immigration Reform.

Last year, immigration arrests had only limited impact on the home-building industry, other than in select pockets of the country, such as South Texas. That changed this summer as the Trump administration scaled up immigration arrests to record levels.

Over the summer, the Jacksonville, Fla., area became a hot spot for arrests of residential-construction workers, with some subcontractors estimating 70% of their workforce has been lost. But the immigration crackdown has also started hitting builders in wider swaths of the country.

“It seems to have become ubiquitous,” said Jim Tobin, NAHB chief executive.

Paul Schwinghammer, co-founder of Hallmark Homes and former president of the Indiana Builders Association, estimated that in Indiana, immigration enforcement has taken out up to 25% of the residential-construction workforce.

In 2024, immigrants accounted for 26% of construction workers, a record high, according to the NAHB.

“Talking with large shell contractors, large builders, I don’t think they fully anticipated or estimated the amount of immigrant labor that are on our job sites every day,” said Brannock, the Jacksonville builder.

Builders report that immigration agents have frequently pulled over workers driving with construction materials and raided building sites.

“These operations target illegal employment networks that undermine American workers, destabilize labor markets, and threaten American communities,” a Department of Homeland Security representative said.

David Hacker, owner of subcontractor Construction Specialties of North Florida, said they have had to pick up trailers on the side of the road after crews were detained.

Hacker said he has seen a 70% reduction of crews available for certain jobs, forcing the leftover crews to double their rates in most cases. He has also started a campaign to recruit new workers, offering significant cash bonuses for applications.

“These are massive hits to the cost to build,” he said, “not to mention the delays.”The arrests have also spooked foreign-born workers with legal status, some of whom have been caught up in raids and repeatedly detained, taking them away from the job site for days at a time, builders and subcontractors say. Many legal residents have stopped showing up to work or fled to other parts of the country out of fear.

“People are afraid. If there’s any wind of anything happening, they’re not coming in for a week,” said Matt Risinger, a builder in Austin, Texas.

A June National Bureau of Economic Research paper found that each single arrest by U.S. Immigration and Customs Enforcement causes an average of seven likely undocumented immigrant men to stop working.

Building a house is largely a linear process—one stage can’t occur until another has been completed. “That costs money in interest, site labor, site management, rental equipment, utilities, insurance,” said Schwinghammer, the Indiana builder.

And even if contractors are able to find workers, the labor shortage means they have to pay them significantly more. Brannock, the Jacksonville builder, said local framers’ wages used to work out to around $20 an hour. Now, that is up to $40 to $45 an hour. For the average 2,500-square-foot house, framing labor, which used to cost $10,000, now runs builders $25,000 to $28,000, he said.

For 18 years, Eddy Velazquez did siding, framing, roofing and bathroom remodels in Florida. He typically worked on 10 different homes at any given time and had plans to build his own house to sell later this year. “I really love the construction,” he said.

He was pulled over in March on his way to a job. After a month-and-a-half in detention, Velazquez was deported to his native Mexico, leaving the crew he was part of without one of its most experienced workers.

Builders and subcontractors say it will be costly and time-consuming to replace workers like Velazquez with U.S. citizens. “There’s no one lining up to do a lot of these jobs,” said Josh Putnal, the owner of a siding-and-soffit subcontractor in Jacksonville.

Write to Nicholas G. Miller at nicholas.miller@wsj.com