By Emanuella Grinberg and Jay Croft CNN

(CNN) -- Jim Serowski knew his employees planned to skip work last Thursday.

They told him they wanted to participate in "A Day Without Immigrants," the nationwide day of protest. Serowski, founder of JVS Masonry in Commerce City, Colorado, said his position to them was clear:

"If you're going to stand up for what you believe in, you have to be willing to pay the price."

When Serowski's foreman and some 30 bricklayers failed to show up for work, he fired them all with no regrets, he said.

People skipped work and businesses closed on Thursday to demonstrate immigrants' combined contribution to the nation's economy in the face of President Donald Trump's tough stance on immigration.

Many simply went back to work Friday. Others, including Serowski's employees, received phone calls or text messages telling them not to come back.

Several business owners and managers across the country told CNN their decision to fire protesters was not about politics or immigration. They said they are running their businesses responsibly.

'A slap in the face'

As news of the firings spread, backlash was swift. Serowski said he received phone calls throughout the weekend criticizing his decision, followed by messages of support.

Serowski stands by the decision, saying he was not motivated by politics.

"It's about work ethic," he said by phone Sunday. "They were warned: 'If you do this, you're hurting the company, and if you go against the team, you're not a member of the team.'"

His foreman told CNN affiliate KDVR that it was a cause close to his heart. He and his masonry workers know people who are afraid to leave their homes out of fear they may get arrested or deported, he said.

But Serowski said it was a slap in the face to him, the grandson of immigrants from Poland, someone who has long supported immigrant labor.

He said he's known many of his employees for nearly two decades and has always taken care of them financially, even when business was slow.

"I've gone above and beyond for these people," he said, seemingly distraught. "No one is going to dictate how my company is run."

'We don't make money'

Steve Deese, owner of Encore Boat Builders in Lexington, South Carolina, confirmed firing 21 employees who didn't show up Thursday.

Deese said they broke company policy by not giving notice and not calling in to say they would be away from work. The employees were given an opportunity to provide a doctor's note, but failed to do so, he added.

The company does not tolerate violations, no matter who the employees are, he said.

Deese's company builds boats "to make money, so when we can't build boats, we don't make money," Deese said. "We run an assembly line, and when 21 people call out, that seriously affects our operation."

Those 21 employees were replaced by new workers who are being trained.

Warned of the consequences

Robert Peal, attorney for Bradley Coatings Inc., based in Nashville, confirmed to CNN that 18 employees were terminated for failing to show up Thursday.

Peal said one of the line supervisors, an immigrant, was alerted to the possibility of the no-shows and spoke to each employee individually, warning them of consequences.

Bradley Coatings is a commercial painting company that works on tight deadlines and depends on employee attendance to meet its obligations, he said.

The company owner is a longtime supporter of immigrant causes and makes mission trips to Central America, Peal said. The company previously had donated materials and labor to one of the employees who was terminated Thursday after that worker lost his home to a fire.

Peal said it was a business decision that had nothing to do with the nature of the protests.

At the I Don't Care Bar and Grill

Bill McNally, owner of I Don't Care Bar and Grill in Catoosa, Oklahoma, said his 12 line cooks gave him no heads-up. They didn't even call to say they were not coming in.

So they were fired, just like anyone else who's a no-show for work, he said.

"I'm on their side, but we have rules at I Don't Care Bar and Grill," he said. "If you're going to be late, call in. If you're not coming to work, call us. That's the American way."

News of the firings sparked criticism and heated debate on the restaurant's Facebook page.

Beneath a job posting to replace the fired employees, some customers vowed to never return to the restaurant. Others stood by the restaurant, saying the workers deserved to be fired for not showing up.

As one critic of the decision wrote, addressing the business, "You missed the larger picture. What you fired were not employees who did not show up. You fired employees who are willing to fight for their principles and yours. Because when one group of people is harassed the next group may be yours."

And, as someone else noted in response, "The larger picture: Those who stand with lawbreakers and condone lawbreaking need to suffer consequences for their actions."

Six of the fired Hispanic immigrant workers told CNN affiliate KTUL they felt they had been unfairly terminated. They wanted to stand in solidarity with other immigrants, they said through a translator. They didn't think it would cost them their jobs.

Under US labor law, at-will workers can be fired with no warning and without a stated reason.

McNally also said his action had nothing to do with politics. Had he known about the protests, he said he would have closed the restaurant in solidarity.

Why fire them, then?

"They just forgot about the 50 other people who work here," he said. "If the cooks don't show up, then servers don't have jobs, and customers can't eat."

Not so fast in New York

A restaurant owner in New York disputed local media reports that he similarly had fired a group of protest participants. The owner of Ben's Kosher Delicatessen Restaurant & Caterers disputed the story and said he is a longtime supporter of minority rights.

Ronnie Dragoon, of Ben's Kosher Delicatessen, said he fired just one employee -- and only because that worker pressured others to walk out and physically threatened two of them.

"All other employees involved with the walkout were, and still are, invited to return to their positions with the company," the local restaurant chain said.

Added Dragoon, "And yes, I am the grandchild of immigrants, and I have been a progressive my entire life and will continue to be a progressive."

TM & © 2017 Cable News Network, Inc., a Time Warner Company. All rights reserved.