By Laura Jarrett

NEW YORK (CNNMoney) -- The victims' stories were heartbreaking, like the 85-year-old woman in San Diego who was fleeced out of $12,300.

Or the man in Hayward, California, who was repeatedly terrorized into paying up until his account was drained of $136,000.

They were among an estimated 15,000 people who were allegedly swindled by a network of call center workers in India who posed as IRS or other federal agents and used the threat of arrest, fines or deportation to intimidate their victims.

An indictment unsealed Thursday revealed that 61 defendants have been swept up in the fraud scheme that prosecutors say stole more than $300 million. Twenty individuals across eight states in the U.S. have been arrested so far, and five call centers in India have been charged.

It was the second time this month that Indian call centers have been implicated in a scam where fraudsters pose as IRS agents. Police in India arrested 70 people working at nine call centers on similar charges.

The scam unveiled Thursday specifically preyed on the elderly and immigrants who feared deportation.

Using information obtained from data brokers and other sources, call center operators allegedly called potential victims and threatened arrest, imprisonment, fines or deportation if they did not pay taxes or penalties to the government, according to the indictment.

"If you get one of these calls, it is not the U.S. government calling you.... It's a scam," cautioned Assistant Attorney General Leslie R. Caldwell of the Criminal Division.

Because the alleged crimes took place in the U.S. where victims were called, prosecutors say they have federal jurisdiction even though many of the defendants are in India.

TM & © 2016 Cable News Network, Inc., a Time Warner Company. All rights reserved.