By Jeanne Sahadi

NEW YORK (CNNMoney) -- Many people complain about paying too much in federal income taxes. But it turns out that a majority of U.S. tax filers actually pay less in income taxes than they do in payroll taxes, which fund Social Security and Medicare.

That's according to Roberton Williams, a senior fellow at the Tax Policy Center.

Williams estimates that this year close to two-thirds of all households (62.3%) will end up owing more in payroll taxes than federal income taxes. Generally speaking, the less you make, the more likely you are to be in this group.

For Social Security the payroll tax amounts to 12.4% of your first $118,500 in wages. Half of that is deducted directly from your paycheck and half is paid by your employer. The Medicare tax, meanwhile, is 2.9% on all your wages -- again with your employer picking up half the tab.

Williams made his estimate based on the assumption that workers effectively bear the burden of both the employee and employer share. His reasoning: a worker's wages would be higher if his company didn't have to pay any Social Security or Medicare taxes on his behalf. If you assumed the worker bore just the employee share, the percentage of households with a higher payroll tax liability than income tax liability drops to 45%.

In any case, what does all this mean in dollars and cents?

For households with $50,000 in income, the total payroll tax tab comes to $7,650 a year, with $3,825 of it coming directly out of your paycheck.

The total bite on a $100,000 income would be $15,300 for the year, with half paid directly by the wage earner.

Why are payroll taxes a bigger bite for so many?

A lot of attention is given to how many tax breaks high-income households get.

But the truth is there are also a lot of valuable tax breaks available to lower and middle income households, which have the effect of greatly reducing -- if not eliminating entirely -- their income tax bite.

Williams estimates that 44% of all households will end up with no income tax liability this year. But a majority of those do pay payroll taxes because members of the household have jobs.

The rest end up owing neither income nor payroll taxes. While that may seem like a great deal, it's not. "More than half [of people in this situation] are retirees living on Social Security and too little other income to owe income tax. The rest have no jobs and not much income. That isn't a club many would want to join," Williams wrote in TaxVox.

TM & © 2016 Cable News Network, Inc., a Time Warner Company. All rights reserved.